mixing open and commercia tools · companies use open source because… cost is the main reason why...

30
Mixing Open and Commercial Tools Mauro Garofalo, Maveryx W22 Automation

Upload: others

Post on 08-Oct-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Mixing Open and Commercial

Tools

Mauro Garofalo, Maveryx

W22 – Automation

Page 2: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Summary

Costs, Advantages and Risks of mixing open

source and commercial software

A case study of blending open source and

commercial tools for testing Java applications

Some practical rules to combine open and

commercial test tools

2

All trademarks referenced herein are the properties of their respective owners.

Page 3: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

So, what is Open Source?

First, no official definition exists (http://www.opensource.org/osd.html)

Open source doesn't just mean access to the source

code

Open source rules are, simply enough:

Allow free redistribution

Allow source code access

Permit derived works

Protect the integrity of the author‟s source code

….

3

Page 4: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Key Predictions

By 2012, at least 80% of all commercial software solutions will

include elements of open source technology (Gartner Highlights Key

Predictions for IT Organizations and Users in 2008 and Beyond)

By 2010, Global 2000 IT organizations will use open-source products

in 80% of infrastructure-focused software investments and 25% of

business software investments (Gartner's Positions on the Five Hottest IT

Topics and Trends in 2005)

4

Page 5: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Survey: Do you use open source software?

Do you use…

Firefox to browse the web?

OpenOffice to write documents?

Linux or Ubuntu as server or end user OS?

Subversion or CVS for the source code?

MySQL as database?

Eclipse to write code?

FileZilla for file transfer?

http://downloadpedia.org/Open_Source_Alternative_to_Commercial_Software

5

Page 6: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Companies use open source because…

COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research]

But cost is not the only reason for open source‟s growing popularity:

many firms now know that it offers more FLEXIBILITY than

proprietary programs [Matthew Aslett, 451 Group]

Improving quality of products a/o processes

Achieving faster time-to-market

Higher level of security

6

Page 7: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

OSS vs. Commercial

Commonly it is framed as

Linux vs. Microsoft

Security vs. Innovation

7

Rewards of OSS

availability of source code

freedom to customize, enhance or repair

lower price, sometimes free

does not depend on vendor

security

Risks

lack of support, training and docs

frequent updates & forks

backwards compatibility

Rewards of Commercial software

vendor professional services

easier to adopt in organizations

comply with industrial standards

more user friendly

mature products offer more features

Risks

costs

dependent upon vendor

do not allow users to alter or customize the product

.…

Page 8: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

A ‘mixed source’ strategy

The greatest chance for success with open source software involves a

strategy of mixing open with commercial software

A mixed-source strategy means identifying and integrating both open

and commercial products that

are best-of-breed technologies

fits the requirements

minimizes integration „headaches‟

rather than using “good enough” products

included in legacy solutions or

available (for free…) on SourceForge!

8

Page 9: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Integration

Mixing open & commercial tools requires integration and interoperability

Integrating any two programs is often challenging…

Integrating commercial software & OSS can be difficult because:

variety of technologies

(commercial) tools use proprietary „approaches‟

data integration is not friendly (OpenOffice vs. Microsoft Office file formats)

no design for interoperability

different products have varying requirements for integration

many products are intended for use with specific combinations of tools or designated platforms

The result is that integration of OSS with commercial offerings is often partial

or missing completely, or must be provided by a third-party vendor…

9

Page 10: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Test Tools

Application Test Tools for:

Source testing

Functional & GUI testing

Load & Performance testing

Embedded testing

Database testing

Supporting Tools for:

Test Management

Bug Tracking

Requirements Management

Configuration / version control

10

Page 11: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Can they cooperate?

11

Open Source

Commercial

Page 12: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Case Study

Testing a Java™ stand-alone, GUI-based application for the

Telecom market

The application-under-test has

„legacy‟ functionalities (taken from past project / application)

new features continuously changing (specially at early stage)

Reusing of many automated functional tests built with IBM Rational

Functional Tester for legacy functionality testing

12

Page 13: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

The Test Environment

13

Page 14: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Subversion

Subversion is a free & open source version control

system

Industry standard

Easy to use

Local baseline

Atomic commits

One repository to maintain (single url for all projects)

There are several plug-ins implementing SVN support into

Eclipse (→ Subversive)

14

Page 15: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

JIRA

JIRA is a commercial issue, bug and project tracking tool

Easy to use

Highly configurable (e.g. to manage a complex workflow)

Flexible for different kind of issues (e.g. for test reporting)

Accessible from tests scripts through Java API

Extensible and customizable (source code is available to customers)

Eclipse integration

Subversion integration

Fully supported

Quite cheap

15

Page 16: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Eclipse

Eclipse is a free & open source development environment

comprising an IDE and an extensible plug-in system

Industry standard

Fully supported

Extensible and configurable more than 1200 plug-in available at the

Marketplace (most free!)

Simple and powerful Workbench → Views, Editors and Perspectives

16

Page 17: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

IBM Rational Functional Tester

Rational Functional Tester is a commercial tool

for automated functional and regression testing

Built-in support for Java applications

Java scripting language

Eclipse rich client application

ScriptAssure technology to accommodate UI changes

Data-driven & Keyword-driven testing

17

Page 18: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Maveryx

Maveryx is a free & open source tool for automated

functional, regression, GUI and data-driven testing

Built-in support for Java applications

No „GUI Map‟ needed to create and run tests

Intelligent GUI objects recognition (including fuzzy matching)

Available as Eclipse plug-in

Extensible and configurable through plug-ins

Integrated with all Java IDEs and testing frameworks (Eclipse,

NetBeans, JUnit, IBM Rational Functional Tester, …)

18

Page 19: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Test Strategy

Legacy functionalities tested by reusing test scripts

developed with IBM Rational Functional Tester

Needed to update / re-capture the GUI Maps for the

current application

New functionalities tested by using Maveryx

Maximizing the Rational Test Manager usage

Refactoring of some tests by mixing, where

possible, Rational code with Maveryx code

19

Page 20: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Mixing open & commercial tools

You must decide which components be open vs. closed source

A possible strategy:

closed source for “commodity” components

open source for “differentiating” components

Following such a strategy:

classify features as commodity or differentiating

determine for each feature whether using open or closed SW based on:

budget

technology

know-how

requested quality

20

Page 21: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Evaluation Process

21

Identify candidates

Read existing reviews

Compare attributes to your needs

Analyze top candidates

Wrap-up

Page 22: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Identify Candidates

First step : find out your options

Search for open and commercial tools:

ask friends and co-workers

search using specialized sites

use search engines

thumb specialized magazines

22

Page 23: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Read existing reviews

After you've identified your options, read existing evaluations about the

alternatives:

use search engines

search for specialized web sites

read blogs, forum

thumb specialized & independent magazines

look at social networks

23

Page 24: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Evaluation Criteria

24

TOOL #1 TOOL #2 TOOL #3

Functionality

Cost

Support

Market Share

Maintenance

/ Longevity

Reliability

Performance

Scalability

Usability

Security

Flexibility /

Customizability

Interoperability

Page 25: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Analyze top candidates

After the initial evaluation, pick the top candidates, and perform a more

in-depth analysis

Try any contender in non-critical situations first, by evaluating the

functionality that you‟re interested in using (use a checklist)

Examine also the program‟s documentation, source code (for OSS),

and other related materials

If the tool had some but all the functions you need, examine what it

would take to add those functions

25

Page 26: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Evaluating Software

In general, a mature platform will meet the following

criteria: Project extensions are available

The project has reached a one-year maturity mark

Security patches, bug fixes and new features/enhancements are delivered

separately

The software has reasonable automated unit and functional tests with code

coverage in the 30 percent - 80 percent range

The software easily integrates with external tools

The component‟s bug database is kept up-to-date with revision numbers for each

product enhancement

The solution has been ported across multiple platforms (Linux, Windows, Solaris

and Mac)

Large-scale adoption, including both public and well-known large-scale

organizational deployments exists

There is documentation purpose separation: User guide (or Getting Started

Guide), Installation guide, Admin guide and Development guide

26

Page 27: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Important factors influencing testing tools

Factors influencing the choice of Software Testing Tools

Programming Language

Support

Long feedback

Evaluation Period

Selling is driven by Marketing

Vendor locking

Specialization & Generalization

27

Page 28: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Wrap-up

Review and present to your manager the results of your evaluation

Identify the recommended solution

Identify the main alternatives, first of all the recommended alternative

Once a decision has been made, get the software & go!

28

Page 29: Mixing open and commercia tools · Companies use open source because… COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research] But

Conclusion

A “mixed-source” approach combines open source & proprietary software,

by taking the best of both worlds

The right mixed solution offers advantages in flexibility, functionality,

performance, and management that aren't available when either open

source or commercial technologies are used alone

But, integrating commercial software and OSS can be difficult…

For testing purposes, evaluate open source and commercial tools by making

decisions based on cost, technical and integration factors

29