mixing open and commercia tools · companies use open source because… cost is the main reason why...

Mixing Open and Commercial

Tools

Mauro Garofalo, Maveryx

W22 – Automation

Summary

Costs, Advantages and Risks of mixing open

source and commercial software

A case study of blending open source and

commercial tools for testing Java applications

Some practical rules to combine open and

commercial test tools

2

All trademarks referenced herein are the properties of their respective owners.

So, what is Open Source?

First, no official definition exists (http://www.opensource.org/osd.html)

Open source doesn't just mean access to the source

code

Open source rules are, simply enough:

Allow free redistribution

Allow source code access

Permit derived works

Protect the integrity of the author‟s source code

….

3

Key Predictions

By 2012, at least 80% of all commercial software solutions will

include elements of open source technology (Gartner Highlights Key

Predictions for IT Organizations and Users in 2008 and Beyond)

By 2010, Global 2000 IT organizations will use open-source products

in 80% of infrastructure-focused software investments and 25% of

business software investments (Gartner's Positions on the Five Hottest IT

Topics and Trends in 2005)

…

4

Survey: Do you use open source software?

Do you use…

Firefox to browse the web?

OpenOffice to write documents?

Linux or Ubuntu as server or end user OS?

Subversion or CVS for the source code?

MySQL as database?

Eclipse to write code?

FileZilla for file transfer?

…

http://downloadpedia.org/Open_Source_Alternative_to_Commercial_Software

5

Companies use open source because…

COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research]

But cost is not the only reason for open source‟s growing popularity:

many firms now know that it offers more FLEXIBILITY than

proprietary programs [Matthew Aslett, 451 Group]

Improving quality of products a/o processes

Achieving faster time-to-market

Higher level of security

…

6

OSS vs. Commercial

Commonly it is framed as

Linux vs. Microsoft

Security vs. Innovation

7

Rewards of OSS

availability of source code

freedom to customize, enhance or repair

lower price, sometimes free

does not depend on vendor

security

…

Risks

lack of support, training and docs

frequent updates & forks

backwards compatibility

…

Rewards of Commercial software

vendor professional services

easier to adopt in organizations

comply with industrial standards

more user friendly

mature products offer more features

…

Risks

costs

dependent upon vendor

do not allow users to alter or customize the product

.…

A ‘mixed source’ strategy

The greatest chance for success with open source software involves a

strategy of mixing open with commercial software

A mixed-source strategy means identifying and integrating both open

and commercial products that

are best-of-breed technologies

fits the requirements

minimizes integration „headaches‟

rather than using “good enough” products

included in legacy solutions or

available (for free…) on SourceForge!

8

Integration

Mixing open & commercial tools requires integration and interoperability

Integrating any two programs is often challenging…

Integrating commercial software & OSS can be difficult because:

variety of technologies

(commercial) tools use proprietary „approaches‟

data integration is not friendly (OpenOffice vs. Microsoft Office file formats)

no design for interoperability

different products have varying requirements for integration

many products are intended for use with specific combinations of tools or designated platforms

…

The result is that integration of OSS with commercial offerings is often partial

or missing completely, or must be provided by a third-party vendor…

9

Test Tools

Application Test Tools for:

Source testing

Functional & GUI testing

Load & Performance testing

Embedded testing

Database testing

…

Supporting Tools for:

Test Management

Bug Tracking

Requirements Management

Configuration / version control

…

10

Can they cooperate?

11

Open Source

Commercial

Case Study

Testing a Java™ stand-alone, GUI-based application for the

Telecom market

The application-under-test has

„legacy‟ functionalities (taken from past project / application)

new features continuously changing (specially at early stage)

Reusing of many automated functional tests built with IBM Rational

Functional Tester for legacy functionality testing

12

The Test Environment

13

Subversion

Subversion is a free & open source version control

system

Industry standard

Easy to use

Local baseline

Atomic commits

One repository to maintain (single url for all projects)

…

There are several plug-ins implementing SVN support into

Eclipse (→ Subversive)

14

JIRA

JIRA is a commercial issue, bug and project tracking tool

Easy to use

Highly configurable (e.g. to manage a complex workflow)

Flexible for different kind of issues (e.g. for test reporting)

Accessible from tests scripts through Java API

Extensible and customizable (source code is available to customers)

Eclipse integration

Subversion integration

Fully supported

Quite cheap

…

15

Eclipse

Eclipse is a free & open source development environment

comprising an IDE and an extensible plug-in system

Industry standard

Fully supported

Extensible and configurable more than 1200 plug-in available at the

Marketplace (most free!)

Simple and powerful Workbench → Views, Editors and Perspectives

…

16

IBM Rational Functional Tester

Rational Functional Tester is a commercial tool

for automated functional and regression testing

Built-in support for Java applications

Java scripting language

Eclipse rich client application

ScriptAssure technology to accommodate UI changes

Data-driven & Keyword-driven testing

…

17

Maveryx

Maveryx is a free & open source tool for automated

functional, regression, GUI and data-driven testing

Built-in support for Java applications

No „GUI Map‟ needed to create and run tests

Intelligent GUI objects recognition (including fuzzy matching)

Available as Eclipse plug-in

Extensible and configurable through plug-ins

Integrated with all Java IDEs and testing frameworks (Eclipse,

NetBeans, JUnit, IBM Rational Functional Tester, …)

…

18

Test Strategy

Legacy functionalities tested by reusing test scripts

developed with IBM Rational Functional Tester

Needed to update / re-capture the GUI Maps for the

current application

New functionalities tested by using Maveryx

Maximizing the Rational Test Manager usage

Refactoring of some tests by mixing, where

possible, Rational code with Maveryx code

19

Mixing open & commercial tools

You must decide which components be open vs. closed source

A possible strategy:

closed source for “commodity” components

open source for “differentiating” components

Following such a strategy:

classify features as commodity or differentiating

determine for each feature whether using open or closed SW based on:

budget

technology

know-how

requested quality

20

Evaluation Process

21

Identify candidates

Read existing reviews

Compare attributes to your needs

Analyze top candidates

Wrap-up

Identify Candidates

First step : find out your options

Search for open and commercial tools:

ask friends and co-workers

search using specialized sites

use search engines

thumb specialized magazines

…

22

Read existing reviews

After you've identified your options, read existing evaluations about the

alternatives:

use search engines

search for specialized web sites

read blogs, forum

thumb specialized & independent magazines

look at social networks

…

23

Evaluation Criteria

24

TOOL #1 TOOL #2 TOOL #3

Functionality

Cost

Support

Market Share

Maintenance

/ Longevity

Reliability

Performance

Scalability

Usability

Security

Flexibility /

Customizability

Interoperability

…

Analyze top candidates

After the initial evaluation, pick the top candidates, and perform a more

in-depth analysis

Try any contender in non-critical situations first, by evaluating the

functionality that you‟re interested in using (use a checklist)

Examine also the program‟s documentation, source code (for OSS),

and other related materials

If the tool had some but all the functions you need, examine what it

would take to add those functions

25

Evaluating Software

In general, a mature platform will meet the following

criteria: Project extensions are available

The project has reached a one-year maturity mark

Security patches, bug fixes and new features/enhancements are delivered

separately

The software has reasonable automated unit and functional tests with code

coverage in the 30 percent - 80 percent range

The software easily integrates with external tools

The component‟s bug database is kept up-to-date with revision numbers for each

product enhancement

The solution has been ported across multiple platforms (Linux, Windows, Solaris

and Mac)

Large-scale adoption, including both public and well-known large-scale

organizational deployments exists

There is documentation purpose separation: User guide (or Getting Started

Guide), Installation guide, Admin guide and Development guide

26

Important factors influencing testing tools

Factors influencing the choice of Software Testing Tools

Programming Language

Support

Long feedback

Evaluation Period

Selling is driven by Marketing

Vendor locking

Specialization & Generalization

…

27

Wrap-up

Review and present to your manager the results of your evaluation

Identify the recommended solution

Identify the main alternatives, first of all the recommended alternative

Once a decision has been made, get the software & go!

28

Conclusion

A “mixed-source” approach combines open source & proprietary software,

by taking the best of both worlds

The right mixed solution offers advantages in flexibility, functionality,

performance, and management that aren't available when either open

source or commercial technologies are used alone

But, integrating commercial software and OSS can be difficult…

For testing purposes, evaluate open source and commercial tools by making

decisions based on cost, technical and integration factors

29

30

info@maveryx.com