mixing open and commercia tools · companies use open source because… cost is the main reason why...
TRANSCRIPT
Mixing Open and Commercial
Tools
Mauro Garofalo, Maveryx
W22 – Automation
Summary
Costs, Advantages and Risks of mixing open
source and commercial software
A case study of blending open source and
commercial tools for testing Java applications
Some practical rules to combine open and
commercial test tools
2
All trademarks referenced herein are the properties of their respective owners.
So, what is Open Source?
First, no official definition exists (http://www.opensource.org/osd.html)
Open source doesn't just mean access to the source
code
Open source rules are, simply enough:
Allow free redistribution
Allow source code access
Permit derived works
Protect the integrity of the author‟s source code
….
3
Key Predictions
By 2012, at least 80% of all commercial software solutions will
include elements of open source technology (Gartner Highlights Key
Predictions for IT Organizations and Users in 2008 and Beyond)
By 2010, Global 2000 IT organizations will use open-source products
in 80% of infrastructure-focused software investments and 25% of
business software investments (Gartner's Positions on the Five Hottest IT
Topics and Trends in 2005)
…
4
Survey: Do you use open source software?
Do you use…
Firefox to browse the web?
OpenOffice to write documents?
Linux or Ubuntu as server or end user OS?
Subversion or CVS for the source code?
MySQL as database?
Eclipse to write code?
FileZilla for file transfer?
…
http://downloadpedia.org/Open_Source_Alternative_to_Commercial_Software
5
Companies use open source because…
COST is the main reason why companies are turning to open source [Jeffrey Hammond, Forrester Research]
But cost is not the only reason for open source‟s growing popularity:
many firms now know that it offers more FLEXIBILITY than
proprietary programs [Matthew Aslett, 451 Group]
Improving quality of products a/o processes
Achieving faster time-to-market
Higher level of security
…
6
OSS vs. Commercial
Commonly it is framed as
Linux vs. Microsoft
Security vs. Innovation
7
Rewards of OSS
availability of source code
freedom to customize, enhance or repair
lower price, sometimes free
does not depend on vendor
security
…
Risks
lack of support, training and docs
frequent updates & forks
backwards compatibility
…
Rewards of Commercial software
vendor professional services
easier to adopt in organizations
comply with industrial standards
more user friendly
mature products offer more features
…
Risks
costs
dependent upon vendor
do not allow users to alter or customize the product
.…
A ‘mixed source’ strategy
The greatest chance for success with open source software involves a
strategy of mixing open with commercial software
A mixed-source strategy means identifying and integrating both open
and commercial products that
are best-of-breed technologies
fits the requirements
minimizes integration „headaches‟
rather than using “good enough” products
included in legacy solutions or
available (for free…) on SourceForge!
8
Integration
Mixing open & commercial tools requires integration and interoperability
Integrating any two programs is often challenging…
Integrating commercial software & OSS can be difficult because:
variety of technologies
(commercial) tools use proprietary „approaches‟
data integration is not friendly (OpenOffice vs. Microsoft Office file formats)
no design for interoperability
different products have varying requirements for integration
many products are intended for use with specific combinations of tools or designated platforms
…
The result is that integration of OSS with commercial offerings is often partial
or missing completely, or must be provided by a third-party vendor…
9
Test Tools
Application Test Tools for:
Source testing
Functional & GUI testing
Load & Performance testing
Embedded testing
Database testing
…
Supporting Tools for:
Test Management
Bug Tracking
Requirements Management
Configuration / version control
…
10
Can they cooperate?
11
Open Source
Commercial
Case Study
Testing a Java™ stand-alone, GUI-based application for the
Telecom market
The application-under-test has
„legacy‟ functionalities (taken from past project / application)
new features continuously changing (specially at early stage)
Reusing of many automated functional tests built with IBM Rational
Functional Tester for legacy functionality testing
12
The Test Environment
13
Subversion
Subversion is a free & open source version control
system
Industry standard
Easy to use
Local baseline
Atomic commits
One repository to maintain (single url for all projects)
…
There are several plug-ins implementing SVN support into
Eclipse (→ Subversive)
14
JIRA
JIRA is a commercial issue, bug and project tracking tool
Easy to use
Highly configurable (e.g. to manage a complex workflow)
Flexible for different kind of issues (e.g. for test reporting)
Accessible from tests scripts through Java API
Extensible and customizable (source code is available to customers)
Eclipse integration
Subversion integration
Fully supported
Quite cheap
…
15
Eclipse
Eclipse is a free & open source development environment
comprising an IDE and an extensible plug-in system
Industry standard
Fully supported
Extensible and configurable more than 1200 plug-in available at the
Marketplace (most free!)
Simple and powerful Workbench → Views, Editors and Perspectives
…
16
IBM Rational Functional Tester
Rational Functional Tester is a commercial tool
for automated functional and regression testing
Built-in support for Java applications
Java scripting language
Eclipse rich client application
ScriptAssure technology to accommodate UI changes
Data-driven & Keyword-driven testing
…
17
Maveryx
Maveryx is a free & open source tool for automated
functional, regression, GUI and data-driven testing
Built-in support for Java applications
No „GUI Map‟ needed to create and run tests
Intelligent GUI objects recognition (including fuzzy matching)
Available as Eclipse plug-in
Extensible and configurable through plug-ins
Integrated with all Java IDEs and testing frameworks (Eclipse,
NetBeans, JUnit, IBM Rational Functional Tester, …)
…
18
Test Strategy
Legacy functionalities tested by reusing test scripts
developed with IBM Rational Functional Tester
Needed to update / re-capture the GUI Maps for the
current application
New functionalities tested by using Maveryx
Maximizing the Rational Test Manager usage
Refactoring of some tests by mixing, where
possible, Rational code with Maveryx code
19
Mixing open & commercial tools
You must decide which components be open vs. closed source
A possible strategy:
closed source for “commodity” components
open source for “differentiating” components
Following such a strategy:
classify features as commodity or differentiating
determine for each feature whether using open or closed SW based on:
budget
technology
know-how
requested quality
20
Evaluation Process
21
Identify candidates
Read existing reviews
Compare attributes to your needs
Analyze top candidates
Wrap-up
Identify Candidates
First step : find out your options
Search for open and commercial tools:
ask friends and co-workers
search using specialized sites
use search engines
thumb specialized magazines
…
22
Read existing reviews
After you've identified your options, read existing evaluations about the
alternatives:
use search engines
search for specialized web sites
read blogs, forum
thumb specialized & independent magazines
look at social networks
…
23
Evaluation Criteria
24
TOOL #1 TOOL #2 TOOL #3
Functionality
Cost
Support
Market Share
Maintenance
/ Longevity
Reliability
Performance
Scalability
Usability
Security
Flexibility /
Customizability
Interoperability
…
Analyze top candidates
After the initial evaluation, pick the top candidates, and perform a more
in-depth analysis
Try any contender in non-critical situations first, by evaluating the
functionality that you‟re interested in using (use a checklist)
Examine also the program‟s documentation, source code (for OSS),
and other related materials
If the tool had some but all the functions you need, examine what it
would take to add those functions
25
Evaluating Software
In general, a mature platform will meet the following
criteria: Project extensions are available
The project has reached a one-year maturity mark
Security patches, bug fixes and new features/enhancements are delivered
separately
The software has reasonable automated unit and functional tests with code
coverage in the 30 percent - 80 percent range
The software easily integrates with external tools
The component‟s bug database is kept up-to-date with revision numbers for each
product enhancement
The solution has been ported across multiple platforms (Linux, Windows, Solaris
and Mac)
Large-scale adoption, including both public and well-known large-scale
organizational deployments exists
There is documentation purpose separation: User guide (or Getting Started
Guide), Installation guide, Admin guide and Development guide
26
Important factors influencing testing tools
Factors influencing the choice of Software Testing Tools
Programming Language
Support
Long feedback
Evaluation Period
Selling is driven by Marketing
Vendor locking
Specialization & Generalization
…
27
Wrap-up
Review and present to your manager the results of your evaluation
Identify the recommended solution
Identify the main alternatives, first of all the recommended alternative
Once a decision has been made, get the software & go!
28
Conclusion
A “mixed-source” approach combines open source & proprietary software,
by taking the best of both worlds
The right mixed solution offers advantages in flexibility, functionality,
performance, and management that aren't available when either open
source or commercial technologies are used alone
But, integrating commercial software and OSS can be difficult…
For testing purposes, evaluate open source and commercial tools by making
decisions based on cost, technical and integration factors
29