the forrester wave application life cycle management 2012

8/11/2019 The Forrester Wave Application Life Cycle Management 2012

1/20

Forrester Research, Inc., 60 Acorn Park Drive, Cambridge, MA 02140 USATel: +1 617.613.6000 | Fax: +1 617.613.5000 | www.forrester.com

The Forrester Wave: Application Life-Cycle Management, Q4 2012by Tom Grant, Ph.D., October 23, 2012

FOR: Application

Development

& Delivery

Professionals

KEY TAKEAWAYS

Leaders Build Product Strategy Around ALM Use Cases

Forrester Wave evaluation Leaders IBM, Rally Sofware, PC, CollabNet, Microsof,

and Serena Sofware offer strong solutions or ALM problems that no singlepoint solution could address. Teir product strategies ocus on such challenges as

delivery and the disruptions o Agile adoption, not just throwing new capabilities at

the customer.

Strong Performers And Contenders Find A Specific Audience

Atlassian and Rocket Aldon dont ocus as much on specific use cases that span

many ALM activities. Instead, they build tools to address smaller but widespread

problems, such as release management and developer collaboration. Meanwhile,

HP is expanding its ALM offering beyond Quality Center.

ALM Is Going Through A Period Of Redefinition And Innovation

Hot ALM issues such as DevOps, embedded sofware, and requirements definition

show how the concept o ALM has expanded and changed. During this transition,

the seemingly crowded ALM market will benefit rom the innovation that comes

rom a wealth o competing and complementary product offerings.


2/20

2012, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best availableresources. Opinions reflect judgment at the time and are subject to change. Forrester, Technographics, Forrester Wave, RoleView, TechRadar,and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. Topurchase reprints of this document, please email [email protected]. For additional information, go to www.forrester.com.

FOR APPLICATION DEVELOPMENT & DELIVERY PROFESSIONALS

WHY READ THIS REPORT

In Forresters 116-criteria evaluation o application lie-cycle management (ALM) vendors, we identified

the nine most significant sofware providers in the category Atlassian, CollabNet, HP, IBM, Microsof,

PC, Rally Sofware, Rocket Aldon, and Serena Sofware and researched, analyzed, and scored them.

Tis report details our findings about how well each vendor ulfills our criteria and where they stand in

relation to each other to help application development and delivery proessionals select the right partner

or their sofware-ueled business innovation efforts.

Table Of Contents

Innovation Drives Interest In Application Life-

Cycle Management

Application Life-Cycle Management

Evaluation Overview

The ALM Forrester Wave Finds Many

Leaders And Different Directions

Vendor Profiles

Supplemental Material

Notes & Resources

Forrester conducted lab-based evaluations

in September and October 2011 and

interviewed nine vendor and user companies:

Atlassian, CollabNet, HP, IBM, Microsoft,

PTC, Rally Software, Rocket Aldon, and

Serena Software.

Related Research Documents

The Time Is Right For ALM 2.0+

October 19, 2010

The Forrester Wave: Agile Development

Management Tools, Q2 2010

May 5, 2010

The Forrester Wave: Application Life-CycleManagement, Q4 2012IBM, Rally Software, PTC, CollabNet, Microsoft, And Serena SoftwareLead The Pack, With HP, Atlassian, And Rocket Aldon Following

by Tom Grant, Ph.D.with Kyle McNabb and Alissa Anderson

2

6

10

14

17

OCTOBER 23, 2012
http://www.forrester.com/go?objectid=RES56832http://www.forrester.com/go?objectid=RES48153http://www.forrester.com/go?objectid=RES48153http://www.forrester.com/go?objectid=BIO1887http://www.forrester.com/go?objectid=BIO1887http://www.forrester.com/go?objectid=RES48153http://www.forrester.com/go?objectid=RES48153http://www.forrester.com/go?objectid=RES56832http://www.forrester.com/


3/20


The Forrester Wave: Application Life-Cycle Management, Q4 2012 2

2012, Forrester Research, Inc. Reproduction Prohibited October 23, 2012

INNOVATION DRIVES INTEREST IN APPLICATION LIFE-CYCLE MANAGEMENT

Business innovation now drives the application lie-cycle management (ALM) market. Te

contribution that application development and delivery makes to a companys business goals making

workers more productive, creating engaging customer experiences, and bringing new sofware

products to market more successully must be more direct and successul. Sofware increasingly

plays a central role in a firms ability to deliver new products and services or exploit new channels.

Firms can no longer accept historical guls between business and application development and delivery

teams as, increasingly, firms now expect to manage application development and delivery as a business

and treat it as a competency.1

Unortunately, everyone has their long list o reasons why they struggle with better managing

application development and delivery. Te work is creative, and teams must figure out how to work

with one another. ools or techniques that impose an assembly-line model will ail. Application

development and delivery still need management, but the tools o management must fit the natureo the creative and collaborative work.

Sofware and application development work itsel is changing. Tis Forrester Wave evaluation

examines the most significant developments that affect both ALM strategy and ALM tools. For

example, the definition o ALM has stretched to include not just development but also delivery (see

Figure 1). Some o the vendors we assess have expanded their tools portolio to add or enhance their

support or release management and other delivery-related activities.

Te need to support the business management o application development and delivery underlies

this Forrester Waves evaluation criteria:

Delivering business value, not just driving process efficiency.ALMs original aim wasprocess improvement. Firms turned to ALM to do a better job o managing project timelines,

dependencies, and deliverables; ensure that every requirement could be traced to a real

sofware capability; and test to ensure that the sofware worked. While process improvement is

still important, increasing efficiency in building and delivering sofware is as important a goal

or ALM as proving the value o the sofware, as it contributes to business outcomes.

Supporting individual, team, and institutional management.ALMs first wave o toolsocused on in truth application development management (ADM), designed to increase

developer and tester productivity. Now, firms seek ALM tools to improve team success while

linking these teams to the larger application development and delivery activity within the

organization or, in many cases (such as offshore partners), outside o it.

Extending lie cycles to include delivery.Organizations now recognize that the lie cycleo ALM does not stop with check-in or build phases. Te real lie cycle and process goes on

much longer, arguably all the way to the moment the application goes live and even beyond to

support continuous eedback.


4/20




Managing sofware that gets deployed everywhere.Sofware is truly everywhere, rom datacenter servers to laptops, rom mobile devices to cars and rerigerators. Regardless o where

the sofware winds up, sofware development and delivery management is still necessary.

Integrating across tools.Vendors have reluctantly admitted that customers are not interestedin their suite o tools, no matter how impressive or cost effective they might be. Switching

costs can be high, and many capabilities, such as issue tracking and source control, are so

commoditized that switching makes little sense. Some tools, such as HP Quality Center, are so

widely adopted that other vendors must integrate with them in some ashion. Te speed o tool

innovation in some areas, such as continuous delivery and requirements, outpaces what any

single vendor can keep up with, providing another reason or integrating across tools.

Using reporting and dashboards as instruments o change.Many application development

and delivery organizations aspire to be learning organizations and see their investmentin ALM tools in that light. Te data in an ALM system provides a yardstick or a teams

continuous improvement efforts. Some teams use this inormation to communicate

with outside parties about the shape and value o new Agile and Lean practices. Larger

organizations may use the same inormation to track sofware value streams across projects

and teams. In these and other scenarios, reporting and dashboards provide the ultimate payoff

or ALM investments.

Figure 1 ALM Boundaries Expand o Encompass Te Sofware Value Chain

Source: Forrester Research, Inc.60080

Production planningclosed loop

Change management Service management

Portfolio management

Change-awarecontinuous integration

Just-in-timedemand

management

Releasemanagement

Testing andquality assurance

Build andsoftware

configurationmanagement

Deployment

Projectmanagement

Production controlclosed loop


5/20




ALM Vendors Have Taken Many Steps Forward, But A Long Road Remains

ALM tools have made significant progress since out last evaluation:

ALM vendors have discovered macro differentiators.In past years, ALM vendors did ar lessin their product strategy and product marketing to clearly differentiate their products. With

notable exceptions such as Rally Sofware, a company that built its ALM strategy around Agile

support, ALM vendors had noticeable differences only i you careully examined the details o

their product eatures. oday, ALM vendors more clearly identiy the types o customers they

serve and the problems they can help these customers address. Serena Sofware, or example,

tells a very clear story about the challenges orchestrating development and delivery in large I

organizations and how its products help organizations address those challenges.

Vendors help improve the efficiency o the larger ALM process.As noted earlier, the

boundaries o ALM have expanded to include more o the delivery phase. In response,ALM vendors have expanded their capabilities in this area to remove the inefficiencies that

arbitrarily lengthen and complicate the delivery phase.

Requirements, a part o ALM, need innovation attention. Recent innovations inrequirements tools, such as visualization and social media integration, have increased the

depth and accuracy o insights and the speed o collecting them. Tese innovations have

occurred in requirements-ocused companies such as Balsamiq Studios, Blueprint Sofware

Systems, iRise, and Visure Solutions. Even though ALM vendors provide requirements tools,

they have not kept pace with these innovations.

Mobile and embedded development scenarios get insufficient attention.Embeddedsofware developers still get scant attention, with the exception o MKS (PC) and IBM.

Surprisingly, given how hot the mobile development market is, ALM vendors by and large do

nothing special or mobile developers, including in areas such as testing where there is a large

need or better tools.

Collaboration is the lightning that ALM vendors cant capture in a bottle. Many o thebiggest ALM challenges, such as coordinating the work across geographically distributed teams,

boil down to collaboration problems. Collaboration presents some extremely vexing puzzles or

tools vendors, particularly given how idiosyncratically a specific set o people may collaborate

with each other. Still, some basic collaboration eatures desktop sharing or team members in

different locations and wikis or capturing and updating inormation, or example are ar less

common than expected.


6/20




The Market Is Still Open For Innovation And New Contenders

Te vendors we assessed in this evaluation provide an impressive array o tools with a broad range

o capabilities. Nevertheless, they do not represent every possible innovation in the expanding and

evolving realm o ALM. Te ALM market is still wide open or anyone to devise a better way to

address a problem, identiy new and unmet challenges, or give a particular class o customers more

attention than the bigger, general-purpose vendors can.

Vendors listed in this section were not evaluated but represent various innovative approaches to

ALM, demonstrating how much room remains within this market or innovation and specialization:

Blueprint ocuses on requirements.Some aspects o ALM pose tough challenges that donthave an easy solution. Smaller vendors ocused exclusively on some o these challenges can

innovate in ways that larger suite vendors may not. Blueprint, a requirements tools vendor,

provides a good example o the important role that these smaller companies with a muchnarrower ocus play in todays ALM market.

Hansof caters to video game developers.Since the video game market is nearly as large as themovie industry, it should be no surprise that at least one vendor, Hansof, has identified this

market as its core constituency.

Polarion tackles difficult compliance challenges.Few o the vendors we evaluated have takensteps to address complex compliance requirements, leaving no small amount o opportunity or

a smaller vendor such as Polarion Sofware.

SmartBear Sofware makes quality the starting point.While many ALM vendors are shifingtheir attention to later parts o the development and delivery process, SmartBear Sofware

makes those stages the starting point or its product strategy. Tis strategy includes both code

quality improvement through review and testing and quality o service maintained through web

monitoring.

asktop is the Switzerland o integration.Each ALM vendor has its own set o integrationswith other tools, both open source and commercial. Each ALM vendor also has its own

approach to integration. Unsnarling this tangle sometimes requires going to a neutral third

party, such as asktop, or cross-ALM integration.

ToughtWorks provides thought leadership in continuous delivery.A real thought leaderin continuous delivery, ToughtWorks provides a modest range o tools but a very established

consulting business (primarily aimed at helping with Agile adoption).

VersionOne supports Agile.Rally has built Agile and Lean support into its tools, andother vendors such as Microsof and IBM provide Agile-specific configurations. With Agile


7/20




continuing to spread, there is still room or another Agile-ocused ALM vendor such as

VersionOne to meet the needs o Agile teams in a different way than its competitors.

APPLICATION LIFE-CYCLE MANAGEMENT EVALUATION OVERVIEW

o assess the state o the application lie-cycle management (ALM) market and see how the vendors

stack up against each other, Forrester evaluated the strengths and weaknesses o nine top vendors.

Tese companies ell into two broad categories:

Large sofware companies that, among other products, also offer ALM tools.In this category,HP, IBM, and Microsof maintain a strong position in the ALM market.

Smaller sofware companies that specialize in ALM.Other companies included in thisevaluation, such as Atlassian, CollabNet, Rally Sofware, Rocket Aldon, and Serena Sofware,

have made ALM their exclusive business. Until its recent acquisition by PC, MKS also ell into

this group.

Evaluation Criteria Measured Breadth Of Portfolio And Support For Use Cases

Afer examining past research, user need assessments, and vendor and expert interviews, we

developed a comprehensive set o evaluation criteria. We evaluated vendors against 116 criteria,

which we grouped into three high-level buckets:

Current offering.ALM covers a vast expanse o unctionality or developers, business analysts,

testers, project managers, and people in a wide array o other roles. Tereore, we developedmore than 86 distinct criteria to measure the strength o each vendors tools.

Strategy.Te value that a vendor brings to the table goes beyond just its current tools, so weincluded 18 criteria that assessed each vendors ability to support customer implementations, its

commitment to the ALM market, its ALM product strategy, and its corporate strategy.

Market presence.We also took into account the size o the ootprint each vendor has in thesofware market. Forresters clients want to know how sae a bet it will be that a given ALM

vendor will be in business several years rom now in order to gauge how saely they can invest in

that vendors ALM tools.

Tis years ALM evaluation represents a departure rom the 2010 Forrester Wave evaluating Agile

development management solutions in three ways:2

Opening the aperture to encompass all ALM scenarios.Te 2010 evaluation ocusedexclusively on Agile support among ALM vendors. Tis year, we include both Agile and non-

Agile use cases in the criteria.


8/20




reating Agile and Lean as a critical test o ALM offerings.At the same time, this yearsevaluation by no means leaves Agile behind. Instead, we treat Agile and Lean as critical tests o

an ALM vendors offering.

Incorporating systems engineering and embedded sofware.Given the increasing quantityo hybrid products those incorporating sofware into physical products ranging rom cars

to medical devices we have also included criteria covering sofware engineering outside o

corporate I and independent sofware vendors (ISVs).

Evaluated Vendors Have Strong Tools And Substantial Market Presence

Forrester included nine vendors in the assessment: Atlassian, CollabNet, HP, IBM, Microsof, PC,

Rally Sofware, Rocket Aldon, and Serena Sofware. Each o these vendors has (see Figure 2):

Depth and breadth o ALM offering.While there are many commercial and open source pointsolutions or particular activities (test management, build management, issue tracking, etc.),

the title ALM vendor goes to companies that do more than provide a single capability (source

control, deect tracking, etc.), however good it might be. We expect these tools to cover the

majority o ALM use cases, as Forrester defines ALM.

A broad ALM customer base.We also looked at the number o customers or a vendors ALMproducts in 2011 (at least 750) as well as the growth in the vendors customer base rom 2010 to

2011.

Definitive revenue rom ALM.Finally, the criteria included the revenue rom ALM. Onceagain, we included both the numbers or 2011 (at least $19 million) and the growth since 2010.


9/20




Figure 2Evaluated Vendors: Product Inormation And Selection Criteria

Source: Forrester Research, Inc.

Vendor

Atlassian

CollabNet

HP

IBM

Microsoft

PTC

Product evaluated

Atlassian OnDemand:JIRAGreenHopperBonfireConfluenceSharePoint Connector for Confluence

Team Calendars for ConfluenceBambooFishEyeCrucibleCloverCrowdBitbucket

Atlassian IDE Connector for EclipseAtlassian IDE Connector for IntelliJ IDEAAtlassian IDE Connector for Visual Studio2008 and 2010HipChatStashSourcetreeJIRA Mobile ConnectJIRA OnDemandGreenHopper OnDemandBonfire OnDemandConfluence OnDemand

Team Calendars for ConfluenceOnDemandSource OnDemand Bundle (FishEye,

Crucible, and Subversion)

TeamForgeScrumWorks ProSubversion EdgeLab Management

HP Application Lifecycle Management (HPALM) 11.5

Rational Collaborative LifecycleManagement Solution (Rational CLM):Rational Requirements Composer 4.0Rational Team Concert 4.0Rational Quality Manager 4.0

Microsoft Visual Studio

Integrity

Product version

evaluated

5.05.102.24.2

1.5.32.34.12.72.7

3.1.62.4.2

Weekly builds

3.5 or later7 and later

N/A

Weekly builds1.1

Weekly buildsN/A

Weekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly builds

Weekly builds

6.26.03.02.5

11.5

4.0

2010

10.0

Version

release date

June 2012

Weekly builds

Weekly builds

Weekly builds

Weekly buildsWeekly buildsWeekly buildsWeekly buildsWeekly builds

Weekly builds

June 2012May 2012April 2012May 2012

June 2012

June 2012

May 2010

December 2011


10/20




Figure 2Evaluated Vendors: Product Inormation And Selection Criteria (Cont.)


Vendor

Rally Software

Rocket Aldon

SerenaSoftware

Product evaluated

Rally Community EditionRally Enterprise EditionRally Unlimited EditionRally Quality ManagerRally Support ManagerRally Product ManagerRally Community ManagerRally Time TrackerRally Idea ManagerRally Portfolio Manager

Aldon Lifecycle ManagerAldon Deployment Manager

Aldon Community ManagerAldon Report ManagerAldon Agile Manager 1.0

Development ManagerRelease ManagerRequirements ManagerService ManagerDemand ManagerRequest CenterDashboardChangeMan ZMFAgile Planner

Product version

evaluated

Weekly builds

(e) 6.1/(i) 7.6N/A

9.5.41.51.0

1.21.02.13.13.13.12.17.12.3

Version

release date

Weekly builds

May 2011N/A

November 2010June 2011January 2011

May 2012

Vendor selection criteria

The vendor provides designated ALM product(s) and a stated ALM road map.

The vendor has more than 750 customers.

The vendor provides tools that cover the majority of ALM use cases, as Forrester defines ALM.

The vendor generates more than $19 million in revenue.


11/20




THE ALM FORRESTER WAVE FINDS MANY LEADERS AND DIFFERENT DIRECTIONS

Te vendors we evaluated or the ALM Forrester Wave provide a wide range o capabilities.

Superficially, they look very much alike. Project management? Check. Issue tracking? Check. With

such eature breadth, you might conclude that ALM is a mature market.

But where are the vendors heading? Te answer is, in very different directions. Sofware may be

everywhere, but the orm o sofware development and delivery varies widely across organizations.

While this market now has many genuine leaders, they have reached this point only by identiying

their core market. Tereore, the market is less mature than it might seem on the surace. When

evaluating ALM vendors, its important to consider not just the capabilities they provide but also the

use cases or which they are optimized.

Nowhere is this differentiation more clear than in the overarching principles that guide the

evaluated vendors product strategies (see Figure 3):

Leaders IBM and Microsof provide broad ALM capabilities.Both IBM and Microsof havemaintained a strong portolio o ALM tools designed to support common use cases or sofware

development and delivery. Te two companies both provide broad swaths o capabilities or

common ALM activities, such as project definition and management, reporting, issue tracking,

and task assignment.

IBM and Microsof come rom a more general-purpose perspective on ALM, and both

treat Agile practices as a special instance o ALM workflows, metadata, and reporting. Both

companies support Agile as a template, a special configuration o their ALM tools.

Leaders PC and Rally Sofware lead by ocusing on specific use cases.In contrast to IBMand Microsof, both Rally and PC (originally MKS, beore PCs acquisition o the company in

2011) have succeeded in the ALM market by finding a niche in which to specialize or Rally,

Agile, and or PC, compliance. While this approach may limit their ability to support a broader

swath o scenarios, their ocus gives them the opportunity to provide thought leadership in their

respective areas. Rallys ocus on Agile and Lean, or example, has led the company to build

top-notch planning capabilities into its tools. PCs ocus on compliance, including the highly

regulated world o embedded sofware, has made it the only ALM vendor to provide out o the

box many o the workflows and reports needed to satisy regulatory authorities.

Leaders CollabNet and Serena lead through delivering differentiated and broad capabilities.Both CollabNet and Serena have tailored their ALM offerings to more-general value

propositions and needs. CollabNet sees its opportunity in the cloud, where ALM tools support

has been lagging behind Is need to manage cloud development, testing, and deployment more

effectively. Serena has recognized how important it is or organizations to effectively orchestrate

the larger meta-process o sofware development and delivery, which spans the typical


12/20




activities (coding, testing, building, etc.) that guide the development efforts o other commercial

and open source tools.

Strong Perormers Atlassian and HP provide ocused capabilities.Atlassian and HP showhow ar an ALM vendor can go with strong support or a piece o the overall lie cycle. However

capable individual tools such as JIRA and Quality Center may be, they are no substitute or

a broader ALM offering. Te two companies view this gap differently. Rather than deocus

its product strategy, Atlassian preers to provide a smaller range o strong capabilities than

potentially do a mediocre job tackling a wider range o ALM challenges. Tis strategy makes it

easier or Atlassian to deal with one o its biggest challenges, perception as an enterprise-ready

vendor, without the added complexity o too many products and eatures.

In contrast, HP has pushed hard to fill in the gaps in its ALM offering, moving ar past Quality

Center. Te current version, ALM 11.5, provides a very robust set o tools that can compete with

other ALM suites on a wide range o capabilities. Some o these new capabilities, such as the

executive dashboard eatures, demonstrate that HP understands many tough ALM challenges

and will invest in providing credible, differentiating eatures to address them. However, these

tools are relatively new, and Forrester has not encountered many implementations o them yet.

Te next year will be critical or HPs ALM offering in gaining market traction.

Contender Rocket Aldon is the operational dark horse o ALM.While most other ALMvendors strive to move down the value chain to the point o release and deployment, Rocket

Aldons product strategy starts with these later stages o the application lie cycle. Automation

plays a prominent role in Rocket Aldons offerings, as do I-governance-related eatures. Te

Rocket Aldon road map is, to some extent, extending upstream to cover scenarios (or example,product owner review o the backlog) that it currently does not address as thoroughly as

production-related activities.

Tis evaluation o the ALM market is intended to provide a starting point or your own evaluation.

We encourage readers to view the detailed product evaluations and adapt the criteria weightings to

fit their individual needs through the Forrester Wave Excel-based vendor comparison tool.


13/20




Figure 3Forrester Wave: Application Lie Cycle Management ools, Q4 2012


Go online to download

the Forrester Wave tool

for more detailed product

evaluations, feature

comparisons, and

customizable rankings.

Risky

Bets Contenders Leaders

Strong

Performers

StrategyWeak Strong

Current

offering

Weak

Strong

Market presence

Atlassian CollabNet

HP

IBMPTC

Rally

Rocket Aldon

Serena

Microsoft


14/20




Figure 3Forrester Wave: Application Lie Cycle Management ools, Q4 2012 (Cont.)


CURRENT OFFERING

Platform support

Installing the product and

getting going

Administration

Security

Integration and customization

Management

Change elements

Life-cycle traceability

Reporting

Analytics

Running a project

General collaboration capabilities

Process configuration

STRATEGY

Support for implementations

Product strategy

Corporate strategy

Price Commitment

MARKET PRESENCE

Installed base

Financial strength

Employees

Support services

Channel partnerships

Global presence

Atlassian

3.47

3.40

4.80

4.00

2.20

4.80

3.95

2.47

3.63

2.75

1.38

4.75

4.70

2.24

2.92

2.50

2.90

4.50

0.005.00

3.94

3.30

4.00

2.00

4.10

5.00

3.00

Forresters

Weighting

50%

5%

5%

2%

3%

10%

5%

10%

15%

5%

10%

10%

10%

10%

50%

15%

80%

5%

0%0%

0%

35%

5%

0%

35%

20%

5%

CollabNet

3.51

4.10

4.40

4.75

2.60

4.90

3.89

3.74

3.26

3.70

2.28

4.30

2.90

2.36

3.95

4.90

3.70

5.00

0.005.00

4.42

4.20

3.00

1.00

5.00

4.00

5.00

HP

3.53

4.30

4.20

5.00

3.80

4.00

3.96

5.00

3.85

3.85

3.36

2.90

2.80

1.18

3.22

4.50

2.90

4.50

0.005.00

4.95

5.00

4.00

4.00

5.00

5.00

5.00

IBM

4.38

3.80

4.00

5.00

4.60

5.00

3.79

4.48

4.38

4.60

5.00

4.30

4.80

3.20

4.76

5.00

4.70

5.00

0.00

5.00

4.51

4.50

5.00

3.00

4.10

5.00

5.00

Microsoft

3.91

3.90

3.60

5.00

4.20

4.75

3.96

4.74

3.47

4.30

3.20

4.30

4.20

2.56

3.25

5.00

2.90

3.50

0.00

5.00

4.32

3.20

4.00

3.00

5.00

5.00

5.00

PTC

4.01

1.50

3.60

4.00

5.00

4.75

4.42

4.21

3.64

4.65

5.00

3.70

4.25

3.31

3.72

5.00

3.40

5.00

0.00

5.00

3.23

2.20

3.50

3.00

4.10

3.00

5.00

RallySoftware

3.95

3.20

4.60

5.00

3.40

5.00

3.97

4.75

4.08

4.55

4.28

4.15

3.80

1.26

4.39

4.10

4.40

5.00

0.005.00

4.26

3.10

3.50

1.00

5.00

5.00

5.00

RocketAldon

1.97

0.10

2.82

1.50

5.00

1.05

2.06

1.44

0.66

1.80

2.44

3.10

1.05

4.45

2.04

2.10

1.90

4.00

0.005.00

2.22

2.20

1.50

1.00

3.20

1.00

1.00

SerenaSoftware

3.62

3.10

2.80

3.75

2.20

5.00

4.14

3.38

3.38

4.15

4.26

3.25

3.15

3.61

3.44

3.10

3.40

5.00

0.005.00

3.73

2.90

2.50

2.00

4.10

5.00

3.00

All scores are based on a scale of 0 (weak) to 5 (strong).

ALM Vendors Have Yet To Fully Address Many Common Challenges

While many aspects o the ALM tools market have improved substantially in the past two years

since the Agile development management Forrester Wave evaluation, other areas remain ripe or

innovation and improvement:


15/20




esting.With one or two notable exceptions, the vendors we included in this study did not excelat any aspect o testing support. o some extent, the weakness o testing eatures is characteristic

o the ALM tools market, not just the vendors we assessed. However, its still worth noting that

some specialty vendors have done a better job o connecting testing to other ALM activity.

Some requirements tools vendors, or example, provide rapid test generation rom requirements,

something that many purveyors o bigger ALM suites have yet to implement.

Collaboration.While we have seen some progress in this area, thats damning by aint praiseconsidering how ofen collaboration is at the core o an organizations ALM challenges. ake,

or example, the very common challenge o collaborating with an offshore partner. While some

eatures, such as eeds or wikis, make it easier or onshore and offshore teams to eavesdrop on

each others activities, these are not vehicles or communicating product vision, mentoring

people on the other team, or suggesting ideas or process improvement.

Mobile support.At the risk o stating the obvious, many aspects o mobile development userexperience, testing, deployment, etc. are significantly different rom the elements involved

in building middleware. Yet the typical ALM vendor today has done very little to address these

needs, despite the continued expansion o the mobile market.

Integration.ALM vendors have acknowledged the inevitability o integration with othercommercial and open source tools. However, were a long way rom arriving at a common

integration strategy that makes it easier or customers to select ALM tools based on their

individual merits outside o integration. Some vendors provide an application programming

interace (API) with ew or no prebuilt connectors. Others provide plenty o connectors, with

perhaps a weaker API or custom-built integrations. Te act that small companies, such asKovair and asktop, have established themselves as third-party providers o ALM connectors is

a testament to the weaknesses o the overall ALM markets integration efforts.

VENDOR PROFILES

Leaders

IBMs offering is becoming more than the sum o its parts.IBMs position in this yearsassessment o the ALM market is a testament to an important principle: Its not enough just

to have good products the products, in combination, must support common activities

in sofware development and delivery. In previous years, IBM has certainly provided strong

solutions, such as Rational RequisitePro, Rational eam Concert, and Rational DOORS in

specific unctional areas such as requirements, planning, and project management. However,

IBM provided multiple solutions in some o these areas, leading to conusion about what exactly

each o these tools was supporting and how they all worked together.


16/20




Tat conusion has largely dissipated with IBMs current ALM offering. Not only has IBM

continued development o its strong suite o products, but it has also stitched them together in

a more coherent way. Te company has also made clearer the use cases it supports with its tools,

such as Agile teams and embedded sofware development. Because o the companys strongproduct and portolio strategy, it is possible to write about IBMs ALM offerings taken as a

whole with ar less irony.

Rally Sofware continues its leadership in Agile/Lean.Rally targets a very healthy andgrowing opportunity: the expanding number o organizations that have adopted Agile and

Lean. Rallys tools are optimized or Agile planning, project management, status reporting, and

other actions that happen within and outside sprints. Te companys acquisition o AgileZen,

a Lean project management tool, was a natural fit or both Rally and its customers. So too was

the addition o Rally Portolio Manager, a tool or planning, decision-making, and management

above the level o an individual project or product.

Rally also continues to provide thought leadership in the Agile and Lean community. However,

this ocus on Agile has a downside: non-Agile teams will find Rallys products and services ar

less attractive than other general-purpose ALM tools. Rallys leadership rests with its breadth and

depth o capabilities or Agile teams, combined with a strong and ocused corporate strategy.

PC continues its leadership addressing regulated and digital products needs.PCprovides strength where product-based development and compliance intersect. No other

company, with the possible exception o IBM, has done as much to support teams o embedded

sofware developers and systems engineers. PC helps firms address digital product

development needs or products such as appliances, vehicles, and medical devices that havesofware components while still maintaining support or teams building and delivering

sofware-only products and projects.

PCs strength rests on its long-standing support or addressing governance and compliance

challenges. Burrowing into this niche has its concomitant cost: a more complex product

that is harder to justiy adopting or teams that dont have to satisy the needs o auditors or

government regulators.

CollabNet provides a flexible ALM ramework optimized or the cloud.As already noted,one o the strengths o CollabNets ALM suite is the flexibility with which users can define

ALM content. We also see this flexibility pay off when trying to effectively address the needs o

complex teams or complex projects, where there is less chance or a one-size-fits-all design.

CollabNet has also invested heavily in supporting cloud development and deployment. Not

only are CollabNets tools available in the cloud (CloudForge), but they also include eatures

optimized or developing, testing, and deploying cloud systems.


17/20




Microsof shortens cycle times.Starting with Visual Studio at the ront end and through toeam Foundation Server (FS) at the back end, Microsofs ALM portolio provides broad

lie-cycle support. Microsofs overarching goal is to shorten cycle times, which requires a

combination o individual measures (task management, test automation, etc.).

Microsof deserves recognition or understanding how central collaboration is to ALM. While

the M word, management, is part o the ALM acronym, ALM initiatives requently ail because

they emphasize control over collaboration. Features such as visualization via PowerPoint and

translating recorded manual testing into automated tests emphasize collaboration in these

two examples, with business users.

Serena Sofware orchestrates I projects rom idea to deployment.Since retooling its ALMstrategy, Serena has made impressive advances at both the individual tool and the suite level.

Central to its product strategy is orchestration, a vision o ALM as an ongoing, rhythmicactivity. Not surprisingly, given this vision o a regular flow o activity (conceive, build, test,

deploy), Serena has one o the best workflow designers available. Serenas recent enhancements

to its ALM portolio, such as the Serena Release Manager, provide more support in this end-to-

end orchestration model.

Strong Performers

For HP, quality is truly central, but other ALM capabilities are growing.HP Quality Center isthe ulcrum on which HPs ALM strategy rests. HP Quality Center is a ubiquitous part o many

organizations ALM inrastructure. Consequently, other ALM vendors integrate with HP Quality

Center as a matter o course, almost as a price o entry into any potential customers shortlist.

HP has improved upon and expanded its ALM capabilities beyond test management and test

automation in the two years since the previous Forrester Wave evaluation o this space. HP

now provides improved requirements management and traceability. Another example, HPs

Application Liecycle Intelligence (ALI), offers a broad set o ALM data aggregation and

analysis eatures, with strong reporting and dashboarding components. Given the importance

o this inormation or everything rom team-level continuous improvement to executive-

level portolio management, it represents one o many areas where HP has moved quickly to

distinguish itsel as a serious ALM player.

Atlassian ocuses on issue tracking and collaboration within teams.Atlassians JIRA, a widelypopular issue tracking tool, ofen serves double duty as a requirements management system.

Te companys Confluence Wiki is equally popular with development teams. Te two products,

in tandem, support the various kinds o day-to-day work done within sofware development

teams, which are always looking or ways to better collaborate (or example, through a wiki).


18/20




While Atlassian has decided to provide a narrower set o ALM tools than other vendors, it

continues to hone the capabilities in the areas where the company believes it can excel, such as

providing a marketplace or its developer community to provide plug-ins. Te company has

made moves into more enterprise-level scenarios, with new capabilities such as Stash, a tool orthe management o Git repositories that enterprise customers demand. Atlassian has also made

enhancements to its support, sales orce, and other aspects o its business needed to gain ground

in the enterprise market.

Contenders

Rocket Aldon provides strong operational support.Unlike other ALM vendors, RocketAldons roots lie in sofware delivery, not sofware development. Rocket Aldon has ocused rom

the beginning on the operational side, including both automation and reporting. Unlike other

vendors, Rocket Aldon is expanding backward, not orward, in the sofware timeline, adding

support or more upstream ALM activities, such as demand management and requirements.

SUPPLEMENTAL MATERIAL

Online Resource

Te online version o Figure 3 is an Excel-based vendor comparison tool that provides detailed

product evaluations and customizable rankings.

Data Sources Used In This Forrester Wave

Forrester used a combination o three data sources to assess the strengths and weaknesses o each

solution:

Hands-on lab evaluations.Vendors spent one day with a team o analysts who perormed ahands-on evaluation o the product using a scenario-based testing methodology. We evaluated

each product using the same scenario(s), creating a level playing field by evaluating every

product on the same criteria.

Vendor surveys.Forrester surveyed vendors on their capabilities as they relate to the evaluationcriteria. Once we analyzed the completed vendor surveys, we conducted vendor calls to clariy

and confirm their products capabilities.

Customer reerence calls.o validate product and vendor qualifications, Forrester alsoconducted reerence calls with two o each vendors current customers.


19/20




The Forrester Wave Methodology

We conduct primary research to develop a list o vendors that meet our criteria to be evaluated

in this market. From that initial pool o vendors, we then narrow our final list. We choose these

vendors based on: 1) product fit; 2) customer success; and 3) Forrester client demand. We eliminate

vendors that have limited customer reerences and products that dont fit the scope o our evaluation.

Afer examining past research, user need assessments, and vendor and expert interviews, we develop

the initial evaluation criteria. o evaluate the vendors and their products against our set o criteria, we

gather details o product qualifications through a combination o lab evaluations, questionnaires,

demos, and/or discussions with client reerences. We send evaluations to the vendors or their review,

and we adjust the evaluations to provide the most accurate view o vendor offerings and strategies.

We set deault weightings to reflect our analysis o the needs o large user companies and/or

other scenarios as outlined in the Forrester Wave document and then score the vendors basedon a clearly defined scale. Tese deault weightings are intended only as a starting point, and we

encourage readers to adapt the weightings to fit their individual needs through the Excel-based

tool. Te final scores generate the graphical depiction o the market based on current offering,

strategy, and market presence. Forrester intends to update vendor evaluations regularly as product

capabilities and vendor strategies evolve.

ENDNOTES

1 Forresters research indicates that the dependence on sofware to achieve business outcomes will only

continue through the next decade. Tereore, while packaged applications will continue to be important,

sofware development and delivery or building new applications, customizing packaged systems, or

perorming custom integrations will be critical or bridging the gap between sofware capabilities

and business needs. See the January 30, 2012, B 2020: o hrive In he Empowered Era, Youll Need

Sotware, Sotware Everywhere report.

2 Te previous Forrester Wave contained many similar questions but ocused more narrowly on Agile team

support. See the May 5, 2010, he Forrester Wave: Agile Development Management ools, Q2 2010

report.
http://www.forrester.com/go?objectid=RES61257http://www.forrester.com/go?objectid=RES61257http://www.forrester.com/go?objectid=RES48153http://www.forrester.com/go?objectid=RES48153http://www.forrester.com/go?objectid=RES61257http://www.forrester.com/go?objectid=RES61257


20/20

Forrester Research, Inc. (Nasdaq: FORR) is an independent research company that provides pragmatic and forward-thinking advice to

global leaders in business and technology. Forrester works with professionals in 17 key roles at major companies providing proprietary

Forrester Focuses On

Application Development & Delivery Professionals

Responsible for leading development and delivery of applications

that support your companys business strategies, you also choose

technology and architecture while managing people, skills, practices,

and organization to maximize value. Forresters subject-matter expertiseand deep understanding of your role will help you create forward-thinking

strategies; weigh opportunity against risk; justify decisions; and optimize

your individual, team, and corporate performance.

ANDREA DAVIES, client persona representing Market Insights Professionals

About Forrester

A global research and advisory firm, Forrester inspires leaders,

informs better decisions, and helps the worlds top companies turn

the complexity of change into business advantage. Our research-

based insight and objective advice enable IT professionals to

lead more successfully within IT and extend their impact beyond

the traditional IT organization. Tailored to your individual role, our

resources allow you to focus on important business issues

margin, speed, growth first, technology second.

FOR MORE INFORMATION

o find out how Forrester Research can help you be successul every day, please

contact the office nearest you, or visit us at www.orrester.com. For a complete list

o worldwide locations, visit www.orrester.com/about.

CLIENT SUPPORT

For inormation on hard-copy or electronic reprints, please contact Client Support

at +1 866.367.7378, +1 617.613.5730, or [email protected] . We offer

quantity discounts and special pricing or academic and nonprofit institutions.
mailto:[email protected]://www.forrester.com/mailto:[email protected]