research in action: vendor selection matrix -- next generation application performance management...

© 2015, Research In Action GmbH Reproduction Prohibited 1

Vendor Selection Matrix – Next Generation Application Performance Management: Ranking Of Selected Vendors

Strategy Execution Total1 Dynatrace 4,28 4,43 8,702 CA 3,90 4,08 7,983 AppDynamics 4,00 3,85 7,854 New Relic 3,93 3,73 7,655 HP 3,88 3,70 7,586 Microsoft 3,30 3,68 6,987 IBM 3,48 3,35 6,838 Oracle 3,63 3,13 6,759 AppNeta 2,90 3,55 6,45

10 Riverbed 2,80 3,48 6,28


Vendor Selection Matrix – Next Generation Application Performance Management: Key HighlightsScope: Global 2015

Dr. Thomas Mendel Ph.D.Managing Director September 2015


The Research In Action GmbH Vendor Selection Matrix MethodologyData Summary: Unique, primarily survey-based methodology for comparative vendor evaluation. Roughly 60% of evaluation results are based on enterprise buyers’ survey results. Analyst’s opinion accounts for roughly 40% of evaluation results (not 100% as in most

other vendor evaluations). More than 40000 data points were collected. Data was collected in Q1 of 2015, covering 900 IT buyers in a telephone survey, and in

Q2 of 2015, covering 700 IT buyers in an online survey of Next Generation Application Performance Management solutions.

20 vendors of Next Generation Application Performance Management solutions (selected by the buyers in the survey) were evaluated. This document provide the highlights of the complete report.

The evaluations results and forecasts are based on customer and vendor feedback, publicly available information, triangulation, as well as the analyst’s opinion.


Decision Makers use a mix of traditional and online tools

N=931 IT and Business Managers in Enterprises (Companies with more than 10,000 employees only)

What Tools Do You Use To Create The Vendor Longlist?

MQ/VSMGooglePeersSocial MediaPress


Market Overview: Requirements For Next Generation Application Performance ManagementWhat are the key requirements for a Next Generation Application Performance Management solution?

N=1600 IT Managers in Enterprises


Market Overview: A Market In Transition The importance of Application Performance Management is at an all time high. This is a

very dynamic market, highly complex and with many different facets. Application Performance Management solutions today have many faces and definitions, few vendors can be compared on an apples to apples basis. There are more than 400 active software and SaaS vendors generating around $ 4,6 B in annual revenue. However, the overall importance of Application Performance Management for the IT Service, Application and Operation Management space as a whole, is still growing every year. This makes it worthwhile to look ahead to see what the future will look like for the Next Generation of Application Management solutions.

SaaS will be the platform of choice for RoI conscious buyers. Next Generation Application Performance Management will be predominantly a SaaS play. The vast majority of new vendors are already SaaS-only. A much better RoI as well as ease of deploy and upgrade are the major drivers for this.

Next Generation Application Performance Management will be more streamlined. The technical requirements for Next Generation Application Performance Management will ultimately lead to a vendor landscape that is much more comparable than the one in today’s highly fragmented market. For the IT decision makers in our survey, the key requirements for the future are: (1) Hybrid Cloud management, (2) Analytics and Big Data, (3) End-to-end and real-time SLAs, (4) Application Quality Life Cycle and DevOps integration, (5) Virtual CMDB integration and (6) Application discovery and application component deep-dive capabilities.


Market Overview: Market Trends 2015What is your number one investment area in the Application Performance Management space for 2015?

N=900 IT Managers in Enterprises


Market Overview: The End-User Pain Points Today The Cloud challenge in the extended enterprise. The Cloud phenomenon is

driving investment from IT buyers to address issues in Application Performance Management around new applications in the Cloud, as well as extended real-time and Big Data requirements. More traditional challenges around N-tier Applications and instrumentation, are further exemplified by the Cloud.

More business orientation. DevOps is only the beginning, but already high on the agenda for 2015. As the business becomes more entrenched in Application Management as a whole, business SLAs are a key focal point for 2015 as well.

More integration issues than one can handle. IT decision makers are still struggling with the many areas of integration in today’s Application Performance Management implementations. Very few companies have worked this out completely. The most important integration challenges come from the areas of: (1) Cost, (2) Capacity, (3) BPM/BTM, (4) Business process and (5) The CMDB.


Market Overview: Key Points To Remember A changing market with new and clearer requirements. Application

Performance Management has a long history in many enterprises. Many implementations are already more than 25 years old. Vendors and IT buyers alike a struggling with updating their solution sets to accommodate the quickly changing requirements. The Next Generation Application Performance Management solutions, however, will be based a clearer set of requirements, making it easier to implement and maintain, as well as to compare vendors.

SaaS will be the future delivery platform. SaaS is now widely accepted. While new players are already predominantly SaaS-only, established players are either revamping their solutions, or are at least offering SaaS options.

One clear market leader and the comeback of an established player. Dynatrace, the re-branded former Compuware solution set is the clear winner in our evaluation and well positioned for future market dominance. BMC Software, with a new strategic direction for its Application Performance Management solution, is making serious headway in the market and will be very likely the long-term number two.


Vendor Selection Matrix – Next Generation Application Performance Management: Evaluation Criteria

StrategyVision & Go-To-Market 30% Does the company have a coherent vision in line with the most probable future market scenarios?

Does the go-to-market and sales strategy fit the target markets and customers?Innovation & Partner Ecosystem 20% How innovative is the company?

How is the partner ecosystem organized and how effective is the partner management?Company Viability & Execution Capabilities 15% How likely in the long-term survival of the company?

Does the company have the necessary resources to execute the strategy?Differentiation & USP 35% Does the solution have a Unique Selling Proposition (USP) and clear differentiators?ExecutionBreadth & Depth Of Solution Offering 30% Does the solution cover all necessary capabilities expected by the customers?Market Share & Growth 15% How big is the market share and is it growing above market rate?Customer Satisfaction & Mindshare 25% How satisfied are customers with the solution and the vendor?Price Versus Value 30% How do customers rate the relationship between the price and perceived value of the solution?



Strategy Execution Total1 Dynatrace 4,28 4,43 8,702 CA 3,90 4,08 7,983 AppDynamics 4,00 3,85 7,854 New Relic 3,93 3,73 7,655 HP 3,88 3,70 7,586 Microsoft 3,30 3,68 6,987 IBM 3,48 3,35 6,838 Oracle 3,63 3,13 6,759 AppNeta 2,90 3,55 6,45

10 Riverbed 2,80 3,48 6,28



Strategy Execution Total Revenue ($m)* Market Share*AppDynamics 4,00 3,85 7,85 85 1,85%AppNeta 2,90 3,55 6,45 45 0,98%CA 3,90 4,08 7,98 250 5,43%Dynatrace 4,28 4,43 8,70 400 8,70%HP 3,88 3,70 7,58 132 2,87%IBM 3,48 3,35 6,83 250 5,43%Microsoft 3,30 3,68 6,98 280 6,09%New Relic 3,93 3,73 7,65 115 2,50%Oracle 3,63 3,13 6,75 85 1,85%Riverbed 2,80 3,48 6,28 240 5,22%

* Research In Action forecast CY2015, in Application Performance Management


Vendor Selection Matrix – Next Generation Application Performance Management: The Number One Dynatrace: The champion of Application Performance Management General: The re-branding from Compuware to Dynatrace, was a risky endeavor

indeed, but it is paying off. Clients value he renewed focus of the company, in particular with the Keynote integration which completes this impressive portfolio.

Strategy: Dynatrace is highly differentiated, clients give Dynatrace the highest scores when it comes to innovation. Strategically, Dynatrace is second to none.

Execution: Dynatrace is the current market share leader by a wide margin and will continue to be so. The company already covers all major growth areas for the future. Clients are highly satisfied and see the value in Dynatrace’s solutions.

Customer Quote: ”Dynatrace continues to impress us. This going private thing was a concern for sure, but now I cannot even imagine going with another vendor.” CIO North-American retail company.

Bottom Line: Being focused is paying off. Dynatrace will be the reference company in Next Generation Application Performance Management for the foreseeable future.


Vendor Selection Matrix – Next Generation Application Performance ManagementCA Technologies: Integration will be a key competitive advantage for future growth General: CA’s mission is to cover all aspects of the IT Service, Application and Operation

Management space and the company does this well. CA is addressing all key client requirements of Next Generation Application Performance Management.

Strategy: CA is highly differentiated, the vision remains strong. However, Clients would like to see a clearer articulated strategy for Next Generation Application Performance Management.

Execution: CA is one of the market share leaders in Application Performance Management, still capitalizing on the Wily acquisition. Clients are very satisfied with CA today, but there were of course some complaints regarding CA’s overall pricing.

Customer Quote: ”When it comes to platform integration, CA is second to none. This works well for us and we will continue to work with CA in the future.” CIO North-American financial services company.

Bottom Line: The overall IT Service, Application and Operation Management champion is back on track in Application Performance Management.


Vendor Selection Matrix – Next Generation Application Performance Management AppDynamics: The fall of a rising star General: Founded in 2008 by former Wily leaders, and well-funded indeed, the rise of

AppDynamics has recently come to a more or less abrupt halt. Strategy: AppDynamics is highly differentiated, clients still like the strategic approach

which is very much in-line with where the market is going, but would like to see a more mature approach to company growth and profitability forecasts.

Execution: AppDynamics is liked by their customers for their capability of delivering value for money. Clients also like the SaaS and non-SaaS options, as the all SaaS approach is still questioned by many IT decision makers today, despite the fact that almost everyone agrees that this is the strategic future direction.

Customer Quote: ”We are happy with the AppDynamics solution. However, we are concerned about the accounting and reporting principles of the company.” CIO North-American financial services company.

Bottom Line: AppDynamics needs to grow-up to become a long-term competitor in the Next Generation Application Performance Management space.


Vendor Selection Matrix – Next Generation Application Performance Management New Relic: A top Application Performance Management player in-the-making General: New Relic, founded in 2008, also from former Wily Technology leaders,

and like AppDynamics well-funded, is a pure-play SaaS Application Performance Management vendor.

Strategy: Clients see New Relic as highly differentiated, they value the company’s innovation capabilities and future-proof architecture, in particular as it comes to Big Data integration.

Execution: New Relic today has above average top 20 scores for client satisfaction as well as on a price versus value level. However, as clients become more familiar with the SaaS-only approach of the company, Research In Action believes that the scores will improve even more.

Customer Quote: ”We decided to buy-into SaaS wholeheartedly. New Relic is the therefore the number one choice for us.” CIO North-American services company.

Bottom Line: New Relic is moving up the ranks quickly. Though still a larger niche player, the future looks bright for New Relic.


Vendor Selection Matrix – Next Generation Application Performance Management HP: The ITIL champion is becoming more important in Application Performance Management General: HP has been a key player in the IT Service Management market almost from

its inception. HP’s legacy in Network Performance Management is helping the company to expand its reach into Application Performance Management.

Strategy: HP’s differentiation is only at a medium level. However, clients like the partnership approach and the strategic direction. Clients see also huge value in HP’s Big Data and Analytics’ solutions.

Execution: HP is still extremely strong in Infrastructure Performance management, Network Node Manager still being the market reference. Clients are therefore excited about the future prospects in the Application Performance Management market.

Customer Quote: ”We like HP’s flexibility and customer orientation. This helps us a lot when talking to our business leaders.” CIO European manufacturing company.

Bottom Line: HP is now focusing more on this growth market. The future looks bright.


Vendor Selection Matrix – Next Generation Application Performance Management: Detailed Results

Weighting

ScoreWeighted

Score ScoreWeighted

Score ScoreWeighted

Score ScoreWeighted

Score ScoreWeighted

Score

StrategyVision & Go-To-Market 30% 4 1,20 3,5 1,05 4 1,20 4 1,20 4,5 1,35Innovation & Partner Ecosystem 20% 4,5 0,90 3 0,60 3 0,60 4 0,80 4 0,80Company Viability & Execution Capabilities 15% 4 0,60 4,5 0,68 3 0,45 3,5 0,53 4,5 0,68Differentiation & USP 35% 4,5 1,58 4,5 1,58 5 1,75 4 1,40 3 1,05

100% 4,28 3,90 4,00 3,93 3,88

ExecutionBreadth & Depth of Solution Offerring 30% 4,5 1,35 4,5 1,35 4 1,20 4 1,20 3,5 1,05Market Share & Growth 15% 5 0,75 4,5 0,68 4 0,60 4 0,60 4 0,60Customer Satisfaction & Mindshare 25% 4,5 1,13 4 1,00 4 1,00 3,5 0,88 4 1,00Price versus Value 30% 4 1,20 3,5 1,05 3,5 1,05 3,5 1,05 3,5 1,05

100% 4,43 4,08 3,85 3,73 3,70Scale Explanation: 1 (Low) to 5 (High)

AppDynamicsCA HPDynatrace New Relic


The Research In Action GmbH Vendor Selection Matrix MethodologyVendor Selection Matrix Disclaimer:

Research In Action GmbH does not endorse any vendor, product or service depicted in our research publications, and does not advise technology users to select only those vendors with the highest ratings. The information contained in this research has been obtained from both enterprise as well as vendor sources believed to be reliable. Research In Action GmbH’s research publications consist of the analysts’ opinions and should not be considered as statements of fact. The opinions expressed are subject to change without further notice. Research In Action GmbH disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.


Contact

Dr. Thomas Mendel [email protected]+49 160 99492223

RESEARCH IN ACTION

Research In Action GmbHHauptstraße 9

56244 HartenfelsGermany

Office: +49 2626 9249845Fax: +49 2626 9249845

Email: [email protected]