EDITOR’S NOTE INVEST IN THE RIGHT QA AND TEST MANAGEMENT SOFTWARE

COMPARING TODAY’S TOP APM SOFTWARE

ADDRESS ALL YOUR APM SOFTWARE NEEDS

Uncover Your Application Performance Management NeedsIt’s challenging to work through the noise in the application performance test and management world. Get advice on identifying problem areas, matching business needs and the importance of flexibility.

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS2

EDITOR’SNOTE

Navigating the Crowded APM/Test Market

Choosing the right application perfor-mance management and testing tools takes time, the ability to ask the right questions and sometimes a willingness to be flexible. It’s hard, if not impossible, to find a single APM/test tool that will do every single thing an orga-nization needs. So bring your “out of the box” thinking skills to this task to ensure everything goes smoothly.

In the first article in this guide, testing expert Gerie Owen explains why companies need to identify explicitly what it is they’re trying to solve before it’s possible to choose the right APM/test management tool. Compa-nies that have regulatory obligations have one set of requirements, while those following an Agile or continuous integration methodology will have other issues to consider.

Next, testing and Agile consultants Justin

Rohrman and Matthew Heusser address the challenges of choosing the right APM/test tool in a very crowded field of options. Every com-pany wants to improve performance, and an APM/test tool can help, but choosing the cor-rect one means prioritizing the importance of features like stack visibility, variety and dash-boards, data mining and scheduling, reporting and system security.

In our final article, Rohrman and Heusser weigh in again, focused on more specific APM product choices and how to best match your business needs with the solutions available.

When it comes to application performance management and testing, like in life, it’s clear that flexibility is key. n

Valerie SilverthorneSite Editor, SearchSoftwareQuality

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS3

COORDINATION

Invest in the Right QA and Test Management Software

Finding the right quality assurance and test management software for your organiza-tion can be a daunting process. The market for QA tools is very mature; there is a dazzling array of products from complete application lifecycle management (ALM) to niche software with support for testing various commercial off-the-shelf (COTS) packages. Since all of these products offer different combinations of features, it is critical to understand what an organization actually needs before making a purchasing decision.

The best way to understand your organi-zation’s needs is by looking at what led to considering a purchase of QA and test man-agement software. Examining issues and per-forming a needs analysis will illuminate what specific features are most important to your organization. Understanding the demograph-ics of the company and size of the test team reveals the kind of scope QA software should

have. Analyzing the types of testing and the testing methodologies will point out needs for specific types of QA and test management tools, such as supporting Agile or integrating with automation tools. Finally, researching the business climate and regulatory requirements will provide information on potential test pro-cess changes and reporting requirements that may impact the purchasing decision.

IDENTIFYING PROBLEM AREAS

The business case for QA and test management software is developed from evaluating the orga-nization’s problem areas. This, along with the needs analysis, will be useful in narrowing down the choice of vendors and their software. There are many specific use cases for purchasing QA and test management software, but there are some common themes that an organization may fit into. If so, that company’s specific obstacles

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS4

COORDINATION

can be added in to create the business case. The common themes are as follows.

Large organizations often need a standard-ized, repeatable process. Generally in these sit-uations, testing is decentralized and business analysis, development and release management teams may be using many different tool stacks. This type of environment indicates the need for a product that integrates with many other products and could potentially become the basis of an ALM suite. When organizations find a continuing pattern of critical defects in pro-duction that jeopardizes their reputation and sales, it indicates a clear need for more effec-tive testing. This scenario suggests that QA and test management tools that focus on trace-ability to requirements, test execution tracking and defect management would be of value.

More specialized businesses cases include organizations in regulated industries and orga-nizations undergoing an Agile transformation

or moving to continuous integration (CI). Reg-ulated industries have a need for test manage-ment software that provides smooth reporting features as these organizations must report to regulatory authorities early and often. Busi-nesses that use or are planning to use Agile or CI methodologies require QA and test manage-ment software that provides collaboration and work management features geared to quick but effective testing. In the sections below, you’ll find products and vendors that fit various busi-ness cases.

QA AND TEST MANAGEMENT SOFTWARE

For large organizations with distributed teams that need test process and efficiency, Hewlett Packard Enterprises’ (HPE) ALM and Quality Center, Microsoft’s Visual Studio Team Foun-dation Server (TFS), and Borland’s Silk Central are good options. These tools provide not only

There are many specific use cases for purchasing QA and test management software, but there are some common themes that an organization may fit into.

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS5

COORDINATION

strong QA and test management functionality but also complete application lifecycle manage-ment and integrations with many tool stacks including automation tools and collaboration features that distributed teams require.

Quality Center is the centerpiece of HPE’s ALM software and provides 360-degree trace-ability from requirements through test cases and defects. In the most recent versions, HPE refers to Quality Center as ALM; however, this tool provides all the test management func-tions and integrates with other tools in the HPE suite to form a true ALM product. It provides integration with HPE’s Performance Center, Load Runner and Storm Runner for performance and load testing and HPE’s United Functional Test (UFT) as well as other third-party tools. UFT includes Service Test, which is used for testing web services, and Quick Test Professional, which is a premier tool for auto-mated functional testing. Although licensing is pricey and complicated, the tool is easy to use and most testing organizations throughout the world have at least some familiarity with it.

Microsoft’s TFS is both an ALM and DevOps product, providing version control, work item

tracking, reporting and automated builds. Through its integration with Microsoft’s Visual Studio Developer tools, TFS offers a collabora-tive environment in which to manage the entire requirements gathering, development, test and release management processes. TFS’s team rooms facilitate communication among decen-

tralized team members. Although TFS is geared more at QA testers with more technical skills than Quality Center, it is a particularly good choice for organizations that are already using Visual Studio.

Borland’s Silk Central is a complete test management product that provides applica-tion lifecycle management by offering out-of-box integrations with many requirements management, development, versioning and defect management tracking tools. Through

Microsoft TFS is a particularly good choice for Agile and CI shops because it features integration with DevOps as well as ALM tools.

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS6

COORDINATION

integration with Silk Performer, Silk Test and Silk Mobile, it provides complete performance and automation capabilities. Silk Central’s Test Book feature tracks test activities and allows for collaboration among test teams.

SOFTWARE FOR AGILE OR CI

METHODOLOGIES

Organizations that are currently using an Agile or CI methodology or are engaged in an Agile transformation will need QA and test man-agement software that features collaboration, support for exploratory testing, and integra-tion with development, automation and CI tools. Product choices for this scenario include Microsoft’s TFS as well as Telerik’s Test Stu-dio, QA Symphony’s qTest and Jira’s Clover.

Microsoft’s TFS provides integrations and collaboration features as described above. TFS is a particularly good choice for Agile and CI shops because it features integration with DevOps as well as ALM tools. With its require-ments management features, product owners become part of the team with developers and testers.

Telerik’s Test Studio is a great choice for Agile organizations because it features strong exploratory testing and provides a Visual Stu-dio plug-in for collaboration between testers and developers through automation. Test Stu-dio supports CI by providing functionality that allows for integrating any build server with recorded tests.

QA Symphony’s qTest is a QA and test man-agement tool developed specifically to support Agile methodology. It provides integration with Agile ALM tools including Rally and Ver-sionOne as well as with Jira and other defect management tools. QTest creates seamless test management for manual, exploratory and automated tests by integrating with QA Sym-phony’s tool suite, qTest eXplorer and qTest Scenario. Exploratory testing capabilities fall under the purview of qTest eXplorer, and qTest Scenario provides support for shift left test approaches including test-driven development. The tools are intuitive and easy to use, and qTest has strong collaboration features that can share test cases.

For Agile and continuous integration shops that are focused on code coverage, Jira’s Clover

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS7

COORDINATION

offers complete code coverage that easily exposes incomplete test coverage. Clover also includes a test optimization feature that auto-mates tests in order to run them faster; it also runs integrated development environments and continuous integration systems. Although Clo-ver is not a complete QA and test management tool, it offers test management value, especially in the CI arena.

ORGANIZATIONS WITH SPECIALIZED NEEDS

Organizations with specialized testing needs, including mobile applications or COTS soft-ware, and organizations within regulated indus-tries, should evaluate vendors’ products that have superior capabilities in those areas.

Telerik’s Test Studio, Borland’s Silk Cen-tral and SmartBear’s QAComplete have strong capabilities in mobile testing. Test Studio sup-ports many platforms and browsers by using object identification and allows for automating tests directly on the device or through emula-tion. Silk Central provides support for mobile testing for both web and native applications.

SmartBear’s QAComplete and its companion

products strongly support both mobile and web testing. It has integrations with Jira and Bug-zilla for defect management and integrates with SoapUI and Selenium as well as SmartBear’s companion automation tools, TestComplete, MobileComplete and LoadComplete.

Although both HPE’s Quality Center and Borland’s Silk Central provide support for COTS testing, if your organization needs to test specific COTS packages, it is worthwhile to look at vendors that provide support specifi-cally for COTS packages. Worksoft has a suite of tools for analyzing and testing business pro-cesses. Worksoft Analyze is a tool within that suite that documents business processes, and Worksoft Certify including Certify Mobile and Certify Performance are Worksoft tools geared specifically to automate business process testing.

Organizations in safety critical or regulated industries should look at vendors whose trace-ability, documentation and reporting capa-bilities are strong. Although all of the market leaders including HPE’s Quality Center, Micro-soft’s TFS and Borland’s Silk Central have effi-cient dashboards, Seapine Software’s TestTrack

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS8

COORDINATION

provides all of the traceability and reporting capabilities that regulated organizations need. Requirements, test plans, source code and a help desk are linked in one platform, providing visibility and traceability. TestTrack’s report-ing capabilities include analyzing bugs across releases, tracing project work from design and development through testing as well as test coverage and effectiveness.

If your organization needs QA and test man-agement software, but the budget is limited, Gurock’s Test Rails or SmartBear’s QACom-plete would be perfect choices.

SmartBear’s QAComplete is a comprehensive test management package that has traceability between requirements, test cases and defects as well as slick reporting features. SmartBear pro-vides a complete package much like the market leaders at a much lower price. Gurock’s Test

Rail also provides complete test management including integrations to defect management tools. Test Rail’s metrics and reporting features are efficient and easy to use with nice collabo-ration features for distributed teams. Although they have fewer bells and whistles than the market leaders, Test Rail and QAComplete offer core test management functionality at a reasonable cost.

Buying QA and test management software is not difficult when it is approached in three steps. First, perform a needs analysis to docu-ment the organization’s requirements. Then, build the business case by reviewing the issues that the purchase of QA and test management software will mitigate. Finally, use these analy-ses to narrow down the choice of vendors that best meet the needs of the organization.

—Gerie Owen

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS9

JUSTIFICATION

Comparing Today’s Top APM Software

Managing the performance of an appli-cation is easier said than done. With many avenues to explore and many trends to inves-tigate, application performance management can go from simple to complex in a hurry. The same can be said about APM software. APM software’s many options, all of which perform a variety of functions at different levels of suc-cess, make the question of which to procure less of a question and more of a riddle.

This article removes some of that mystery from APM products by covering a number of different APM tools, the features they offer and how they fit in certain business scenarios.

BUSINESS SIZE AND FOCUS MATTERS

Some tools are built for an enterprise with a variety of environments and others for smaller or newer companies that have a simpler setup. If you have a simple environment that consists

of, say, a couple of servers and databases, an APM tool like Sensu might work perfectly. This open source tool lacks some of the fea-tures sets, security options and scalability that other open source tools offer, but it is perfectly suited to a simple product environment.

Other tools—AppDynamics and BMC True-Sight—are better for larger and more com-plicated product environments. These two in particular can monitor hardware and software across many different tiers. For example, a product like Facebook has many virtualized servers, databases that are constantly being replicated and backed up, and lots of different services to support users. AppDynamics, True-Sight and other APM software with a large-scale focus can quickly map the environment and help discover all the pieces that make the product run.

Somewhere in the middle, APM products like FogLight, AppPulse and AppNeta allow

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS10

JUSTIFICATION

organizations to purchase only the parts of the product that they would need. You can pur-chase these modular products based on the functionality needed at the time. There are two benefits here—one is that companies get only the functionality they want and pay for, which reduces bloat. The second benefit is that they can get a product specialized for their environ-ment. For example, a company that is heavily interested in transactional data might need a different product than a company that only has mobile products or another company that uses cloud servers only.

KEY FEATURES IN APM SOFTWARE

Bloated software with more features than someone could ever use can be a nuisance. Learning to use the software is confusing, and it also represents something you paid for and didn’t really need. But if that tool is missing

the one or two pieces of functionality an orga-nization really needs, it is completely use-less. No matter what other functions vendors offer with their APM tools, data collection and reporting are must-haves in any APM software, and every product does it a little differently. Products like FogLight and TrueSight specialize in end user experience monitoring by collect-ing information about every transaction, such as latency on the pages those transactions hap-pened. Dynatrace and AppResponse, found in Riverbed’s SteelCentral suite, can collect every piece of transactional data. To display this data, Dynatrace and NewRelic offer a single pane of glass approach, an approach where data is col-lected from all over and displayed with one dashboard. The two APM tools provide sets of preconfigured dashboards with data points that many people find useful.

Getting up and running with monitoring products is challenging sometimes. There is

If product security and restricting who has access to report data are important, AppDynamics and FogLight both offer a role-based security system.

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS11

JUSTIFICATION

configuration, and sometimes changes to your software, that have to happen before any data can be collected. Products like AppDynam-ics, Dynatrace and SAP Extended Diagnos-tics claim zero, or very little, configuration is needed before you can start collecting data. These products use mapping technology to discover the different dependencies required to make your product go. If you have three virtual machines pointed to a database that is backed up to a data warehouse where you are getting analytics from, these products will find all of those and create a map of how the systems connect.

If product security and restricting who has access to what report data are important, App-Dynamics and FogLight both offer a role-based security system. With this, companies can create user groups to manage and control data access. This is important for people in regu-lated environments like healthcare.

PLATFORM SUPPORT

Ten years ago, software lived on a few servers and a few databases and that was that. Things

are different now; there are virtual machines, the cloud, mobile and even platform as a ser-vice. For APM tools, remember to ask these two questions: Will it work with the language our company’s software is written in? And will it run on our servers?

There are a few very popular programming languages—such as Java, C#, Ruby and Java-script—but hundreds more are in use. There is some variety in server software, too, especially Linux. Most APM products have a specific technology set that they work with. It’s vital that an organization selects APM software that works with the programming language it devel-ops with.

Windows and Linux are the most popular environments, so it’s no surprise that most APM products mentioned in this article will work in these environments. Sensu has a more limited platform of Debian, Ubuntu or CentOS, while FogLight will run on almost anything, including Red Hat Enterprise Linux, Solaris, Advanced Interactive eXecutive and HP-UX.

On the programming technology side, Java and .NET are fairly standard. Just about any APM software will support these languages.

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS12

JUSTIFICATION

Some products go further—Dynatrace sup-ports PHP and AppPulse supports Python.

PRICE POINTS

Some vendors offer free trials or free versions of APM tools, but they also have a subscrip-tion plan. While this means companies would pay monthly fees, they also receive the benefit

of a support department and knowing that the APM software will be actively developed.

Finding the right APM tool, one where the right feature set overlaps with the right price and the right technology, can be challenging. When a company understands its own needs and investigates the APM tool, it will find the answer to the riddle more easily.

—Justin Rohrman and Matt Heusser

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS13

HARMONIZATION

Address All Your APM Software Needs

An organization has decided to put an end to app crashes and disappointed custom-ers for good by purchasing an application per-formance management tool. Here’s the hard part: picking the right package. From platform compatibility to platform type to data col-lection and reporting, you’ll find plenty of fundamental issues to look at, including hard-to-measure items like ease of use and support options. APM customers have to balance the features with team skills and price. It’s unlikely that they’ll find a single tool that does every-thing an enterprise needs, aligned with the skills the team currently has, at the right price. Instead, this article covers different aspects of software used to monitor application per-formance and how it affects one’s decision on purchasing an APM tool.

STACK VISIBILITY

The term solution stack is the set of technol-

ogies, strung together, that makes the software work. Linux and Microsoft form two different sets of tools; a stack includes every tool from top to bottom. At the top of the stack are the user interface and the tools it was built with. Today, that probably means one of the many JavaScript frameworks that are available. Going below that leads to the web server; database; and server-side programming languages like PHP, python, .NET or Java. In between those are the integration tools that send data from a company’s software to another third-party product; APIs and web services; and other tools like virtualization, the load balancer, continu-ous integration and version control. Each layer, and the technologies within, are a source of information.

Some products that monitor application performance specialize in just a layer or two in the stack, while others reach deeper. Select-ing the right depth can help solve issues much

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS14

HARMONIZATION

faster. Some APM tools are also designed to run on specific technologies, such as Linux/Java or Microsoft/TFS. Tools that specialize in one or two technologies may have access to more debugging information more easily. It’s important to pick a tool that matches the stack. Organizations with several different sys-tems, and different stacks, might want a more general tool—but those tend to have weaker integration.

VARIETY AND DASHBOARDS

Complex data centers don’t want to monitor one stack; they want to monitor all stacks. That might mean physical servers in one place, plat-form as a service in another, some cloud appli-cations and some pure-database systems in yet another. One common pattern is to have plug-ins that run and monitor on each system, then report back to a central server. From the central server, operations can create dashboards at dif-ferent levels—from red (crash), yellow (slow) and green (healthy) at the transaction level—to detailed reports on each system. In some cases, it’s possible to connect dashboards, so when

technical leaders see yellow, they can click in to find out that the delay is caused by one spe-cific system that isn’t meeting its service-level commitment.

The right APM software will ideally cover most, if not all, of the platforms that are important to an enterprise and also inte-grate them into a holistic view of system performance.

SCHEDULING AND DATA MINING

Most hand-created monitoring systems were polling systems that ran on a schedule. Every five minutes, a hand-created script might wake up and ping every server in the network; if any response was slow, it would email an admin-istrator. If a response timed out, the system

The right APM tool ideally covers most, if not all, of the platforms that are important to an enterprise and integrates them into a holistic view of system performance.

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS15

HARMONIZATION

would send a text message. This is known as taking the heartbeat of the system. Most APM tools continuously take the heartbeat of the system, creating a huge amount of data, which can be incredibly powerful.

Consider, for example, a system that is used during the business day. The “average” response time of a 24-hour day will include all the quick responses overnight when system traffic is low; it can also be thrown off by a few very slow responses at peak traffic times. Managers that are looking at this data likely want only the median response time of that eight-hour day.

On the other hand, those few slow responses could be saying something. There might be a few peaks in usage during the day—in the morning when everyone logs in, for example—that could lead to lost sales and renewals or, for internal software, complaints and lack of productivity. Worse, the peak time could be the start of a trend that will only grow if the software is popular. If those are issues in your organization, dig into what features the APM software has for data mining, for looking at, say, the average of 10% of the worst responses.

The ideal APM tool will proactively point out problems as they inch toward a services com-mitment breach, not after.

REPORTING

Collecting data is a big step to improving appli-cation performance; knowing what to do with it is another. There are lots of different ways to report and notify, but the first place com-pany’s employing APM software should start is deciding what they want to know about their product. To start, many people are interested in downtime, or periods of time where a product is not accessible, and extreme resource usage like high memory and CPU usage. These few things are the blood pressure and temperature of a product, vitals that show when something is right on or way off.

Software that can monitor application per-formance may provide many answers, but dif-ferent APM tools answer different questions. It’s important for companies to first come up with a list of questions they want answers to—such as what is the average transaction time for login, where their users are and when they are

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS16

HARMONIZATION

active, which database processes are the slow-est, what is the average response time for their API. Then, the company needs to find which APM products answer those questions.

Once an IT or DevOps department knows if the APM software can capture the data they want, they should consider if visualization, dashboards, alerts or warnings, along with email lists and responsibility by system are core needs. Some APM software provides the ability to visualize system performance on a monitor as it is happening. If developers or tes-ters want people to see performance on public screens, then find out what particular informa-tion certain individuals in certain roles on cer-tain teams need and see if the APM tools can provide reports that drill down to the level the organization requires.

SYSTEM SECURITY

Anyone that has dealt with software has prob-ably come across security tools. APM software is no exception. There are administrative func-tions; data that might be sensitive, especially

for companies in regulated industries like healthcare; and reports that people should not be privy to. In healthcare systems, data is care-fully guarded. Someone seeing data that they shouldn’t have access to often means large fines. If an enterprise is one that is concerned about who is accessing data, then security is a must have.

Nearly all APM software can be configured for security and access levels; the trick is to figure out if it will integrate seamlessly with the directory system you have without requir-ing an administrator to add and remove per-missions for users. Consider, too, if the data is sensitive and if it could somehow emerge in reports. For example, if webpage links include transaction IDs and those appear in reports of slow pages, it might be a problem.

Platform compatibility and integration, reporting ability, and security are all aspects of APM software that can help guide your pur-chase. An enterprise should start by figuring out what it needs from the tool and how it will support its business goals, and then move from there. —Justin Rohrman and Matt Heusser

HOME

EDITOR’S NOTE

INVEST IN THE RIGHT

QA AND TEST MANAGEMENT

SOFTWARE

COMPARING TODAY’S

TOP APM SOFTWARE

ADDRESS ALL YOUR

APM SOFTWARE NEEDS

UNCOVER YOUR APPLICATION PERFORMANCE MANAGEMENT NEEDS17

ABOUT THE

AUTHORS

MATT HEUSSER is a consultant at Excelon Development, where he recruits, trains and does software testing and development. Read more about Heusser on his blog, “Creative Chaos,” or follow him on Twitter: @mheusser.

GERIE OWEN manages projects involving multiple appli-cations and coordinates test teams across multiple time zones. She also is a Certified Scrum Master and has pre-sented at conferences, including Better Software, Bel-gium Testing Days, CAST, TestKit and STPCon. Find her on LinkedIn.

JUSTIN ROHRMAN has been a professional software tester in various capacities since 2005. He currently is a con-sulting software tester and writer working with Excelon Development. He also serves as president on the Associ-ation for Software Testing Board of Directors. Check his webpage for more about him.

Uncover Your Application Performance Management Needs is a SearchSoftwareQuality.com e-publication.

Scott Wallask | Editorial Director

Ron Karjian | Managing Editor

Moriah Sargent | Associate Managing Editor

Jan Stafford | Executive Editor

Valerie Silverthorne | Site Editor

Brein Matturro | Group Managing Editor

Linda Koury | Director of Online Design

Martha Moore | Senior Production Editor

Doug Olender | Publisher dolender@techtarget.com

Annie Matthews | Director of Sales amatthews@techtarget.com

TechTarget 275 Grove Street, Newton, MA 02466

www.techtarget.com

© 2016 TechTarget Inc. No part of this publication may be transmitted or repro-duced in any form or by any means without written permission from the pub-lisher. TechTarget reprints are available through The YGS Group.

About TechTarget: TechTarget publishes media for information technology pro-fessionals. More than 100 focused websites enable quick access to a deep store of news, advice and analysis about the technologies, products and processes crucial to your job. Our live and virtual events give you direct access to independent ex-pert commentary and advice. At IT Knowledge Exchange, our social community, you can get advice and share solutions with peers and experts.

COVER: ISTOCK

STAY CONNECTED!

Follow @SoftwareTestTT today.