faster sap testing means business...
Post on 28-Mar-2018
223 Views
Preview:
TRANSCRIPT
FASTER SAP TESTING MEANS BUSINESS AGILITY
02.04.2012
AN EASY-TO-USE, CLOUD-BASED, SECURE SOLUTION FOR TESTING SAP SYSTEMS
- 1 -
FASTER SAP TESTING MEANS BUSINESS AGILITY If you have an SAP ERP installation at your company, or run one for a client, one of the biggest
challenges is avoiding negative side effects to the production system when changing or adding
business processes, customizations, patches, or upgrades. This is the main reason SAP requires
businesses to run a separate instance of its ERP system in order to conduct testing. Regression
testing helps ensure that changes do not impact production systems.
Thorough testing slows the pace of needed changes and reduces agility. If you could speed up
the testing process significantly, you could move faster without increasing risk.
Most organizations test their SAP applications manually before they move packages of changes,
called transports, to production systems. Manual testing is labor-intensive, requiring hours of
tedious work. There are a few automated testing solutions on the market, but for the most part
they are expensive, difficult to implement, and time-consuming to use and maintain. Many
administrators would like a more cost effective and simplified solution to test SAP applications.
There is now a third, emerging generation of testing solutions that focuses specifically on
packaged applications and reduces the effort of capturing and running tests scenarios by a
factor of 10. This new breed of testing solutions also offers increased peace of mind, knowing
that test coverage is complete and that everything will run smoothly.
In this paper, we track the development of SAP testing techniques through these three
generations and explain how the latest generation of testing tools represents a giant step
forward in addressing a challenge that all SAP customers face.
THE THREE GENERATIONS OF SAP TESTING Software testing methodologies and tools have evolved to support the demands of the business
world. In this section we discuss the pros and cons of three generations of testing solutions:
manual, automated, and intelligent.
- 2 -
THREE GENERATIONS OF SAP TESTING TOOLS
GENERATION OF TESTING
TOOLS
CREATING SCRIPTS RUNNING SCRIPTS MAINTAINING SCRIPTS
FIRST GENERATION:
MANUAL TESTING
Manual Manual Manual
SECOND GENERATION:
AUTOMATED TESTING
Semiautomated Automated Manual
THIRD-GENERATION:
INTELLIGENT TESTING
Automated Automated Automated
FIRST GENERATION: MANUAL TESTING The first generation of testing, manual testing, is still the method most companies use for the
bulk of their testing. Because business processes trigger functions in SAP, manual tests are
often created and run by power users or SAP functional experts, who view testing as an extra
step in the release process. They see it as time-consuming and tedious, with very little payoff.
Manual testing causes friction between end users, power users, functional experts, and IT
managers. While IT managers worry about system stability, users in all categories see testing as
a burden that is added on top of their regular responsibilities.
More than half of the organizations surveyed don‟t use a testing solution at all. Of those who
do test, 74% don‟t use an automated solution, and 53% use general-purpose business software
such as Microsoft Word and Excel. (Source: 2011 SAP Testing Survey)
Despite the investment level of most companies in SAP ERP and its importance to their business,
testing still gets short shrift.
Manual testing remains the most common method of SAP testing. On the one hand, it is
relatively economical, at least from the perspective of purchasing software and staffing.
Because no special skills—besides understanding the SAP application and having a great deal of
patience—are required, and no special software needs to be purchased, many organizations
continue to test this way, creating test plans and executing them.
- 3 -
On the other hand, manual testing is a costly and ineffective way to limit risk to an expensive
investment. Manual testing is time-consuming, inefficient, and highly prone to errors. It‟s
impossible to achieve 100% coverage for all possible usage scenarios. The result is that
frequently little quality testing gets done when manual testing is the only option.
In many ways, this slows down a business‟s ability to innovate and to keep up to date on the
latest software releases. Concerned about the cost and delays involved with upgrades and
testing, many organizations are slow to make changes to their SAP installations. This limits
opportunities provided by SAP‟s „best practices‟ enhancements to innovate and become more
efficient on a business level; on the technical system front it actually heightens the risk of a
catastrophic system failure or non-supported technical problems.
PROS AND CONS OF MANUAL REGRESSION TESTING
PROS CONS
Can handle any script and playback
permutation scenario using “human
intelligence”
Economical in terms of software purchases—
most manual testers use standard business
software that is already in-house
Simple—the only requirement is to accurately
record the results of changes made to the SAP
software
Economical on a per-resource basis—little skill
required
Not comprehensive—limits the ability of an
organization to roll out changes to SAP
Labor-intensive—high labor cost relative to level
of risk control provided
Risky and error prone because of human
limitations related to extended focus, accuracy,
and repetition
Slow—Since all testing is done by humans,
thorough testing takes time. The tradeoff is
whether to sacrifice quality or delay the release
SECOND GENERATION: AUTOMATED TESTING The second generation of SAP testing delivered a significant leap forward. Quality assurance
(QA) platforms were built to test software, offering several advantages. Business users or
functional experts no longer needed to devote significant time to running the tests, and the
tests could be run continuously with minimal disruption. The ability to automate a far greater
number of tests offered new potential to diagnose issues with SAP installations and accelerate
rollout schedules.
There were still significant disadvantages, however.
- 4 -
COMPLEXITY AND EFFORT
Test automation is a resource-intensive undertaking, requiring programmers for home-baked or
vendor solutions. The typical test automation project includes setting up dedicated, on-premise
hardware and software and hiring a full-time test automation programmer and administrator to
manage the project. The deployment of these systems offers substantial configurability but it is
quite complex.
EXPENSE
Organizations could spend close to $300,000 before a single test is run. Often, the tests run
aground before any useful information is collected.
PROS AND CONS OF AUTOMATED REGRESSION TESTING
PROS CONS
Business users not as heavily consumed in
tedious testing execution
Increased test execution efficiency as
greater volume of test repetitions can be run
in a given time period
Testing depth and accuracy improves—for the
constructed scripts
Once tests are created, they can be run
again and again (subject to maintenance
work)
Offshore testing resources can help develop
tests
Complexity—Takes up to a year to implement. Coverage is
limited because of time and effort to create scripts for
each business process
Effort—full-time test automation developers are needed
Expense—as much as $300,000 before the first test is run
Tools are general-purpose; not explicitly designed for
testing SAP systems—requires tedious specification and
programming work by both functional experts and test
engineers
Disconnect between functional experts and testing pros
(external and offshore) can lead to coordination issues and
even logical errors in the created test scripts
Deployment costs—an expensive solution with regard to
effort to deploy and system landscape requirements
Brittle scripts—Once created, scripts are hard to maintain
because even a small change breaks them
Developing scripts using test automation tools requires
scarce and expensive engineering talent
- 5 -
THE STATE OF TESTING TODAY According to the 2011 SAP Testing Survey, only about 26% of organizations have implemented
automated testing of their SAP systems. Even at organizations that have adopted such tools,
automated tests still do a fraction of the work. Those organizations using automated testing
tools automate only about 20% of their testing processes, meaning that the bulk of testing is
still done manually—organizations write test scripts and keep track of results using Word
documents and Excel spreadsheets.
Even more telling with regard to testing is the apparent lack of satisfaction with the solutions.
Nearly half of the respondents to the survey are unhappy with their current situation; of the
26% who have tried automated solutions, 15% are not satisfied with what they've achieved using
them. Of the 74% who have yet to automate testing, about 34% plan to do so.
With nearly half unhappy with their current situation, there is a need for a new take on the
very old problem of testing SAP landscapes.
THE STATE OF TESTING AUTOMATION (SOURCE: 2011 SAP TESTING SURVEY)
- 6 -
THIRD GENERATION: INTELLIGENT TESTING FOR SAP SYSTEMS The third generation of SAP testing is a software-as-a-service (SaaS) testing solution designed
specifically for SAP. First, this solution does an impact analysis to show you what areas of the
system are impacted by the changes and where to focus testing. Then, business users or
functional experts generate testing scripts on the fly by running the business process once on
their actual system. From then on, anyone can automatically run the test scenarios whenever
needed. The generated test scripts can be executed automatically and they self-adjust, based
on test results, with no need to re-record. Alongside the automated script, a natural-language
step-by-step document is created, which is useful for management, reviews, and manual
execution if needed. As a result, customers can reduce the effort associated with running SAP
ERP tests by up to 80%.
PROS AND CONS OF INTELLIGENT TESTING
PROS CONS
Reduced time for testing staff—up to 80% less time
spent testing
Reduced time-to-market for SAP changes of all
types (days instead of months)
Software-as-a-Service—no hardware or software to
install—set-up in 20 minutes
Cloud-based library stores 70,000+ test scripts, and
adds more every day
Regenerative learning—learns from and logs errors;
skips them on the next run
Simple and intuitive—business users can record
their daily activities, then automate test scripts
Organizations with an investment in automated
testing might find it challenging to justify a second
investment
- 7 -
THE PANAYA PROPOSITION Panaya is the leading third-generation SAP testing solution.
Panaya‟s cloud approach to SAP testing is different. Instead of weeks or months to begin
testing, Panaya can be up and running in 20 minutes.
Panaya is an intelligent testing system that combines the human intelligence from manual
testing with the efficiency and adaptability of computers. Panaya‟s customers perform their
manual tests only once. Panaya records their keystrokes as they work through business
processes and creates scripts. The next time that they want to test the process, the scripts run
automatically.
Panaya creates both manual and automated test scenarios in minutes. You don‟t need test
automation engineers on staff to replicate and test your business processes. The money you
save will come from spending less on dedicated testing staff. The time you save will reduce the
time-to-market for the next iteration of your SAP ERP application. And finally, the increased
test coverage improves the quality of your SAP ERP application and provides additional peace of
mind.
Here are a few of the value propositions of Panaya.
BUSINESS CLASS The level of effort required to set up most testing scripts is beyond the reach of the business
user and requires dedicated programmers. Panaya is different. With Panaya, business users—the
people who use SAP every day—can start recording and testing within minutes. The key here is
that the business user doesn‟t feel like they‟re preparing tests because they are simply using
the ERP application, just like they always do.
SAAS = SIMPLICITY, SECURITY, AND STABILITY Panaya is delivered as a SaaS solution. That means no machines to host, no dedicated
administrators, and no installation routines. Screens are simple and intuitive. Setup takes about
20 minutes.
Creation and execution of tests is always initiated and controlled by the customer; Panaya
doesn‟t have any access to the customer landscape or to customer data. Panaya runs on
Amazon.com‟s EC2 cloud, which has the strongest security certification of any cloud offering
available. Even FDA-regulated companies and financial institutions with strict security
requirements use Panaya.
- 8 -
THE WISDOM OF THE CROWD There are more than 20,000 companies using SAP. Before rolling out changes, they need to
generate regression-testing scripts and perform regression testing. Because there are many
shared processes across SAP instances, many of the scripts generated at individual installations
are redundant. So why spend time creating them?
At Panaya, we have analyzed more than 2,500 SAP instances and have created, stored, and
catalogued more than 70,000 test scripts. Our algorithm uses the wisdom of the crowd, more
thoroughly automating your scripts than any automation test engineer who could possibly
work for you.
Think of Panaya as the Google Trends of ERP testing. The entire SAP community benefits from a
knowledge base that is more powerful than any one company could create.
REGENERATIVE LEARNING AND SELF-HEALING No test run is likely to be perfect, but every glitch is a learning opportunity. Panaya can learn
from errors, and avoid repeating them the next time. Here are three common scenarios where
Panaya excels and other solutions fall short.
SMART PARAMETERIZATION
Suppose you are creating a purchase order and then use this purchase order for goods receipt.
SAP always generates unique purchase order numbers, so every time you run this test, you will
have to use the order number you received while running the test. Manual testers simply take
notes of the order number they received and enter it manually. Test automation engineers must
have the business knowledge and do some fair amount of programming to get such processes
working, parameterizing the purchase order number. Panaya automatically knows to pass
document numbers exactly as you would expect it to work without additional effort on anyone's
part.
CROWD-BASED WORKAROUNDS
Test recordings naturally cover only one specific case. When running the test, the system may
behave slightly differently. This is where the cloud can work for you. Panaya‟s software already
knows how other customers reacted in the very same situations. So when unexpected behavior
occurs, Panaya automatically looks up a way to handle it via the community knowledge base
and keeps running. If there is no workaround available, it switches to manual mode to let the
user deal with the unexpected situation.
- 9 -
For example, if the user name field content is different from the recording, Panaya knows
that‟s not a problem. However, if you suddenly have a different item category, it will alert you
of a potential error.
TESTING AT THE PUSH OF A BUTTON
There may be no better example of Panaya‟s advantage than the following diagram. This
diagram represents the difference between Panaya and a generic testing solution. We‟ve
collapsed some of the sub-tasks for clarity, but think you‟ll get the picture.
To capture a business process and run a test script in a general-purpose test-automation tool, a
test-automation expert must negotiate his way through 32 steps and the business user must be
involved in 11 of those steps. With Panaya‟s SAP-specific testing solution, the same process can
be automated, by business user alone, in just two steps: record and play.
PANAYA STREAMLINES TESTING
“AN INCH WIDE AND A MILE DEEP” Generic testing solutions are made to work on every Windows application in the enterprise;
they are a “mile wide and an inch deep.” Panaya was built using domain-specific knowledge,
explicitly for SAP ERP systems—we‟re “an inch wide and a mile deep.” Our solution brings a new
depth to testing software—the way people actually work on SAP.
- 10 -
DOMAIN AWARE: INHERENTLY UNDERSTANDS SAP PROCESSES
Let‟s go back to that purchase order for a minute. Suppose you are attempting to test the
integrity of a purchase order process in SAP. With a generic testing solution, you would write a
long script that would run the same process, with allowances for every expected amount
($10,000 increments, for example). Now, suppose there are certain business rules that must
occur in order to accept certain numerical increments: If an order is more than $10,000, it
should be approved by the site manager; if an order is more than $20,000, it needs the regional
manager‟s approval. In a generic testing solution, you would need to write a highly complex
branch script to cover these business cases. If the approval protocol changes, you have to
change the script. That‟s a lot of “if‟s.”
With Panaya, you quickly record a short script for each approval condition—so changing the
protocol for any one increment won‟t break the entire order processing routine.
CONCLUSION: FROM QUANDARY TO QUALITY Panaya is a high-quality, low-impact, easy-to-use ERP testing solution. Manual testing is time
consuming and an ineffective use of resources. Homegrown and generic commercial automated
testing is expensive and difficult to set up. With Panaya, the intelligent testing solution for SAP,
you don‟t have to choose between time-to-market, cost reduction, and risk reduction. You can:
Reduce the effort and duration of regression tests by a factor of 10 Increase the number of useful scripts by a factor of 10 Commit internal IT staff to higher-value projects Let SAP users test their own functions Introduce changes more frequently, while maintaining or reducing risk levels Test 10 times a year, rather than two
By using Panaya to test your SAP ERP system, you can be more innovative, agile, and meet
changing business conditions head-on. By helping you move faster, Panaya speeds up business
and moves testing from a roadblock to a competitive advantage.
- 11 -
DISCLAIMER AND TRADEMARK NOTICES This report is provided by Panaya Ltd. It is completely independent of and not affiliated with
Oracle or SAP AG. Oracle is a registered trademark of Oracle. Oracle and other Oracle products
and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of Oracle. SAP is a registered trademark of SAP AG. SAP and other SAP products and
services mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP AG in Germany and in several other countries. All other product and service
names mentioned are the trademarks of their respective companies.
DISCLAIMER OF WARRANTY
Panaya Ltd. makes no representation or warranties, either express or implied by or with respect
to anything in this document, and shall not be liable for any implied warranties of
merchantability or fitness for a particular purpose or for any indirect special or consequential
damages.
COPYRIGHT NOTICE
No part of this publication may be reproduced, stored in a retrieval system or transmitted, in
any form or by any means, photocopying, recording or otherwise, without prior written consent
of Panaya Inc. No patent liability is assumed with respect to the use of the information
contained herein. While every precaution has been taken in the preparation of this publication,
Panaya Inc. assumes no responsibility for errors or omissions. This publication is subject to
change without notice.
© 2012 Panaya Ltd. All rights reserved.
ABOUT PANAYA Panaya‟s software-as-a-service helps companies that use SAP or Oracle reduce 80% of their
upgrade and testing risk and effort. Utilizing a cloud-based supercomputer, Panaya simulates
the upcoming upgrade, automatically pinpointing which custom programs will break as a result
of the upgrade and automatically fixing most of these problems.
Panaya‟s testing solutions dramatically expedite ERP testing and eliminate the need for manual
test script maintenance. Seamlessly capturing business knowledge in the background, as users
work with the ERP applications, Panaya automatically generates plain-English test scripts that
are rapidly executed and continually self-adjust based on test results.
top related