test_mgt

7/31/2019 Test_Mgt

1/27

Test Management

Application Lifecycle Management

Best Practice Process Document

7/31/2019 Test_Mgt

2/27

2012 SAP AG. All rights reserved. 2

Best Practice Processes:

Lean Test Management with SAP Solution Manager

Core Business Processes are documented with assigned Transactions and test cases in a project to be used to define the testscope for intended change activity. Test Plans and Test Packages are created based on the project . One or more testers areassigned to each Test Package to be able to execute the test and set test status. Test status is evaluated based on TestPackage, Plan or Projects.

Advanced Test Management with SAP Solution Manager

In addition to the lean testing approach attribute-based test planning is possible, test-workflow is realized via Release Status andTest Sequences. Change Impact Analysis with Business Process Change Analyzer (BPCA) is used to optimize test scope forregression tests and automated tests to decrease additional effort for repeating tests.

Advanced Test Management with integrated SAP Quality Center by HP and SAP TAO

Solution Documentation includes various documents like Business Requirements and Test Objects which are transferred to QCto be treated as Test Scope. Automated Tests are created with SAP Test Acceleration and Optimization (TAO) and updated incombination with Business Process Change Analyzer (BPCA).

Scenario / ALM ProcessTest Management

SAP Standards

Test Management

Problem Description:How to define, execute and sign-off tests that are based on newrequirements as well as on intended changes and potentially impactedprocesses

How to ensure that tests are reflecting business requirements, process, functional& technical designHow to define test scopeHow to plan test phases, assign testersHow to access and execute tests, document the results and manage defects

How to analyze project coverage, test planning status and test execution status
http://service.sap.com/~sapidb/011000358700001995812008Ehttp://service.sap.com/~sapidb/011000358700001995812008E

7/31/2019 Test_Mgt

3/27


Lean Test Management with SAP Solution Manager

Process DescriptionTest Scope Definition:

The business process structure has been created within a SAP Solution Manager project. At least transactions and testcases (preferred type: Test Document) are assigned to the appropriate structure nodes (in most cases: business processsteps). The transactions are assigned as Test Objects to the test cases. This provides a repository of test cases to cover thecurrent project scope as well as for future projects for re-use. It is recommended to check the coverage of businessprocesses by test cases before starting with Test Planning.

Test Planning:

For each test phase one or more Test Plans are created while selecting test cases from the project to be part of the testplan. Test Packages are created with a subset of test cases to be assigned to the relevant tester(s). Test cases to beexecuted twice by different testers should be assigned twice to 2 different test packages. It is recommended to check thateach test case of the defined scope is assigned to Test Plans and Test Packages before starting test execution.

Test Execution:The tester is launching his work list that contains all assigned test packages. If Test Packages are assigned to more than 1tester, these are handled as a common work list, so relevant test cases are intended to be executed by one of the assignedtesters. Detected defects can be addressed via messages using the integrated Service Desk. To get full transparency abouttest status and related messages it is recommended to frequently perform the related reports during the test phase as well

as for sign-off.

7/31/2019 Test_Mgt

4/27


Lean Test Managementwith SAP Solution Manager Test Scope definition

Project

Scenario

Business Process 1

Business Process 2

Process Step 1

Process Step 2

Process Step 4

Process Step 1

Process Step 2

Upload of existing Test Cases

For new Test Cases a templatedocument can be defined inproject standards

= Transaction

= Test Case Description

7/31/2019 Test_Mgt

5/27


Test Plan

Test Package 1

Test Package 2

Lean Test Managementwith SAP Solution Manager Test Planning

Project

Scenario

Business Process 1

Business Process 2

Process Step 1

Process Step 2

Process Step 4

Process Step 1

Process Step 2

Process Step 1

Process Step 2

Process Step 4

Tester 1

Tester 2

Tester 3

Process Step 1

Process Step 2

7/31/2019 Test_Mgt

6/27


Test Plan

Test Package 1

Test Package 2

Lean Test Managementwith SAP Solution Manager Test Execution

Project

Scenario

Business Process 1

Business Process 2

Process Step 1

Process Step 2

Process Step 4

Process Step 1

Process Step 2

Process Step 1

Process Step 2

Process Step 4

Tester 1

Tester 2

Tester 3

Process Step 1

Process Step 2

Test Status

7/31/2019 Test_Mgt

7/27 2012 SAP AG. All rights reserved. 7

Service Desk

Test Workbench

Yes

No

Yes

No

Role

Tester

Service

Desk Employee

1st Level

Developer

2nd Level

Lean Test Managementwith SAP Solution Manager Test Execution

7/31/2019 Test_Mgt


Lean Test Managementwith SAP Solution Manager Test Reporting

Business Blueprint

Test Planning

Test System SetupTest Execution

Apply Changes

to Production

Change ImpactAnalysis

SAP SolutionUpdate

New SAPSolution

Test Scope

IdentificationType of Change

Coverage ofBusiness

Processes byTest Cases ?

Assignment of TestCases to

Test Plans andTest Packages ?

Test Statusand Progress ?

Detected Errors andrelated messages ?

Sign-OffTest Plans

Sign-OffTest Scope Sign-OffTest Execution

TestCompleteness ?Defect Message

resolution ?

7/31/2019 Test_Mgt


Lean Test Managementwith SAP Solution Manager

Advantages

Quick and easy setup, preparation and planning of tests

Test Execution allows guided navigation to application to be tested via Test Object

Integrated Service Desk to address and follow-up defects

Powerful and flexible reporting on different levels: Project, Test Plans, Test Packages for Test Status and related messages

No additional license fees for Test Management Tools needed

Disadvantages

Limitations for larger organizations and projects regarding process variants and high volume test scope

Difficulty to analyze impact of change and related test scope for regression test

Increasing effort for repeating Tests due to missing test automation

Additional effort for coordination and notification of testers during larger cross-organizational scenario tests

Integrated with the ALM process*

Solution Documentation

Solution Implementation Upgrade Management

Change Control Management

IT Service Management

* ALM processes are reflected in scenarios in the Business Process Repository

7/31/2019 Test_Mgt


Advanced Test Managementwith SAP Solution Manager

Process Description

The Lean Test Management approach can be enhanced based on customers needs step by step to the full extent calledAdvanced Test Management. The following description and slides should be treated as delta information compared to the Lean

Test Management approach.

Test Scope Definition:In addition to the lean approach business processes and test cases will be classified via attributes to be able to allow a morefocused test planning and reporting for the different types and triggers for tests. This also allows to manage high volume of testcases with reasonable effort. To be able to perform change impact analysis and risk based test scope definition so-called

Technical-Bill-Of-Material (TBOM) need to be created for each process step / transaction.

Test Planning:Based on the preparation done above the actual test planning can be done much more efficient based on relevant filter criteria.In addition it is possible to use Release Status Schema to allow a phased approach for test planning, execution and relatedsign-off. This includes notification of assigned tester when starting or ending test phases. For process / scenario based testingthat requires the processing of individual test cases in a specific order Test Sequences can be defined to support the workflowand handover via mail notification from one to the other tester.

Test Execution:During test execution test coordinators and testers will benefit from using Release Status and Test Sequences.For repeating test activities like regression tests it is highly recommended to setup automated test cases to avoid additionaleffort for every repetition of tests.Enhanced flexibility and visualization is provided via additional Web-Reports based on SAP BW. These reports can be used toevaluate the current test status, the test progress as well as status and priority of test related messages. Also here theattributes defined during preparation can be used as filter criteria.To fulfill legal requirements like FDA a Test Report can be created for the different milestones to compile all test relevant

information per test plan.

7/31/2019 Test_Mgt


Advanced Test Management

with SAP Solution Manager Test Scope definition (advanced)

Classification of Business Processes via attributes

Core: Yes/No

Development Layer: Customer, Modification, Enhancement, Standard

Customer attributes: e.g. Country, Legal impact (FDA),

Responsible Person

Classification of Test Cases via attributes

Status: In Process / Released

Priority

Time Required

Customer attributes: e.g. Country, Legal impact (FDA), Test Type/Phase,

Links: e.g. to Process description, Specification documents for traceability reason

Connect Test Case to Test Objects (Transactions)Change Impact Analysis with Business Process Change Analyzer

Create TBOM at least for all transactions that are part of the Core Business Processes

For intended changes perform Change Impact Analysis to identify running processesthat might impacted

7/31/2019 Test_Mgt


Advanced Test Managementwith SAP Solution Manager Test Planning

Business Process Attribute Filter

Test Case Attribute Filter Allows to filter on Test Cases Attributes like: Test Case Type Search Terms Priority Statusassigned to the individual Test Case

Allows to filter Test Cases based on Business Process Attributes like Status Team Members Keywords SAP/Customer

Attributes

7/31/2019 Test_Mgt


Planned change

BPCA Execution

BPCA Analysis

Test Plan Generation

Customizing change Coding change

BPCA Analysis

Test Plan generated basedon impacted process stepsand assigned test cases(manual, automated)


with SAP Solution Manager Business Process Change Analyzer

7/31/2019 Test_Mgt


Test Plan

Test Plan

Test PackageTest Package

Test PackageTest Package

Advanced Test Managementwith SAP Solution Manager Test Planning with Test Sequences

Test Package

Tester 1

Tester 2Test Case 1

Test Case 2

Test Case 3

Tester 3

Standard approach:Several Testers (Tester Pool)are assigned to a collectionof Test Cases (Test Package)

Sequence approach: In addition to standardapproach you can assigneach Test Case to a Testerand the sequence of testcases can be processed asworkflow

Test Package

Tester 1

Tester 2Create Sales Order

Outbound Delivery

Tester 3

Example: As soon Test Case 1 has been processed successfully by Tester 1, Tester 2 will be notif ied by E-Mail that Test Case 2 is ready to be tested

Benefit: Less coordination effort during complex scenario tests and accelerated test execution

Post Goods Issue

Create Billing Doc.

7/31/2019 Test_Mgt


Advanced Test Managementwith SAP Solution Manager Test Planning with Status Schema

Test plan is closedfor changes

Test plan is closed

for execution

Test plan is open forchanges

Test plan is closed

for execution

Test plan is closedfor changes

Test plan is open for

execution

ProtectedNew Released

Simplified example to highlight the benefit of Release Status Schema for test plans

After TestTest Planning Test ExecutionPhase

Status

Impact

Automated E-Mail notificationto all Tester that test execution phaseis started

Automated E-Mail notificationto all Tester

that Test execution phase has ended

Workflow active:

Sign-OffTest Scope

Sign-OffTest Execution

Benefit:

Allows phased approach with Q-Gates and Sign Off with electronic signature to meet legal requirements

7/31/2019 Test_Mgt


Advanced Test Managementwith SAP Solution Manager Test Automation with eCATT

Testing of complex business process scenarios like Order-to-Cash and Procure-to-Pay

Advanced functionality to test backend business logic (function modules, SOA services, testresult verification at SAP table / database level, result comparison with previous runs, )

Central definition via SAP Test Workbench combined with execution in distributed systems

Central repository for test data

Strengths

Sales OrderBilling &PaymentReceipts

Return &Complaint Handling

Delivery &Shipment

Business Processes

eCATT Test AutomationeCATT

Script 1

eCATT

Script 2

eCATT

Script 3

eCATT

Script 4

Test Data Container

SAP System Landscape

Landscape 1 (Dev)

Landscape 2 (Test)

ParameterParameter Parameter

CRM Test 2 R/3 Test 2 ERP Test 2

CRM Dev 1 R/3 Dev 1 ERP Dev 1

Import

included in 1TestConfiguration

Test Automation of SAP Business Processes via eCATT

7/31/2019 Test_Mgt


Backend

Database

Table operations

Data

Table

Cust.

Table

ABAP Objects

BC Sets

SOA

eCATTBackendApplication Server

Frontend

Dynpro based UI

Function Library & BAPIs

SAP GUI SAP CRM, Portal, 3rd applications

?Web Dynpro

via integraded 3rd party test tool

Advanced Test Managementwith SAP Solution Manager Test Automation with eCATT

7/31/2019 Test_Mgt

18/27


Test Design

Test Execution

Launch eCATT and call HP QTP Record non-SAPGUI application via HP QTP + save script within eCATT

Work Center Test Management

User executes assigned Tester Worklist

Test Worklist: Executionof eCATT TestConfiguration

eCATT executes HPQTP script

QTP scripts executesnon-SAPGUI application

eCATT & QTP Test logs

Test Workbench: Test result reporting

Manual tests

Automated eCATT / HP QTP tests

Advanced Test Managementwith SAP Solution Manager Test Automation with integrated HP QTP

7/31/2019 Test_Mgt

19/27


GoalMeet legal requirement regardingTest Documentation

Test ReportingTest Report: Documenting Test Coverage and Results

Scope

Provide all test related

information in one document: Project / Solution Test Plan - Responsible Person Test Plan - Overall Results Status - Percentage

and absolute view

Related Messages System(s) under Test Involved Processes Keywords

Tester assignment Test Case Description incl. attributes Status history per Test Case

BenefitEfficient solution to get preparedfor audits.

7/31/2019 Test_Mgt

20/27


Advanced Test Managementwith SAP Solution Manager

Advantages

Reduced effort and complexity for repeating Test Planning with high volume.

Transparency about affected processes and reduced risk for planned changes

Reduced effort for regression tests

System guided Test Execution with workflow to support complex test scenarios

Integrated Service Desk to address and follow-up defects

Powerful, flexible and visualized reporting on different levels: Project, Test Plans, Test Packages for Test Status, TestProgress and related messages

Better support for internal and external audits and to meet legal requirements

No additional license fees for Test Management Tools needed

Disadvantages

Increased effort for initial preparation

Classification of processes and test cases

Creation of TBOMs for Change Impact Analysis

Setup of Automated Tests



Solution Implementation

Upgrade Management




7/31/2019 Test_Mgt

21/27



with integrated SAP Quality Center by HP* and SAP TAO*

Process Description

Test Scope Definition:The Advanced Test Management approach with SAP Quality Center integrated into SAP Solution Manager is based onRequirements Descriptions (documents) and Test Objects (Transactions) defined in a SAP Solution Manager project. TypicallyUser Acceptance Tests are referring to the Business Requirements. To be able to manage additional test types like String Test,Integration Test it is recommended to setup additional document types in SAP Solution Manager like Specification and Processdescription. The assigned Test Objects/Transactions are recommended to be used to trigger Regression Tests. This allows

traceability for all types of tests between SAP QC by HP and SAP Solution Manager. To define the Test Scope for a specific testthe related documents and transactions need to be assigned to the tab Transfer to SAP QC by HP in the SAP Solution

Manager Project. Then the Test Scope can be transferred to SAP QC by HP and appears in the Requirements Module. Therequirements appear in the same process oriented structure like in the SAP Solution Manager. Documents are created bydefault as Business Requirements and Transactions as Test Requirements. New Requirements need to be converted to tests,as most likely no test cases exist. Results of the Change Impact Analysis performed via BPCA can be used to define the testscope for regression tests based on the impacted process steps / transactions but also to identify existing test cases built bySAP TAO that might be affected too and so need to be updated.

Test Planning:In the Test Plan Module for the new requirements you can now define the test steps for each test. In case of automated tests the

test cases are build based on test components with SAP TAO. Existing Test Cases can be mapped to requirements that are partof the test scope. In the Test Lab Module you then define Test Sets and assign relevant Test Cases from the Test Plan Module.Test Execution:

Tests are executed in the Test Labs module and in case of failures defects can be to be reported directly.Test Status as well as Defects are transferred to SAP Solution Manager. Changes on the related Service Desk Message arereplicated with the Defect in QC until it is completed and re-test is performed successfully.At the various phases and milestones QC provides adequate reporting capabilities to evaluate coverage, status, progress oftest activities as well as defect status.In SAP Solution Manager a report is available to evaluate Test Status for each transferred Business Requirement or Test Object.

* additional license required

7/31/2019 Test_Mgt

22/27


SAP Solution Manager Project

Scenario

Business Process 1

Business Process 2

Process Step 1

Process Step 2

Process Step 4

Process Step 1

Process Step 2

Advanced Test Managementwith integrated SAP Quality Center by HP Test

Scope Definition

Requirements- SAP Solution Manager Blueprints

- Project Name- Business Scenarios

- Business Process

- Business Process Step 1

Outbound Delivery

VL01NVL02N

Quality Center ProjectRequirements Module

Transfer to SAP QC by HP

Outbound Delivery

VL01N

VL02N

Adapter for SAP QC by HP

7/31/2019 Test_Mgt

23/27



Scenario

Business Process 1

Business Process 2

Process Step 1

Process Step 2

Process Step 4

Process Step 1

Process Step 2


with integrated SAP Quality Center by HP Test Planning

Requirements- Solution Manager Blueprints


- Business Process


Outbound Delivery

VL01NVL02N


Test Plan Module

Test Plans

- Integration Test

- String Test - Delivery

VL01N

VL02N

Outbound Delivery


Outbound Delivery

VL01N

VL02N


7/31/2019 Test_Mgt

24/27



Scenario

Business Process 1

Business Process 2

Process Step 1

Process Step 2

Process Step 4

Process Step 1

Process Step 2


with integrated SAP Quality Center by HP Test Execution

Requirements- Solution Manager Blueprints


- Business Process


Outbound Delivery

VL01NVL02N


Test Plan Module

Test Plans

- Integration Test


VL01N

VL02N

Outbound Delivery

Test Lab Module


- Integration Test Del.

VL01N

VL02N

Outbound Delivery


Outbound Delivery

VL01N

VL02N


7/31/2019 Test_Mgt

25/27


Advanced Test Managementwith integrated SAP Quality Center by HP

Solution Manager: Application Lifecycle

Design Deploy OperateRequirements Optimize

Quality Center by HP*

Test PlanTestComposition

Test SetsBusiness / TestRequirements Defects

TestExecution

SAP Solution Manageradapter for SAPQuality Center by HP*

* additional license required

Build

Change ImpactAnalysis

SAP TAO: Test Acceleration and Optimization*

Generate newTest Components

Inspect new SAPBusiness Processes

Inspect updated SAPBusiness Processes

TestSign Off

Update existingTest Components

7/31/2019 Test_Mgt

26/27


Advanced Test Managementwith integrated SAP Quality Center by HP

Advantages

Complete and integrated Application Lifecycle Management using the advanced capabilities of SAP QC by HP and SAP TAO

Transparency about affected processes and reduced risk for planned changes

Reduced effort for regression tests

Reduced effort to create and maintain automated Tests while using SAP TAO

Integrated Defect Management in QC with Service Desk in SAP Solution Manager to address and follow-up defects

Powerful, flexible and visualized reporting inside QC for: Requirements Coverage, Test Status and Defects

Test Status reporting based on transferred requirements in SAP Solution Manager

Disadvantages

Additional software licenses to be purchased

Increased effort for initial preparation

Creation of TBOMs for Change Impact Analysis

Setup of Automated Tests



Solution Implementation

Upgrade Management




7/31/2019 Test_Mgt

27/27

No part of this publication may be reproduced or transmitted in any form or for anypurpose without the express permission of SAP AG. The information containedherein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors containproprietary software components of other software vendors.

Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks ofMicrosoft Corporation.

IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5,System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries,zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390

Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6,POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes,BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF,Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere,Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBMCorporation.

Linux is the registered trademark of Linus Torvalds in the U.S. and othercountries.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks orregistered trademarks of Adobe Systems Incorporated in the United States and/orother countries.

Oracle and Java are registered trademarks of Oracle and/or its affiliates.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, andMultiWin are trademarks or registered trademarks of Citrix Systems, Inc.

HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C,World Wide Web Consortium, Massachusetts Institute of Technology.

2012 SAP AG. All rights reserved.

SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjectsExplorer, StreamWork, and other SAP products and services mentioned herein aswell as their respective logos are trademarks or registered trademarks of SAP AGin Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects, CrystalReports, Crystal Decisions, Web Intelligence, Xcelsius, and other BusinessObjects products and services mentioned herein as well as their respective logosare trademarks or registered trademarks of Business Objects Software Ltd.Business Objects is anSAP company.

Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and otherSybase products and services mentioned herein as well as their respective logosare trademarks or registered trademarks of Sybase, Inc. Sybase is an SAPcompany.

All other product and service names mentioned are the trademarks of theirrespective companies. Data contained in this document serves informationalpurposes only. National product specifications may vary.

The information in this document is proprietary to SAP. No part of this documentmay be reproduced, copied, or transmitted in any form or for any purpose withoutthe express prior written permission of SAP AG.