SAP BLU

E PR

INT

FOR T

ESTI

NG

DES IGNED BY:STEPHEN MISSETT

CONTENTS1> Introduction

2> Check List

3> SAP Unit Test ing

4> SAP System Testing

5> SAP Scenario & Str ing Testing

6> SAP Integrat ion Testing

7> SAP Interface Testing

8> SAP E2E Testing

9> SAP UAT

10> SAP Stress/Load/Performance Testing

11> SAP Usabi l i ty Test ing

12> SAP Securi ty & Authorisat ion Testing

13> SAP Cutover/Dry Run Test ing

14> Appl icat ion Testing

15> DITL Testing

16> Regression Testing

17> SAP Blue Pr int For Test Inputs & Outputs

18> SAP Technical Blue Print

INTRODUCTION

It’s important to create usable working definitions that fit your project.  And remember not every project will need to perform every one of the types of testing in this presentation.

In general the type of testing is tied to a project phase or system environment, but project phases and system environments can be a hot topic, too (and a subject of future blog entries).  Who does the testing provides an additional clue where it fits in the overall project lifecycle.  What follows are a few broad definitions of testing types.

CHECK LISTA short checklist to review when thinking about each type of testing:

> What systems (SAP, non-SAP) are needed?

> What environments (development, QA, training, etc.) are needed?

> What data (master data, transaction data, and historical data) is needed?

> Who does the testing (development team, test team, end users, etc.)?

> What are the testing success criteria?

> How are results documented and able to be audited?

> What test cases (positive and negative) are required?

> Who provides sign-off?

SAP UNIT TESTING

This tests isolated pieces of functionality, for example, creation and save of a sales order. 

The test is done in the development by a configuration specialist and confirms that the sales order can be saved using the SAP organisation elements:

> sales organisation

> company code

> credit control area

> customer master data set up

> partner functions material master data

 The above points establish a baseline of SAP functionality

For ABAP development, for example, unit testing shows that a report can be created from developer generated data 

Assistance in data generation may come from a functional consultant

SAP SYSTEM TESTING

Testing where elements of related SAP functionality are linked together:

Where?

> In the development environment to ensure all the pieces work together. 

Examples:

> A quote-to-cash flow can create a sales order

> A delivery can be created

> Processed from the order

> Delivery can be billed

> The billing released to accounting

> Customer payment applied against the accounting invoice

Each of the component parts is unit tested ahead of time and the data used in testing is usually fabricated based on the knowledge of the project team.

SAP SCENARIO & STRING TESTINGTests specific business cases 

For example:

> There may be configuration and business process design that is unique to a certain customer set or a given product line or a set of services.

> Tangible products and services are processed very differently from each other, so you might have different scenarios you need to test.  

> Again this testing is usually done in the development environment to prove out a requirement – an argument can be made to say this is a test case you would cover in system testing. 

> Scenario testing can also happen in the QA environment, also known as string testing.

> This testing also includes execution of interfaces and other development objects, e.g. reports, with fabricated data.

SAP INTEGRATION TESTINGIntegration testing is similar to scenario testing except it is typically done in the QA environment and uses more realistic data. 

Ideally the data has come from a near real data extraction, conversion and load exercise so the data has a certain familiarity to it for a business end user:

> Recognisable customers

> Materials pricing

> Vendors

> Contracts

> Industry formats

The testing shows that the business process as designed and configured in SAP runs using representative real world data. 

In addition the testing shows:

> Interface triggers

> Reports generated

> Workflows are working

> IDOCs generated

> Objects loaded

> Replication

> BADi triggered

> BAPI executed

> Correct Configuration

SAP INTERFACE TESTING

Testing of interfaces typically occurs at different points in a project so it is important to know what you are testing when. If some interfaces not available recommendation is Service Virtualisation.  During the project development phase isolated interface testing usually refers to unit testing activities where you confirm that your code can consume a file of your own making. 

You might have two development systems – one SAP, one non-SAP – where you run a test to show that the sender can generate a file and the receiver can consume it. 

In the QA environment interface testing might involve execution of business transactions on the sending system followed by looking for automatic generation of the interface output.

This is then followed by the receiving system consuming that file and proving that a business process continues on the receiver. 

Your interface testing might prove that the whole process runs automatically with business events triggering the outbound interface correctly, automatic transfer and consumption by the receiver.

This testing and it’s definition can become even trickier if you use a message bus where the idea of point-to-point interfaces doesn’t apply and you need to consider publish-and-subscribe models.

Be clear about the scope of the tests and the success criteria.  Typically interface testing becomes part of larger testing activities as a project progresses. 

Interface testing shows that the triggering works, the data selection (and exclusion) is accurate and complete, data transfer is successful, and the receiver is able to consume the sent data. 

Wrapped around this is showing that all the steps run automatically and that error handling and restart capability (e.g. data problems, connectivity failures) are in place.

SAP END-TO-END TESTING

This is similar to scenario testing in that a specific business case is tested from start to finish

> Running of interfaces

> Reports

> Manual inputs

> Workflow

> Replication

> IDOC creation

> Message Business Documents

> Invoicing and billing

In short it is attempting to simulate a real world business process and, in order to make it as real as possible, it is done using the most realistic data. 

Ideally the data used was the result of a data extract, conversion and load process. 

Execute Testing in a QA environment: at some level it can be seen as a way of validating that the individual unit tests, scenario tests, integration tests and interface tests produced results that work together.

SAP END USER TESTING & USER ACCEPTANCE TESTING

These two are grouped together because they are closely related, if not identical. 

The goal here is to ensure that end users are able to perform their designated job functions with the new system(s).

  A crucial part of this testing is referring back to the business requirements and blueprint to ensure that the expected features, functions and capabilities are available. 

As part of the project user involvement along the way should have been providing feedback to ensure the design met the requirements, so there should not be any big surprises.

Again this is activity that usually occurs in a QA environment with realistic data and the inclusion of end user security and authorisations.

SAP STRESS / LOAD / PERFORMANCE TESTING

This kind of testing examines:

> The system response time is acceptable

> Periodic processes run quickly enough

> The expected concurrent user load can be supported

> Identifies processing bottlenecks and ABAP coding inefficiencies

It is rare for a project to have worked out all the system performance tuning perfectly ahead and to have every program running optimised code. 

Consequently the first stress test on a system can be painful as lots of little things pop up that weren’t necessarily an issue in isolated testing.

The testing is geared towards:

> Simulating peak loads of activity

> Either online users or periodic batch processing

> Identifies the steps needed to improve performance

Given that the initial test reveals lots of areas for improvement you should expect to run through this a couple of times to ensure the results are good.

SAP USABILITY TESTING

This testing is usually concerned with:

> How many key strokes

> Mouse clicks it takes to perform a function

> How easy and intuitive it is to navigate around the system and find what you need

 

In a SAP implementation using the standard GUI there isn’t much scope for this kind of testing:

> End user training shows how to navigate

> How to create short cuts and favourites

> Modify screen layouts, etc. 

On the other hand a project that involves building portals may well need to perform this kind of testing, not just for reasons mentioned earlier, but also for consistency of look and feel.

SAP SECURITY AND AUTHORISATIONS TESTING

Ensuring that users are only able to:

> Execute transactions

> Access appropriate data that is critical

> Unable to view data that is not assigned the their user role

 This testing is typically done in a QA environment against:

> Near-final configuration

> Data from a full extract

> Conversion and load exercise 

> Test IDs for job roles are created

> IDs to both confirm what a user can do and what a user cannot do

More often than not this kind of testing is combined with end user or user acceptance testing.

SAP CUT OVER / DRY RUN TESTING

This kind of testing is simulating and practicing certain major one-time events in the project lifecycle. 

Typically the terms “dry run” and “conversion” together to mean a full scale execution of the all tasks involved to extract data from legacy systems, perform any kind of data conversion, load the results into SAP (and any other systems) and fully validate the results, including a user sign-off. 

Most projects have several dry run conversions which progress from an exercise in capturing all the steps, checkpoints and sign-offs in data conversion to a timed exercise to ensure everything can be accomplished in the time window for go-live. 

Once it becomes a timed event a dry run data conversion readily rolls into a cut over test, where it is one component of an overall cut over activity sequence.

A cut over test usually ensures that all the necessary tasks, e.g. importing transports; manual configuration; extracting, converting and loading data; unlocking user IDs; starting up periodic processing for interfaces, etc. are all identified and can be executed in the go-live time window.

APPLICATION TESTING

This term can be construed as so broad it has no meaning as an “application” can mean a lot of things.  I have only ever heard it as generic blanket term for another kind of testing, e.g. SAP application testing, so it needs to be refined and given context to be of use.

SAP DAY-IN-THE-LIFE (DITL) TESTING

This is one of my favourite kinds of testing – it really is what is says it is.  Run the system the way you expect it to be run during a regular business day. 

> Real users

> Real data

> Real volumes

> Real authorisations

> Real interface

> Periodic job execution

> The closest you can get to a production environment before you go-live with the system.

Not every day in business is the same so you might want to run a few DITL tests.  However these can be difficult to organise because of the need to have end users trained and available for extended periods of time as well as having all partner systems able to participate in the activities with real and synchronised data across the systems, real users, real data volumes, etc.

SAP REGRESSION TESTING

Each time you put a new release of code and configuration into your production system we want to be sure we don’t cause any changes in any processing beyond what you expect to change. 

Regression testing:

Test your existing functionality to be confident it still works as expected with the newly updated configuration and code base.  Clearly you don’t want to find you have issues in production after you make the changes.

Regression testing in a QA environment that has similar data to production is a good test bed.  In some cases automated testing can be effectively deployed as a fast and regular method to ensure core business processes are not adversely affected by new releases of code and configuration.

CONCLUSION

It is worthwhile having good definitions that are commonly understood and communicated across your project

APPLICATION BLUE PRINT FOR TEST INPUTS & OUTPUTS

SAP PI

SAP ERP

SAPCRM

Replication

SAP/Non-SAP interface

Raw data files loaded

FICA/FICO Invoices

BI Reports

XML fileIDOC

IDOC

CRM UI

Customer Service calls

Back Office Teams

mB

Doc

umen

ts

mBDocumentsMiddleware

SI_C

ontr

act

SI_Contract

Web based beta access

Android access

PDF Customer email

Iphone access

SE16 Config Tables SE38 ABAP Batch Program

Objects

BADiSE16 Config Tables

IDOCIDOC

Function Modules

Function Modules

SE38 ABAP Batch Program

Industry Files

Make Payments

Call Centre Staff

Tech Teams

View Invoices

SAP ISU Process Documents

SE16 Config Tables

SE38 ABAP Batch Program

Back Office Teams

Function Modules

Tech Teams

Function Modules

TECHNICAL LEVEL BLUE PRINT

ERP

BAP

Data base

ERP CRM

IDOC

BAPi

BADi

ABAP

SAP WEB APP SERVER

DDIC

DB ASL

QUESTIONS…..

Thank you