©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

Session ID: BTOT-TU-1700/6 Twitter hashtag #HPSWU

©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

Speaker Name: Martin Moser, PhDDate: 30.11.2010Session ID: BTOT-TU-1700/6

SAP ® Best Practices for Performance TestingHow SAP ® uses HP LoadRunner ® and HP QuickTime Professional ® for performance measurements

3 HP Confidential

Software Performance & Speed Bumps

4 HP Confidential

SP0 SP1 SP2 SP3 SP4

Expectation

SP0 SP1 SP2 SP3 SP4

Observa-tion

Software Performance & Speed Bumps

Expectation and observation regarding response timeR

esp

onse

tim

e

Months/Release

Removing the Speed BumpsHow Agile Methods Help

6 HP Confidential

– 4 week sprint run without (management) interference

– Sprints must deliver software that satisfy quality requirements

Agile development @ SAP

Potentially shippable product

increment

Sprint Backlog of

Tasks

4 week sprint

Daily Scrum Meeting and Artifacts Update

Cross-Functional Team

Product Owner

SAP Best PracticesMeasurement Methodology and Process

8 HP Confidential

SAP Performance Map

Full picture of system performance

- KPIs for all potential bottlenecks

- Software architecture dependant

- Measured in two-stage process

Full performance map includes front-end

Server-only performance map excludes front-end

ABAPserve

r

Java serve

r

Java DB

ABAP DB

9 HP Confidential

Stage 2Stage 1

SAP performance test process

SAP Performance Test Process

ReportingAnalysisReportgeneration

Test execution System restartLoadRunner script recording

System check

Test executionSystem restart

Tool configSystem check

Stage 1: Single-user tests

- Measure full performance map of a scenario

- Resource consumption per dialog step

- For one single user using the server exclusively

- Focus on end-to-end response time

Detect ~80% of all performance issues (mostly simple issues)

Stage 2: Multi-user tests- Measure server-only performance

map of a scenario- Average resource consumption of

the scenario- For multiple users accessing the

server in parallel- Focus on scalability and stability of

the system

Detect ~20% of all performance issues (mostly difficult issues)

10 HP Confidential

Stage 1: Single-Users Tests

The SUPA tool

- Generic UI

- Plug-in based server

- Plug-ins collect full performance map:- End-to-end response

time- Garbage collection

data- CPU consumption- Network traffic- DB load

ABAPserve

r

Java serve

r

Java DB

ABAP DB

UI

Server

E-t-E

GC

CPU

...

11 HP Confidential

Single-Users Tests: Follow-up

– The SAP Performance Standard defines KPIs that are a prerequisite for good performance

– Violations of the KPIs are identified using the SUPA tool

– Specific tools are available to investigate the root-cause of KPI violations, among others:- SAP Java Virtual Machine Profiler- Memory Analyzer Tool (Open source tool sponsored by SAP)- ABAP transactions such as ST05, STAD, ST02

– The findings are reported back to development for fixing

12 HP Confidential

Stage 2: Multi-User Tests

HP LoadRunner used to check

- Server response time for fixed number of users

- Linear scaling of throughput with number of users

- Stability over longer periods of time

Follow-up as for single-user tests.

ABAPserve

r

Java serve

r

Java DB

ABAP DB

13 HP Confidential

HP LoadRunner strengths and weaknesses

– Strengths:- High load generation with off-the-shelf hardware- Supports most well-known technologies, among others: HTTP, Flash, Silverlight, SOAP, TCP/IP

- Scriptable in various languages, such as: C, Java, Visual Basic

- Extendable monitoring

– Weaknesses:- Correlation is complex (see TruClient below)- No monitoring for: Java garbage collection, CPU break-down & HTTP message lengths

14 HP Confidential

SAP internal extensions to HP LoadRunner

Scripted extensions:

- Java garbage collection

- CPU break-down

- HTTP Request and response lengths

ABAPserve

r

Java serve

r

Java DB

ABAP DB

Controller

Monitors

LRScript

SAP monitoring routines

15 HP Confidential

Stage 2Stage 1

SAP Performance Test Process

SAP Performance Test Process Automation

Sem

i-auto

mate

d

Sem

i-auto

mate

dSUPALoadRunner

ControllerLoadRunner

Analysis

NetWeaverTestSuite

HP QTP

NetWeaverTestSuite

HP QTP

ARIES suite (full automation)

ReportingAnalysisReportgeneration

Test execution System restartLoadRunner script recording

System check

Test executionSystem restart

Tool config

LoadRunner VUGen

NetWeaver

System check

Getting Even BetterBudgets and Price Tags

17 HP Confidential

The Way Forward to Predicting Performance

UI design

Architecture

Implementation

Measurement

Typical waterfall model

- Design the UI

- Design the architecture

- Build the system

- Measure the performance

Long and expensive feedback cycle.

18 HP Confidential

The Way Forward to Predicting Performance

UI design

Architecture

Implementation

Measurement

Budget and price tags- Define resource budget for

dialog steps

- Measure resource consumption of

- Major UI elements- Major backend operations

- Attach price tags to them

UI designers and architects can estimate performance of designs

19 HP Confidential

LoadRunner TruClient: First experiences @ SAPStrengths:

- Runs fully embedded in the browser

- No correlation required

Weaknesses:

- Significantly higher hardware requirements

- Instrumentation extension possible, but cumbersome

Continue the conversation with your peers at the HP Software Community hp.com/go/swcommunity