agile pt

8/6/2019 Agile PT

1/15

AgilePerformance

Testing

Cognizant Technology Solutions

Plot No. 26,

Rajiv Gandhi Infotech Park,

MIDC, Hinjewadi,

Pune-411057 (INDIA)

+91-20-22931100

Lokesh GulechhaAssociate

AbstractIncrease in competition and leaps in technology

have forced companies to adopt innovativeapproaches to have a system which performs well.

Today performance testing happens just beforeproduct going live with very short timeframe

allocated for it which leads to missing performance

problems.With small extra efforts, efficiency ofperformance testing can be increased significantly

and these extra efforts usually pay off multi-fold,which can be achieved usingAgile Performance

Testing.

This paper addresses performance testing using

agile methodology, its approach, advantage, risk

mitigation and many others.

8/6/2019 Agile PT

2/15

8

AgilePerformanceTest i

ng

Table of Contents

1.0 Todays Market need ............................................................................................ 3

2.0 Current Performance Testing .............................................................................. 42.1 Objective ...............................................................................................................................4

2.2 Approach ..............................................................................................................................4

2.3 Risks ......................................................................................................................................4

2.4 Drawbacks ............................................................................................................................5

3.0 Agile Performance Testing .................................................................................. 6

3.1 Objective ...............................................................................................................................6

3.2 Approach ..............................................................................................................................6

3.3 Benefits .................................................................................................................................9

4.0 Effort Metrics for a Release ............................................................................... 10

4.1 With No Change in Requirement ............................................................................... .......10

4.2 With Change in Requirement .......................................................................................... ..10

Note: Value may vary based on the data used for calculation of effort saving. 12

5.0 Risk Mitigation .................................................................................................... 12

6.0 Advantage ...........................................................................................................13

7.0 Pre-requisite, Challenges and Risk .................................................................. 13

8.0 Conclusion .......................................................................................................... 149.0 About Author .......................................................................................................14

8/6/2019 Agile PT

3/15

8


ng

AGILE PERFORMANCE TESTING

1.0 Todays Market needIncrease in competition and leaps in technology have

forced companies to adopt innovative approaches to

have an application which performs well. Having

more features and functionality may not necessarily

translate into success for a system - what ultimately

count is how stable the consumer finds these

features.

Thus performance testing is the testing conducted to evaluate the performance of

a system with specified performance requirements.

Today performance testing happens just before product going live with very short

timeframe allocated for it. This may lead to missing performance problems and

propagate to production and have unhappy customer.

To have customer satisfaction in regards to performance, one can have Agile

Performance Testing which states: -

With small extra efforts, making the process more agile, efficiency of performance

testing increases significantly and these extra efforts usually pay off multi-fold

even before the end of performance testing.

8/6/2019 Agile PT

4/15

8


ng

2.0 Current Performance Testing

2.1 Objective

To validate/determine the performance behaviour of the system with respect to

the end-user experience.

2.2 Approach

2.3 Risks

When major issues are found at the end of test life cycle, it becomes much more

expensive to resolve them

Major performance issues identified during later stages may hold up important

releases and delay the project delivery

8/6/2019 Agile PT

5/15

8


ng

2.4 Drawbacks

Performance tuning is applied to server-configurable performance parameters

(Hardware/Software). Thus, scalability issues are addressed by expensive

hardware/software resizing techniques

If code optimization is done, then one needs to go through the complete Testing

Lifecycle.

8/6/2019 Agile PT

6/15

8


ng

3.0 Agile Performance Testing

3.1 Objective

To determine the performance bugs/bottlenecks in key business processes as

early as possible during the project development cycle

Identify, isolate and fix performance bottlenecks at the component/code level

3.2 Approach

Agile Performance Testing is a three phases approach: -

Unit Level Tuning For optimizing

o Execute tests to isolate and fix bottlenecks at code level

Component Level Tuning For testing Components

o Execute tests to isolate and fix bottlenecks in application components

level.

Application Load Tuning For testing Application Flows

o Test the critical application flows for user-experience under normal and

peak loads

8/6/2019 Agile PT

7/15

8


ng

Lets have detail look at each of the phase: -

Unit Level Tuning

Component Level Tuning

8/6/2019 Agile PT

8/15

8


ng

Application Load Tuning

8/6/2019 Agile PT

9/15

8


ng

3.3 Benefits

Iterative performance testing leads to better code, optimized for performance

Early detection of performance bugs/bottlenecks reduces the effort & duration for

performance tuning and retesting of application

8/6/2019 Agile PT

10/15

8


ng

4.0 Effort Metrics for a Release

What will be the efforts required to incorporate Agile Performance Testing?

To evaluate this, effort required is calculated based on following criteria during a

release: -

With No Change in requirement

With Change in requirement

4.1 With No Change in Requirement

Testing Lifecycle Current Approach Agile PT Approach

Unit Testing 10 10

Unit Level Performance Tuning 0 2

Integration/Functional Testing 3 3

Component Level Tuning 0 2UAT Testing 4 4

Application Load Testing 2 (Script Re-Used)1

Tuning Performance bottleneck at

Code Level.

1 0

Tuning Performance bottleneck at

Component Level.

0 1

Unit Testing after Tuning 2 Not Required,

because Code Level

Tuning has been

already done in

step2

Integration/Functional Testing after

Tuning

1

UAT Testing after Tuning 2

Application Load Testing after Tuning (Script needs to re-

create due to code

tuning)2

(Script Re-used)1

Total Efforts Required (Man Day) 28 24

Saving in Efforts = 28 24 = 4 Man Days = 14% (approx.) for one module

4.2 With Change in Requirement

When change in requirement is invoked, efforts required for testing lifecycle is less

as compared to the efforts for first time testing of an application. And also Unit level

tuning for CR depends on how changes made impact the working of application.

Example: -

Considering 2 CRs are made, there are following 3 scenarios for effort saving: -

8/6/2019 Agile PT

11/15

8


ng

Scenario 1:

Performance Testing (Unit Level Tuning) is performed in DEV Environment

considering major changes impacting the application for both the CRs.

Effort Saving

0

5

10

15

20

25

30

35

First Run Run after CR1 Run after CR2

Testing Phase

Savingin%

Scenario 2:

Performance Testing (Unit Level Tuning) is not performed in DEV Environment

considering minor changes not impacting the application for both the CRs.

Effort Saving

0

5

10

15

20

25

30

35

40

45

50


Testing Phase

Savingin%

Scenario 3:

CR1 - Performance Testing is performed in DEV Environment considering

major changes impacting the application.

CR2 - Performance Testing is not performed in DEV Environment considering

minor changes not impacting the application.

8/6/2019 Agile PT

12/15

8


ng

Effort Saving

0

5

10

15

20

25

30

35

40

45

50


Testing Phase

Savingin

%

Note: Value may vary based on the data used for calculation of effort saving.

5.0 Risk Mitigation

Agile Performance Testing has positive outcome in risk mitigation with respect to: -

Customer Satisfaction

Customers/users have higher expectation about the quality of application.

Code Tuning

8/6/2019 Agile PT

13/15

8


ng

Unit Level tuning is done early so no need for code optimization at later stage

on Testing Lifecycle.

Application Rejection

Reduces risk of the application being rejected because of issues such as Memory

Leaks, Hard Coded Values in code, Database Locking

Release Dates

Iterative Testing leads to constant awareness of the application performance and

therefore more confidence in meeting release dates.

Performance Bugs

Less Number of Performance bug(s) in subsequent phase(s).

Performance Testing Efforts

Performance Script can be re-used thus saving 60% of the efforts of Performance

Cycle.

6.0 Advantage

One Good decision at Start can cover a lot

Earlier detection of performance bottlenecks

Reduces the effort and duration for performance tuning and retesting

Project management has constant awareness of the application

performance and therefore more confidence in meeting release dates.

Faster resolution of identified issues

Bottom-up approach to identify bottlenecks helps narrowing down the

root-cause faster

Better performance tuning

Development team gets more time for performance tuning activities

Cost Effective

Identified bottlenecks are resolved much earlier in Testing Lifecycle

saving expenses required to a great extend.

7.0 Pre-requisite, Challenges and Risk

Pre-requisites

Full time Performance Tester.

Must be technically sound.

8/6/2019 Agile PT

14/15

8


ng

Closer interactions between developers & testers

Access to monitor all the servers.

Challenges

Defining accurate performance requirements.

Risks

Incorrect performance requirements may jeopardise the testing

8.0 Conclusion

Applications are backbone of business; hence performance of application becomes

one of the key differentiators in success or failure of business. Thusperformance

testing is conducted to evaluate the performance of a system with specified

performance requirements.

In current approach, performance testing happens just before application going live

with very short timeframe allocated for it. When major issues are found at the end of

test life cycle, it becomes much more expensive to resolve them which may hold up

important releases.

This calls for the solution which states - Investigate performance early and Validate

performance last

Agile Performance Testing does just that. With Agile Performance Testing, Project

management is constantly aware of the application performance and therefore more

confidence in meeting release dates.

Agile Performance Testing has positive outcome in risk mitigation because systems

are getting more and more complex with a dynamic technology landscape and high

standards of customer satisfaction

One Good decision at Start can cover a lot

9.0 About Author

A Cognizant India Test Engineer with over 3.5 years of proven Quality and Test

management experience in the Financial Services sector. Consistently developed

8/6/2019 Agile PT

15/15

8


ng

and implemented new ideas and techniques which have lead to dramatic Quality

improvement within the projects. Published following whitepapers:

1. Customer Satisfaction through Quality Index

http://www/projectperfect.com.au/info_quality_index.php

2. Sanity Testing

http://www.stickyminds.com

3. Software Testing Metrics

http://www.stickyminds.com/s.asp?F=S14789_ART_37

To reach email at: - [email protected]
http://www/projectperfect.com.au/info_quality_index.phphttp://www.stickyminds.com/http://www.stickyminds.com/s.asp?F=S14789_ART_37mailto:[email protected]://www/projectperfect.com.au/info_quality_index.phphttp://www.stickyminds.com/http://www.stickyminds.com/s.asp?F=S14789_ART_37mailto:[email protected]

agile pt

Documents