agile load testing in the real world
DESCRIPTION
Recent and constant changes in technology have resulted in a transition from the traditional Center of Excellence model, to an agile development model, which has made its way into the mainstream and caused a shift in how testing, particularly performance testing, is done. Join our Webinar to learn how a top tier manufacturer transformed their ability to deliver high quality projects, faster, by adapting their load testing to fit an agile development approach. In this webinar you’ll learn: Why you need to think differently about performance testing in an agile development environment How to shift Performance Testing to the left in the development cycle What adjustments to consider in your performance testing centers of excellence model Technology platform considerations for faster, more collaborative load testing Utopia Solutions Founder and CTO, Lee Barnes, will dive into tips and techniques for bringing effective performance testing to your agile organization.TRANSCRIPT
© Utopia Solutions
Agile Load Testing in the Real World
Lee Barnes, CTO Utopia Solutions @USI_LeeBarnes
© Utopia Solutions 2
Key Questions
What needs to change?
How do we make those changes?
Why don’t traditional performance testing models fit nicely with agile?
© Utopia Solutions 3
Case Study
Background
• Large global manufacturer • Performance Testing CoE in operation for ~10 yrs • Seen as a huge value to the enterprise • Began moving toward agile a few years ago…
and the luster of the CoE began to dull…
© Utopia Solutions 4
Traditional Performance Testing Model
Center of Excellence
Project 1
Project 2 Project N
Provides Consistency in… • Resources / Skills • Processes • Deliverable Quality • Technology
Request Test
Project Team
Center of Excellence
Perform Discovery
Develop & Execute
Deliver Results Evaluate Results
Info / Environment
© Utopia Solutions 5
• One size fits all • Strict entry criteria • Performed late in SDLC • One-way communication of
results
Service Characteristics
CoE in the Waterfall Years
Plan Design Build Test Deploy
Performance Testing
Too late!
© Utopia Solutions 6
Agile Development
Analyze
Develop
Integrate
Acceptance Test
Deploy
Where does performance testing fit?
© Utopia Solutions 7
Early Attempts on Agile Projects
Sprint 1 Sprint 2 Sprint 3 … Deploy
Agile
Performance Testing
Same Process…
Same Risk!
© Utopia Solutions 8
Early Attempts on Agile Projects
Sprint 1 Sprint 2 Sprint 3 … Deploy
Agile
Performance Testing
Performance Testing
Performance Testing
Performance Testing
CoE Approach • Rigid • 4 week test cycle
Agile Approach • Flexible • 2 week sprints
+ =
© Utopia Solutions
Back to the drawing board…
© Utopia Solutions 10
Early Cycle / Ongoing
Performance Analysis
Establish Objectives
Ultimate Goal…
Late Cycle Performance
Testing Shift Left
© Utopia Solutions 11
Identify Gaps
What’s stopping us from analyzing performance continuously?
• Performance objectives weren’t part of user stories
• Agile teams lacked performance engineering expertise
• Performance testing knowledge and tools were locked in the CoE
• APM tools / skills were non-existent
© Utopia Solutions 12
Disclaimers
It was not easy.
The changes discussed on the next few slides were implemented over a two year duration.
It was not painless.
© Utopia Solutions 13
Performance Testing in an Agile Environment
Build performance objectives in to user stories
Make performance everyone’s job
Create a role to coordinate performance activities
CoE performance test architects become mentors
Begin developing a performance focused culture
© Utopia Solutions 14
Performance Testing in an Agile Environment
Map business and end user requirements onto system components
Create both single and multi user tests
Benchmark in development / QA environment
Look for trends in measurements vs. absolutes
Use service virtualization to stub out 3rd party or yet to be developed components
Execute unit performance tests as part of continuous integration
Implement Unit / Component Performance Testing
© Utopia Solutions 15
Waterfall Agile
Responsibility for Performance
Performance CoE
Role of CoE End of cycle PT
Role of development
Throw system over wall Wait for results
Tools Traditional load test tools
Environment Dedicated / production-like Deliverable Boiler plate results delivered
just prior to release
Waterfall Agile
Responsibility for Performance
Performance CoE Everyone
Role of CoE End of cycle PT Provide mentorship to agile team Pre-deployment system PT
Role of development
Throw system over wall Wait for results
Continuous focus on performance
Tools Traditional load test tools Traditional load test tools APM tools Service virtualization Continuous integration
Environment Dedicated / production-like Development / desktop Deliverable Boiler plate results delivered
just prior to release Continuous feed of actionable data
Performance Testing in an Agile Environment
Transformation Summary
© Utopia Solutions 16
Summary
Key takeaways… • Agile performance testing is not more
/ quicker performance tests • Success requires an shift in people,
process and technology
© Utopia Solutions 17
Questions…
…. and Answers!
Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: [email protected] Twitter: twitter.com/USI_LeeBarnes LinkedIn: linkedin.com/in/leebarnes Blog: www.RaiseYourMQ.com
?