loadtesting wuc2009v2
TRANSCRIPT
![Page 1: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/1.jpg)
Loadtesting Web(GUI) sites:why, what and how
Mark Rotteveel (Pluton IT)
![Page 2: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/2.jpg)
Who am I?• Mark Rotteveel• Born in 1979• Hobbies: reading,
working with Firebird and more
• In IT since 1999• Working as a tester
since 2006• Joined Pluton in
2008• Loadtesting CRM
systems at T-Systems / T-Mobile NL
![Page 3: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/3.jpg)
Overview• Introduction testing• Why : reasons for testing
– Verifying performance requirements– Resource planning
• What : targets for testing– Designing tests– Metrics to use
• How : methods of testing– Tools– Recording / writing tests– Execution– Results
![Page 4: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/4.jpg)
Introduction testing
• Finding errors– Verifying software matches requirements– Validating software meets user needs– Other errors
• Mitigating (business) risks• Minimizing cost
![Page 5: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/5.jpg)
Loadtesting
• Verifying performance requirements• Checking behavior under load• Resource planning• Representative load (eg normal, peak, max
transaction rate with 'normal' resource use)
![Page 6: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/6.jpg)
Stress testing
• Finding limits of software (or hardware)• Behavior beyond expected load or with limited
resources– Bugs under high load / constrained resources– Error handling / graceful degradation or failure
• Exaggerated load (eg 2x peak, until application breaks)
![Page 7: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/7.jpg)
Why : Reasons for testing
![Page 8: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/8.jpg)
Verifying requirements (1)
• Requirements state objectives for the application (and the test)
– Response time– Load to handle– Etc
• Test to verify application meets those requirements!
![Page 9: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/9.jpg)
Verifying requirements (2)
• Response time• Transactions per
second• Concurrent users
• List of non-functional requirements
• Usage model– Operational profile– Load profile
Target metrics (example) Source
![Page 10: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/10.jpg)
Requirements : Example
• Scenario : login– Process 5 logins per minute– 500 ms response time target
![Page 11: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/11.jpg)
Resource planning (1)
• Resource planning: knowing when to upgrade your hardware or add new servers
• Test to check or enhance your planning for upgrades,
• or to make a contingency plan for unexpected increase of traffic
![Page 12: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/12.jpg)
Resource planning (2)
• Current usage: required resources
• Expected growth• Determine triggers /
warnings on resource usage
• Plan ahead• Contingency plan for
unexpected growth• Resource impact of
new version
What Why
![Page 13: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/13.jpg)
What : Targets of testing
![Page 14: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/14.jpg)
Designing tests (1)
• Select requirements to test
• Select (or write) use cases
• Determine user count or load target
• Amount of 'thinktime'
![Page 15: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/15.jpg)
Designing tests (2)
• Order tests by business risk or importance
• Most important tests first!
• Test individual and combined
• Aggregate test cases
• Realistic load for combined scenario
![Page 16: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/16.jpg)
Test : Example
• Login• Target : 5 logins per
minute• Determine response
time (should be < 500 msecs)
• Access main page• Enter login
information• Click login
![Page 17: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/17.jpg)
Metrics (1)
Metrics are measurements or performance indicators used to monitor the test, and determine
success or failure of the test.
![Page 18: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/18.jpg)
Metrics (2)
• Select metrics for the requirements• Too much is better than too little• But, more measurements influence test (eg
additional load)
![Page 19: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/19.jpg)
Metrics (3): examples
• Resource planning: memory, CPU, bandwidth• User experience: response time, latency,
throughput• Correctness: % failed tests
![Page 20: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/20.jpg)
How : Methods for testing
![Page 21: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/21.jpg)
Testing needs
• Tools : open source or proprietary• Recording / writing actual test• Preparation• Execution• Reporting on results
![Page 22: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/22.jpg)
Tools
• Open source tools: Jmeter, The Grinder (and more)
• Proprietary tools: HP LoadRunner, SilkPerformer (and more)
• Select the tool that matches your budget and needs
![Page 23: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/23.jpg)
Open source vs proprietary
• Cheap• Extendable (not
always easy)
• Support, documentation
• More functionality
Open source Proprietary
![Page 24: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/24.jpg)
Recording / writing tests• Translate test design to executable test• Record and playback vs 'handwritten'• Parametrize user data
– Scenario 'consumables' (eg usernames, order information)
– Dynamic session data (eg cookies)• Add assertions for expected results• Verify playback (both single and multiple user)
![Page 25: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/25.jpg)
Using Jmeter
• Simple example: recording login into WebGUI
![Page 26: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/26.jpg)
Preparation for execution
• Configure scenario (number of users, 'thinktime', ramp up, duration)
• Combine scenarios for 'realistic' load• Setup monitoring of target application(s) /
server(s)• Collect data for scenario 'consumables'
![Page 27: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/27.jpg)
Execution
• Start the test• Keep an eye on metrics• Manually check the application during test• Verify that loadtest server(s) is not overloaded
![Page 28: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/28.jpg)
Results (1)
• Correlate metrics : dependencies and bottlenecks
• Draw conclusions
![Page 29: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/29.jpg)
Results(2)
• Verify requirements ('did we reach our targets')
![Page 30: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/30.jpg)
Results (3)
• Communicate results and findings to development, management
• Record lessons learned for future reference• Preserve testware for future use
![Page 31: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/31.jpg)
Pitfalls (1)• Using one loadserver: bottleneck is the
testserver• Flooding the target machine• Not parametrizing data (caching effects, errors
due to duplicate or incorrect data)
![Page 32: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/32.jpg)
Pitfalls (2)
• Loadtest software is not a browser (usually no or limited javascript support!)
• Not checking for expected data (or absence of errors) in response of server
• Test environment not enough production-like (impossible to predict behavior in production)
![Page 33: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/33.jpg)
WebGUI specifics• Manage session cookies (eg use HTTP
Cookie Manager in JMeter)• Parameter separator of WebGUI (';') not
supported by all software; only standard '&' (minor problems recording, WebGUI does support '&' in playback)
• JMeter bug: recording proxy sometimes incorrectly encodes GET requests (stop and start proxy before recording next step)
![Page 34: Loadtesting wuc2009v2](https://reader035.vdocuments.site/reader035/viewer/2022062514/559a999d1a28ab683d8b4619/html5/thumbnails/34.jpg)
Questions?