![Page 1: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/1.jpg)
@ Smals_ICT#DV14 # AgileContextTesting @Smals_ICT#DV14 #AgileContextTesting
Beyond the urban legend
–
Testing in an agile contextJochim Van Dorpe
![Page 2: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/2.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
In-house ICT shared services for e-government
• Focus on social security and e-health services
• For federal, regional, local & European institutions
• Based in Brussels
• Software development, ICT-operations & staffing
• >1700 people
Introduction: What is Smals?
![Page 3: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/3.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Introduction: Who Am I?
(QA) ( (lead) (technical) test(er)
analyst/coördinator/manager) (automator/engineer)
I am a tester
![Page 4: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/4.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Introduction: What do I have to tell?
![Page 5: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/5.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
What is testing?
It's not about proving that the software works, it's about finding bugs
![Page 6: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/6.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Why is testing necessary?
To prevent:
• Death
• (Serious) injury
• Loss of business
• Loss of reputation
![Page 7: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/7.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Why do we test?
![Page 8: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/8.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
The test project in waterfall
![Page 9: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/9.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Agility
Agility is the continuous
delivery of prosperity (value) to
stakeholders of a system in a
sustainable and balanced
manner -Schalk Cronjé (@ysb33r)-
![Page 10: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/10.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Agile fundamentalists are like …
Did you even read
our manifesto?
![Page 11: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/11.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Agile manifestoOf course …
• Individuals & interactions over
processes and tools
• Working software over
comprehensive documentation
• Customer collaboration over
contract negotiation
• Responding to change over
following a plan
![Page 12: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/12.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Urban legends
• Agile doesn’t need testers
• Agile = de facto good code, built-in quality
• Testing is dead
• Agile = TDD, ATDD, BDD
• Testers should be able to code
• Agile hasn’t time for testing
• 100% automation
• Faster, shorter, better!
• Developers and Testers are like oil and water
• You only need to unit test
• User Acceptance Testing is no longer necessary
![Page 13: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/13.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
So…
• The Agile manifesto shouldn’t stand between you &
good quality
• No documentation, few documentation or poor
documentation is no excuse for “not testing”
• Don’t put an Agile stamp on your team, just to avoid
the things you dislike, but find a way of doing them
so you like them …
![Page 14: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/14.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
No difference at all?
![Page 15: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/15.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Many ways to be agile …
![Page 16: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/16.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
… but I’m no salesman
![Page 17: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/17.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Principles of testing in an agile
context
-Elisabeth Hendrickson (@testobsessed)-
![Page 18: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/18.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
1. Testing moves the project forward
Shifting focus:
No longer the last line of defense
![Page 19: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/19.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
2. Testing is not a phase
Integrated & continuous:
develop & test together
![Page 20: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/20.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
3. QA = responsability of the team
![Page 21: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/21.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
4. Reduce feedback latency
No long gaps between development
& testing feedback
![Page 22: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/22.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
5. Don’t let bugs crawl around
Deal with bugs as soon as possible
![Page 23: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/23.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
6. Test should represent requirements
Don’t ASSUME, it makes an ASS of
U & ME
SCHOOLBUS
![Page 24: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/24.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
7. Reduce overhead
Think high level: don’t waste time on
too much detail
![Page 25: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/25.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
8. Don’t believe in fairy tales
If something
seems to good to be true, it probably is
![Page 26: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/26.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
9. Done means done
Done means implemented and tested
![Page 27: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/27.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Concrete practices
![Page 28: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/28.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate More
• The testing triangle: This will drown your testers:
Manual tests
Acceptance tests
System &
integration tests
Comp.
tests
![Page 29: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/29.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate More
• The testing triangle: Make sure your testers can
cope with the pace
Component tests
System &
Integration tests
Acc.
Tests
Manual
testsManual tests
Acceptance tests
System &
integration tests
Comp.
tests
![Page 30: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/30.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
A. Automate Testing, not only tests
Test
execution
ch
ec
kin
g
Load
Data-
sets
Reset
DB
Continuous integration
Logging
resultsGenerate
reports
Live
dashboards
Calculating
‘metrics’
Use the tools that fit your purpose
AND NOT THE OTHER WAY AROUND!
![Page 31: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/31.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
B. Explore
Leave some room for
exploratory testing
![Page 32: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/32.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
C. Diversify
• …in the test levels
• Unit-tests only won’t find
everything
• System test only, and you’ll
drown
• … in the way of testing
• Automation is good, automation
is fun, but in the end, an
automated test isn’t better than a
human based test (manual test)
![Page 33: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/33.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
D. Test the tests
• GREEN == GOOD ?
• DO: Review tests
• DONT: No test coverage fairy tales
• DONT: Mock lasagna
• DONT: I should adapt the test cases so it’s green again
![Page 34: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/34.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
E. Crowdsource …
• Crowdsource in your team to find
the good & passionate tester
• Accept that there are learning curves
• Testing is a craftmanship
• A passionate tester will tend to be more
effective than the chinese-volunteer-
tester
• Encourage collaborative testing
![Page 35: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/35.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
… or ask a testjumper
• Ask a ‘test jumper’ to join the
team
• May spend some hours as a consultant
• Or months as an ordinary tester
• Coaches or contributes himself
• Encourages testability
• Help the developers think
productively about testing
![Page 36: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/36.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
The test project in agile
![Page 37: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/37.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
What should the PL not expect?
• Metrics such as:
• # of bugs found
• Bugs per functionality
• Bugs per 1000 lines of code
• DDP (defect detection percentage)
• We can however show our QA-efforts
• QA-survey
• Test case progress
• Fixed test cases
![Page 38: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/38.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
What should the PL not expect?
• Qualification (good / bad) of the devs
• 100% green charts
• Upfront fixed number of test cases
• Automated self-generating auto-code-correcting
tests
![Page 39: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/39.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
conclusion
There are no agile testers,
but we need testers who
adapted their mindset to an
agile context
![Page 40: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/40.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Read & experience more 1/2• Books:
• Lisa Crispin, Janet Gregory (2009). Agile Testing: A Practical Guide for Testers and
Agile Teams
• Lisa Crispin, Janet Gregory (2014), More Agile Testing: Learning Journeys for the
Whole Team
• Blogs:
• Huibschoots.nl
• LisaCrispin.com
• http://testobsessed.com : Agile testing overview
• Http://www.satisfice.com : Test jumpers: one vision of Agile testing
• pascaldufour.wordpress.com
• agile-and-testing.chriss-baumann.de
![Page 41: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/41.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Read & experience more 2/2• Presentations:
• 40 agile methods in 40 minutes by Craig Smith
• Agile testing quadrants explained by @RubyTester
• Certification:
• CAT-training: Certified Agile Tester
• ISTQB foundation level add-on: Agile tester
• Conferences:
• Agile testing day(s)
• More references:
• Word clouds generated on wordle.net
• Images found with google image search
![Page 42: 20141113 devoxx2014 jochim van dorpe testing in agile](https://reader034.vdocuments.site/reader034/viewer/2022042514/559868f61a28ab9f738b46ef/html5/thumbnails/42.jpg)
@ Smals_ICT#DV14 # AgileContextTesting
Questions