agile testing - university of helsinki · agile transformation is demanding → currently testing...
TRANSCRIPT
08.10.08 1
Agile TestingAgile Testing
Tiina KiuruReaktor Innovations
08.10.08 2
Agenda● What is agile?
– Agile values, principles and practices● How to organize testing?● How to implement testing?
08.10.08 3
What is agile?
08.10.08 4
Why Agile?● Top 5 reasons:
– Software based on real and most valuable requirements - business needs and value drive development
– Shorter time to market– You always know where your projects are - more
transparency and controllability– Focus on quality– Lower project risk
➔ Better return on investment
08.10.08 5
Agile Manifesto
AgileValuesAgile
Values
AgilePrincipl
es
AgilePrincipl
es
Copyright Reaktor Innovations 2007 6
These are important
Processes and tools
Comprehensive documentationContract negotiation
Following a plan
These are even more importantIndividuals and interactions
Working software
Customer collaboration
Responding to change
Agile ValuesAgile ManifestoAgile
Values
AgilePrinciple
s
08.10.08 7
Agile Principles● Our highest priority is to satisfy the customer through early and continuous
delivery of valuable software.
● Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
● Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
● Business people and developers must work together daily throughout the project.
● Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
● The most efficient and effective method of conveying information to and within a development team is face-to-face conversation
Agile ManifestoAgileValues
AgilePrinciple
s
08.10.08 8
Agile Principles● Working software is the primary measure of progress.
● Agile processes promote sustainable development.The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
● Continuous attention to technical excellence and good design enhances agility.
● Simplicity--the art of maximizing the amount of work not done--is essential.
● The best architectures, requirements, and designs emerge from self-organizing teams.
● At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly
Agile ManifestoAgileValues
AgilePrinciple
s
08.10.08 9
Values
Principles
Practices
Defines a set ofways to meet
the values
Defines in detail howthis is implemented
in practice
Defines the setof most importantValues of what istruly important
Copyright Reaktor Innovations 2007 10
Product Backlog Sprint
Backlog
Potentially shippable product increment
Sprint1 - 4 weeks
24 hours
Daily Scrum meeting
S printP lanningM eeting
Demo, S printR eview &R etros pec tiveM eeting
Scrum
08.10.08 11
Scrum in 5 Bullets
● Iterative, incremental, cyclic process● Time-boxed development● Evidence-based planning● Inspect and adapt - feedback loops● Cross-functional, self-managing teams
08.10.08 12
How to Organize Testing?
08.10.08 13
Basics of Agile Testing● Individuals and interactions over processes and tools
➔ Tester integrated part of the team● Working software over comprehensive documentation
➔ Testing right after feature has been implemented● Customer collaboration over contract negotiation
➔ Customer participates in testing, continuous feedback● Responding to change over following a plan
➔ Exploratory testing and test automation
08.10.08 14
Test Process● Principles:
– Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale
➔ Purpose is to help in building quality into system➔ No separate testing phase, continuous testing ➔ Test early and often: when a feature is implemented, it is
tested
08.10.08 15
Test Process● Principles:
– Working software is the primary measure of progress.– Business people and developers must work together daily
throughout the project➔ Feature is not ready before it has been tested➔ Direct feedback as soon as possible➔ Bugs are fixed as they are found
“If you don’t have meaningful feedback then you’re not agile. You’re just in a new form of chaos.”
-Brian Marick
08.10.08 16
Between Agile and Iterative● Agile transformation is demanding → currently testing is
often something between iterative and agile, eg:● Testing done in the end of sprint or after it
– This is not agile!– Customer most probably does not get working
software– Bugs remain → tasks are not actually finished by the
end of sprint – Creates endless quality debt → stabilization sprints
08.10.08 17
Between Agile and Iterative● Few test releases in each sprint
– A lot better than testing in the end of sprint– Slow feedback loop → tasks get ready late, pressure
at the end of sprint– Creates quality debt, if not enough time to fix bugs
08.10.08 18
The Role of Tester ● Principles:
– Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
➔ Tester is like any other team member➔ Quality belongs to everyone, tester is just a facilitator➔ Quality contributor instead of gate keeper
”Testaajien yleisin virhe on kaapata laatukeskustelu kokonaan itselleen”
-Erkki Pöyhönen
08.10.08 19
The Role of Tester● Principles:
– Business people and developers must work together daily throughout the project.
➔ Testers work with business representatives to understand their needs
➔ Testers work with developers to understand the system and to provide feedback
➔ All essential stakeholders contribute to testing
08.10.08 20
Do You Need Specialized Testers?
● No
– In a crossfunctional team everyone does everything and the responsibility is shared, tester is just another team member
– Developers have the technical knowledge and customer has domain knowledge → they can do testing
● Yes
– Testing of complex systems is demanding and requires testing expertise
– Testers need a different mindset → it is hard to switch between development and testing
➔ Responsibility and knowledge is shared, but also specialized experts are needed
08.10.08 21
Test Reporting and Documentation
● Principles: – The most efficient and effective method of conveying
information to and within a development team is face-to-face conversation
➔ Direct ongoing feedback to developers➔ Communication with business representatives in order to
understand their needs
08.10.08 22
Test Reporting and Documentation
● Principles:– Simplicity--the art of maximizing the amount of work
not done--is essential. ➔ Forget the test documentation templates and IEEE 829!➔ Automate reporting as much as possible➔ Diaries, tables, mind maps instead of word documents
08.10.08 23
Do You Need a Bug Management Tool?● No
– Feedback is given directly– A failing automated test is a bug report– Bugs are fixed as they are found → no need to manage their state– An unmanageable amount of bugs is just an indication of bad practices
● Yes
– All bugs are not found only with automated tests– Often testing starts too late → lots of bugs found in one sprint– All bugs are not fixed → they have to be controlled in some way
➔ In an ideal agile project bug management tool is not needed, but in reality it often has to be taken into use.
08.10.08 24
Challenges and Common Mistakes
● Tasks marked as done before testing is completed (Definition of Done)
● Testing is not done from the beginning of the project
● Only features are tested, not the system as a whole
● Regression testing is neglected● Testers are not part of the team
08.10.08 25
Test Process and Scrum
Product Backlog Sprint
Backlog
Potentially shippable product increment
Sprint1 - 4 weeks
24 hours
Daily Scrum meeting
S printP lanningM eeting
Demo, S print R eview &R etros pec tiveM eeting
Testing activities defined in backlogs
Tester participates and contributes asa team members
Testing tasks distributed among teammembers, not only to tester
Only testedfeatures in demo
08.10.08 26
Scrum and Testing in 5 Bullets
● Iterative, incremental, cyclic process➔ Testing in the same pace with development
● Time-boxed development➔ Time-box includes QA, QA included in the
Definition of Done
08.10.08 27
Scrum and Testing in 5 Bullets● Evidence-based planning
➔ Concerns testing as well; tools and practices are evaluated by the team
● Inspect and adapt - feedback loops➔ Testing practices changed as needed
● Cross-functional, self-managing teams➔ Testers are team members, quality belongs to
everyone
08.10.08 28
How to do it in Practice?
08.10.08 29
Agile Testing Areas
By Brian Marick
08.10.08 30
Unit Testing● Cheapest way to build quality● Done by developers● Continuous integration● Unit tests ignore user's point of view → unit
tests do not substitute system level tests!● Frameworks: JUnit, TestNG, RSpec,
JDave
08.10.08 31
Test Driven Development● Tests written first● Tests serve as documentation
– Behavior Driven Development● Red-Green-Refactor
– Red: Failing test– Green: Write code so that test passes– Refactor: Code with better quality
08.10.08 32
Acceptance Testing● Should be automated and run frequently● Every team member should have access to
test automation tools and scripts● Automated regression tests
”Traditional test automation tools don’t work for an Agile context because they solve traditional problems, and those are different from the challenges facing
Agile teams.”.- Elisabeth Hendrickson
08.10.08 33
Acceptance Test Driven Development (ATDD)
● Tests for a feature are defined before development starts
● Tests are specifications or examples of the feature
● Feature can be verified right after it has been implemented
● Usually keyword driven
08.10.08 34
Keyword Driven Automation● Tests defined in natural language, eg. in
html tables● Customers and business people can
specify tests
● Testers or developers take care of the implementation
● Frameworks: Fit, Robot, Concordion
08.10.08 35
Example: Keyword Driven Automation
Test case:Test case:
Test implementation:
08.10.08 36
Acceptance Testing - Challenges● Maintenance costs of automation● Automated system tests are slow● It's not possible to automate everything
cost-efficiently
➔ Unit tests and acceptance tests support each other
➔ Continuous running
08.10.08 37
Usability and Exploratory Testing● Tests that cannot be automated● Question acceptance tests
– Are acceptance tests valid?● Finding new essential features
08.10.08 38
Exploratory Testing “Exploratory software testing is a style of software
testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the value of her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.”
08.10.08 39
Exploratory Testing● Purpose to find bugs, new features and to
learn● No detailed predefined tests● Based on testers' expertise● Can be done in pairs with customer,
developers, users etc.● Not equal to ad hoc!
08.10.08 40
Example: Exploratory Testing
Session handling & authentication
Credentials
OK Wrong password Wrong user
name
Expiration
Configuration
Re-opening
Accountlocking
Configuration
Basic case
08.10.08 41
Usability and Exploratory Testing - Challenges
● Keeping testing exploratory, not ad hoc● Finding appropriate level of documentation● Getting usability feedback as early as
possible → protos, simulations, early usability tests
08.10.08 42
Property Testing● ~ Non-functional testing, excluding
usability● Reliability, Portability, Supportability,
Security● Early testing and feedback● These issues have to be
considered during development
08.10.08 43
Property Testing - Challenges● Usually requires very specialized
knowledge● Has to be done also on complete system● Specialized tools, that are not available for
all team members➔ Open source tools➔ Early profiling➔ Awareness
08.10.08 44
Summary● Collaborate● Adapt● Explore● Automate