test design made easy (and fun) rik marselis eurostar
TRANSCRIPT
Test Design Made Easy (and Fun!)
Rik Marselis 5 November 2015
Let’s walk the testing talk
Question 1
Applying Test design techniques is
easy
Question 2
Applying Test design techniques is
fun
Question 3
How many Test Design Techniques exist?
Test design techniques & coverage types TMap NEXT • Equivalence classes • Boundary value analysis • Decision points (CC, DC, CDC, MCDC,
MCC) • Orthogonal arrays • Pairwise testing • Data cycle test (CRUD) • Statistical usage (op.prof. • Right paths / Fault paths • Checklist • Decision table • Data Combination Test • Elementary comparison tst • Process Cycle test • Real-life test • Semantic test • Syntactic test • Use case test • Error Guessing • Exploratory testing • Reviews • Walkthroughs • Inspections
ISTQB • Equivalence partitioning • Boundary value analysis • Decision table testing • Cause Effect graphing • State transition testing • Use case testing • User story testing • Structure based (SC, CC, DC, CDC,
MCDC, MCC) • Error Guessing • Exploratory testing • Orthogonal arrays • Pairwise testing • Domain analysis • Defect based techniques • Checklist • Combinatorial / Class. tree • Informal review • Technical review • Walkthrough • Inspection • Static analysis
Torbjorn Ryber • Exploratory testing
(this is an approach, not a technique) • Inspection • Walkthrough • Technical review • Informal review • Modelling • Equivalence partitions • Boundary values • Domain tests • Business process testing • Use case testing • State based testing • Decision tables & trees • Elementary comparison • Combinatorial testing • Data cycle testing • Syntax testing • Time cycle testing • Program Logic (SC, BC, PC) • Data flow testing
Boris Beizer • Control-flow testing
• Loop testing
• Data-flow testing
• Transaction-flow testing
• Domain testing
• Syntax testing
• Finite-state testing
And more…
Question 4
How many Test Design Techniques do you apply in your day-to-day work??
Question 5:
Which test design techniques do you use in your day-to-day work?
TMap HD test approaches
Why coverage?
What’s your testing challenge? Select from 4 groups of Coverage types
Better know & use one technique of each group
than
all techniques of only one group
Why experience based?
So it needs a skilled tester !
Use results of previous tests to create new tests
Always use a mix
What’s your testing challenge? Select an experience based approach
Largely intuitive & ad hoc
Structured, chartered
Experience brought together
Checklist: Test Heuristics Cheat Sheet • Abstract (remove details) • Never and Always • Interrupt • Reverse • Some, none, all • Starve • Too few, too many
Source: “Explore It!” by Elisabeth Hendrickson, appendix 2
• Useful approximations • Violate data format rules • Zero, one, many • Zoom • Back, forward, history • Bookmark it
More checklists from Explore it!
Numbers:
• Zero
• One
• Many
• Too many
• Too little
Selecting:
• None
• One
• Some
• Many
• All
Timing:
• Always
• Sometimes
• Never
CRUD:
• Create
• Read
• Update
• Delete
Source: “Explore It!” by Elisabeth Hendrickson
What is the goal of testing?
• To report about quality and risks
• To build confidence
• So you need to log and report
Example of a test log
How to mix test approaches
• Create an Exploratory Testing charter (this can be done long before the test object is available)
• Use coverage based techniques as “test ideas” (thus your start-set of test cases can be made long before …)
• Execute your test based on prepared test cases complemented with exploratory testing
Charter
Explore <target>
With <resources>
To discover <information>
simple
Extensive:
Source: “Explore It!” by Elisabeth Hendrickson, page 13
Use a tool to get a start
Source: the test design tool “COVER” of Sogeti
Calculate Public Transport tariff
• Based on age, profession and discount card the tariff for public transport is calculated.
SPRINT 1: Calculate Transport tariff
• What kind of testing challenge is this?
Assignment: Test the public transport tariff decision
• Think of some test idea’s
• Use the DCT test cases as a starting point
• Execute the test
• Use the logsheet Groups of 2-3
20 minutes
Retrospective
• What is your conclusion about quality & risks?
• How much confidence do you have?
• Have you found defects?
Coffee break
What is the goal?
• With preparing test ideas and some test cases; You have a good start for exploring
– Most people get better coverage than with purely experience based test cases
– Proof of coverage at the spots where you need it
– Faster progress of testing compared to only exploring
– It fuels your imagination for the real exploration
Workshop: The Triangle exercise
In his 1979 book “The art of software testing” Glenford J. Myers starts with a simple test to see if you are suited as a tester.
The challenge is to test an application that gets 3 integers as input which represent the lengths of the sides of a triangle. Then the program tells what type of triangle it is.
So we start thinking…
• What is/are our challenge(s)?
• What possible inputs?
• What possible outputs?
• What’s the process?
• How can we test this?
• How can we be efficient?
Process for triangle application
Input check
start
A = B ?
A = C ?
A = C ? B = C ?
A + B < C OR A + C < B OR B + C < A ?
Input numeric
?
Not a triangle
yes
yes
yes yes yes
yes
no
no
no no no
no
Inspired by Glenford J. Myers
Isosceles Equilateral Scalene Not a triangle
Exercise 2a: Create a charter
• Include at least 3 test design techniques
Explore <target> With <resources> To discover <information>
Example of an extensive charter
Source: “Explore It!” by Elisabeth Hendrickson, page 13
Example of a simple charter
Groups of 2-3 5 minutes
Use MCDC for the complex decision
FALSE (0) (scalene triangle) X Y Z
TRUE (1) (not a triangle) X Y Z
(X) A + B < C
(Y) A + C < B
1 . . . 1 . . . 1
0 . . . 0 . . . 0 (Z) B + C < A
0 0
0 0
0 0
0 0
0 0
0 0
D2.1
D2.2
D2.3
D2.4
Result = Condition X OR Condition Y OR Condition Z
Neutral value (AND = 1, OR = 0 ) 0 0
MCDC = Modified Condition Decision Coverage
Exercise 2b: Execute your charter
Test the triangle application using:
• Your charter
• Your test ideas and tools
• The logsheet
Groups of 2-3
20 minutes Note: application and handouts will be provided
Retrospective
• What is your conclusion about quality & risks?
• How much confidence do you have?
• Have you found defects?
• How many of the tests that Glenford Myers has listed, did you do?
Tooling: Of course ! But beware…
Source: the test design tool “COVER” of Sogeti
Conclusion
• Exploratory testing can be very well combined with test design techniques
• Exploratory tests can be prepared before test execution starts
• Combining scripted and exploratory tests can be both easy and fun!!
Presentation will be available at EuroSTAR-site and on Slideshare.
The end. Thank you !!
Happy testing
This whole presentation copyright © Sogeti 2015
