atdd half day_new_1_up
TRANSCRIPT
ATDDAcceptanceTest Driven Development
J a r e d R i c h a r d s o n A g i l e A r t i s a n s
@ J a r e d R i c h a r d s o n
1
Who are you?
2
Jared RichardsonCoAuthored GROWS Methodology
Author Ship It! and Career 2.0
Screencast editor PragProg.com
2nd public signatory of the Agile Manifesto
Started AgileRTP in 2007!
Coach
AgileArtisans.com
3
Who are you?
4
What would you say you do here?
Roles?
Responsibilities?
Why are you here?
What are your tests for this session?
5
ObjectivesDefine ATDD
Why requirements are not enough
Who to involve when
Turning requirements into tests
What to test
How ATDD can make your entire company more productive
6
Outline
Terms and definitions
Requirements and tests
Software development
ATDD examples
Anatomy of a test
Tables (Grid Driven Development)
7
Take Aways
What’s a test?
Who writes them?
When are they written?
What do they verify?
Provide an example
Primary benefit?
Secondary benefits?
8
Breaks
Halfway (90 minutes)
9
Background
Lean-Agile Acceptance Test Driven Development
by Ken Pugh
10
One Rule
There are exceptions to every statement
Except this one
11
Start
Familiar
Working
Smooth
Teaching
Dreyfus ModelofSkillsAcquisition
en.wikipedia.org/wiki/Dreyfus_model_of_skill_acquisition
12
Start
Familiar
Working
Smooth
Teaching
Dreyfus ModelofSkillsAcquisition
Steps
Recipes
Intuition
Stuck
Principles
13
Key Points
Skills don’t map
dilbert.com/strip/1990-09-13
14
Key Points
Beginners need steps
15
What is ATDD?
Acceptance Test Driven Development
External view of the system
Examines visible effects
Inputs
Outputs
State changes
External interfaces16
Definitions
Acceptance criteria -> General idea
Acceptance test -> specific. Pass/fail.
Implementation independent
Triad -> Customer, developer, tester
17
Fast Car
Who wants one?
Criteria
Closed course. Measure acceleration.
Test
Accelerate 0 to 60 in X seconds
flickr.com/photos/vodcars/4132650949
18
Fast Car
flickr.com/photos/fordeu/14296246942
0 to 60 in 30 seconds
0 to 60 in 5 seconds.
Top speed is 61.
0 to 60 in 5. Top speed is 120.
60 to 120 in 5 min
0 to 60 in 5. 60 to 120 in 20 sec.
Range of 1/4 mile.
19
0 to 60 in 30 seconds
0 to 60 in 5 seconds.
Top speed is 61.
0 to 60 in 5. Top speed is 120.
60 to 120 in 5 min
0 to 60 in 5. 60 to 120 in 20 sec.
Range of 1/4 mile.
Range of 1,000 miles. Cost 7 billion
Fast Car
flickr.com/photos/alpi-costerni/8161111001
20
Why ATDD?Rework down from 60% to 20%
Workflows work 1st time
Little room miscommunication
Saves time
Getting business rules RIGHT
Tighter cross functional team integration
Crisp visible story completion criteria
Automation yields reduced testing time21
Requirements and Tests
22
Requirement
Needed by a user to solve a problem
Needed by a system to satisfy contract, standard, spec, regulation...
23
Requirements Include
Functions (what software does)
Constraints (limits)
24
Requirements Should not
Include
Implementation details
Specific manifestation
25
A Brownie Spec
Ingredient Parts by weight
Sugar 23.0Flour 21.0Shortening 16.8 Nuts 16.0whole eggs 13.0Cocoa 5.5Dextrose, anhydrous 4.4Salt
26
Brownie Spec
a. Whip eggs in large bowl on high speed until light and fluffy.b. Combine sugars, cocoa, salt, and leavening; add to beaten eggs, and whip on high speed until thick.c. Add shortening slowly while mixing on low speed.d. Scrape bowl and whip on high speed until thick.e. Mix flour, nuts, and flavors together and fold into batter; mix until uniform.f. Pour batter into pan at a rate that will yield uncoated brownies which, when cut such as to meet the dimension requirements specified in 3.4f, will weigh approximately 35 grams each. g. Bake at 3500F until done (45 minutes).h. Cut into 2 inch squares
27
Requirements and tests
Inter-related
Can't have one w/o the other
Failing test is a requirement
Passing test valid system spec
28
ATDD as a Prism
Requirements
Tests
ATDD
29
Software Development
30
Exercise What Issues?
Current software dev process
Write down your org’s issues
Groups of 3 to 8
Pass to front
31
Developers Testers
Customers
The Team
32
Customer Unit
Product owner
Business analysts
Users
QA
33
Developer/Tester Unit
Programmers
Testers
QA
34
Developers Testers
Customers
The Triad
35
Traditional Value Stream
Elicit&Requirements&
Analyze&Requirements& Design&&
Code& Test&
Why go back?
Deploy&
36
Requirements
Write Code
The Traditional Approach
Demo
Rewrite
“Discussion”
Typically a confrontation &
argument
The demo forces a discussion of the
requirementsAn isolated event,
primarily worked by the customer proxy
37
Demo
Write Code
Requirements
The Agile Approach
Rewrite
Before significant time and $$ is
invested
Discuss
Include the Golden Triad
Dev QA
Customer38
Mistakes?
Misunderstandings
Missed requirements
Mis other
39
Mistakes
Feedback corrects misunderstandings
Quick feedback is better than slow feedback
40
Desired Actual
Slow Feedback
Actual
Fast Feedback
41
Agile Value Stream
Elicit&Requirements&
Analyze&Requirements&With&Tests&
Design&&
Code&With&Tests& Deploy&
42
Move Testing Forward
43
Two Types of Testing
To find defects
To prevent defects
44
Finding Defects
Waste
Identifying mistakes
flickr.com/photos/mag3737/444557606
45
preventing Defects
Saves development time
Saves QA time
Saves money
46
Steps
Author tests (write)
customer, tester, developer together
Connect to the system (automate)
developer
Run the tests (execute)
developer, testers, customers, CI system
Test% write% bind% run%
47
Two Ways
Code%write% run%bind%
Code%write%
run%
bind%
48
Test First
Write the test first
Write the code after
One molds the other
First in wins
49
Testing Strategies
Meszaros, XUNIT TEST PATTERNS: REFACTORING TEST CODE, Fig 6.1 "Purpose of Tests" p. 51, © 2007 Pearson Education, Inc
50
Testing Focus
51
Verification vs
ValidationVerification Are the requirements right Focus of acceptance tests
Validation Have we got the right requirements? Customer really do it this way? How do you test this?
52
Where to test?
53
Acceptance Testing
Applica'on*/**Middle*'er*
Core*Business*Logic**
UI*
Persistence*
Acceptance Tests
54
Tests all the way down
Requirement* Requirement**
Test* Test*
Requirement*
Test*
Test*
Service*Implementa4on*
Test*
Customer Acceptance Tests
Developer Acceptance Tests
Service*Implementa4on*
Requires services to implement
55
Acceptance Test Examples
56
Temperature
Input in Celsius
Output in Fahrenheit
How long to code?
57
Temperature
F = C x 9/5 + 32
58
Temperature
Tests?
59
Temperature
60
Test Binning
Celsius' Fahrenheit' Notes'0" 32"100" 212" Needed?"
Celsius' Fahrenheit' Notes'*273.15" *459.67" Precision"
Formula Tests
Precision Tests
Celsius' Fahrenheit' Notes'*273.15" *459.67" 0"Kelvin""*273.151" Error" Below"0"Kelvin""500" 932" Maximum"–"Needed?"
Limit Tests
61
Temperature
Input in Celsius
Output in Fahrenheit
How long to code?
62
Is This A Good Spec?
63
Business Rules
If a Customer Rating is Good and the Order Total is less than or equal $10.00, Then give no discount Otherwise give a 1% discount
If Customer Rating is Excellent, Then give a discount of 1% for any order
If the Order Total is greater than $50.00, Then give a discount of 5%
64
Rule Table => Test
?
?
65
Another Business Rule
Email coupon codes
If a customer enters the code, provide a 20% discount
Discount an item or the order?
66
Implementation
Testing script
Program interface
Xunit framework
ATDD framework
67
Testing Script
Tools for testers
(Usually) GUI based
68
Testing Script
Log in as a Customer who is rated GoodStart an OrderAdd items totally $10.01Complete order Verify $0.10 discount
Repeat for 10 other similar cases
69
Program Interface
Code something (CLI or GUI)
Screen
Discount Method
Discount Percentage Screen Customer Type: Good Order Total: 10.01 Percentage: 1 %
C:>DiscountPercentage Good 10.01 Percentage: 1%
70
Xunit Test
class TestCase { testDiscountPercentageForCustomer() { SomeClass o = new SomeClass()
assertEquals(0, o.computeDiscount(10.0, Good)); assertEquals(1, o.computeDiscount(10.01, Good)); assertEquals(1, o.computeDiscount(50.01, Good)); assertEquals(1, o.computeDiscount(.01, Excellent)); assertEquals(1, o.computeDiscount(50.0, Excellent));
assertEquals(5, o.computeDiscount(50.01, Excellent));
} }
71
Fit Test
72
Tables as Requirements and Tests
Discount Rule Customer Rating Order Total Discount Percentage
Good <= $10.00 0% Otherwise 1%
Excellent Any 1% > $50.00 5%
Discount))Test)Order)total) Customer)ra3ng) Discount)percentage?)$10.00% Good% 0%%$10.01% Good% 1%%$50.01% Good% 1%%$.01% Excellent% 1%%$50.00% Excellent% 1%%$50.01% Excellent% 5%%
Requirement
Test
73
Context
74
Context Diagram
Process'
User
Reports
Commands Views
75
Another Context Diagram
Process'
User
Another System Shared Database
Commands
Views
Requests
Responses
Stores
Retrievals
76
Engine Controller
Engine&Controller&
Engine Rotation Speed
Gasoline Flow Control
Spark Timing
Acceleration / De-acceleration
Air Flow Control
77
Embedded Contexts
Sub$ System Two
Input Output
System One
Internal I/O
78
Car Context
Drive&sha*&&
Acceleration De-acceleration Movement
Trans-mission&
Engine&& Wheels&
&
What to test?RPMs?Miles per hour
79
Software Context
Core or API
Developer Acceptance
Test
Customer Acceptance
Test
Middle-tier
GUI
What to test?Elements in an array?Behavior of the system
80
What To Test
BehaviorActions
Changesvia the API
via the same mechanism your GUI doesNot elements in a data structure
Not fields in the database
81
Use Cases and Tests
82
Use Case Template
Name—Identifier to easily reference it by Description—Brief note Actor—Who initiates the use casePre-conditions—What must be true before the use case is initiated Post-conditions—What’s true if the use case successfully executesMain course—Steps that show the sequence of interactions
83
Example: Part One
Name—Check Out CD.Description—Check out a CD for a customer.Actor—Clerk. Pre-conditions—The customer has an identification. The CD
has an identity. Post-conditions—The CD is recorded as rented. The rental
contract is printed.Main Course:
1. The clerk enters the customer identification and CD identifier into the system.
2. The system records the information.3. The system prints a contract that the customer signs.
84
Example: Part Two
Exceptions:1a. Customer identification is not recognized.
Clerk repeats step 1. 1b. The customer violates the CD Rental Limit business rule.
The clerk notifies the customer of the violation.The use case is abandoned.
Business Rule:CD Rental Limit
A customer can rent only three CDs at any one time. Alternatives
3a. The printer jams.The clerk fills out the contract by hand. The use case exits.
85
Acceptance Testing
The Triad create acceptance tests for stories
Tests from general (expected path) ...
... to specific (exceptions)
86
Examples
Rent a CD (expected path)
Bad Customer ID (Enter it wrong)
CD Rental Limit (already has 3. Try 4.)
Print jam (simulate by removing paper)
87
Test Anatomy
88
Test Flow
Setup
Use
Verify
Setup
Use
Verify
Test Flow
89
Test Flow
Ini$al'System'State'
Test'
Setup'(Given))'
Trigger'(When)'
Assert'(Then)'
Final'System'State'and'Output'
Action or Event
ExpectedSystem'State'and'Output'
90
Given/When/Then
Given = Use case pre-conditions When = Main course (or exceptional course)Then = Use case post-conditions
91
Given/when/Then
Given (Setup)Customer has ID (initial system state)CD has ID (initial system state)CD is not currently rented (initial system state)
When (Trigger)Clerk checks out CD (action)
Then (Verify)CD recorded as rented (final system state)Rental contract printed (output)
92
Given/when/Then
GivenCustomer has IDCD has IDCD is not currently rented
WhenClerk checks out CD
Then (Verify)CD recorded as rentedRental contract printed
93
Context Diagram
94
Internal or External Persistence
Output is response to change address
Change address Send mail
External repository for address
Input for send mail
95
Internal or External Persistence
How to test internal persistence? Through another action Through a “see-through” method APIs should be symmetricalAdd/DeleteModify/View
Change address
Internal persistence of address (state)
Send mail
Addressed mail
96
Exercise
Write questions (tests) for:“copying a file from one directory to another”
Describe in terms of:Given: some stateWhen: copy (file, source_directory,
destination_directory)Then: expected state
Source'Directory'Contents'
Des/na/on'Directory'Contents'
Copy'File''
Des/na/on'Directory'Contents?'
Anything'else?''
X' X' X'
Given When Then (Expected)
97
Scenario
B
A X
Copy%
B X
A X
98
Example
Source'Directory'Contents'
Des/na/on'Directory'Contents'
Copy'File''
Des/na/on'Directory''Contents?'
Source'Directory'Contents?'
Content'Comparison?'
Notes'
X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#
Standard#copy##
X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#
X#overwri?en#
X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Des-na-on.X)##
X#not#overwri?en#
What about modification date and other attributes? Make more columns
Given When Then (Expected)
99
Example
Source'Directory'Contents'
Des/na/on'Directory'Contents'
Copy'File''
Des/na/on'Directory''Contents?'
Source'Directory'Contents?'
Content'Comparison?'
Notes'
X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#
Standard#copy##
X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#
X#overwri?en#
X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Des-na-on.X)##
X#not#overwri?en#
What about modification date and other attributes? Make more columns
Given When Then (Expected)
100
Example
Source'Directory'Contents'
Des/na/on'Directory'Contents'
Copy'File''
Des/na/on'Directory''Contents?'
Source'Directory'Contents?'
Content'Comparison?'
Notes'
X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#
Standard#copy##
X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#
X#overwri?en#
X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Des-na-on.X)##
X#not#overwri?en#
What about modification date and other attributes? Make more columns
Given When Then (Expected)
101
Example
Source'Directory'Contents'
Des/na/on'Directory'Contents'
Copy'File''
Des/na/on'Directory''Contents?'
Source'Directory'Contents?'
Content'Comparison?'
Notes'
X,#Y,#Z# D,E,f# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#
Standard#copy##
X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Source.X)#
X#overwri?en#
X,#Y,#Z# D,E,F,X# X# D,E,F,X# X,Y,Z# (Des-na-on.X#==#Des-na-on.X)##
X#not#overwri?en#
What about modification date and other attributes? Make more columns
Given When Then (Expected)
102
Tables
103
Tables
Table-driven requirements can reduce ambiguityTables can be testsDiscovery
Tables bring to light overlooked assumptionsDraws out unstated requirements
104
Table Types
Calculation
Action
Data
105
Calculation Table
106
Calc Table Example
Discount Calculation
Item Total Customer Rating Discount Percentage?
$10.00 Good 0%
Given
When
Then
107
Data
! Exists&(or&should&exist)&–&for&Given&and&Then&
Title Data
Value Name 1 Value Name 2
Value for 1 Value for 2
Another value for 1 Another value for 2
108
Data Table Example
Customer Data
Name ID
James 007
Maxwell 86
Customer Data Name Begins with="’J"
Name ID
James 007
109
Text Requirements
User can log in
What are the rules?
110
Login Grid
User Name Password Result Notes
111
Action Table
Enter enters data into an entry fieldPress initiates a process, such as a Submit buttonCheck sees if a result is equal to an expected
value
112
Action Table Example
Check Out CD
Enter Customer ID 007
Enter CD ID CD2
Press Rent
Check Rented True
Can be just “When” or a stand-alone Given-When-Then
Given
When Then
113
Copy File ExampleCopy%a%file%(happy%path)%%! Given&a&source&&! and&target&directory&&&&&&&
File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F
Copy File
Enter Source Directory A
Enter Target Directory B
Enter File name X
Press Copy
! When©&request&is&made&
&&&&&&
! Then&file&is&copied&&&&&&&
File system Directory Name Directory Contents A X,&Y,&Z&B D,E,F,X
114
The Action
! Can$drive$a$GUI$
! Or$a$method$$CopyFile(Directory source, Directory target, Filename file)
115
Business Rules Test
Usually many
One test per rule
116
Example
Rates&
Type& Days& Cost?&Regular( 2( $2(Golden(Oldie( 3( $1.50(Hot(Stuff( 6( $12(Hot(Stuff( 50(( IGBTYOT((
CD(Rental(Rates(Regular(($2(/(2(days(plus($1(/(day((Golden(Oldie($1(/(2(days(plus($(.50(/(day(Hot(Stuff(($4(/(2(days(plus($2(/(day(
117
UI & Business Rules
Email&Valida)on&(Middle/Tier)&
Value&& Valid?&&[email protected], yes,someone@@somewhere.com, no,
!#$%&*+7/=?^_,{|}[email protected], yes,
Ajax call to middle-tier
Submit,
Middle-tier test
118
Alternatives
Can use "script" in tables or separately
119
System Boundaries
120
External Interfaces
! Connec&ons(to(external(systems(need(to(have(test(doubles((mocks)(
! Random(events(may(need(to(be(simulated((! Test(doubles(give(repeatability(and(speed((
External system, device, service
Random events
Time
121
External Interface Example
CD Rental System
Credit Card Processor
Sam’s Bank
Charge Transfer To
Confirmation or Denial Bank
Statement
122
Example of a Credit Card Test Double
CD Rental System
Credit Card Processor Test Double
Charge
Confirmation or Denial
123
Example of a Email Test Double
CD Rental System
Email Server Test Double
Confirmation or Denial
Test verifies that email sent to proper address with proper contents
124
Complex Business Rule and
Separation
125
Complex Business Rule
Field&One& Field&Two& Field&Three& Field&Four& Result&?&>&20& <&50&or&blank& >=100& Y& Allow&Otherwise&or&blank&
DNC& DNC& DNC& Disallow&
DNC& Otherwise& DNC& DNC& Disallow&DNC& DNC& Otherwise&or&
blank&DNC& Disallow&
DNC& DNC& DNC& N&or&blank& Disallow&
A business rule determines whether a user is allowed to perform a certain operation Fields may contain values or be blank
126
Complex Business Rule
Field&One& Result?&>&20& Allow&Blank& Disallow&Otherwise& Disallow&
Field&Two& Result?&<&50& Allow&Blank& Allow&Otherwise& Disallow&
Field&Three& Result&?&>=100& Allow&Blank& Disallow&Otherwise& Disallow&
Field&Four& Result?&&Y& Allow&N& Disallow&Blank& Disallow&Otherwise& ??&
127
Complex Business Rule
Field&One& Field&Two& Field&Three& Field&Four& Result&?&Allow& Allow& Allow& Allow& Allow&Disallow& DNC& DNC& DNC& Disallow&DNC& Disallow& DNC& DNC& Disallow&DNC& DNC& Disallow& DNC& Disallow&DNC& DNC& DNC& Disallow& Disallow&
128
Google Maps
What can you test w/o a UI?
129
Google Maps
No UI?
Search phrase => List of places
Place => Position
Zoom Level => List of maps
Position => List what's there
130
Test Evaluation
131
WARNING!!
Acceptances tests do NOT replace interactive communication!
They provide focus for the communication
132
Test Evaluation
Remove redundant redundancyBusiness domain terms
Shared between customer unit and developer/tester units
Avoid lots of input and output columnsBreak into smaller tables
133
Guidelines
Develop tests & automation separately First, understand Second, automate (if appropriate)
Automate tests for regression Run in continuous integration
Cover 100% of function (if practical)
134
Separation of Concerns
Separate business rules from how results of business rules are displayed
Separate calculation of a business rule (such as a customer rating) from the use of that business rule (such as to give a discount)
Separate each use case or step in a workflow Separate out validation of an entity from use of entity
135
Separation and Abstraction
One story together:When the build fails, send an email to the
administrator Separate and abstracted:
When the build fails, generate an alert When an alert is generated, notify the
responsible individual When the responsible individual is notified, use
that individual’s notification preference (email) Test the notification itself works (e.g. email
works)
136
What is a Good Test?
Fails reliably (for a known reason)
No false positives
No other test covers the same area
NOTE: Tests at the "top" often forced to assume business rules... aim for 1 failure at each level
137
Use Alternate Tables
Sudoku Puzzle 1 4 7
2 5 8 3 6 9
4 7 1 5 8 2
6 9 3 7 1 4
8 2 5 9 3 6
Sudoku Solution 1 6 5 4 9 8 7 3 2 9 2 4 3 5 7 6 8 1 8 7 3 2 1 6 5 4 9 4 9 8 7 3 2 1 6 5 3 5 7 6 8 1 9 2 4 2 1 6 5 4 9 8 7 3 7 3 2 1 6 5 4 9 8 6 8 1 9 2 4 3 5 7 5 4 9 8 7 3 2 1 6
Create tables in alternative forms, if standard not appropriate
Required(Performance((
Puzzle(Type( Time(to(Solve?( Notes(
Easy% .1%seconds%% See%easy%determina2on%
Impossible% 1%second% No%solu2on%%
138
Other uses for Tests
Estimation Aids
Degree of Doneness
139
Process note
Requirement Testable Requirement
Developable Requirement
At least one example for each test
Not schedulable Preferably all examples for all tests
140
process Note
Code%write%
run%
bind%
Code%write%
run%
bind%
Write previous iteration
Write within iteration
141
SCMDevelopers
Watch
Build
Unit Tests
Deploy to Dev
Integration Tests
First Look
Continuous Testing
On Demand Deploy to QA
Manual Testing
142
Not the End.... The Beginning!
143
Objective Review
Understand ATDD in context of SDLC
Learn how to turn requirements into tests
What is a good acceptance test?
ATDD as a communication vehicle
144
Our OutlineTerms and definitions
Requirements and tests
Software development
ATDD examples
Anatomy of a test
Tables
System boundary145
Recap
Primary goalsDiscover ambiguous requirements and gaps in requirements
early onCreate a record of business/development understandingGive feedback on quality
Secondary goalsUse acceptance tests as an executable regression testMeasure your progress towards "done” completenessMeasure the complexity of requirements.Use the tests as a basis for user documentation
146
Rules
Convert requirements into tests
Write tests before code
Write tests in grids (spreadsheets)
Never write code without tests
Include the Golden Triad
147
Further Reading
GROWS Method http://GrowsMethod.com
BDD Behavior Driven Development (by Dan North)
Cucumber http://cukes.info
Tellurim (hosted Cucumber) http://te52.com
Junit http://jaredrichardson.net/articles/junit-tutorial.html
FitNesse fitnesse.org
148
Exercise
You listed your development issues...
Would ATDD help, hurt, or be neutral
Why?
Write down your answer and pass it up
149
Take Aways
What’s a test?
Who writes them?
When are they written?
What do they verify?
Provide an example
Primary benefit?
Secondary benefits?
150
151
151