requirements and acceptance tests: yes, they go together

Lean-Agile Acceptance Test-Driven Development: Better Software Through Collaboration

Copyright © 2011 Net Objectives 1

[email protected] www.netobjectives.com

1 Copyright © 2007 Net Objectives. All Rights Reserved. 2 June 2015

Requirements and Tests – Yes

They Go Together

May 2015


Ken Pugh

ken.pugh @netobjec2ves.com

Photo Size: Height: 2.25 Position: from top left corner Horizontal 0.75 Vertical 1. Picture Style: Simple Black Frame

No code goes in till the test goes on. A journey of two thousand miles begins with a single step.

§  Fellow Consultant §  OOA&D, Design PaAerns, Lean, Scrum, Test-‐Driven

Development

§  Over 2/5 century of soLware development experience

§  Author of seven books, including: –  Prefactoring: Extreme Abstrac4on, Extreme

Separa4on, Extreme Readability (2006 Jolt Award)

–  Interface Oriented Design

–  Lean Agile Acceptance Test-‐Driven Development: BeDer SoEware Through Collabora4on




Objec2ves

§  Show how requirements and tests are complementary facets

§  How to create requirements and tests collabora2vely


Overall Rule

§  There are excep2ons to every statement, except this one

4




Flow

§  Introduc2on §  Acceptance Test Examples §  Test Anatomy §  Supplemental

5

6 2 June 2015

Introduc8on




5 W’s and an H

§  ATDD – What are they – Who creates and uses them – When are they created – Where are they used – Why use them –  How to create them


What Are Acceptance Tests?

§  Acceptance Tests: –  External view of system

§  Examine externally visible effects –  Inputs and outputs –  State changes –  External interfaces User

Server

Service Provider




Defini2ons

§  Acceptance criteria –  General ideas

§  Acceptance tests –  Specific tests that either pass or fail –  Implementa2on independent

§  Triad – customer unit, developer unit, tester unit


Fast Car Example

§ Who wants a fast car? §  Criteria –  Run on a closed course, measure accelera2on

§  Test –  Detail accelera2on (0 to 60 mph in X seconds)




ATDD as a Prism

Requirement

Tests

ATDD

Fast Car

Acceleration

Top Speed


Why?

§  Rework Down from 60% to 20% § Workflows Working First Time §  LiAle Room for Miscommunica2on §  Saving Time §  Gefng Business Rules Right §  Game Changing §  Tighter Cross-‐Func2onal Team Integra2on §  Crisp Visible Story Comple2on Criteria §  Automa2on Yields Reduced Tes2ng Time




Value Stream Map – Classical Development

Elicit Requirements

Analyze Requirements Design

Code Test

Why go back?

Deploy


Why Mistakes?

§ Misunderstandings, missed requirements, mis-‐other

§  Feedback helps to correct misunderstandings §  Quick feedback beAer than slow feedback

Desired Actual Actual




Value Stream Map – Agile Development

Elicit Requirements

Analyze Requirements With Tests

Design

Code With Tests Deploy


Move Tes2ng Forward

§  Two types of tes2ng §  AAemp2ng to find defects – is WASTE §  AAemp2ng to prevent defects – is ESSENTIAL

§ When are defects found? –  Preven2on is just early detec2on




The Team

§  Customer Unit -‐ develops the requirements –  Product Owner –  Business analysts –  Users –  Quality Assurance

§  Developer / Tester Unit – develops the implementa2on –  Programmers –  Testers

§  The Triad –  Customer, developer, tester


Who Does What

§  Author the tests (write) –  Triad -‐ customer, tester, developer

§  Connect tests to system (bind) –  Developer

§  Run the tests (run) –  Developers, testers, customers –  Automated – part of build

Test write bind run




Usually the Case

Code write run bind


A BeAer Way

Code write

run

bind




Requirements and Tests

§  Requirements and tests are inter-‐related –  You can’t have one without the other

§  Failing test is a requirement –  Passing test denotes specifica2on on how system works

22 2 June 2015

Acceptance Test Examples




First Example

§  Input Temperature in Celsius, Output Temperature in Fahrenheit §  What tests would you run?


Input and Output Example (continued)

Celsius Fahrenheit Notes 0 32 100 212 Needed? -‐273.15 -‐459.7 Precision? -‐273.151 Error Below 0 Kelvin 500 932 Maximum – Needed?




Input and Output – “Unit Tests”

Celsius Fahrenheit Notes 0 32 100 212 Needed?

Celsius Fahrenheit Notes -‐273.15 -‐459.67 Precision

Formula Tests

Precision Tests

Celsius Fahrenheit Notes -‐273.15 -‐459.67 0 Kelvin -‐273.151 Error Below 0 Kelvin 500 932 Maximum – Needed?

Limit Tests


A Business Rule Example

If Customer Ra2ng is Good and the Order Total is less than or equal $10.00, Then do not give a discount, Otherwise give a 1% discount.

If Customer Ra2ng is Excellent, Then give a discount of 1% for any order.

If the Order Total is greater than $50.00, Then give a discount of 5%.




Business Rule Table = Test

Discount

Order total Customer ra8ng Discount percentage?

$10.00 Good 0%

$10.01 Good 1%

$50.01 Good 1%

$.01 Excellent 1%

$50.00 Excellent 1%

$50.01 Excellent 5%


Ways To Implement Test

§  Tes2ng script §  Program interface §  Xunit framework §  ATDD framework




Tes2ng script

§  Tester creates script (usually GUI based). §  Example: –  Logon as Customer who is rated Good –  Start order –  Put items in the order un2l the total is exactly $10.01 –  Complete order –  Check it shows a $.10 discount

§  Repeat for other five cases


Program interface

§  Create a command line or graphic user interface

Screen

Discount Method

Discount Percentage Screen Customer Type: Good Order Total: 10.01 Percentage: 1 %

C:>DiscountPercentage Good 10.01 Percentage: 1%




XUnit Test

class TestCase { testDiscountPercentageForCustomer() {

SomeClass o = new SomeClass()

assertEquals(0, o.computeDiscount(10.0, Good));



assertEquals(1, o.computeDiscount(.01, Excellent));

assertEquals(1, o.computeDiscount(50.0, Excellent));

assertEquals(5, o.computeDiscount(50.01, Excellent)); }

}


Fit (Table = Test)

Discount


10.00 Good 0

10.01 Good 1

50.01 Good 1

.01 Excellent 1

50.00 Excellent 1

50.01 Excellent 5




Fit Test

Discount


10.00 Good 0

10.01 Good 1

50.01 Good Expected 1 Actual 5

.01 Excellent 1

50.00 Excellent 1

50.01 Excellent 5


Tables As Requirement and Test

Discount Rule Customer Rating Order Total Discount Percentage

Good <= $10.00 0% Otherwise 1%

Excellent Any 1% > $50.00 5%

Discount Test Order total Customer ra8ng Discount percentage? $10.00 Good 0% $10.01 Good 1% $50.01 Good 1% $.01 Excellent 1% $50.00 Excellent 1% $50.01 Excellent 5%

Requirement

Test

(Seems unclear)



35 2 June 2015

Test Anatomy


Scenario Flow

Ini2al System State

Scenario

Setup (Given)

Trigger (When)

Assert (Then)

Action or Event Expected System State and Output




Example of Scenario

§  Given (Setup) –  Car is not moving

§ When (Trigger) –  Accelerator pressed

§  Then (Assert) –  60 MPH reached before 4.5 seconds


Test Flow

Ini2al System State

Test

Setup (Given)

Trigger (When)

Assert (Then)

Final System State and Output

Action or Event

Expected System State and Output

Compare




Example of Test Against Scenario

§  Given (Setup) –  Car is not moving

§ When (Trigger) –  Accelerator pressed

§  Then (Verify) –  Check that 60 MPH is reached before 4.5 seconds

40 2 June 2015

Workflow Example




Context

§  Sam’s Lawn Mower Repair and CD Rental Store –  A custom soLware package for CD rentals.


User Story Examples

§ As the clerk, I want to check out a CD for a customer.

§ As the clerk, I want to check in a CD for a customer.

§ As the inventory maintainer, I want to know where every CD is—in the store or rented.

42




Use Case Example

§  Name—Check Out CD. §  Descrip8on—Check out a CD for a customer. §  Actor—Clerk. §  Pre-‐condi8ons—The customer has an iden2fica2on. The CD

has an iden2ty. §  Post-‐condi8ons—The CD is recorded as rented. The rental

contract is printed. §  Main Course:

1. The clerk enters the customer iden2fica2on and CD iden2fier into the system.

2. The system records the informa2on. 3. The system prints a contract that the customer signs.

43


Given / When / Then Example

§  Context: Sam’s Lawn Mower Repair and CD Rental Store §  Given (Setup)

Customer has ID (ini2al system state) CD has ID (ini2al system state) CD is not currently rented (ini2al system state)

§  When (Trigger) Clerk checks out CD (ac2on)

§  Then (Verify) CD recorded as rented (final system state) Rental contract printed (output)




Full Example (1)

Check Out CD §  Given Customer has ID and CD has ID and CD is not currently rented

Customer Data

Name ID

James 007

CD Data

ID Title Rented

CD2 Beatles Greatest Hits No


Full Example (2)

§ When a clerk checks out a CD:

Check Out CD

Enter Customer ID 007

Enter CD ID CD2

Execute Rent




Full Example (3)

§  Then the CD is recorded as rented and a rental contract is printed:

CD Data

ID Title Rented Customer ID

CD2 Beatles Greatest Hits Yes 007

Rental Contract

Customer ID Customer Name CD ID CD Title

007 James CD2 Beatles Greatest Hits


Full Example – Extended §  Given

§  When a clerk checks out a CD on:

§  Then a rental contract is printed: Rental Contract

Customer ID Customer Name CD ID CD Title Due Fee 007 James CD2 Beatles Greatest Hits 1/3/2014 $3

Rental Fee Business Rule

Fee

$3

Today

1/1/2014

Rental Time Business Rule

Time

2 days




The Ac2on

§  Can drive a GUI

§  Or a method Rent(CustomerID aCustomer, CDID aCD);


Example of Business Rule

CD Rental Rates Business Rule Regular $2 / 2 days plus $1 / day Golden Oldie $2 / 4 days plus $ .50 / day Hot Stuff $2 / 1 days plus $2 / day

_s

CD Rental Rates As Table

Type Base Rental Period Days

Base Rental Fee Extra Day Fee

Regular 2 $2 $1 Golden Oldie 4 $2 $.50 Hot Stuff 1 $2 $2




Example of Business Rule Test

CD Rates Test

Type Days Cost? Regular 2 $2 Golden Oldie 5 $2.50 Hot Stuff 6 $12 Hot Stuff 50 IGBTYOT

CD Rental Rates Business Rule Regular $2 / 2 days plus $1 / day Golden Oldie $2 / 4 days plus $ .50 / day Hot Stuff $2 / 1 days plus $2 / day

_s


Business Rule and Flow Tests

§  Use cases / scenarios usually include business rules. §  Test every business rule separately §  Flow test (of scenario) –  Uses one varia2on of business rule




Full Flow Example Revisited Check Out CD

Given Customer has ID; CD has ID and not currently rented

and one varia2on of business rule for Rental Rates

Customer Data Name ID James 007

CD Data ID Title Rented Type CD2 Beatles Greatest Hits No Regular

CD Rental Rates As Table

Type Base Rental Period Days

Base Rental Fee Extra Day Fee

Regular 2 $2 $1


Full Flow Example – Extended

§ When a clerk checks out a CD on:

§  Then a rental contract is printed: Rental Contract

Customer ID Customer Name CD ID CD Title Due Fee

007 James CD2 Beatles Greatest Hits 1/3/2014 $2

Today

1/1/2014

Check Out CD

Enter Customer ID 007

Enter CD ID CD2

Execute Rent



55 2 June 2015

Guidelines


Tests

§  Acceptance tests are not a subs2tute for interac2ve communica2on –  They can provide focus for that communica2on

§  Tests in business domain terms –  Shared between customer unit and developer unit




Guidelines

§  Tests and automa2on should be developed separately. –  Understand the test first –  Then explore how to automate it.

§  Automate tests for regression –  Use in con2nuous build

§  As much as prac2cal, cover 100% of the func2onal requirements by acceptance tests.

§  Can break down stories by acceptance tests –  One acceptance test per story


Tes2ng Strategies

Meszaros, XUNIT TEST PATTERNS: REFACTORING TEST CODE, Fig 6.1 "Purpose of Tests" p. 51, © 2007 Pearson Education, Inc



59 2 June 2015

Not an Ending, But a Beginning


Recap

§  Primary goals –  Discover ambiguous requirements and gaps in requirements early on.

–  Create a record of business/development understanding.

§  Secondary goals –  Use acceptance tests as an executable regression test. – Measure the complexity of requirements. –  Use the tests as a basis for user documenta2on.





Go Forth and Become

Acceptance Test Creators

Thank You



Supplemental Slides



63 2 June 2015

Context Diagram


Context Diagram

§  A context diagram shows scope

Process

External Actors

Outputs

Input / Output

Inputs




CD Rental Diagram

Process

Clerk

Credit Card Processor

Database

Check-out/Check-in

Response

Send Transaction

Response

Update Rental Info

Retrieve Rental


External Interfaces

§  Connec2ons to external systems need to have test doubles (mocks) –  They are stand-‐ins for real system during tes2ng –  Random events may need to simulated

§  Test doubles give repeatability and speed –  Cheaper if have to pay for service

External system, device, service

Random events

Time




CD Rental Diagram (Revisited)

Process

Clerk Test Double (UI or Mid-tier)

Credit Card Processor Test Double

Database Test Double

Check-out/Check-in

Response

Send Transaction

Response

Update Rental Info

Retrieve Rental


Context of Automobile

Drive shaL

Acceleration De-acceleration Movement

Trans-‐mission

Engine Wheels




Alterna2ve Design of Automobile

Acceleration De-acceleration Movement

Engines

Wheels

Engine Controller

70 2 June 2015

Cucumber Version




Cucumber Version

Scenario Outline: Compute discount Given total is <OrderTotal> and rating is "<CustomerRating>" When I compute discount Then percent is <DiscountPercentage> Examples: |OrderTotal|CustomerRating|DiscountPercentage| |10.00 |Good |0 | |10.01 |Good |1 | |50.01 |Good |1 | |0.01 |Excellent |1 | |50.00 |Excellent |1 | |50.01 |Excellent |5 |


Alterna2ve (2)

Cucumber / Specflow with Tables

Feature: Check Out As a store clerk I want check out a CD for a customer So that I keep track of where the CDs are Scenario: Normal Check Out Given customer exists: |ID | Name | |007| James| And CD exists: |ID | Title | Rented | |CD2 | "Beatles Greatest Hits" | No | When CD "CD2" is checked out to Customer "007" Then CD is now: |ID | Title | Rented | CustomerID | |CD2 | "Beatles Greatest Hits" | Yes | 007 | And rental contract produced: |CustomerID | CustomerName | CDID | CDTitle | |007 | James | CD2 | Beatles Greatest Hits |



73 2 June 2015

Sequence


Usually the Case

Code write run bind




A BeAer Way

Code write

run

bind

76 2 June 2015

Process Notes




Process Note (1)

Requirement Testable Requirement

Developable Requirement

At least one example for each test

Not schedulable Preferably all examples for all tests


Process Note (2)

Code write

run

bind

Code write

run

bind

Write previous iteration

Write within iteration



79 2 June 2015

ReseTng Tests


Resefng Tests

§  Changes in system state need to be reset before running test again

§  Some ways –  Another test that reverses state (e.g. Check In CD) –  Set persistent storage back to original state

§  For just affected records (CD) §  For all records (en2re database)




Full Example (1)

Check In CD §  Given CD is rented

CD Data

ID Title Rented Customer ID

CD2 Beatles Greatest Hits Yes 007


Full Example (2)

§ When a clerk checks in CD:

Check Out CD

Enter CD ID CD2

Execute Return




Full Example (3)

§  Then the CD is recorded as not rented

CD Data

ID Title Rented

CD2 Beatles Greatest Hits No

requirements and acceptance tests: yes, they go together

Software

net objectives

collaboration copyright

better software

acceptance tests

extreme separa4on

photo size

simple black

left corner horizontal