From user stories to acceptance tests

What we learned about Specification by example

Morten Berg and Jakub Holy present:

Iterate :: 1/2012

Quick fly-through

The Key Concerns of SbE

Deliver true business value

Build shared understanding

(Evolve living documentation)

"Doing the right thing vs. doing the thing right."

Shared understanding in business terms

Shared understanding + business terms and relations

Aligned business, SW, test models

Small change in business => small in SW, test

Much, much better maintainability, evolvability

(also less rework)

Build the right thing

The Key Patterns of SbEBusiness

goalScope

(UC, US)

Key examples

Executable specification

Living documentation

derive

Specify collaboratively

automate w/o changing

ilustrate

Often, evolve

1. Derive scope from business goal2. Specify collaboratively3. Ilustrate using examples4. Refine the specification5. Automate validation w/o changing specification6. Validate frequently7. Evolve into a documentation system

http://specificationbyexample.com/key_ideas.html

Goal: Shared understanding0. Clarify the goal

1. Build the shared understanding○ Collaborative specification... ○ ... with examples

2. Keep it○ Automate validation

3. Profit from it○ Evolve into living documentation

"The devil is in the details"

Example specification with examples

Title: Free book delivery

Objective: VIP users who order more books get free shipping in Norway. Examples: Customer Books Country Free shipping?

VIP 5 Norway Y

VIP 4 Norway N

VIP 10 Danmark N

Regular 10 Norway N

Sidenote: Automization

class FreeBookDeliveryTest { void setCustomer(..) {..} void setBooks(..) {..} void setCountry(..) {..} boolean isFreeShipping() {..}}

Customer Books Country Free shipping?

Showtime: Some real-world specifications with example

What makes a good specification with examples?

Good● Good google-able title● Clear objective - how to read the examples?● Uses business language, unified terms● Couple of key, focused examples, completeBad● Script ("click here, find css class X, ...)● Too detailed, too verbose, duplication● Many examples (bad understanding, bad

representation, more concepts mixed)

Show the poster!

Introducing SbE & Its Applicability

Strategy: What's the team's top pain? Can something from SbE fix it?

Examples:● Lot of rework devs - testers => better shared

understanding via Specification workshop● Late feedback from testers => automateKey: Introduce what you need, when you need it, step by step. Shift culture first. It's easy!

9-15 mo

buzz-words

Big Bang

Why to specify collaboratively?

Examples Tests

Requirements

can become

elaborate verif

y

Single source of truth

Devs: Truth in codeTesters: Truth in test casesBusiness analyst: Truth in documents Use a single source of truth, avoid translation! => Start with business analyst

Extend with examples from devsExtend with examples from testers

Random highlights

How to Make Acceptance Tests Maintainable 1. Aligned business, software, and test models

○ => small change in one is small in the other too

2. Test under the surface level, if possible○ Service > Servlet > UI - decide by risk vs. cost

3. Isolate tests from implementation by layers of test abstraction (What>Instrumentation>DSL)

See The Holy Java: How to Create Maintainable Acceptance Tests (http://wp.me/pTHiC-tl)

Why Are We Doing This?The Effect Maps Technique

Simple mind-mapping technique that incredibly facilitates agile product design, planning, prioritisation, and scoping. Levels:1. Why - the measurable business goal2. Who can help/hinder achievel of the goal?3. How can they do it?4. What are the SW features or business

activities that support it?

demo

Techniques: Spec Workshop and Diverge & Merge

Specification Workshop (½ day → 2w sprint)● All: business + devs + testers

○ Business: What, why○ Devs: What's technically possible/troublesome○ Testers: How to test (break) this? Corner cases

● Surface hidden complexities, discover assumptions● Build shared understanding (=> feedback exercise)

Diverge & Merge● Groups of 3-5 work in parallel 10-20 min, then merge

results● More divergent => richer & better ideas● Prevent paralysation by confining troublesome people

Acceptance Test Automation Tools

● FitNesse - wiki-based○ Cons: Long learning, encourages copy&paste○ Pros: Wiki, mature

● Concordion - new, HTML+JUnit○ Pros: Contrary to FitNesse disables copying, quick to

learn, integrated anywhere via JUnit, free-form○ Cons: No live editing, no copying => not for 104 tests

● Cucumber - new, websites, Ruby, ecosystem○ Pros: Simple text format (RegExp), easy2learn, web○ Cons: Only test execution - other tools for the rest

● Robot Framework - tabular, keyword-driven○ Pros: Great for testers w/o devs, libraries (SQL,..)

● Commercial - GreenPepper (Confluence), ...

The end is coming, run for your life!

Other Key Lessons

● When the time is short ... do communicate!● The key benefit of SbE aside of improved

communication is living documentation; regression tests are only a minor by-product

● (and many others :-))

Quiz

What have we learned today?

Resources

Gojko Adzic: Specification by example, 2011- condensed experience of 50 highly successful teams

Gojko Adzic: Agile product management using Effect Maps (paper)

Eric Evans: Domain Driven Design, 2003