acceptance test driven development - (lasse koskela at scrum gathering orlando 2010)

Copyright Reaktor 2009

Acceptance Test Driven DevelopmentScrum Gathering Orlando 2010

Lasse Koskela

@[email protected]

1

Tuesday, March 9, 2010


Why should I care?

With acceptance test driven development...

...we collaborate between dev, test, biz, etc.

...we build a shared a language and vocabulary.

...we understand what we're implemenFng.

...scope of our work is clear and understood.

...we get feedback earlier.

2



3



ATDD. What is it?

4



"acceptance tes9ng"

5

0

2

4

5

7

ATDD STDD CTDD FTDD



Conversa9ons about the product and its features.

6



What were they doing?

1. SpecificaFon

2. ValidaFon

3. VerificaFon

8



❝❞Mary Poppendieck

If you can't predict the results of a test, you don't understand

the system.

9



Rela9on to TDD.

10



TDD

Developer acFvity.

Code-‐facing.

Local. Isolated.

Automated tests.

ATDD

Team acFvity.

Business-‐facing.

HolisFc.

May be automated



12



What does it look like?

13



14



15



As a competition organizerin order to show the judging to the

audience on several monitorsI want to have the scores for the

current pair on a web page.

16



"With 3 judges giving scores 7, 11, and 2, the displayed score should be 20."

"When the first 2 judges have given their scores, e.g. 7 and 11, the intermediate

score of 18 should be displayed already."

These examples and any additional business rules we describe become the acceptance criteria or tests.

17



The examples are translated into executable tests,allowing a computer to run them against the product.

18



19



http:/ /bit . ly/atddoverview

To me, the essen9al characteris9cs of ATDD, by whatever name you want to call it, are:

1. We use test ideas to elicit details from the business stakeholder(s) about their expecta9ons. By proac9vely discussing test ideas -‐ like boundary condi9ons or configura9ons or varying sequences of user ac9ons, etc -‐ we can come to a shared understanding of the business stakeholder’s real expecta5ons for the system instead of having testers file a whole bunch of bugs late in the process.

2. We dis5ll acceptance criteria into automatable tests expressed in a natural language rather than a programming language. This enables us to completely separate the ar5cula5on of expecta5ons from any technical details or dependencies.

3. We write fixtures or libraries to wire the keywords in the tests to the soMware under development during implementa9on. That’s wiring, not transla9ng. And doing it as part of the implementa9on effort, not aOemp9ng to retrofit automa9on aMer the code is wriOen.

Elisabeth Hendrickson(emphasis mine)

20



21



Cri9que

22



"The Problems With Acceptance TesFng"

http:/ /bit . ly/a lternatives2at

My experience with Fit and other agile acceptance tes9ng tools is that they cost more than they're worth. There's a lot of value in geUng concrete examples from real customers and business experts; not so much value in using "natural language" tools like Fit and similar.

The planned benefit of those tools is that the customers are supposed to write the examples themselves, thus improving communica9on between customers and programmers. In prac9ce, I found that customers (a) weren't interested in doing that, and (b) oMen couldn't understand and didn't trust tests that were wriOen by others. Typically, responsibility for the tests gets handed off to testers, which defeats the whole point.

Furthermore, acceptance tes9ng tools are almost invariably used to create end-‐to-‐end integra9on tests, which are slow and briOle. Fit works best for targeted tests that describe the domain, but that's not how it's used. Also, tools like Fit don't work with refactoring tools. Once you have a lot of tests, they become a real maintenance burden.

These two problems-‐-‐that customers don't par9cipate, which eliminates the purpose of acceptance tes9ng, and that they create a significant maintenance burden, means that acceptance tes9ng isn't worth the cost. I no longer use it or recommend it.

Instead, I involve business experts (on-‐site customers) closely throughout the itera9on. I do have them create concrete examples, but only when faced with par9cularly complex topics, and never with the inten9on that they be run by a tool like Fit. Instead, the programmers use those examples to inform their test-‐driven development, which may or may not involve crea9ng automated tests from the examples, at the programmers' discre9on.

I also have the on-‐site customers conduct ad-‐hoc review of the soMware as programmers complete it. They pair with a programmer, look at what's been created, and ask for changes that the programmer implements on the spot.

James Shore

23



❝❞George Dinwiddie

I’ve had some success using automatable examples as a tool to enhance the conversa>on between the

customer, the tester, and the developer. [To add] precision to the agreement about what needs to be accomplished. The goal is for the customer to look at

the test and say “Yes, that’s what I want.”

http:/ /bit . ly/real ityofaat24


http://blog.gdinwiddie.com/2010/02/25/a-lingua-franca-between-the-three-or-more-amigos/





❝❞Ron Jeffries

I’m concerned about the poten>al loss of understanding and agreement between on-‐site

customer / product owner and the team. Examples are the best way to do that and while it is more work for the customer, the examples are more likely to be

understood, and to be correct.

http:/ /bit . ly/problemswithat25



❝❞Ron Jeffries

I’m OK that [acceptance tests] are not customer-‐understandable — though I would prefer that they were if it were close to free. I am less comfortable with the no>on that they are not automated. My concern would be that if they are not automated,

doors are opened to regressions.

http:/ /bit . ly/problemswithat26



❝❞Ian Cooper

With tests that are evaluated by an impar>al tool, “done” is really “what everyone agreed on”, not “almost done with just a few things to fill in

tomorrow”. I’m not sure whether an on-‐site review is enough to guard against this completely.

http:/ /bit . ly/areattoolsevi l27



❝❞James Shore

When it comes to tes>ng, my goal is to eliminate defects. [...] I think of defects as coming from four

sources: programmer errors, design errors, requirements errors, and systemic errors. When trying to eliminate defects, I look for prac>ces that address

these four causes.

http:/ /bit . ly/a lternatives2at28



For catching requirements errors Jim uses the following pracFces:

• Whole Team

• Customer Examples

• Customer Review

• Bring Testers Forward

http:/ /bit . ly/a lternatives2at29



Conversa9ons about the product and its features.

30



How to succeed with ATDD?

31



How to fail with ATDD?

32



1. No collaboraFon

2. Focusing on how instead of what

3. Tests unusable as live doc

4. ExpecFng acceptance tests to be full regression suite

5. Focusing on tools

6. Not considering acceptance tesFng as value added

7. Test code not maintained with love

8. ObjecFves of team members not aligned

9. No management buy-‐in

10. UnderesFmaFng the skill required to do this well

http:/ /bit . ly/atddfai l33



#FollowFriday

34


acceptance test driven development - (lasse koskela at scrum gathering orlando 2010)

Documents

automated copyright

cri9que copyright reaktor

vericafoncopyright reaktor

lot of tests

automated tests

acceptance tes9ng7

automatable tests

targeted tests