acceptance test driven development with spec flow and friends

Acceptance Test Driven Development

with SpecFlow and Friends

Acceptance Test Driven Development

with SpecFlow and Friends

Christopher Bartling

Joel Levandoski


Joel Levandoski

Contact informationContact information


[email protected]

Twitter: @cbartling

Joel Levandoski

[email protected]

Twitter: @joellevandoski


[email protected]

Twitter: @cbartling

Joel Levandoski

[email protected]

Twitter: @joellevandoski

Administrative stuffAdministrative stuff

Presentation and demo are available on BitBucket.org

https://bitbucket.org/cebartling/presentations

Presentation and demo are available on BitBucket.org

https://bitbucket.org/cebartling/presentations

TopicsTopics

Acceptance Test Driven Development (ATDD)

SpecFlow for .NET

Behavior Driven Development (BDD) tool

WatiN and WebAii

Live demonstration

Questions

Acceptance Test Driven Development (ATDD)

SpecFlow for .NET

Behavior Driven Development (BDD) tool

WatiN and WebAii

Live demonstration

Questions

Acceptance Test Driven DevelopmentAcceptance Test Driven Development

ATDD

Acceptance tests are executable specifications of desired behavior and functionality of the system

Expressed in language of the problem domain

Automated

Regression suite

Development is driven from the outside-in

ATDD

Acceptance tests are executable specifications of desired behavior and functionality of the system

Expressed in language of the problem domain

Automated

Regression suite

Development is driven from the outside-in

ATDD vs. TDDATDD vs. TDD

TDD is extremely valuable, but you need more

Achieve great unit test coverage and still not deliver value to the customer

ATDD focuses on complete features and functionality

ATDD: macro view

TDD: micro view

TDD is extremely valuable, but you need more

Achieve great unit test coverage and still not deliver value to the customer

ATDD focuses on complete features and functionality

ATDD: macro view

TDD: micro view

SpecFlow for .NETSpecFlow for .NET

BDD testing framework

Integrates nicely with Visual Studio

Acceptance tests manifest themselves as features and scenarios

SpecFlow generates automated test from features

SpecFlow tests run as normal xUnit tests

Visual Studio test runner, build server

BDD testing framework

Integrates nicely with Visual Studio

Acceptance tests manifest themselves as features and scenarios

SpecFlow generates automated test from features

SpecFlow tests run as normal xUnit tests

Visual Studio test runner, build server

FeaturesFeatures

Describe some piece of functionality of the system

Maintained in a .feature file

Plain readable text

Understandable by all parties, including business

Gherkin format, popularized by Cucumber

Features contain one or more scenarios

Describe some piece of functionality of the system

Maintained in a .feature file

Plain readable text

Understandable by all parties, including business

Gherkin format, popularized by Cucumber

Features contain one or more scenarios

Feature file exampleFeature file example

Feature: Calculate Net Present Value

In order analyze the profitability of a project

As a project manager

I want to be able to calculate the project’s Net Present Value

Scenario: A project is rejected

Given a project to evaluate

When data is entered reflecting a failing project ROI scenario

Then the net present value calculator will reject the project

Feature: Calculate Net Present Value

In order analyze the profitability of a project

As a project manager

I want to be able to calculate the project’s Net Present Value

Scenario: A project is rejected


When data is entered reflecting a failing project ROI scenario


ScenariosScenarios

• A scenario describes a single acceptance test for a feature

• Most features are composed of multiple scenarios

• SpecFlow generates a test for each scenario

• The name of the test is generated from the scenario title

• A scenario describes a single acceptance test for a feature

• Most features are composed of multiple scenarios

• SpecFlow generates a test for each scenario

• The name of the test is generated from the scenario title

Scenario exampleScenario example

Scenario: A project is rejected when its NPV is negative


When data is entered reflecting a failing ROI scenario


Scenario: A project is rejected when its NPV is negative


When data is entered reflecting a failing ROI scenario


Scenario stepsScenario steps

• Used to compose a scenario

• Custom code to automate your application

• Describe preconditions, triggering action, and verification of outputs for the acceptance test

• Given: preconditions

• When: triggering action

• Then: behavior and state verifications

• Used to compose a scenario

• Custom code to automate your application

• Describe preconditions, triggering action, and verification of outputs for the acceptance test

• Given: preconditions

• When: triggering action

• Then: behavior and state verifications

Scenario step exampleScenario step example


is matched to the following step definition binding…

[Given(@"a project to evaluate")]

public void GivenAProjectToEvaluate() {

. . .

}


is matched to the following step definition binding…

[Given(@"a project to evaluate")]

public void GivenAProjectToEvaluate() {

. . .

}

Set up and tear downSet up and tear down

• Attributes for before and after events

• TestRun

• Feature

• Scenario, ScenarioBlock

• Step

• Specificity achieved when used with tags

• Attributes for before and after events

• TestRun

• Feature

• Scenario, ScenarioBlock

• Step

• Specificity achieved when used with tags

TagsTags

• Markers that can be applied to features and scenarios

• Useful for selectively mixing in behavior

• Used to categorize scenarios

• Some unit test frameworks support this categorization tagging

• Custom tags you define

• Markers that can be applied to features and scenarios

• Useful for selectively mixing in behavior

• Used to categorize scenarios

• Some unit test frameworks support this categorization tagging

• Custom tags you define

Tag exampleTag example

@WatiN

Scenario: Save valid sample size mid range

Given the user enters 10 as sample size

When the user selects save

Then the value is stored

@WatiN

Scenario: Save valid sample size mid range

Given the user enters 10 as sample size

When the user selects save

Then the value is stored

Tag example used with BeforeScenarioTag example used with BeforeScenario[BeforeScenario("WatiN”)]

public void BeforeScenarioUsingWatiN()

{

...

}

[BeforeScenario("WatiN”)]

public void BeforeScenarioUsingWatiN()

{

...

}

BackgroundBackground

• Common preconditions for all scenarios in a feature

• Contains one or more scenario steps

• Executed before any other steps in the scenario

• SpecFlow generates a method from the background element in the feature file in the test class

• Invoked from each scenario test case in the test class

• Common preconditions for all scenarios in a feature

• Contains one or more scenario steps

• Executed before any other steps in the scenario

• SpecFlow generates a method from the background element in the feature file in the test class

• Invoked from each scenario test case in the test class

Background exampleBackground example

Background:

Given that the welcome page is displayed

Scenario: Add a comment to a book being reviewed

. . .

Background:

Given that the welcome page is displayed

Scenario: Add a comment to a book being reviewed

. . .

Scenario outlinesScenario outlines

• Data-driven scenarios or scenario templates

• Consists of

• Scenario template specification with data placeholders

• Set of examples providing values for placeholders

• SpecFlow generates parameterized test logic for the scenario outline and individual test method for each example set

• Data-driven scenarios or scenario templates

• Consists of

• Scenario template specification with data placeholders

• Set of examples providing values for placeholders

• SpecFlow generates parameterized test logic for the scenario outline and individual test method for each example set

ScenarioContextScenarioContext

• ScenarioContext.Current.Pending()

• Causes the step definition to return pending

• Used to signal a stubbed step definition

• ScenarioContext.Current dictionary

• Allows you to carry context/state across step definitions participating in scenario

• ScenarioContext.Current.Pending()

• Causes the step definition to return pending

• Used to signal a stubbed step definition

• ScenarioContext.Current dictionary

• Allows you to carry context/state across step definitions participating in scenario

WatiNWatiN

Open source browser automation tool for IE and Firefox

Similar to Selenium, WebDriver, Watir

Browser abstraction

Find elements using CSS selectors

Interact with AJAX web elements

Handle popup dialogs (native and HTML)

Open source browser automation tool for IE and Firefox

Similar to Selenium, WebDriver, Watir

Browser abstraction

Find elements using CSS selectors

Interact with AJAX web elements

Handle popup dialogs (native and HTML)

WebAiiWebAii

Browser automation tool from Telerik

Automates both web 2.0 and Silverlight applications

HTML element wrappers

WaitForElement(s) support when using AJAX

Identifying elements using LINQ

Invoke JavaScript directly from test code

Browser automation tool from Telerik

Automates both web 2.0 and Silverlight applications

HTML element wrappers

WaitForElement(s) support when using AJAX

Identifying elements using LINQ

Invoke JavaScript directly from test code

Best practicesBest practices

• Write high-level specifications

• Specifications should remain stable over time

• Build a scripting interface for manipulating your system under test (SUT)

• Focus specifications on isolated behaviors

• Think of specifications in Given-When-Then format

• Use the Page Object pattern

• Write high-level specifications

• Specifications should remain stable over time

• Build a scripting interface for manipulating your system under test (SUT)

• Focus specifications on isolated behaviors

• Think of specifications in Given-When-Then format

• Use the Page Object pattern

SmellsSmells

• Specifications are constantly changing

• Specifications are composed of “sequential command executions”

• Specifications have a lot of instrumentation or fixture code

• Specification examples exhibit same structure

• Specifications are constantly changing

• Specifications are composed of “sequential command executions”

• Specifications have a lot of instrumentation or fixture code

• Specification examples exhibit same structure

Net Present Value Calculator DemoNet Present Value Calculator Demo

Used in capital budgeting

Measures the excess or shortfall of cash flows, in present value terms, once financing terms have been covered

Demo has two implementations

ASP.NET MVC

Silverlight

Used in capital budgeting

Measures the excess or shortfall of cash flows, in present value terms, once financing terms have been covered

Demo has two implementations

ASP.NET MVC

Silverlight

Questions?Questions?

Literature citedLiterature cited

http://www.concordion.org/Technique.html

http://www.telerik.com/automated-testing-tools/webaii-framework-features.aspx

http://watin.org/

http://www.specflow.org/

http://code.google.com/p/selenium/wiki/PageObjects

http://www.concordion.org/Technique.html

http://www.telerik.com/automated-testing-tools/webaii-framework-features.aspx

http://watin.org/

http://www.specflow.org/

http://code.google.com/p/selenium/wiki/PageObjects

acceptance test driven development with spec flow and friends

Documents

scenario example scenario

scenario specflow

watin scenario

scenario test case

scenario steps

scenario step example

scenario templates

scenario title