tw specifications for-testing1

December 2011

Specifications For Enterprise Testing

Presenters

Sathyan Sethumadhavan & Shalabh Varma

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Agenda

Where do we spend more time

A new approach for collaboration

Implement Specifications

Gauge Progress

Q&A

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Do you experience any of this

Long Regression Cycles

Defects Leakage - Missing Requirements

Test and Requirements documents becomes stale

Spends more time on defect management and triage

Less confidence - Existing test coverage

Frequent changes - Breaks existing functionalities

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Do you spend more time here

Requirements document

Creating test plan and strategy

Creating test Cases/Steps

Defect Management

Traceability Matrix

Fix productionDefects

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Specifications Driven Development

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

EXAMPLES

REQUIREMENTS

TESTSCan Become

Elaborate

Verify

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Specifications – Focus Area

Requirementsdocument

Creating test plan and strategy

Creating test cases

Defect management and analysis

Traceability Matrix

Fix production defects

Specifications for both humans and machines

Tests as Documentation

Managing stale artifcats

Specifying Collaborately

Validating Frequently

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

A Good Specification

Examples should be complete

Precise and Testable

True Specification – Not a script

About business functionality – Not software design

Self-explanatory and focused

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

A Good Specification

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

A Good Specification

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Benefits

Distributed teams - Necessary artifacts and re-use artifacts

One-time specification – Referred by stakeholders and team

Artifacts remain as living documents

Focus on collaboration and communication gap

More time on exploratory/business testing

Time-to-market availability

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Specifications – Key Process Patterns

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Goals

Understand customer’s business requirements

Understand the “Why” and “Who”

Understand the intent and value

Understand the Output

Team delivers complete feature

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Collaboration

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Examples

Requirements - Don’t provide full and unambiguous context

Don’t have Yes/No answers in your example

Ask for alternate way for validation

Avoid making up your own data

Avoid temptation to explore all possibility

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Refining

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Refining

Don’t create workflow like descriptions

Don’t get trapped in user interface details

Show and lets the user understand

Refined Examples = Acceptance criteria for delivery

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Refining

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Automation

Plan for automation upfront

Don’t delegate automation towards the end

Avoid automating existing manual scripts

Automate along system boundaries with necessary context

Automated Specification = Executable Specification

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Automation

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns - Validation

Fix the most annoying thing, fix it and repeat

Simpler test doubles for external systems

Isolated environment and fully automated deployments

Look for ways for faster feedback

Don’t just disable failing tests

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Key Process Patterns – Living Documentation

Don’t create long specifications

Avoid using technical concepts in tests

Base specification on personas

Evolve the ubiquitous language and use it consistently.

Organize based on functional areas

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Implementing Specifications

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Greenfield

Implement as part of the big process change

Use TDD as a first step towards Specifications

Follow the key process patterns

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Existing Projects

Start with automated tests

Don’t talk process or jargon words

Don’t make test automation end goal

Our Focus - Collaboration and process-change

Executable specification in the same codebase

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Gauge Progress

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Gauge Progress

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Take Away

Distributed teams - Necessary artifacts and re-use artifacts

One-time specification – Referred by stakeholders and team

Artifacts remain as living documents

Focus on collaboration and communication gap

More time on exploratory/business testing

Time-to-market availability

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Case Studies

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Case Studies

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

Tools

Twist - http://studios.thoughtworks.com/twist-agile-test-automation/ Cucumber – http://cukes.infoConcordion – http://www.concordion.orgFitNesse - http://fitnesse.orgJbehave - http://jbehave.orgGreenPepper: http://www.greenpeppersoftware.com. Robot Framework: http://www.robotframework.orgSpecFlow: http://www.specflow.orgTextTest: http://www.texttest.org

Contact us

Sathyan Sethumadhavansatnair@thoughtworks.com

Shalabh Varmasvarma@thoughtworks.com

Private and Confidential. © 2011 ThoughtWorks, Ltd. All rights reserved.

References

① Specification by example and Manning the specification- Gojko Adzic

② Projects implemented SDD in real-time