�

ML PM�HalfͲday�Tutorial�11/11/2013�1:00�PM�

�����

"The Developer's Guide to Test Automation"

���

Presented by:

Dale Emery, DHE George Dinwiddie, iDIA Computing, LLC

���������

Brought�to�you�by:��

��

340�Corporate�Way,�Suite�300,�Orange�Park,�FL�32073�888Ͳ268Ͳ8770�ͼ�904Ͳ278Ͳ0524�ͼ�sqeinfo@sqe.com�ͼ�www.sqe.com

Dale Emery DHE

Since 1980, Dale Emery has worked in both IT organizations and software product development

companies as a developer, manager, process steward, trainer, and consultant. He helps people apply the agile values of communication, feedback, simplicity, courage, and respect to software development. Dale's combination of deep technical expertise and extensive organizational development experience makes him particularly effective in working with software teams. In 2007 Dale received the Ward Cunningham Gentle Voice of Reason Award, which the Agile Alliance created to recognize Dale’s unique contribution to the agile community. Dale's personal mission is to help people create joy, value, and meaning in their work. Learn more about Dale at dhemery.com

George Dinwiddie iDIA Computing, LLC

George Dinwiddie is an independent software development consultant who helps organizations, large and small, increase the effectiveness of their software development efforts. As a coach, George he provides guidance over a broad range, at the organizational, process, team, interpersonal, and technical levels. As a trainer, he offers experiential education in technical practices and agile methods. George is currently crusading to break down the barriers that hinder effective collaboration between the business, the programmers, and the testers. George is a frequent speaker at agile conferences and contributor to online publications. Learn more about his work at his company website idiacomputing.com and read his blog.

The Developer’s Guideto

Test Automation

Dale Emery@dhemeryhttp://dhemery.com

George Dinwiddie@gdinwiddie

iDIA Computing, LLChttp://blog.gdinwiddie.com

http://idiacomputing.com

1

HELP!These tests are too slow!

These tests are flaky!

There are too many changes!

What is this test testing?

What does this failure mean?

These tests don’t prevent bugs!

What tools should I use?2

What Makes TestsToo Slow?

Avoidable delays- Avoidable use of slow interface- Avoidable use of unnecessary technology- Pessimistic fixed waits

Run-on tests

Large setup

The tests don’t pull their weight

3

“Avoidable” DelaysBrowserDriver

SauceLabs

Browser

Web GUI

LegacySystem

Database

3rd PartyWeb Service

WebServer

System Under Test

Test

4

What Makes TestsFlaky?

Depends on variables not under test control

- Database contents, clock time, calendar, ...

- Interference from other tests

Asynchrony

- Variable response times

- Race conditions, threading

Identifiers overly restrictive or permissive

Untested test helpers5

What Causes“Too Many” Changes?

Changes in requirements

Changes in system implementation

Changes in execution environment

Changes in test tools and libraries

Growth of test code

Test code that is hard to change

Test automation as a separate activity6

What Makes Test CodeHard To Change?

Difficult to understand

- Cryptic code

- Large blocks of code

Details in inappropriate places

Duplication

Coupling

Poorly organized7

How To Make Test CodeEasy To Change

Write small, understandable blocks of code

Eliminate duplication

Name every important idea

Hide incidental details- System implementation details- Test implementation details- Interface between test and system

Put code where you will look for it8

What Is This Test Testing?Essential details are hidden

- Implicit data and conditions

- Essence obscured by incidental details

Test does not describe its intentions clearly

- Vague terminology

- Missing abstractions

- Key ideas left unnamed

- Inaccurate description

Run-on tests9

What Does This Failure Mean?

Omits useful information

Swamped with distracting information

Misleading or irrelevant message

Delays between problem and discovery

Implicit or incidental dependencies

Failures from test environment and tools

10

What Do Our Tests Not Do?

Test everything

Test the things we’re changing

Find all unintended effects of our changes

Let us know what’s not tested

Test the things we care most about

Test the things our customers care about

Test the things they’re testing11

What TestsDo We Need?

Tests for important system responsibilities

Tests for areas that change frequently

Tests for changes that might be error prone

12

What Can TestsDo For Us?

Describe the system’s responsibilities

Demonstrate what the system does

Detect unintended changes in behavior

Indicate when a feature is done

Expose defects

Home in on the default

Illuminate which parts of the system are tested and which are not 13

Criteria For Choosing ToolsSupports writing examples before implementation

Supports expressing examples “naturally”

Fits development tools and languages

Can keep examples under version control

Supports traceability

Strong community support

No license fees

No limit of concurrent users 14

Common WaysTo Express ExamplesGherkin: Given / When / Then

Tables

- Each row is a scenario

- Each row is a set of data

Keywords

Arbitrary text with no predefined structure

15

Some Tools That Use Gherkin

• Cucumber (Ruby, Java, JVM languages)

• SpecFlow (.NET)

• Behat (Python)

16

Example Toolstack For Testing Web Applications

CucumberCucumber

Capybara Page-object

Selenium WebDriverSelenium WebDriver

BrowserBrowser

17

Example Toolstacks For Testing Mobile Applications

CucumberCucumber

AppiumAppium

iOS / AndroidSimulator

iOS / AndroidDevice

CucumberCucumber

FrankFrank

iOSSimulator

iOSDevice

JUnitJUnit

VictorVictor

iOSSimulator

iOSDevice

18