agile testing.pdf

30
Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 1 Agile Testing Elisabeth Hendrickson Quality Tree Software, Inc. www.qualitytree.com

Upload: janicejia

Post on 26-Dec-2015

184 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 1

Agile Testing

Elisabeth HendricksonQuality Tree Software, Inc.

www.qualitytree.com

Page 2: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 2

Poll: How Agile Is Your Testing Now?

How many of you…Would call your test practices “Agile”?Participate in defining the acceptance criteria?Routinely test incomplete software to provide feedback to your stakeholders sooner?Have adopted a lightweight documentation style?Have found that your test artifacts are relatively easy to update even when there are radical changes to the code?Have access to the source code and to the same set of tools as the programmers?Have ensured the programmers have full access to all the artifacts and tools you use?Are co-located with the project team?

Page 3: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 3

What’s “Agile?”

Agile is more than a buzzword. It is a relentless focus on providing business value, usually by adopting one or more Agile methodologies such as Scrum or XP.– See the Agile Manifesto:

http://www.agilemanifesto.org– And the Agile Alliance:

http://www.agilealliance.org

Page 4: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 4

Examples of Agile Methodologies

LeanLean manufacturing concepts applied to software development.

ScrumLightweight management framework.

CrystalLightweight set of development practices.

Extreme Programming (XP)

Rigorous set of practices designed to keep both the code and team agile.

Page 5: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 5

Agile Synthesized

• Communication and collaboration

• Visible indicators• Disciplined development

practices

• Feedback• Whole team thinking• Short iterations• Low overhead, high

productivity

Page 6: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 6

Calling It “Agile” Doesn’t Make It So

This is NOT Agile:

1. Compress the schedule2. Toss out the

documentation3. Code up to the last minute

The organization may gain short term speed but at the cost of long term pain.

Page 7: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 7

How Traditional Test Practices Evolved

Analyze Design Code Test/Bug Fix

With great optimism and the best of intentions, The Project Plan is announced:

ReleaseRequirements handed off to Dev

Completed Code handed off to Test

Page 8: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 8

How Traditional Test Practices Evolved

Analyze, Design, & Code Test/Bug Fix

Inevitably, The Project Plan is revised:

ReleaseCompleted Code handed off to Test

Page 9: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 9

The Result: Practices Intended to Control ChaosTraditional test practices attempt to manage the chaos (or at least avoid the blame):• “Last Defender of Quality” stance• Strict change management • Detailed preparation and up front planning• Heavyweight documentation suitable for outsourcing

the test effort• Strict entrance and exit criteria with signoffs• Heavyweight test automation focused on regression

Page 10: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 10

Becoming Agile: Shifting Roles

“Fear not! I’ll protect

you!”

“Hey, would this help?

from last line of defense… …to team support

Page 11: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 11

Testing Directions: Marick’s Model

Source:http://www.testing.com/cgi-bin/blog/2003/08/21#agile-testing-project-1

Business Facing

Technology Facing

Supp

ort P

rogr

amm

ing

Critique Product

Page 12: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 12

Traditional & Agile Testing Contrasted

All levels, built by anyone, an integral part of the project.

System-level, built by tool specialists, created after the code is “done.”

Automation

It’s not a relay race. Collaborate.

Formal entrance and exit criteria with signoffs.

Handoffs

Only as much as absolutely necessary.

Verbose.Documentation

Plan as you go.Comprehensive up front test design.

Planning

Accept it.Manage & control it.Change

Agile TestingTraditional Testing

Page 13: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 13

Embrace Change: Plan Tests for Maintainability

When creating test artifacts…

• Minimize duplication.Thought exercise: if a feature were removed from your software, how many test artifacts would have to be updated?

• Use tools designed for change.Hint: if the vendor says “stabilize the interface first,” run away!

Chan

gePl

annin

gDo

cume

ntatio

nHa

ndoff

sAu

tomati

on

Page 14: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 14

How Not to Plan a Test Effort

Unfortunately, this is common practice:• Download a generic, standard, formal test plan

template from the Internet.• Fill in every section in the template even if you

don’t understand it and have to make stuff up.• Kill the maximum possible number of trees by

distributing copies to every team member.The result is a large and mostly irrelevant planthat must now be maintained. Yuck.

Chan

gePl

anni

ngDo

cume

ntatio

nHa

ndoff

sAu

tomati

on

Page 15: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 15

Plan as Little as Possible

A little planning is good. More is not better.• Know who you serve (primarily).

Technology-facing or business-facing? • Plan for the current iteration.

Speculative planning means rework.• Have a strategy that fits on one page.

If it is still relevant in 6 months, it’s probably at the right level of detail.

• Keep an up-to-date, prioritized risks list.What kind of information are the testers looking for? The risks list covers it.

Chan

gePl

anni

ngDo

cume

ntatio

nHa

ndoff

sAu

tomati

on

Page 16: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 16

Favor Informal, Collaborative Tools

DatabasesGantt/PERT ChartsPolished Documents

Formal

WhiteboardsSticky NotesIndex CardsWikis

InformalChan

gePl

anni

ngDo

cume

ntatio

nHa

ndoff

sAu

tomati

on

Page 17: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 17

Monitor Documentation Costs

How Much Does Documentation Cost?Informal polling of 162 software testers from 65 companies revealed that most spend more than 33% of their time documenting tests.

How Much Time Are We Spending on Test Documentation?

0510152025303540

0 10 20 30 40 50 60 70 80 90 100

% Time

# Te

ster

s R

epor

ting

Chan

gePl

annin

gDo

cum

enta

tion

Hand

offs

Autom

ation

Page 18: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 18

Keep the Documentation Simple

• Capture the essence, not the details.Step-by-step instructions cost time without providing value (usually).

• Point to other project documents.If it’s in the user guide, requirements, specs, etc., leave it there.

• Centralize generic tests in a checklist.Try this: count the number of times you find a common condition, like invalid dates or null strings, in the test docs. More than once is too many.

Chan

gePl

annin

gDo

cum

enta

tion

Hand

offs

Autom

ation

Page 19: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 19

Watch Verbose Wording

Which would you rather maintain?

Choose the Import option from the File menu. A dialog titled “Import File” appears. Navigate to \\x\y\z\long.dat and click Open. A dialog titled “Importing…” appears with the current status, a progress bar, and a button labeled “Cancel.” Click on the Cancel button. Choose OK on the confirmation dialog. Verify that the import stops.

Or:

Start a long import. Cancel it in the middle. Verify it stops.

Chan

gePl

annin

gDo

cum

enta

tion

Hand

offs

Autom

ation

Page 20: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 20

A Lightweight ExampleCh

ange

Plan

ning

Docu

men

tatio

nHa

ndoff

sAu

tomati

on

Page 21: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 21

Do Away with the Signoff Sheet

Hint: when people joke about signing the handoff form in blood, the signoff process is more about blame than about producing good software.

Chan

gePl

annin

gDo

cume

ntatio

nHa

ndof

fsAu

tomati

on

Page 22: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 22

Integrate Testing

Testing is not a phase. It’s a way of life.• Agile teams are test infected.

The question, “How will we test it?” is as important as “How will we build it?”

• Co-locate testers and programmers. But sitting side by side does not ensure communication.

• Track testing status and programming status all together.Show tests run-passed-failed together with features/stories done and left to do.

Chan

gePl

annin

gDo

cume

ntatio

nHa

ndof

fsAu

tomati

on

Page 23: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 23

Leverage Automation Investments

Automate everything you can, but invest wisely.• Collaborate with programmers on test

infrastructure code.The programmers have already automated the unit tests. Why not reuse the investment where possible?

• Use different types of automated tests for different purposes.Automated system tests should cover end-to-end sequences. Unit tests detect unintended change. Don’t substitute one for the other.

Chan

gePl

annin

gDo

cume

ntatio

nHa

ndoff

sAu

tom

atio

n

Page 24: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 24

Automation & Exploratory Testing

Use test automation to support earlyexploratory testing.

Traditional test wisdom says we can’t start testing a feature until it’s accessible from an external interface (like a GUI). But we don’t have to wait. Test automation can facilitate manual exploration.

Chan

gePl

annin

gDo

cume

ntatio

nHa

ndoff

sAu

tom

atio

n

Page 25: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 25

Agile Testing Synthesized

• Communication and collaboration

• Whole team thinking• Low overhead, high

productivity

• Early involvement• Leveraged automation• Focus on providing rapid

feedback to key stakeholders

Page 26: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 26

Necessary Conditions for Agile Testing

1. The organization is willing to embrace agility as defined by the Agile Manifesto.Saying “Be More Agile” or “Test Faster” isn’t enough.

2. The whole team is responsible for quality, not just the testers or people with “Quality” in their title.Which are you more likely to hear: “How did you miss that bug?!?” or “How did we not catch that?”

3. Everyone tests, not just designated testers.Agile teams are “test infected.”

4. Managers focus on fixing problems, not blame. Agile practices don’t provide CYA paper trails and are unlikely to succeed in a high-blame, high-fear environment.

Page 27: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 27

Agile Testers Wanted: A Job Description

Responsibilities:• Perform manual/exploratory tests on early-stage code• Create automated acceptance tests• Advise the team about overall risks and trends• Assist the business stakeholders define acceptance criteria• Facilitate communication between the technical & business

stakeholdersQualifications:• Experience designing and executing tests• Scripting skills in at least one of the following languages:

Ruby, Perl, Python, or JavaScript. Experience programming in Java, C#, etc. a plus.

• Strong analysis skills• Ability to work in a team (bullpen) environment

Page 28: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 28

Test Teams that Embrace Agility…

• Focus on providing value to our key stakeholders (both business-facing and technology-facing)

• Shift from being the last line of defense to providing an information service.

• Aggressively reduce time and resources spent on anything that does not directly contribute to providing information.

• Collaborate with programmers to improve testability and leverage test automation efforts.

Page 29: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 29

Further Reading…

Beck, K. (1999). Extreme Programming Explained: Embrace Change. Addison-Wesley.

Cockburn, A. (2004). Crystal Clear: A Human-Powered Methodology for Small Teams. Addison-Wesley.

Crispin, L., & House, T. (2002). Testing Extreme Programming. Addison-Wesley.

Poppendieck, M. & Poppendieck, T. (2003). Lean Software Development. Addison-Wesley.

Schwaber, K. & Beedle, M. (2001). Agile Software Development with SCRUM. Prentice Hall.

Page 30: agile testing.pdf

Copyright (c) 2004 - 2005, Quality Tree Software, Inc. 30

Acknowledgements

Many thanks to early reviewers of the ideas presented here: Brian Marick, William Wake, Jonathan Kohl, Jeffrey Fredrick, Daniel Knierim, Marc Kellogg, Danny Faught, Ron Jeffries, Hubert Smits, Rob Mee, Sherry Erskine, Amy Jo Esser, Gunjan Doshi, Dave Liebreich, Janet Gregory, Chris McMahon