the speed to cool: agile testing & building quality in
DESCRIPTION
TRANSCRIPT
Speed to Cool:
Agile Testing &
Building Quality In
Craig Smith
Welcome…Image: © Gracie Films / 20th Century Fox Television http://www.paleodietrecipes.org.uk/paleo-images/content/homer-confession.jpg
Image: Pineapples101 (Flickr) http://www.flickr.com/photos/pineapples101/4292778673/in/set-72157623131356129/
The speed
to cool…
Image: http://brainsyndicate.files.wordpress.com/2011/05/tumblr_lkp6t8505s1qe4wjro1_400.jpg?w=300&h=355
Thumbs
up…
Cool!
Image:http://blog.vidyagamez.com/wp-content/uploads/2010/06/tumbleweed.jpg
Choke…
How Management
See Testing
Image: http://www.craigsfire.com/wp-content/uploads/2010/02/pool.jpg
How Management
Would Like To See
TestingImage: http://us.123rf.com/400wm/400/400/dotshock/dotshock1012/dotshock101200006/8437158-internet-network-server-room-with-computers-racks-and-digital-receiver-for-digital-tv.jpg
Image: http://scitascienda.files.wordpress.com/2013/04/barney-stinson-quote-when-i-get-sad-i-stop-being-sad-and-be-awesome-instead-true-story.jpg?w=500&h=353
Image: http://rnstechnologies.com/content_images/small_image_2.jpg/
So, What is
a Tester?
Look left…
Look right…
Image: http://3.bp.blogspot.com/_RdF7S0TNAHY/TOqn8MLYajI/AAAAAAAAAJI/QDBFdixiq0E/s1600/2006-06-17%252520Falling-piano.gif
Look
up…
Image: © Jerry Bruckheimer Television http://images2.fanpop.com/images/photos/4700000/CSI-Las-Vegas-9x18-Mascara-csi-4772098-2000-1333.jpg
CSI:
Software
Image: http://3.bp.blogspot.com/_ZdyzysU1QPo/TA0kjLMP43I/AAAAAAAABPk/5ckzpw6OVmk/s1600/Peas+and+Carrots.JPG
Agile &
Testing
To Do Analysis Develop Test Done
We reflect work like this…
To Do
Analysis
Develop
Test
Done
Which is really
working like this…
Image: Craig Smith
Remind you of
something?
Let’s
ponder
the Agile
Manifesto
Responding to
change
over
following a
plan
Image: © Sony Computer Entertainment Europe http://www.ultimateps3.fr/images/jeux/buzz-quiz-tv/buzz-quiz-tv_1216759203.jpg
Time to play the “Agile
Manifesto Tester’s Challenge”
Image: © Bright/Kauffman/Crane Productions http://miacannedy.files.wordpress.com/2012/11/friends-couch.jpg
Individuals and interactions over
processes and toolsChallenge
#1
Image: http://blog.agafonov.net.ua/image.axd?picture=2011%2F5%2FvNext+Visual+Studio+2011.jpg
Working software over
comprehensive documentation
Challenge
#2
Challenge
#3
Image: © Touchstone Pictures http://cdn.mymovies.ge/backdrops/ce6/4bc922b5017a3c57fe00dce6/metro-original.jpg
Customer collaboration over
contract negotiation
Image: http://resources3.news.com.au/images/2014/04/01/1226870/743135-26b0c3f6-b92a-11e3-9942-d7dbb3111cd3.jpg
Responding to change over
following a plan
Challenge
#4
Analyst /
Customer
Developer TesterAutomation
Elaboration /
SpecificationAcceptance
Criteria
High
Performing
AGILE
Project
Manager
Testers in an agile team
Developer
Tester
Analyst
More
Technical
Less
Technical
Co
de
De
sig
n
Un
it T
es
t
Au
tom
ate
d
Fu
nc
tio
na
l /
Sp
ec
iali
st
Te
sts
Ma
nu
al
Te
st
Ex
plo
rato
ry
Te
st
Re
qu
ire
me
nts
Cu
sto
me
r
Co
lla
b.
Developers need
more testing involvement
Testers need more
technical involvement
Analysts need more
testing involvement
Testers in an agile team
(noun) the standard of something as measured
against other things of a similar kind; the degree
of excellence of somethingoxforddictionaries.com
Image: http://www.naccho.org/topics/infrastructure/accreditation/images/Quality-Definition.jpg /
Definition of
Quality…
Image: http://upload.wikimedia.org/wikipedia/commons/b/b9/Steve_Jobs_Headshot_2010-CROP.jpg
Quality
means
different
things
Image: http://s3.amazonaws.com/integral-life-home/iceberg.jpg
Implicit /
Explicit
Quality
Testing
≠
Quality
Image: http://www.cupojoy.com/images/image/Quality%20Inn%20Exterior.jpg
Building
Quality
In
Quality is everyone’s
responsibilityImage: © AP http://i.telegraph.co.uk/multimedia/archive/01893/schumaker-dummies_1893704i.jpg
Everyone likes to be
seen to value qualityImage: © Reveille Productions http://blogs.ubc.ca/kdergous/files/2010/10/body-language-the-office.jpg
Image: © United Feature Syndicate http://s458.photobucket.com/albums/qq310/mrbigjohnson/Murals/?action=view¤t=NovemberPics115.jpg&newest=1
Quality from
kickoff
1234
Cost Scope Quality Time
X
XX
X
Fixed
Flexible
Success sliders
1234
Cost Scope Quality Time
X
XXX
Fixed
Flexible
Image: http://4.bp.blogspot.com/-B0hIC1RPJpA/TZjq1IcJF3I/AAAAAAAAGdE/wlJdjuLCWoE/s1600/audience.jpg
1234
Cost Scope Quality Time
X
X
X
X
Fixed
Flexible
Quality almost always at #3
Correctness
Re
lia
bil
ity
Ro
bu
stn
es
s
Av
ail
ab
ilit
y
Inte
gri
ty
Fle
xib
ilit
y
Us
ab
ilit
y
Inte
rop
era
bil
ity
Eff
icie
nc
y
Te
sta
bil
ity
Ma
inta
ina
bil
ity
Re
us
ab
ilit
y
Po
rta
bil
ity
Reliability + + + + - + +
Robustness + + + -
Availability + +
Integrity - - - - -
Flexibility + - - + + +
Usability + - -
Interoperability - + - +
Efficiency - - - - - - - -
Testability + + + + - +
Maintainability + + + - +
Reusability - - + + - + + +
Portability + - + - + - +
Trade
Offs
Quality Advocates
What does quality mean to the different roles in
the team?
Quality Taxonomy
What are quality attributes?
Quality Prioritisation
How do we know what quality attributes to
include?
Quality Tradeoff Risks
What are risks with the quality attributes we are
trading off?
Quality Measurement
How do we test and measure quality?
Success Sliders
How does quality relate to the sliders?
Quality Definition
What does quality mean?
Next StepsHow do we apply
quality to our work
1
2
3
4
5
6
7
8
Defining
Quality
Quality
Definition
Image: © 2008 Melissa McKenney http://www.kernel.org/pub/linux/kernel/people/paulmck/Confessions/Elephant_Team_03.jpg
Shared understanding of quality
Image: http://www.thechildmindingshop.co.uk/ekmps/shops/thecs/images/table-chalkboard-and-white-board-in-one-%5B2%5D-5540-p.jpg
Strategy
& Plan
Simple test
strategy &
plans
Simple
test
strategy
Image: © Sesame Workshop http://images.wikia.com/muppet/images/9/95/Sesame-english-do-you-like-it-apple_count.jpg
Reporting
& Metrics
Understand
Development,
Quality &
Release
Metrics
Report on how
testing is going
Image: http://4.bp.blogspot.com/-B0hIC1RPJpA/TZjq1IcJF3I/AAAAAAAAGdE/wlJdjuLCWoE/s1600/audience.jpg
#1
HEALTH
PROJECT
DEVELOPMENT
TESTING
USER
0
1
2
3
4
5
0
1
2
3
4
5
6
7
1 2 3 4 5 6 7 8
New
Ris
ks &
Issues R
ais
ed
Num
ber
of
Ris
ks &
Issues
IterationTEST
COVERAGE
0
5
10
15
20
25
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8
Num
ber
of
Tests
Num
ber
of
Defe
cts
Iteration
MAINTAINABILITY PERFORMANCE0
200
400
600
800
1000
1200
1400
1600
1800
4700
4800
4900
5000
5100
5200
5300
5400
5500
5600
5700
1 2 3 4 5 6 7 8
Lin
es o
f T
est C
ode
Lin
es o
f C
ode
Iteration
OVERALL
$-
$5
$10
$15
$20
$25
$30
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8
Busin
ess V
alu
e
Num
ber
of
Featu
res
Iteration
BVC
Image: http://artegami.com/wp-content/uploads/2007/08/toysrus_karate_kid.jpg
Agile
Testing
Practices
Software is a Series of LoopsImage: http://2.bp.blogspot.com/_4G2ptSlZZn0/R9nRjimkLII/AAAAAAAAAqA/tzYJjRyhLkk/s400/brekky05011049.jpg
End to end automated tests and cover 10 conditions per
test, manual covers 1 condition per test
Automated tests cover all high priority business flows,
manual tests mostly cover lower priority flows
Metrics need context!
% Regression test cases automated
Automated Manual
How Much
Automated
Testing Is
Enough?
Testing is more than a series
of merely repeatable actionsImage: http://cdn.themis-media.com/media/global/images/library/deriv/71/71335.jpg
Image: © The Curiosity Company / 20th Century Fox Television http://images2.fanpop.com/images/photos/3200000/fry-bender-futurama-3295254-1024-768.jpg
Low value vs high value
Testing Craftsmanship
Image: © Wild Dancer Productions / Touchstone Television http://www.tvacres.com/images/props_tool_time_binford.jpg
Red
GreenRefactor
Red
GreenRefactor
ATDD
ATDD
TDD
Traditionally Test via the UIImage: http://stovallorganizing.com/wp-content/uploads/2013/04/Turtleing.jpg
Manual Exploratory
Collaboratively built around
system behaviour
Developer built optimised for fast
feedback
Co
nfi
de
nc
e
Sp
ee
d /
Fe
ed
ba
ck
Exercises components and
systems
Investment Profile
Image: http://concordion.org/image/example/online-shop/AcceptanceTest.png
Specification
by Example
Image: © Gracie Films / 20th Century Fox Television http://www.widewallpapers.net/mod/cartoons/simpsons/simpsons-1920x1200-021.jpg
Quality
Maturity
Agile
Quality
Practices
Team Self
Assessment
Image: http://4.bp.blogspot.com/-B0hIC1RPJpA/TZjq1IcJF3I/AAAAAAAAGdE/wlJdjuLCWoE/s1600/audience.jpg
No agreed approach to testing
Tests are not considered a valuable asset
An agreed approach to testing that is aligned to a definition of quality
Tests are considered a valuable asset that can be reused as features
are progressively implemented
Test strategy is agreed and understood by the team and includes
targets (coverage, defects, …)
Test strategy is documented and visible as a BVC used by the team
Non-functional requirements are agreed and included in the test
strategy
Test planning is based on risk (technical complexity and business
impact)
Test artifacts are treated with the same importance as code and are
properly maintained and refactored
The relevant ____ility and security tests form part of the test strategy
-1
1
2
3
4
Test
Planning
Image: © Orion Pictures http://sadair.com/wp-content/uploads/2010/07/Robocop_Final012-1024x768.jpg
ROBOCOP: THE FUTURE OF LAW ENFORCEMENT ROBOCOP: THE FUTURE OF LAW ENF
RCEMENT ROBOCOP: THE FUTURE OF LAW ENFORCEMENT ROBOCOP: THE FUTURE O
F LAW ENFORCEMENT ROBOCOP: THE FUTURE OF LAW ENFORCEMENT ROBOCOP: TH Empowering
Testing
Automation:
I work with developers to
automate tests that provide
business value and identify
system risks
Collaboration:
I work with analysts and SMEs to
ensure that testable acceptance
criteria are created for all stories
Strategy & Planning:
I am involved in the project at all
stages to ensure that testing
provides the greatest value and
quality objectives are achieved
Tools & Techniques:
I use both manual and
automated techniques using the
preferred testing tools based on
the situation
Architecture:
I have an understanding of the system
architecture and can create tests that
verify individual components and the
system as a whole
Development:
I take an interest in
development practices and
monitor code quality metrics
Estimating:
I am involved in developing
estimates for projects at a story
level, for tools and infrastructure
and also for deployment/release
activities
Reporting:
I provide metrics that give
insight into project health and
system quality
Agile:
I understand Agile project delivery
and the differences between testing
in the different phases of an phase
Qualifications & Training:
I have recognised qualifications in
testing and continually update and
maintain my skills
Recruitment & Development:
I am attracted to the organisation
because testing is a cool career path
that offers heaps of opportunities and a
way of continually developing my skills
Community & Teams:
I am part of a community of
testers that are embedded
within teams but share
common values
Future Tester
Image: http://houndbound.com/dogblog/wp-content/uploads/2009/09/pug_graduate1.jpg
Testing
Skills
Agile TestingCommand
Line InterfaceContinuous Integration
Version Control
Build ToolsCapture Replay
(Selenium)HTML
Test Maintenance
& Data
Specification By Example / ATDD / BDD
Concordion / Cucumber
SQL Web Services
Test Automation
Course
New Software
Test Engineer
Role
New Graduates,
Recruits, Consultants
& Partners
BA
SME
Other
Up
Skill
As
se
ss
me
nt
Current Test Analysts
Capable + Current Skills +
Desire
Capable + Desire
Not Capable + No Desire
Not Capable + Desire
**Assessment = Technical Test + Interview**
Assist & Assess
Image: http://eventstrategysolutions.com/wp-content/uploads/2010/10/early-bird.jpg
Get
Involved
Early
Image: http://blogs.jcvi.org/wp-content/uploads/2009/11/emperor-queue.jpg
Queue
to Join
Image: http://1.bp.blogspot.com/-pKB3xu6zOc8/TWU1W_4OEqI/AAAAAAAAAzk/ygEOBNumv-Q/s400/nespresso-clooney.jpg
Speed to Cool
Continuous Integration
Watch Code
Build / Compile
Run Tests /
Analysis
Publish Results
Stop The Line!
Failure?
Continuous Delivery
Image: http://www.thoughtworks.com/imgs/drawing-cd-header.png
#notestingImage: http://3.bp.blogspot.com/-jy4Sc0m-oTA/UU9qrGJw56I/AAAAAAAAAC8/zSlPJS9Ed7c/s200/No+testing.JPG
Image: http://www.hackerchick.com/wp-content/uploads/2012/01/2012-01-Kanban-is-the-new-Scrum.jpg
http://media.tumblr.com/8b6564487c4ee9bc0698fa880d183fe9/tumblr_inline_moky6riBro1qz4rgp.jpg
Iterative was
only the
beginning!
Next
???
Wrap Up
Image: © Chuck Lorre Productions http://24.media.tumblr.com/tumblr_ksvfroko3J1qa88nco1_500.jpg
Do you have a testing skillset
(or are you just doing testing?)Image: © Jerry Bruckheimer Television http://photos.telestrekoza.com/var/resizes/Current_shows/CSI/Stills/CSI-Trilogy-Crossover/Miami_8x07/CSI-Miami-Episode-8-07-Bone-Voyage-Promotional-Photos-csi-miami-8507515-1999-1315.jpg
Does everybody understand
their role in quality?Image: http://www.swotti.com/tmp/swotti/cacheDMLSBGFNZSBWZW9WBGU=/imgVILLAGE%20PEOPLE1.jpg
Are you building quality in from
the start?Image: © HiT Entertainment / Hot Animation http://1.bp.blogspot.com/-iLEyvBHn-W0/TglUWW5Or0I/AAAAAAAAAds/HXtPDGPLghA/s1600/bob+the+builder.jpg
Are you measuring testing &
assessing quality?Image: © Talent Associates http://confoundedinterest.files.wordpress.com/2011/06/maxwellsmart_missed_it_by_that_much.jpg?w=600&h=423
Are you hunting down best
practices?Image: © King Features Syndicate / Lee Falk http://www.comicwallpapers.co.uk/images/main/Miscellaneous/Other/1600900/PhantomGen_1010.jpg
Are you or your testers
empowered?Image: © DC Comics http://collider.com/wp-content/uploads/superman_alex_ross2.jpg
The world around us is getting
faster (A… is just the start)
Image: https://drawception.com/pub/panels/2012/5-13/ATX2Tj5p9f-12.png
Image: © Miller-Milkis Productions http://imagetime.co.gp/key/henry%20winkler%20ron%20howard#
Testing is cool…
Image: © Universal Television http://30.media.tumblr.com/tumblr_l6g4d631K31qzp4mwo1_r1_500.jpg
…but I love it when delivery and
quality come together!
