we did it!!? there is place for qas in agile!!?
TRANSCRIPT
• Partner
• Involved in development of core Atlassian products
• Commercial, Open-Source add-ons, customizations
• Developers, Product Managers, Designers, QAs, Agile Team Leads
• Me: JIRA QA Team Lead
QA vs Tester
NOT T
ODAY,
THAN
KS!
My story
2006
!
• My first real Agile Project
• About it later
• Heard about Agile in 2006 on ISTQB straining
• First project - 2006
• No one knew what to do
• Testers in Agile? No experiences available
• Short story
20122013
!
!
• Quality Assistance
The fear20
09
Popularity
Developer Testing Interdisciplinary Technical Skills
How Agile is your team?
“There are now many teams that label themselves as ‘agile’, but some agile teams are more agile than others”!! ! ! ! ! ! ! ! ! ! ! Stuart Reid “Are all pigs equal?” Testing Experience 2009
How Agile is your team?
• Team empowered to improve their process
• Fast feedback and adaptation
• Production ready increment as result of iteration
• Everyone is responsible for the team’s output
My take:
My first “real” Agile project• Test Manager
• Around 10 developers at the beginning, 3 testers
• From time perspective - not good
• No shared quality responsibility, bad fitting, technical debt
• No experience, no enough thinking, no guidance
TOAST FOR QUALITY
TOAST for Quality
• A model helping organise QA process inside Agile project
• Refers to whole quality process - not only QAs
T - Types and Techniques
O - Organisation!
A - Added value
S - Skills !
T - Tracking
Types and Techniques - O - A - S - T
• Which test types do you need to execute within your Agile process?
• Which software characteristics you need to cover (performance, usability)?
• What techniques will you use for those tests?
• Types:
• Functional tests, Performance, Load, Stress, Usability, …
Types and Techniques - O - A - S - T
• Techniques:
• Scripted testing, Automated tests, Exploratory testing
• White-box, Grey-box, Black-box
Feature
StoryStory
Feature
…
Task
Release
Task…
…
TO PRO
DU
CTIO
Nbut also Timing
Feature
StoryStory
Feature
…
Task
Release
Task…
…
STORY TESTING
FEATURE !REGRESSION
PRODUCT !REGRESSION
FEATURE TESTING
PRODUCT !REGRESSION
RELEASE TESTING
PRODUCT !REGRESSION
but also Timing
• How do you plan to organise testing activities?
• Quality is the responsibility of the complete team!
• What is the team setup (How many QAs? How many Developers)?
• What is the QA role in testing activities?
T - Organisation - A - S - T
T - O - Added Value - S - T
• Where do we add value beyond testing?
• Automation, Business Analysis, Acceptance Criteria, Customer Support, Customer Contact, Test Data, Test Environments?
T - O - A - Skills - T
• What skills do we need in the team?
• What do we need to do to fulfil needs?
• Where can QAs specialise?
• Are QAs technical enough?
T - O - A - S - Tracking• How do we estimate QA activities and verify those?
• How do we measure the process?
• How do we measure product quality?
• How do we get inputs for retrospective?
• …
PRACTICES
Types• Which software characteristics are crucial for you?
• Functionality, Usability, Scalability, …ility
• Depends on your context
Types, Techniques, Timing
KEEP!CALM!
ITS NOT IN!SCOPE
Agile Testing QuadrantsTypes, Techniques,
Timing
“Agile Testing: A Practical Guide for Testers and Agile Teams” Lisa Crispin and Janet Gregory
Techniques• Scripted, Exploratory, Ad-hoc, Automated
• White-box, Grey-box, Black-box
!
• Agile moves onto Exploratory, White-Box and Automated
Types, Techniques, Timing
ExploratoryTypes, Techniques,
Timing
“Telling your exploratory story” Jon Bach, Agile 2010
SBTM• Session Based Test Management
• Charters
• Sessions
• Notes
• Debriefing (PROOF)
Types, Techniques, Timing
SBTMTypes, Techniques,
Timing
Organisation
“Adding value in an agile context” !by Henrik Andersson
SBTM• Not only exp. testing technique but a whole
organisational approach!
• Every Testing activity is a Session (~90-minutes)
• Estimate number of possible sessions for every iteration (3-4 sessions a day)
• Fight on the Sprint planning to prioritise
• Take metrics e.g. Sessions burndown chart
Types, Techniques, Timing
Organisation
A word about exploratoryTypes, Techniques,
Timing
• We do not know how to describe the exploratory thinking
• Considered as ‘playing around’ without any test documentation
• Fight that!
• Think about it before encouraging your organisation
AutomationTypes, Techniques,
Timing
Continuous IntegrationUnit Tests
Services
UI
Feature
StoryStory
Feature
…
Task
Release
Task…
…
STORY TESTING
FEATURE !REGRESSION
PRODUCT !REGRESSION
FEATURE TESTING
PRODUCT !REGRESSION
RELEASE TESTING
PRODUCT !REGRESSION
Types, Techniques, Timing
Timing
• Think what you have and what you do not have!
• The testing scope activities represent risks - not all risks must be covered with testing
• … but they must be addressed!
Types, Techniques, Timing
Timing
Release every 2 weeksFEATURES STORIES
Release 2-3 times a yearFEATURES STORIES
TestingTypes, Techniques,
Timing
STORY TESTING FEATURE !REGRESSION
PRODUCT !REGRESSION
FEATURE TESTING PRODUCT !REGRESSION
RELEASE TESTING PRODUCT !REGRESSION
Story
Feature
Release
Autom. (CI) Autom. (CI)
Autom. (CI)
Autom. (CI)
Manual exploratory tests!+ Automated tests
(Optional) Blitz Test
NO
Teams• QA is part of the team (3-4 developers)
• Main responsibility - testing (moving towards exp. testing)
!
• Other responsibilities = Added value
• Everyone has ‘Developer’ role
Organisation
• Tool usage: sub-task of every story, separate testing tasks
• Part of the definition of done
• SBTM
TasksOrganisation
!
Quality is everyones responsibility!
• No satisfying answer
• Sharing partly testing with developers (checking vs. testing)
• Unit tests, TDD, Automated tests
• My answer: Quality Assistance
Organisation
Quality Assistance• 1 QA per 10 developers
• Testing in hands of developers (both manual and automatic)
• Teaches, mentors what and how to test, helps identifying most risk areas for testing
• QA does not test - has time for process improvement, experimenting, bugs analysis, test tools
Organisation
Quality Assistance • Quality is indeed everyones responsibility
• Automated tests from scratch
• Tech camp presentation:
• http://www.slideshare.net/mkujalowicz/1x10-the-role-of-qa-engineer-in-jira
• http://vimeo.com/86691004
Organisation
Keys to succeed• The team must feel QA is mandatory element of the team
and its every activity
• Get the unique strengths and use them
• support Product Owner to transfer customer needs onto backlog and acceptance criteria / automated tests
• technical debt
• test tooling and test coverage
• …
Added Value
Added Value
• Visibility of QAs work!
• “Yesterday testing, today testing”
• Daily stand-ups
• Interesting idea:
• ‘Report on your product’s health’ by Henrik Andersson
(http://tvfortesters.com/video/adding-value-in-an-agile-context-by-henrik-andersson/)
Added Value
• TDD
• ATDD
• BDD
• Specification by example
• Not my cartoon :P - maybe yours?
http://www.fanpop.com/clubs/dexters-laboratory/images/13130720/title/deedee-photo
• Teaching / mentoring
• Test tooling, test coverage
• Process improvement, bug analysis
• Customer view
• We are “linkers” and “sharers”
Added Value
Skills
Technical skillsSkills
• Basics:
• IDE, SCM situation, Reviews
• White-box, Grey-box skills
• Technologies
• Further:
• Depends on your project
Skills
• Comes out of your added values analysis
• Test techniques, Data gathering, Test Tools
• Sorry - no recipe here
!
• Go beyond what is on paper
Technical skills
Tracking
What should we track?• QA Tasks tracking
• Internal and External product quality
• Dev speed, QA speed
• Product health
• Product usage
Tracking
Do we do that?• Not much people sharing it
• Create QA activities backlog and monitor its burn-down - SBTM and Test Sessions
• Bugs and quality - we should have it covered - do we? In most projects - not trivial
Tracking
• Story rejection rate
• OD Incidents, Stable regressions
• Rework time
• SaaS delivery index = JBS * 0.5 + PBS * 0.5 + C2M * 20 + C2DEV * 10 + D8W * -10 + NI * 100
BUT…
… our projects are different• Products in production for several years - not build
from scratch
• Almost full manual regression when switching to Agile
• No automation possible/no automation in place
• Test documentation requirements
• Dislocated teams
No automation
Does not Scale
No automation
“Agile Testing” Tito Linz, Testwarez 2014
No automation
But we require test documentation!
“Telling your exploratory story” Jon Bach, Agile 2010
Test documentation necessary
• What is actually needed? What does one wants to achieve?
• Does anyone read it at all?
• Good exploratory testing charters
• Find your own way!
No automation
• You won’t win without experimenting and taking risks!
• Application specialist - interesting idea
• Going further: White-box QA
White-box QA
• Pick the most technical person
• Let her/him analyse the dependencies, how it works
• Manual regression tests recommendations
• Teach others how to do it
No automation != no work on automation
• Web Applications automation - easy
• Desktop - mature
• Mobile - growing but hard
• Embedded - self-made
!
• Start with quick wins - e.g. Visual Regression
Multiple locations
• Is it still Agile?
• No unicorns here
• Use the available tools, look for tools
• Martin Fowler: “Using an Agile Software Process with Offshore Development”
WHAT DO WE MISS
What does it mean to be Agile QA
Process Product&
Unexplainable expectation
• QA must limit the speed, QA must hold development plans, QA must make sure we do not take risks
Product risks
• What does it mean to leave this area untested?
• What does it mean to leave this bug in release?
• Gather data regarding customer usage
Process risks
• Experiment around your process
• Take a risk to have time on process improvement
• Failure is also a success
• Make sure you know what success means
Our story• QA Testing
• Testing notes
• DoTing
• QA Demo
• Dev as a QA
The train is leaving• Development specificity
changes
• QA specificity changes:
• Analytics, Experiments, A/B Testing, Continuous Deployment
• Explore new paths!
Summary• Think if you merit a TOAST? Not everything is
required but it must be a conscious decision
• QA work visibility
• Think through your Regression test situation
• Quality responsibility of the complete team - Quality Assistance
• Take risks and experiment
The fear20
09
Popularity
Developer Testing
Satisfaction!
Interdisciplinary Technical Skills
Thank you!
Images - credits• Żuraw in Gdańsk - by JM_GD - CC BY 2.0
• 3D Growth Projections - by Chris Potter - CC BY 2.0
• Architetto -- Donna casinista - by francesco_rollandin - CC BY 2.0
• Toast - by Stiefen Schlingen - CC BY 2.0
• Toasting Champagne - Waldo Jaquith - CC BY-SA 2.0
• Thinking - by Wade M - CC BY-SA 2.0
• Stadion Narodowy w Warszawie - by Artur Malinowski - CC BY 2.0
• Thumbup - Pratheepps - CC0 1.0
• Documentation - by Marta Pucciarelli - CC-BY-SA-3.0
• Difficult - Sea Turtle - CC BY-NC-ND 2.0
• Chemia Laboratorium Experyment - pixabay.com - CC0 1.0
• Comic Daredevil Bello Nock - Chris Phutully - CC BY 2.0
• Surprise major - by Achim Hering - CC-BY-SA-3.0
• Train Left - SElefant - CC-BY-SA-3.0