test strategy
DESCRIPTION
Presented on Poznań University of Technology where I had a lecture for students how to choose test approach.TRANSCRIPT
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Test Strategy
Zbyszek Moćkun
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
1. V Model
2. Agile Way
3. Test Planing
4. Control & Execution
5. Metrics
6. Demo
Agenda
Client/partner logo
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Perfect world
V Model
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
V Model
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Preventive or reactive
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Test Pyramid
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
Validation Verification
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
New way?
Agile
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Agile Manifesto
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
Source: blog.twg.ca
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Waterfall
Agile against waterfall
Coding, new functionality
Bug fixing
Testing
Iteration
Requirements
Design
Coding
Testing
Release
Release
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Agile ways
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
Processes Methodologies
XP
Scrum
Kanban
TDD
BDD
ATDD
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Continuous Integration
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
Source: http://wiki.mxunit.org
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Reactive approach?
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Scripting issues
• Scripted testing report
• Not all founded issues are against test case
• Test case number means nothing (James Bach)
• New test scripts added almost each day
• Old onces are updated
phase All Executed passed failed
Test design phase 80 0 0 0
First day 90 (4 updated) 20 16 4
Second day 95 (10) 75 65 10
Third day 120 (14) 75 73 2
Fourth day 125 (16) 110 98 12
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Test Planning
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
What am I doing here?
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
So
urce: h
ttp://a
rboro
sa.org
/tag
/test-p
lan
nin
g/
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Test Strategy / Test Plan
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
Source: http://technet.m
icrosoft.com
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Control and Execution
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Coverage
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
Requirements
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Execution
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
Test Suite
Test Run
Test Case status
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Traceability
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
Source: http://www.tuleap.com
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Entry and Exit Criteria
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Reports
• Coverage
• Do not duplicate tests during regression
• Test specific functionality
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Regression
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Retest or regression?
Source: http://www.softwaretestinghelp.com
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Automation
Source: http://blog.jugnoo.com
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Find what you need
Metrics
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Compare and improve
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Q&A
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Poznań Testing and Quality Group
PTaQ
© 2010 Cognifide Limited. In commercial confidence only.© 2010 Cognifide Limited. In commercial confidence only.
Demo
Test Case Management