W12

Special Topics

10/15/2014 1:45:00 PM

Test Improvement in Our Rapidly

Changing World

Presented by:

Martin Pol

Polteq Testing Services BV

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Martin Pol

Polteq Testing Services BV Martin Pol has played a significant role in raising the awareness and improving the performance of testing worldwide. Martin is experienced in managing testing processes and implementing and improving structured testing in many organizations. He was responsible for creating the structured testing approach TMap® (Test Management Approach) and TPI® (Test Process Improvement), which have both become world standards. As the architect of the method, Martin recently coauthored Testing Cloud Services. A highly regarded presenter at conferences and training sessions on five continents, he remains active in the practice of testing every day.

Martin Pol

Test Improvement in Our Rapidly Changing World

“Continuous Test Improvement for any Situation”

Continuous Test Improvement

for any Situation

Improvement of Improvement

The Evolution

Then Today

S

tru

ctu

re

Pioneering

Structuring

Optimizing

Unstructured

Undisciplined

Flexible

Structured

Disciplined

Inflexible

The Evolution of Testing

Prototyping

Waterfall Agile/Scrum

V-model Spiral model DEVOPS

Iterative RAD TDD ATDD BDD

W-model XP Continuous Integration/Delivery

MBT

Exploratory CDT RST RSTM

TMap ISEB ISTQB TMap-Next

Packages SOA Devices Social media

Virtualization Cloud Mobile Internet of Things

Web Localization Big Data

The Evolution of IT and Testing

SPI and TPI

Software

Process

Improvement Quality

Policy

Test

Process

SPI and TPI

Software

Process

Improvement Quality

Policy

Test

Process

Improvement

SPI and TPI Initiatives

ISO IEEE MIL-STD CMM PSP TSP TI4Agile CMMI TMM TMMI

Juran SEI BS 5750 TSM STEP TPI V2M2 TI4Auto CTPI TPI-Next

TQC Deming IDEAL ISO9000-9126 TIM TOM STEP CTP Cloutest

Models for TPI

• SQR

• TOM

• V2M2

• CTP

• TI4Agile

• TPI (Next)

• STEP

• TSM

• TIM

• TI4Auto

• TMMi

• CTPI

• TPS

• CMMI

How are we doing?

• High scores

– … but does it really help?

• Great improvement plans

– … but poor results

• Solid improvement policy

– … low acceptance

• Ivory tower

– … resistance

Improvement Center

One target, one route, one measure, …

One size fits all?

The best score?

Current TPI-models don’t match

Prototyping

Waterfall Agile/Scrum

V-model Spiral model DEVOPS

Iterative RAD TDD ATDD BDD

W-model XP Continuous Integration/Delivery

MBT

Exploratory CDT RST RSTM

TMap ISEB ISTQB TMap-Next

Packages SOA Devices Social media

Virtualization Cloud Mobile Internet of Things

Web Localization Big Data

More suitable improvement approach required

• Situational

• Continuous

• Joint effort

• Change proof

• ……

Fast

Flexible

Lean

Effective

Continuous

Test Improvement

for any Situation

What helps?

• Learning from the past

• Using successful “recent” innovations

– Agile

– Scrum

– Kanban

– Exploratory

– Context Driven

– Collaborating

– Continuous

• Good practices

Test Improvement at two levels

Improvement backlog

Assessment tasks Improvement sprints

Improvement tasks

Continuous Improvement

Test Improvement Intake (Objective – Scope – Approach

matching) Assessment

Continuous Improvement

‘Release’

Test Improvement using Scrum

Test Improvement Intake (Objective – Scope – Approach

matching) Assessment

Continuous Improvement

‘Release’

Improvement sprints

Improvement backlog

Improvement Stories

Test Improvement using Scrum

Test Improvement Intake (Objective – Scope – Approach

matching) Assessment

Continuous Improvement

‘Release’

Improvement Backlog

Sprint Backlog

2 - x weeks

Sprint DoD

Improvement increment

Scrum

DoD

Improvement Stories

Continuous and Change Friendly

Integrating with Business As Usual

Collaborating

Tearing down the Yvory Tower

Initiating

Acting

Establishing

Learning

Diagnosing

Using IDEAL?

Initiating

Acting

Establishing

Learning

Diagnosing

Using IDEAL?

Improvement Backlog

Sprint Backlog

2- X weeks

Sprint DoD

Improvement increment

Scrum

DoD

Test

Imp

rove

men

t In

take

(O

bje

ctiv

e –

Sco

pe –

Ap

pro

ach

mat

chin

g)

Ass

essm

ent

Co

nti

nu

ou

s Im

pro

vem

ent

‘Relea

se’

Test Improvement Intake (Objective – Scope – Approach matching)

Assessment Continuous

Improvement ‘Release’

Improvement

backlog

Assessment tasks

Improvement sprints Improvement tasks

Continuous Improvement

Roles

Improvement

Architect

Improvement

Owner

Scrum

Master

Team members

Architect level

Test Improvement Intake (Objective – Scope – Approach

matching) Assessment

Continuous Improvement

‘Release’

Test Improvement Intake

• Objectives

– Time

– Costs

– Quality (product)

– Quality (testing)

– …………

• Scope

– Area of consideration

Test Improvement Intake (Objective – Scope – Approach

matching)

Test Improvement Intake

• Stimulus for Change

• Objectives

– Time

– Costs

– Quality (product)

– Quality (testing)

– …………

• Scope

• Approach matching

– Context

– Models

– Selection

Prototyping

Waterfall Agile/Scrum

V-model Spiral model DEVOPS

Iterative RAD TDD ATDD BDD

W-model XP Continuous Integration/Delivery

MBT

Exploratory CDT RST RSTM

TMap ISEB ISTQB TMap-Next

Packages SOA Devices Social media

Virtualization Cloud Mobile Internet of Things

Web Localization Big Data

Context

Models

– TPI Next

– TMMI

– STEP

– CTP

– GQM

– ……

– TI4Agile

– TI4Automation

– CTPI

– SFAI-web

– Belbin

– ……

Bound

Tailor-made

Models

Unbound

Models

Unbound

– Experienced based

– Heuristic

• Questioning

• Discussing

– Implicit knowledge

– Exploring

Using heuristics, etc.

How do you feel?

Ever felt this before?

How about your work?

Doing sports?

How is your family?

In the morning?

When was that?

Need to walk?

At what level?

Simular complaints?

Before breakfast?

…

…

…

…

Heuristics for Test Improvement

• Observation

• Image of testing

• History, earlier improvement attempts

• Discuss potential solutions

Models

Unbound

– Experienced based

– Heuristic

• Questioning

• Discussing

– Implicit knowledge

– Exploring

– Idea raising sessions

– Out-of-the-Box

Idea raising session

• Organize a meeting

(lean, informal, 3-10 people, < 2 hours, coffee, post-it’s)

• Introduction

– Define objectives

– Set the right focus

• Gather ideas

– Individual “brain dump”

– Use post-it’s

– Explain, explore, question, discuss

– Cluster and group ideas, …

– Use Mind-mapping

• Prioritize according to objectives

– E.g. term, costs, impact

• List results, actions, epics, stories, ……

– Who, what, why, …

Advantages

– Efficient

– 100% focus

– Invented here!

– Morale

– Buy-in

Models

Unbound

– Experienced based

– Heuristic

• Questioning

• Discussing

– Implicit knowledge

– Exploring

– Idea raising sessions

– Out-of-the-Box

– Mind mapping

– Good Practices

Models

– Experienced based

– Heuristic

• Questioning

• Discussing

– Implicit knowledge

– Exploring

– Idea raising sessions

– Out-of-the-Box

– Mind mapping

– Good Practices

Unbound

Models

– TPI Next

– TMMI

– STEP

– CTP

– GQM

– ……

– TI4Agile

– TI4Automation

– CTPI

– SFAI-web

– Belbin

– ……

– Experienced based

– Heuristic

• Questioning

• Discussing

– Implicit knowledge

– Exploring

– Idea raising sessions

– Out-of-the-Box

– Mind mapping

– Good Practices

Bound Unbound

Tailor-made

Selection

• Objectives

– T, C, Q(P), Q(T), …

• Scope/Size

• Resources

– Budget

– Time

– TI-skills

• Context

• Maturity (best guess)

• Culture

• Fast

• Flexible

• Lean

• Effective

• Situational

• Continuous

• Joint effort

• Change proof

• ……

Selected ......

Bound (Tailor-made)?

• Apply Scrum for implementation

• Translate recommendations into improvement stories

Assess according to the model, but …

Release backlog

Unbound?

– Interviews

– Heuristics

– Idea raising sessions

– Study

– Walk around

– Test along

– Attend meetings

– Feedback and react

• Apply Scrum for implementation

• Translate recommendations into improvement stories

Release backlog

Assessment

Combine whatever contributes!

– TPI Next

– TMMI

– STEP

– CTP

– GQM

– ……

– TI4Agile

– TI4Automation

– CTPI

– SFAI-web

– Belbin

– ……

– Experienced based

– Heuristic

• Questioning

• Discussing

– Implicit knowledge

– Exploring

– Idea raising sessions

– Out-of-the-Box

– Mind mapping

– Good Practices

Bound Unbound

Tailor-made

Improvement release planning

• Prioritizing in logical portions for Improvement Backlog

– Contribution value (objectives)

– Ease of implementation

– Dependency

– Mark on the horizon

Towards Sprint level

Test Improvement Intake (Objective – Scope – Approach

matching) Assessment

Continuous Improvement

‘Release’

Improvement Backlog

Sprint Backlog

2 - x weeks

Sprint DoD

Improvement increment

Scrum

DoD

Improvement Stories

Conclusion

• Improvement of improvement is required

• The situation sets the improvement approach

• Improvement integrates with BAU using Scrum

Test Improvement Intake (Objective – Scope – Approach matching)

Assessment Continuous

Improvement ‘Release’

Improvement backlog

Assessment tasks

Improvement sprints Improvement tasks

Continuous Improvement