defining quality with the help of your customer• iso 9126 manufacturing based: • good...

1

Defining Quality With The

Help Of Your Customer

Isabel Evans

Testing Solutions Group

EuroSTAR

December 2005

©TSG

2

Quality and Customers

or The Blind Men Meet The Quality Elephant:

How What They See Affects Testers

Isabel Evans

Testing Solutions Group Limited

6th Floor, Walbrook House, 23-29 Walbrook

London EC4N 8BT

email: [email protected]

http://www.testing-solutions.com

tel: +44(0)207 623 2012

fax: +44(0)207 929 0883

EuroSTAR

December 2005

©TSG

3

A number of disciples went to the Buddha and said,

"Sir, there are living here in Savatthi many wandering hermits and scholars who indulge in constant dispute, some saying that the world is infinite and eternal and others that it is finite and not eternal, some saying that the soul dies with the body and others that it lives on forever, and so forth. What, Sir, would you say concerning

them?"

So the exalted one told them the story of the blind men and the elephant…

http://www.cs.princeton.edu/~rywang/berkeley/258/parable.html

EuroSTAR

December 2005

©TSG

4

…When the blind men met the elephant they each felt a different part of it…… what sort of thing is

an elephant?...

The men who were presented with the head answered, 'Sire, an elephant is like a pot.' And the men who had observed the ear replied, 'An

elephant is like a winnowing basket.' Those who had been presented with a tusk said it was a ploughshare. Those who knew only the trunk

said it was a plough; others said the body was a granary; the foot, a pillar; the back, a mortar; the

tail, a pestle, the tuft of the tail, a brush.

EuroSTAR

December 2005

©TSG

5

The exalted one continued…"Then they began to

quarrel, shouting, 'Yes it is!' 'No, it is not!' 'An

elephant is not that!' 'Yes, it's like that!' and so

on, till they came to blows over the matter.

"Just so are these preachers and scholars holding

various views blind and unseeing.... In their

ignorance they are by nature quarrelsome,

wrangling, and disputatious, each maintaining

reality is thus and thus."

EuroSTAR

December 2005

©TSG

6

Then the Exalted One rendered this meaning by

uttering this verse of uplift,

O how they cling and wrangle, some who claim

For preacher and monk the honored name!

For, quarreling, each to his view they cling.

Such folk see only one side of a thing.

Jainism and Buddhism. Udana 68-69:

Parable of the Blind Men and the Elephant

EuroSTAR

December 2005

©TSG

7

Quality is an elephant

• The blind men knew the elephant - it was there, it was big, and they all knew it in different ways

• Quality – it’s big, it’s scary, it’s in the room…And everyone experiences it in different ways…

• We waste time arguing about it - your elephant is different to mine…

• Sometimes we testers argue with each other instead of listening… and we spend more time on our own disputes instead of listening to what our customers need.

EuroSTAR

December 2005

©TSG

8

So what’s the problem?

• We testers concentrate on measuring such items as defect density and product attributes.

• We do this to use testing to help mitigate risk

• We are passionate about testing and about improving testing

• But we have different ideas about how to do it...

• We believe we are working towards providing better solutions for our customers yet they reject our efforts. Why is this?

• Sometimes… testing can increase risk; for the test team, for the project and for the customer organisation

• For example, if we pursue our own view of quality and argue about what we mean we may increase risk.

EuroSTAR

December 2005

©TSG

9

Five groups make up the team

8

EuroSTAR

December 2005

©TSG

10

IT definitions of quality

Product based:

• quality is based on a well-

defined set of software

quality attributes

• these attributes must be

measured in an objective

and quantitative way

• ISO 9126

Manufacturing based:

• good specification,

design and construction

processes

• software product in

conformance with the

original requirements

• quality is based on

inspection and analysis of

faults and failures in

products

See van Veenendaal “Testing Practitioner”

EuroSTAR

December 2005

©TSG

11

Business definitions of quality

User based definition:

• quality is fitness for use

• software quality should

be determined by the

user(s) of a product in a

specific business

situation

• quality can have many

subjective aspects

Value based definition:

• software quality should

always be determined by

means of trade-offs

between time, effort and

cost aspects

• needs communication

with all parties involved,

e.g. sponsors, customers,

developers and

producers


EuroSTAR

December 2005

©TSG

12

A nebulous definition of quality...

Transcendent definition:

• esoteric definition

• quality can, in principle, be recognised easily

• depends on perceptions and feelings of an individual or

group of individuals towards a type of software product

• least operational but should not be neglected in practice

• Recognise this? – it is the killer!

• Are people happy with…?


EuroSTAR

December 2005

©TSG

13

Aren’t we right?

• “After all, as testers, we are the experts on quality and risk…”

• Mistake! – we may only see our own view!

• If testers focus on one definition (our own) we may not communicate well with others in the organization who use a different definition.

• Testers need to listen and understand the quality viewpoints of others, if we are to deliver what is required.

• In the end the risks we take with testing are risks that affect our customer, our colleagues and ourselves.

EuroSTAR

December 2005

©TSG

14

Our audiences and us…

What goes wrong?

Why don’t people listen…?

Berne “Games People Play”

Wagner “Transactional Manager”

Lee Copeland SIGiST talk 1999

Grove Consultants EuroSTAR play 2001

Personal and

organisational

history

Repeating

communication

problems

Games and scripts

Rescuers, persecutors, victims

Co-dependent behaviour

Transactional Analysis

EuroSTAR

December 2005

©TSG

15

Crossed and uncrossed

communication

CP

NP

A

NC

CC RC

LP

CP

NP

A

NC

CC RC

LP

We think we exchange facts...

but actually we exchange … what?

Crossing can change the response state


EuroSTAR

December 2005

©TSG

16

Our colleagues and us...

FactsEmotion

Intuition

Judgement

Pessimism

Logical

Positive

Creative

Alternatives

Overview

of process

Jens Pas EuroSTAR 2001 workshop

De Bono: Six Hats

Black Red White

Yellow Green Blue

EuroSTAR

December 2005

©TSG

17

How can we get closer to

understanding other people’s elephants?

• Our testing may enable or prevent our customer

meeting goals, fulfilling strategy, or even

achieving proper governance.

• If we examine the framework for excellence

used by our customer organisations we will often

find ways to measure our success in relation to

their corporate goals and strategy.

• This enables us to understand how our work

helps the customers’ strategy for quality and

governance.

EuroSTAR

December 2005

©TSG

19

Kaplan & Norton

Balanced Business

Scorecard adapted in

ASWQTT for quality

Evans “Achieving Software

Quality Through Teamwork”18

EuroSTAR

December 2005

©TSG

20

Exploratory v scripted testing

The exalted one continued…"Then they began to

quarrel, shouting, 'Yes it is!' 'No, it is not!' 'An

elephant is not that!' 'Yes, it's like that!' and so

on, till they came to blows over the matter.

"Just so are these preachers and scholars holding

various views blind and unseeing.... In their

ignorance they are by nature quarrelsome,

wrangling, and disputatious, each maintaining

reality is thus and thus."

EuroSTAR

December 2005

©TSG

21

Then the Exalted One rendered this meaning by

uttering this verse of uplift,

O how they cling and wrangle, some who claim

For preacher and monk the honored name!

For, quarreling, each to his view they cling.

Such folk see only one side of a thing.

Jainism and Buddhism. Udana 68-69:

Parable of the Blind Men and the Elephant

EuroSTAR

December 2005

©TSG

22

Just remember..

• Is it wise to join in the procession of “the blind leading the blind”?... It depends …Seehttp://www.guardian.co.uk/online/news/0,12597,1466742,00.html for some accessibility news…

• And http://gallery.euroweb.hu/html/b/bruegel/pieter_e/painting/parable.html for BRUEGEL’s painting

• In the Land of the Blind the one-eyed man is king…

http://www.guardian.co.uk/online/news/0,12597,1466742,00.html


http://gallery.euroweb.hu/html/b/bruegel/pieter_e/painting/parable.html


EuroSTAR

December 2005

©TSG

23

The elephant is everywhere!

• American poet John Godfrey Saxe (1816-1887) based his poem on a fable which was told in India many years ago.

• It is quoted as a tale from Hindu, Jain and Buddhist texts

• See http://www.anekant.org/The%205%20Blind%20Men%20and%20the%20Elephant.htm for an animation

http://www.anekant.org/The 5 Blind Men and the Elephant.htm


EuroSTAR

December 2005

©TSG

24

Referencesvan Veenendaal “Testing Practitioner”

Berne “Games People Play”


De Bono “Six Hats”

Kaplan & Norton “Balanced Business Scorecard”

Evans “Achieving Software Quality Through Teamwork”

Schmaltz “The Blind Men and the Elephant: Mastering Project Work”

http://www.anekant.org/The%205%20Blind%20Men%20and%20the%20Elephant.htm




http://www.anekant.org/The%205%20Blind%20Men%20and%20the%20Elephant.htm

Lee Copeland SIGiST talk 1999

Grove Consultants EuroSTAR play 2001

Jens Pas EuroSTAR 2001 workshop






defining quality with the help of your customer• iso 9126 manufacturing based: • good...

Documents