1

2

� Patterns in real world� Patterns, Heuristics, Lessons Learned, Meta-Patterns� Design Patterns and Test Patterns� Failure of patterns in testing� Usefulness of patterns in testing� Patterns in testing� Patterns in testing

� Data Patterns

� Test Patterns

� Defect Patterns

� Test management Patterns

� People management Patterns

� Control Patterns� What we do not know today � What we might need to create – pattern language

3

4

5

� The sum of the

numbers in any row

is equal to 2n

Source - http://ptri1.tripod.com/

6

� How do you learn your native language?

� How do you learn your second language?

7

� The brain is a pattern-mad supposing machine. Given just a little stimuli, it divines the probable. …it recognizes familiar patterns and acts with conviction.and acts with conviction.

� As much of a confection as our mental maps are, they (Patterns) allow us to speculate, rehearse and make plans.

*http://www.nytimes.com/2004/06/15/science/essay-i-sing-the-body-s-pattern-recognition-machine.html

8

� *A form or model proposed for imitation

� *Something designed or used as a model for

making things

Design pattern - general reusable solution to � Design pattern - general reusable solution to

a commonly occurring design problem

From MW Dictionary*

9

� *Involving or serving as an aid to learning, discovery, or problem-solving by experimental and trial-and-error methods

� *of or relating to exploratory problem-� *of or relating to exploratory problem-solving techniques that utilize self-educating techniques

� Something that borne out of experience that works under most situations

From MW Dictionary*

10

� Heuristic � can be said to be a solution to problem pattern.

� Has a relationship with patterns

� Recognition of a (problem) pattern may result in � Recognition of a (problem) pattern may result in a (mostly) repeatable solution – a heuristic which itself may be a pattern

� Very loosely speaking - Heuristics are mostly “doing” or noticing something that has repeated whereas Patterns are mostly “history”

11

� Lessons can be learned without recognizing

patterns but a lesson learned that WILL NOT

be applied again is not required

Lessons learned may not RECOGNIZE the � Lessons learned may not RECOGNIZE the

underlying pattern explicitly

� Lessons learned may not help understand the

whole picture if patterns are not considered

� Pattern identification is useless if lessons

can’t be learned

12

� Pattern-of-patterns

� Example - History-repeats-itself

� Example pattern - Unit Testing important at one

point of time, has gained importance againpoint of time, has gained importance again

13

� Rediscovery Meta-pattern (almost all domains)� Knowledge – (Legend/Myth/Lost Knowledge) – Rediscovery

Example � *In 1993, an Indian meterologist P.R. Pisharotyvalidated a pattern outlined in a 6th Century encyclopedia “Brahad Samhita” by Varhamihira - that encyclopedia “Brahad Samhita” by Varhamihira - that the Golden Shower tree is a uniquely accurate indicator of rain as it bears abundant bunches of golden yellow flowers just 45 days before the onset of monsoon

� The Bhil tribes of the desert predict drought by extra bushy Khair trees and the growth of wild cucumbers

* http://www.littleindia.com/news/143/ARTICLE/6904/2010-08-08.html

14

� 1970s - Christopher Alexander authored books- The Timeless Way of Building and A Pattern Language

� 1987 - Kent Beck and Ward Cunningham presented paper at OOPSLA on applying patterns to programming

� 1994 - Design Patterns: Elements of Reusable Object Oriented Software by Gamma, Helms, Johnson, and Vlissades1998 – AntiPatterns book releasedSoftware by Gamma, Helms, Johnson, and Vlissades

� 1998 – AntiPatterns book released� 1999 - Q-Pattern by yours truly� 1999 – Bob binder’s book “Testing Object-Oriented Systems:

Models, Patterns, and Tools” released� 2001 - Brian Marick and others had a series of meetings to work on

the idea (http://www.exampler.com/testing-com/test-patterns/)

� 2004 - Java Testing Patterns� A few more books and blog posts then…

silence

15

Composite Pattern

Example

� Folders and files

� Groups & members

http://en.wikipedia.org/wiki/Composite_pattern

16

� cross-checking the testing effort

� scenario testing

� Follow-up testing of minor bugs

� stop the test/fix cycle and redesign

� state based testing

input constraint attack

• blank page test plan

• smoke testing

• self-verifying data

• old bugs regression testing

• use a test harness

monkey testing� input constraint attack

� reality steam roller

� resource starvation attacks

� don't believe development

assurances

� all oracles are heuristic

� use debug mode

• monkey testing

• feature integration testing

• pair testing

• combinatorial testing

• bug fix regression testing

From - http://www.exampler.com/testing-com/test-patterns/w3/summary.html

17

� AbstractTest / AbstractTestCases� BrokenTest� CleanCheckIn� CoverTheGraph� CodeUnitTestFirst (TestDrivenDevelopment)� CrashTestDummy� CrashTestDummy� DefectSeeding� EncapsulateNewForTestability� ExpectedResult� ExploratoryTesting� FakeIthttp://c2.com/cgi/wiki?TestingPatterns

� Most patterns looked like “doing” type of patterns or suggestions or heuristics or lessons learned

18

� Nobody /very few people talk about design patterns these days -Not necessarily because it was a bad idea

� What about test patterns? Why did it fail?

� We were focusing on wrong things

We were taking a wrong approach� We were taking a wrong approach

� We were not at it long enough

� Is there a pattern here? � IMO most patterns were DOING patterns. No

“identifying” patterns here. Doing – how you do things. IDENTIFYING – Looking at information and figuring out possible problems hidden therein

19

� Lack of a pattern language � Advantages of a pattern language

� Once you have a language you can express it

� When you express it you can share it� When you express it you can share it

� Once you share it, it can be applied, improved, extended

� Disadvantage of a pattern language� Can you express everything in words?

� When you use only words and those too in a specific way, you become rigid, closed to other things

20

� Pattern recognition requires

� long term view - Ignoring the specifics and

focusing on general

� Data� Data

� Methods

� Tools

� Patience

21

� I predict that one of the future breakthrough

in testing will come through pattern

recognition

Example� Example

� Patterns of defects arising out of patterns of

errors in code/design/requirements

� Patterns of project failures arising out of data

collected during project execution

22

� Patterns exists everywhere - in defects, code,

design, processes

� Pattern identification helps predict the next

set of events in the chain enabling you to do set of events in the chain enabling you to do

risk mitigation

� Patterns enable you to distinguish the woods

from the trees

23

� Pattern based thinking – effective methods of

implementing innovation by learning from

various projects, industries and disciplines -

both related and unrelatedboth related and unrelated

� Innovation, a talking point for all and pain for

most, can be made easier by applying pattern

based thinking

24

� Data Patterns

� Test Patterns

� Defect Patterns

Test management Patterns� Test management Patterns

� People management Patterns

� Control Patterns

25

� Ross Collard listed a number of data patterns for functional and performance testing -

� Routine Live Data

� Baseline � Baseline

� Batch Volume or Parallel

� Benchmark

� Pristine

� Truncated

� Minimally Redundant etc.

26

� Some patterns listed earlier

� Q-Patterns

� Patterns waiting to be discovered…

27

� People often work on one side of boundary

while testing

� Make them work on both sides/all sides of

boundaryboundary

� Example –

� Triangle Problem negative test cases - no value, 1

value, 2 values, all 3 values

� what about the other side of the boundary - more

number of values?

28

� Patterns can help uncover new test design techniques OR new ways to look at existing test design techniques� Apply test design techniques to create tests� Apply test design techniques to create tests

� Monitor the bugs arising OUT(SIDE) of tests

� Establish the bugs that could have been found using existing techniques

� Defects not “findable” using techniques should be analyzed for patterns and a technique using which these could be found

29

Some patterns in type of bugs not found using tests created using techniques� Interaction of application with the operating

system � Reason – Design specifications hardly go to that level � Reason – Design specifications hardly go to that level

of detail and system specifications never� Defects related to interdependency of features

and code structures causing one portion of code to be bound to another piece of code� Reason – Code dependencies are either not called out

explicitly, born in either assumptions or ignorance (most of the time)

30

� A model

based

techniquetechnique

� Resembles

state

diagrams

31

Entity

Name

Contains Associatio

n

User I/P System

i/p

I/p data Input

Actors

Operations

File

System

Files, Directories - Copy

(Delete,

Add/del/

modify

items)

Memory,

failure

Source

and

destinatio

n

User,

Syste

m

View,

enumerate,

copy,

delete

items)

Director

y

*Files,

*Directories

-

File Blob of bits

occupying

location in

memory.

Path

Memory.

Name +Characters

BLOB Bits at a particular

address

32

� Extension of Noun-and-Verb Technique (http://www.puretesting.com/Noun-and-Verb-Technique.pdf)

� Q-Patterns (http://www.puretesting.com/articles/QPatternsV2.3sm.ppt)(http://www.puretesting.com/articles/QPatternsV2.3sm.ppt)

33

� Bug Taxonomy based

� Patterns of bugs-that-occur-together

� Bug-isolation patterns

34

� How many bugs do you find every month?

� How many of them are type – UI, Feature,

cross-feature, database, security…

Can you sub-classify them?� Can you sub-classify them?

� Do you know what are other testers finding?

� Do you see any patterns in

� What type of bugs you find

� What types of bugs you don’t find?

35

� If you find predictable bugs (based on your

past experience and pattern recognition) can

there be pattern recognition

for code? � for code?

� Design?

� Requirements?

36

� An example –

� When I synchronized my handheld device with the

desktop, it caused a crash when I used “ArvindM”

as the name of the profileas the name of the profile

37

� Are there testers who do good bug isolation?

� Are there those who do not?

� How do they fail? What things they do not

do?do?

� What patterns exist?

� If I know those patterns can I improve their

skills?

38

� Example

� Project Failure pattern –

Communication/delivery/no-interested-client-

stakeholderstakeholder

� Automation Outsourcing Failure – Lack of ~effort

appreciation/well-written tests/domain

knowledge

39

Date Action Issue

24-Jan-01Clear work division in overlapping areas

and ensuring communication between

the overlapping/concerned people.

Performance test cases being written by Joe and Jane.

Test were duplicated. Some missing test cases

because every body was not aware hence did not

review/contribute to the tests.

28-Jan-01Knowledge of the system (WES, WAS

performance, tuning, deployment)

should be available to the team.

Some members of the team did not know what they

had to do, others did not have required technical

knowledge

� What pattern do you see here? � This pattern emerges from long term view - The four

entries here are from a list of 133 entries between Jan and April 2001.� How is pattern recognition different from Root cause analysis?

should be available to the team. knowledge

14-Feb-01Communication of the documents,

Agenda, Minutes to the testing team

should be formal, via mail.

The release meeting with developers happened

without a copy of test plan (esp. dates) being

available.

9-Mar-01User Guides esp. “how to get started”

should always accompany a build

Swami sent a mail asking how to use the ABC app. We

had to write some steps and send him the document.

40

Action Issue

If the understanding on the code base status is

not clear instead of assuming clarify it. The

release under test should be in a code freeze

state in the final stages of testing.

While doing localization testing we had assumed that

the code from English version of the build would not

be migrated to localized builds. But that was a false

assumption. Result was that we had unstable

localized builds.

Always plan for testing of English build along

with localized builds. There are chances that

In case of ABC V3.0 (Localized) planning for English

release was not done. It has to be tested because of

� What Patterns?� Localization related?

� Planning related?

� Platform related?

� Environment related?

with localized builds. There are chances that

English build may have to be released.

release was not done. It has to be tested because of

bugs.

Testing of problems on parallel platforms should

be done when issues are found on one.

V 2.6.3 drag and drop had same problem as V 4.0. We

came to know of V 4.0 problems and should have

tested on V 2.6.3

Always plan for the hardware for all the tasks

and see the resource contentions and how to

resolve them.

When we started product testing there were setup-

sharing issues.

41

� Pattern-a few days before release some

innocuous bug gets fixed and all hell breaks

loose.

Solution - alert teams for field bugs requiring � Solution - alert teams for field bugs requiring

rapid action - web team for upload on the

website, development and test teams for fixing

and testing

� Out of three times, two times I was not rewarded

and only once rewarded

42

� Example – Patterns while Interviewing

� assessing people, forming impressions, validating,

creating and refining patterns

� Remember - you have not discovered the last � Remember - you have not discovered the last

pattern hence you may be wrong

� Keep updating your impression of patterns and be

ready for the worst

43

� Example – “Only a Guest” pattern

� when people (are about to) leave the organization

without information to the management there

are some behavioral patterns that manifest are some behavioral patterns that manifest

themselves.

� Indifference, extra-hard work, sudden forgiveness

and not picking up of battles, unplanned

absenteeism, putting all paper-work in place etc.

44

� Patterns for

� Predicting

� Inventing

� Improving

� Preventing

� …

45

� Reapplication Pattern – What happens in Development follows in Testing

� Examples� Civil Engineering - OO Design – OO Testing

� Design Patterns – Testing Patterns� Design Patterns – Testing Patterns

� Agile Development – Agile Testing

� Pair Programming – Pair Testing

� CMM/CMMi – TMM/TMMi

� Noun-and-Verb OOAD – Noun-and-Verb in Testing

� Closer, quicker, tighter feedback loops result in less wastage – apply it wherever feedback loops exist

46

� *Noun-and-verb technique was first used for

test design by Elisabeth Hendrickson and

extended by yours truly.

ALMOST independently developed � ALMOST independently developed

technique/extension (with help from the

word “noun-and-verb”)

� Can we REDISCOVER it using patterns?

* http://testobsessed.com/2005/02/21/flush-specific-stack-fiercely/

47

� Abbott’s method for design was to extract

the objects and methods from the problem

description

Objects – Nouns� Objects – Nouns

� Methods – Verbs

� Coad-and-Yourdon method talked about

attributes/properties of the nouns

48

*http://www.smartdraw.com/resources/tutorials/yourdon-and-coad-diagrams/

49

� Apply the “Rediscovery” meta-pattern of

testing - “What happens in Development,

follows in Testing” to the Noun-and-Verb

technique and you get the method in testingtechnique and you get the method in testing

50

� Unified Modeling Language – Unified Testing

Language

No guarantee of success and no predictions � No guarantee of success and no predictions

of failure

51

� If brain thinks is visual ways and mind-maps capture things in a visual way and it leads to better learning can this be used as a pattern for learning/doing?

� Can I imitate the way brain works for learning languages? Learning Testing?

� Can I create a mind-map like concept for describing patterns?

� Can these patterns then be used to capture testing patterns?

52

� Applying patterns requires

� Ignoring the small and looking at the large

� Large amount of data (usually)

� Reduction in the number of variables

� Elimination of outliers

� Pattern of outliers

� Variations and taking cognizance of those

� Variations and eliminating them

53

http://www.rense.com/general86/stun.htm

54

http://www.rense.com/general86/stun.htm

55

� “Homo sapiens is about pattern recognition,

he says. Both a gift and a trap.” - William

Gibson – Pattern Recognition

56

� http://www.exampler.com/testing-com/test-patterns/

� http://c2.com/cgi-bin/wiki?TestingPatterns

� http://www.testingreflections.com/node/view/9

57

Thank you everybody <<meta-pattern – help>> for images, text and ideas

58