use test trees to get an overview of your test

Use Test Trees to get an Overview of your Test

INTERNAL USE

Use Test Trees Test Coverage

Test Design and Implementation

Introduction to a Test Tree

Example

When are we done with the test

As much as we like testing, we cannot go on ‘for ever’ – this is why acceptance criteria are defined

When the acceptance criteria are reached we can stop the testing How much do we have to test with this acceptance criteria: No more than 10 unresolved defects

Classification Internal Use Author AMQH Approved by N/A Version 1.0 3

This is more like it

Acceptance criteria should always include a coverage measure Example: •  a coverage of 100 % is achieved for the

requirements on which the test is based

•  a detailed coverage of 60 % (high risk areas) or 40 % (low risk areas) is achieved


Test coverage definitions

ISTQB: the degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.

ISO 29119: degree, expressed as a percentage, to which specified test coverage items have been exercised by a test case or test cases


Specified test coverage items?

So we have to find out what specified test coverage items are. ISO 29119: attribute or combination of attributes that is derived from one or more test conditions by using a test design technique that enables the measurement of the thoroughness of the test execution


Some test case design techniques


Boris Beizer •  Control-flow testing •  Data-flow testing •  Domain testing •  Finite-state testing •  Loop testing •  Syntax testing •  Transaction flow t.

ISTQB •  Boundary value analysis •  Equivalence partitioning •  Cause Effect graphing •  Checklist •  Classification tree •  Decision table testing •  Orthogonal arrays •  Pairwise testing •  State Transition testing •  Structure based •  Use case testing •  User story testing •  and more..

ISO 29119 •  Boundary value analysis •  Equivalence partitioning •  Cause Effect graphing •  Classification tree •  Combinatorial e.g.

•  pairwise •  orthogonal arrays

•  Decision table testing •  Random testing •  State Transition testing •  Scenario testing •  Structure based •  Syntax testing •  and more…

Some coverage items

•  Boundary value analysis Boundary + 1 or 2 nearest values •  Equivalence partitioning Equivalence partition •  Cause Effect graphing Decision rule •  Checklist Entry •  Classification tree Leaf •  Decision table testing Decision rule •  Orthogonal arrays Pair •  Pairwise testing Unique parameter/value set •  State Transition testing States and/or Chow’s n-switch •  Structure based Depends on which one •  Use case testing Unique scenario •  User story testing Acceptance criterium •  and more..


Calculating detailed coverage

For a complex system where a number of test case design techniques are used, all identified and tested coverage items must be included in the calculation of the detailed coverage:

(∑𝑒=1↑𝑛▒ 𝐶𝑇(𝑒)  ) (∑𝑒=1↑𝑛▒ 𝐶𝐼(𝑒)  ) where e = number of test case design techniques used CT = number of tested coverage items per technique CC = number of identified coverage items per technq.


Detailed coverage % = x 100

Small example

Equivalence partitioning were used on 2 requirements, where 5 partitions were identified for one requirement and 3 for the other 2 partitions where tested for one requirement and all 3 were tested for the other

State transition were used on 1 requirement and 4 transitions where identified of which all were tested one at the time 2+3+4 5+3+4


Detailed coverage = x100 = (9/12)x100 = 75%


Test Design and Implementation Introduction to a Test Tree

Example

Test design and implementation

The test design and

implementation process

is a sub-process to the

entire test process

described by ISTQB and

in ISO 29119

Test design and implementation

items

Definitions

A feature set is a logical part of the system, which you can test in isolation, e.g.: •  access to the system, including access right •  a user interface, e.g. a form or a pane •  a set of rules implemented in a component •  general error handling

A test condition is something within a feature set, which you can test in isolation: e.g. •  a requirement •  a user story •  a rule




Introduction to a Test Tree Example

The test tree principle

A test tree is •  a presentation of the test item •  a presentation of the test basis •  an overview of all coverage items for the test item •  akin to classification trees – with coverage items

described under independent classes •  a way of documenting a selection of coverage

items to be tested in a test case •  a mean to calculate detailed coverage •  documented in Excel - the most powerful tool :-)


The contents of the tree


FS-3.PersonandRecidenceInformationpaneRequirements:FR_182,FR_191UISpec:Section5.4UserStory:none

TestcasesCiNo.

Classfilter& Ci

filter& Ci

filter& Ci

filter& Ci 1 2

PersonandRecidenceInformation,expandedcontents

1 topline:[-]PersonandRecidenceInformation xMandatorylabels

whichlabel2 Name: x

Gender:which

3 female x4 male

Test tree can be very ‘tall’



TestcasesCiNo.

Classfilter& Ci

filter& Ci

filter& Ci

filter& Ci 1 2



whichlabel2 Name: x

Gender:which

3 female x4 male5 ID: x6 Addresse: x7 Telephone: x

Citizenship:which

8 UK x9 notUK

DependablecontentswhichlabelortextC/O:

relevant10 yes x11 no

Custodian:infoavailable

12 yes x13 no

[!]CovertNameandAddresspersonsubjecttothis

14 yes15 no x16 buttomline:[+]OtherPeopleatthesameAddress x

actions17 presstopline x18 pressbuttomline

PersonandRecidenceInformation,collapsedcontents

19 topline:[+]PersonandRecidenceInformationactions

20 presstoplineOtherPeopleatthesameAddress,expanded

contents21 Mandatorytext:OtherpeopleattheAdress:<no.>

howmany22 0

morethan0anysubjecttonavne-ogadressebeskyttelse

23 noyesall

24 yes25 no26 Sub-panebuttom:[-]Otherpeopleatthesameaddress

actions27 presstopline28 pressSub-paneltopline

* <no.>lineswith"-"<name><id>ofotherpeople,sorteddescendingbyID** <no.>lineswith"-"<name><id>ofotherpeople,sorteddescendingbyID*** <no.>lineswith"-"<name><id>ofotherpeople,sorteddescendingbyID

This is a relatively small tree with only 3 classes and 28 coverage items Sometimes a test tree can have 50 + coverage items Use ‘divide and conquer’ where possible



Introduction to a Test Tree

Example

Example presentation

The example system is a case management system for case workers, who are responsible for the administration of payments to clients.


Result

Feature sets

The system is broken into the following feature sets (and many more):

•  the framework for the overview of case information for the manual handling

•  a pane within the framework of the overview

•  rules for automatic handling

• 

• 


Framework for overview of case info.


Case Overview for Case: nn

Family Relations

Person & Residence Info

Result Details

Salary Info

Case History

Requirements for Person and Residence Info


Requirement FR_191: Show person information, family relations, history of the case (and more). When a case worker types a person ID, the system shall present the following information related to the citizen:

• Salary Information • Case History • Person and Residence Information

• (and more)

Requirement FR_182: Read and save person information. Upon reception of person information, the system shall be able to store it with a reference to the person’s ID.

Person and Residence Info

Design and notes for the pane with sub-pane


Note Description

1 This pane will be expanded as default. The following is shown for the chosen person: • Name • Gender • ID • if relevant: C/O • Postal address • Country • Telephone number • Citizenship – if not UK • if relevant: Name of custodian

2 If the persons’s name and address are covert, this is marked with a colored text above the name and address, which are shown. Text = “! Name and address are covert”

3 A click on ”[+] Other people at the same address” will show information about other people living at the same address in a sub-pane.

4 If one or more of the other person(s) at the address has covert name and address this will be shown as a list of people without, if any, sorted by ID, followed by a Text as above and a sorted list of the people with, if any.

5 A click on ”[-] Person and Residence Information” will cause the entire pane to collapse, so that only the top line is visible.

6 A click on "[+] Citizen and Residence Information" will cause the pane to expand, with the sub-pane collapsed.

Coverage items are identified

Test case design techniques are used to find coverage items depending on the nature of the test basis, for example:


Equivalence class partitioning

Classification tree

Pane navigation

The navigation in the pane and sub-pane can be treated like a state machine with 3 states and 5 transitions

Classification xxxx Author xxxx Approved by xxxx Version xx 26

pri = “Person and Residence Information” pane both = pri + “Other People at the same Address” sub-pane

Constructing the tree

Fill in the information about the test item and the test basis Identify the highest level of classes for the test item from the test basis For each class, identify the filters that will partition the class into further sub-classes For each sub-class, identify the coverage items using appropriate test case design techniques For each coverage item, describe expected result


FS-3.PersonandRecidenceInformationpane Coverageitems 28Requirements:FR_182,FR_191 40% 11UISpec:Section5.4UserStory:none

CiNo.

Classfilter& Ci

filter& Ci

filter& Ci

filter& Ci

Expectedresult

PersonandRecidenceInformation,expandedonlyP&RIcontents

1 topline:[-]PersonandRecidenceInformation presentMandatorylabels

whichlabel2 Name: shownwithcorrectname

Gender:which

3 female shownwithtext"Female"4 male shownwithtext"Male"5 ID: shownwithcorrectID6 Addresse: shownwithcorrectaddress7 Telephone: shownwithcorrectno.

Citizenship:which

8 UK labelshown,noinfo.9 notUK labelshownwithcorrectcitizenship


relevant10 yes C/Oshownwithcorrectinfo.beforeAddress11 no nothingshownshown


12 yes labelandinfoshownbelowCitizenship13 no nothingshownshown

[!]CovertNameandAddressClassification Internal Use Author AMQH Approved by N/A Version 1.0 29


14 yes textshowninredundertopline15 no nowarningtextshown16 buttomline:[+]OtherPeopleatthesameAddress present

actions17 presstopline([-]) P&RIpanecollapsandtoplinewith[+]18 pressbuttomline([+]) bothpanesexpanded

P&RI(andsub-pane),collapsedcontents

19 topline:[+]PersonandRecidenceInformation presentactions

20 presstopline paneexpandsP&RIandsub-pane,expanded

Sub-panecontents21 Mandatorytext:OtherpeopleattheAdress:<no.> present

howmanywithcovertaddress22 0 textand<no.>=0shown

morethan0anysubjecttonavne-ogadressebeskyttelse

23 no *yesall

24 yes **25 no ***26 Sub-panebuttom:[-]Otherpeopleatthesameaddressshown

actions27 pressP&RItopline([-]) bothpanescollapse28 pressSub-paneltopline sub-panecollapses

* <no.>=actualnumber;lineswith"-"<name><id>ofotherpeople,sorteddescendingbyID** <no.>=actualnumber;warningtext;lineswith"-"<name><id>ofotherpeople,sorteddescendingbyID*** <no.>=actualtotalnumber;linesforwithout;warningtext;linesforwith

The rest of the tree


Select coverage items to test



TestcasesNo. Class filter Ci filter Ci filter Ci 1



whichlabel2 Name: x

Gender:which

3 female x4 male5 ID: x6 Addresse: x7 Telephone: x

Citizenship:which

8 UK x9 notUK


relevant10 yes x11 no


12 yes x13 no


14 yes15 no x16 buttomline:[+]OtherPeopleatthesameAddress x

actions17 presstopline x18 pressbuttomline

FS-3.PersonandRecidenceInformationpane Coverageitems 28Requirements:FR_182,FR_191 40% 11UISpec:Section5.4UserStory:none

CiNo.

Classfilter& Ci

filter& Ci

filter& Ci

filter& Ci

Expectedresult

PersonandRecidenceInformation,expandedonlyP&RIcontents

1 topline:[-]PersonandRecidenceInformation presentMandatorylabels

whichlabel2 Name: shownwithcorrectname

Gender:which

3 female shownwithtext"Female"4 male shownwithtext"Male"5 ID: shownwithcorrectID6 Addresse: shownwithcorrectaddress7 Telephone: shownwithcorrectno.

Citizenship:which

8 UK labelshown,noinfo.9 notUK labelshownwithcorrectcitizenship


relevant10 yes C/Oshownwithcorrectinfo.beforeAddress11 no nothingshownshown


12 yes labelandinfoshownbelowCitizenship13 no nothingshownshown

[!]CovertNameandAddress

Please try this at home


Thank you – and remember

•  Test is difficult

•  Test requires overview

•  Test requires creativity

•  Test requires systematic work

•  Test requires imagination

•  Test requires courage

•  Test is fun 06/06/16 33

[email protected]

use test trees to get an overview of your test

Documents