deliver superior outcomes using hbt visualization tool
DESCRIPTION
HBT Innovation Series webinar presented by T Ashok, Architect-HBT and Founder & CEO, STAG Software on the topic - Deliver Superior Outcomes Using HBT Visualization Tool - on Feb 26, 2014.TRANSCRIPT
Deliver superior outcomes using HBT Quality Visualisation ToolPowered by HBT
T AshokFounder & CEOSTAG Software Private LimitedArchitect - HBT
in.linkedin.com/in/AshokSTAG Ash_Thiru
© 2014 STAG Software Private Limited. All rights reserved.
Delivering superior outcomes requires:
2
good understanding, purposeful strategy, robust test design and rapid execution
Good work -
An excellent orchestration of
Good “mirror” - for a clear & objective feedback rapidly to aid continuous adjustment & adaptation.
© 2014 STAG Software Private Limited. All rights reserved.
The typical “mirror”
3
Requirements traceability, Coverage- Code/Test
#Cycles, TC planned vs executed, Blockers
TC - Pass/fail rate, First pass yieldDefect info- Distribution, Rate, Density, Leakage
!ese are good, but not good enough to be sharply goal focused.
Normally this is done by collecting & analysing typical “extrinsic measures” like:
© 2014 STAG Software Private Limited. All rights reserved.
Good “mirror” needs to tell ...
4
Are test assets good?1
Have we assessed completely?2
How good are the outcomes?3
quality of test cases
quality of execution
quality of product/application
© 2014 STAG Software Private Limited. All rights reserved.
Hypothesis Based Testing - HBT
5
System Under Test
Cleanliness Criteria
Potential Defect Types
Test CasesRequirementstraceability“what to test”
Faulttraceability“test for what”
should satisfy impeded by
© 2014 STAG Software Private Limited. All rights reserved. 6
Quality Levels & PDTs - Powered by HBT
Input cleanliness
Input interface cleanliness
Structural integrity
Behaviour correctness
Environment cleanliness
Attributes met
Flow correctness
Clean Deployment
End user value
L1
L2
L3
L4
L5
L6
L7
L8
L9
!at inputs are handled wellPDTs related to Input data correctness
!at the functional behaviour is correctPDTs related to Functionality
!at the internal structure is robustPDTs related to Structural aspects
!at the user interface is cleanPDTs related to UI
!at end-to-end "ows work correctlyPDTs related to Flow behaviour, Interactions
!at it does not mess up the environmentPDTs related to Resource leaks, Compatibility...
!at the stated a#ributes are metPDTs related to Performance, Load, Volume......
!at it deploys well in the real environmentPDTs related to Compatibility, Migration
!at user expectations are metPDTs related to User "ows, User experience
© 2014 STAG Software Private Limited. All rights reserved.
The W3T
7
QL9 End user value End user value testPDT 57-59End user value test
QL8 Clean deployment Installation test PDT 54-55
Migration testPDT 56Migration test
QL7 A"ributes met LSPS testPDT 44-51
Reliability testPDT 52Reliability testPDT 52
Security testPDT 53
QL6 Environment cleanliness Good citizen testPDT 39-41
Compatibility testPDT 42-43Compatibility testPDT 42-43
QL5 Flow correctness Interaction testPDT 35-38
QL4 Behaviour correctness Functionality testPDT 24-31Functionality test Access control test
PDT 32-34Access control testPDT 32-34
QL3 Structural integrity Structural testPDT 14-23
QL2 Input interface cleanliness API validation testPDT 5-7API validation test GUI validation test
PDT 8-13GUI validation test
QL1 Input cleanliness Input validation testPDT 1-4Input validation test
QL - Quality LevelPDT - Potential Defect Type
Pow
ered
by
HB
T
© 2014 STAG Software Private Limited. All rights reserved. 8
L1 L2L2 L9
T1 T2 T3 Tn
PDT1-2 PDT3-6 PDT7-10 PDT11-14
CC1 CC2 CC3...
CCn
E1 ✓ ✓...
E2 ✓ ✓ ✓
E3 ✓ ✓
E4 ✓ ✓Wha
t-to
-Tes
t (W
2T)
HBT Baseline - The KEY to Visualisation!
Test-for-What(Expectation)
CC Cleanliness Criteria TfW-E Test-for-What(Expectation)
PDT Potential Defect Type TfW-G Test-for-What(Goal)
T* Test Type WhT What-Test
L* Quality Level WhnT When-Test
List the entities under test (E)(Features, Requirements...) in the order of priority/importance.
Map them to the expectations of the customers.
© 2014 STAG Software Private Limited. All rights reserved. 9
L1 L2L2 L9
T1 T2 T3 Tn
PDT1-2 PDT3-6 PDT7-10 PDT11-n
CC1 CC2 CC3...
CCn
E1 ✓ ✓...
E2 ✓ ✓ ✓
E3 ✓ ✓
E4 ✓ ✓Wha
t-to
-Tes
t (W
2T)
HBT Baseline - The KEY to Visualisation!
Test-for-What(Goal)
TfW-E
CC Cleanliness Criteria TfW-E Test-for-What(Expectation)
PDT Potential Defect Type TfW-G Test-for-What(Goal)
T* Test Type WhT What-Test
L* Quality Level WhnT When-Test
© 2014 STAG Software Private Limited. All rights reserved. 10
L1 L2L2 L9
T1 T2 T3 Tn
PDT1-2 PDT3-6 PDT7-10 PDT11-n
CC1 CC2 CC3...
CCn
E1 ✓ ✓...
E2 ✓ ✓ ✓
E3 ✓ ✓
E4 ✓ ✓Wha
t-to
-Tes
t (W
2T)
HBT Baseline - The KEY to Visualisation!
What-Test
TfW-G
TfW-E
CC Cleanliness Criteria TfW-E Test-for-What(Expectation)
PDT Potential Defect Type TfW-G Test-for-What(Goal)
T* Test Type WhT What-Test
L* Quality Level WhnT When-Test
© 2014 STAG Software Private Limited. All rights reserved. 11
L1 L2L2 L9
T1 T2 T3 Tn
PDT1-2 PDT3-6 PDT7-10 PDT11-n
CC1 CC2 CC3...
CCn
E1 ✓ ✓...
E2 ✓ ✓ ✓
E3 ✓ ✓
E4 ✓ ✓Wha
t-to
-Tes
t (W
2T)
HBT Baseline - The KEY to Visualisation!
When-Test
TfW-G
TfW-E
WhT
CC Cleanliness Criteria TfW-E Test-for-What(Expectation)
PDT Potential Defect Type TfW-G Test-for-What(Goal)
T* Test Type WhT What-Test
L* Quality Level WhnT When-Test
© 2014 STAG Software Private Limited. All rights reserved. 12
L1 L2L2 L9
T1 T2 T3 Tn
PDT1-2 PDT3-6 PDT7-10 PDT11-n
CC1 CC2 CC3...
CCn
E1 ✓ ✓...
E2 ✓ ✓ ✓
E3 ✓ ✓
E4 ✓ ✓Wha
t-to
-Tes
t (W
2T)
HBT Baseline - The KEY to Visualisation!
When-What & Test-for-What (W3T)
TfW-G
TfW-E
WhT
WhnT
CC Cleanliness Criteria TfW-E Test-for-What(Expectation)
PDT Potential Defect Type TfW-G Test-for-What(Goal)
T* Test Type WhT What-Test
L* Quality Level WhnT When-Test
© 2014 STAG Software Private Limited. All rights reserved. 13
L1L1 L2L2 L9L9
T1T1 T2T2 T3T3 TnTn
PDT1-2PDT1-2 PDT3-6PDT3-6 PDT7-10PDT7-10 PDT11-14PDT11-14
CC1CC1 CC2CC2 CC3CC3 CC4CC4
E1 #TC-ve
E1 #TCE1 #TC+ve
E1+ve
...
E2 10040
...
500100
E2 10060
500400
E3E3
E4E4E4E4
10060 TOT
#TC-VE
10040
TOT #TC +VE
W2T
W3T
Are test assets good?1quality of test cases
1
2
3
4
Applicable test types considered?1
Any anomalies in #TC wrt importance2
#TC decreasing wrt levels?3
Countable? +ve:-ve ratio ok?4
© 2014 STAG Software Private Limited. All rights reserved. 14
L1L1 L2L2 L9L9
T1T1 T2T2 T3T3 TnTn
PDT1-2PDT1-2 PDT3-6PDT3-6 PDT7-10PDT7-10 PDT11-14PDT11-14
CC1CC1 CC2CC2 CC3CC3 CC4CC4
E1 Tot%-ve%
E1 Tot%E1 Tot%+ve%
E1+ve%
...
E2 90%50%
...
85%75%
E2 90%100%
85%85%
E3E3
E4E4E4E4
97%87%
100%100% TOT
%-VE
97%100%
100%100%
TOT% +VE
W2T
W3T
Have we assessed completely?2quality of execution
4
1 2
3
How are we doing wrt test for a entity?1
How are we doing wrt entity as a whole?2
How are we doing wrt a test & level?3
Are we making progress every cycle?4
© 2014 STAG Software Private Limited. All rights reserved. 15
L1L1 L2L2 L9L9
T1T1 T2T2 T3T3 TnTn
PDT1-2PDT1-2 PDT3-6PDT3-6 PDT7-10PDT7-10 PDT11-14PDT11-14
CC1CC1 CC2CC2 CC3CC3 CC4CC4
E1 CI(%)-CI%
E1 CI(%)E1 CI(%)+CI%
E1+CI%
...
E2 90%50%
...
85%75%
E2 90%100%
85%85%
E3E3
E4E4E4E4
97%87%
100%100% TOT
%-VE
97%100%
100%100%
TOT% +VE
W2T
W3T
How good are the outcomes?3quality of product/application
4
1 2
3
How clean is the entity wrt CC?1
How clean is the full entity?2
How are the entities f faring wrt a CC?3
How clean is the system as of now?4
If TC are proven complete, then CI (Cleanliness Index) =#TC-Passed/#TC-Total
Note that blocked TC would be considered as potentially failed.
Note we are measuring the “intrinsic cleanliness” here.
© 2014 STAG Software Private Limited. All rights reserved.
Summarising : Good “mirror” needs to tell ...
16
Are test assets good?1
Have we assessed completely?2
How good are the outcomes?3
quality of test cases
quality of execution
quality of product/application
© 2014 STAG Software Private Limited. All rights reserved.
HBT OverviewSIX staged purposeful activities, powered by EIGHT disciplines of thinking
17
D1D2
D4
D3
D5D6
D7
D8
Core Concepts
S1
S2
S3S4
S5
S6SIX Stages of DOINGS1: Understand expectationsS2: Understand contextS3: Formulate hypothesisS4: Devise proofS5: Tooling supportS6: Assess & Analyse
EIGHT Disciplines of ThinkingD1: Business value understandingD2: Defect hypothesisD3: Strategy & PlanningD4: Test designD5: ToolingD6: VisibilityD7: Execution & ReportingD8: Analysis & Management
Uses 32 Core ConceptsFor Problem solving - Techniques, Principles, Guideline
Click here to know more about HBT.http://stagsoftware.com/blog?p=570
© 2014 STAG Software Private Limited. All rights reserved. www.stagsoftware.com
Deliver superior outcomes using HBT Quality Visualisation ToolPowered by HBT
Are test assets good? “Quality of assets”1
Have we assessed completely? “Quality of execution”2
How good are the outcomes?“Quality of the product/application”3
HBT is the intellectual property of STAG Software Private Limited.STEMTM is the trademark of STAG Software Private Limited.
@stagsoft
blog.stagsoftware.com
Connect with us...
Thank you.