are your test cases fit for automation?

Are your test cases fit for automation?Powered by HBT

T AshokFounder & CEOSTAG Software Private Limited

in.linkedin.com/in/AshokSTAG Ash_Thiru

Webinar Presentation Nov 27, 2013

© 2013 STAG Software Private Limited. All rights reserved.

Good automation is more than frameworks, tools and scripting.

2

The inherent structure of scenarios/cases plays a vital role in ensuring rapid development and optimised maintenance.

An ill formed structure makes the script complex making it error prone and costly to maintain.

We will discuss an interesting idea "Fitness for automation" wherein we examine properties of test scenarios/cases to ascertain if they are good enough to automate.

Also what needs to be done in the case where it is "unfit" ?

This is one of the applications of "Hypothesis Based Testing" (HBT) to ensure rapid automation and optimised maintenance.


Expectation and Challenges

3

๏ Develop scripts rapidly๏ Adapt to newer functionally quickly

๏ Complexity of scenario


Structure of a script

4

Setup

Execute

Oracle

Cleanup

Prerequisites to be done before “excitation”

“Excite” the system with inputs

Check outcomes to determine pass/fail

Inverse of ‘setup’


What contributes to complexity?

5

Setup

Execute

Oracle

Cleanup





#Steps to do this

#Inputs and the ‘kind of inputs(+/-)’Order, dependency, source

The variety of behaviours to check & #Checks for each behaviour

#Steps to do this


“Clarity of purpose”

6

Setup

Execute

Oracle

Cleanup





What “type of bug” are we looking for?...enables focused & simpler checking


“Clarity of purpose”

7

Setup

Execute

Oracle

Cleanup





... results in ‘just-enough’ steps,enables smaller/crisper script


Characteristics of TS that affect ‘automatability’

8

Clarity of purpose1

Distinctiveness of behaviours (Single output)2

Similarity of inputs3

What types of defects are being targeted by the test cases?

Do TS contain multiple behaviours ? or Single minded?

Are the various inputs for the TS similar ?

© 2013 STAG Software Private Limited. All rights reserved. 9

Quality Levels & PDTs - Powered by HBT

Input cleanliness

Input interface cleanliness

Structural integrity

Behaviour correctness

Environment cleanliness

Attributes met

Flow correctness

Clean Deployment

End user value

L1

L2

L3

L4

L5

L6

L7

L8

L9

That inputs are handled wellPDTs related to Input data correctness

That the functional behaviour is correctPDTs related to Functionality

That the internal structure is robustPDTs related to Structural aspects

That the user interface is cleanPDTs related to UI

That end-to-end flows work correctlyPDTs related to Flow behaviour, Interactions

That it does not mess up the environmentPDTs related to Resource leaks, Compatibility...

That the stated attributes are metPDTs related to Performance, Load, Volume......

That it deploys well in the real environmentPDTs related to Compatibility, Migration

That user expectations are metPDTs related to User flows, User experience

Clarity of purpose1

© 2013 STAG Software Private Limited. All rights reserved. 10

Input cleanliness

Input interface cleanliness

Structural integrity

Behaviour correctness

Environment cleanliness

Attributes met

Flow correctness

Clean Deployment

End user value

L1

L2

L3

L4

L5

L6

L7

L8

L9

Functional automation covers...

That inputs are handled wellPDTs related to Input data correctness

That the functional behaviour is correctPDTs related to Functionality

That the internal structure is robustPDTs related to Structural aspects

That the user interface is cleanPDTs related to UI

That end-to-end flows work correctlyPDTs related to Flow behaviour, Interactions

That it does not mess up the environmentPDTs related to Resource leaks, Compatibility...

That the stated attributes are metPDTs related to Performance, Load, Volume......

That it deploys well in the real environmentPDTs related to Compatibility, Migration

That user expectations are metPDTs related to User flows, User experience


Scripts automate scenarios, not cases

11

C1

C2

C3

C4i1

i2

i3}

Outputs1

2

3

45

Inputs Conditions

Behaviour StimuliEntity under test

Conditions govern behaviourOutcome is Test Scenarios

Input (data) is StimuliOutcome is Test Cases

Distinctiveness of behaviours2

Similarity of inputs3


Fitness for automation

12

Does the TS look for a variety for defects?1

Does the TS check for a specific behaviour?2

What types of defects are being targeted by the test cases?

Is a specific behaviour being checked?


What if TS are “unfit”?

13

The TS is looking for a variety for defects?1

The TS is checking a variety of behaviours2

Simplify a scenario.Simplify it by decomposing it into various levels.“Levelize”

Make the single minded.Split the scenarios such that each TS checks a specific combination of conditions.


Hypothesis Based Testing - HBT

14

System Under Test

Cleanliness Criteria

Potential Defect Types

Test CasesRequirementstraceability“what to test”

Faulttraceability“test for what”

should satisfy impeded by


HBT OverviewSIX staged purposeful activities, powered by EIGHT disciplines of thinking

15

D1D2

D4

D3

D5D6

D7

D8

Core Concepts

S1

S2

S3S4

S5

S6SIX Stages of DOINGS1: Understand expectationsS2: Understand contextS3: Formulate hypothesisS4: Devise proofS5: Tooling supportS6: Assess & Analyse

EIGHT Disciplines of ThinkingD1: Business value understandingD2: Defect hypothesisD3: Strategy & PlanningD4: Test designD5: ToolingD6: VisibilityD7: Execution & ReportingD8: Analysis & Management

Uses 32 Core ConceptsFor Problem solving - Techniques, Principles, Guideline

Click here to know more about HBT.http://stagsoftware.com/blog?p=570

http://stagsoftware.com/blog?p=570

http://stagsoftware.com/blog?p=570

http://slidesha.re/qBMNiy

http://slidesha.re/qBMNiy


HBT is the intellectual property of STAG Software Private Limited.STEMTM is the trademark of STAG Software Private Limited.

www.stagsoftware.com

@stagsoft

blog.stagsoftware.com

Connect with us...

Are your test cases fit for automation? Powered by HBT

Thank you.

http://www.stagsoftware.com

http://www.stagsoftware.com

are your test cases fit for automation?

Technology