software testing methodologies (stm)
DESCRIPTION
Software Testing Methodologies (STM). Unit 3 – Data Flow. Compiled with reference from: Software Testing Techniques: Boris Beizer Craft of Software Testing: Brain Marrick. Narasimha Rao.P. U2. Data - Flow Testing - Basics. We will see in this part of Unit 2: - PowerPoint PPT PresentationTRANSCRIPT
Narasimha Rao.PNarasimha Rao.P
Compiled with reference from:Compiled with reference from: Software Testing Techniques: Boris BeizerSoftware Testing Techniques: Boris Beizer Craft of Software Testing: Brain MarrickCraft of Software Testing: Brain Marrick
Unit 3 – Data FlowUnit 3 – Data Flow
ref boris beizer 2
Data - Flow Testing - BasicsWe will see in this part of Unit 2:
• Concepts of Data flows
• Data-flow testing strategies
U2
ref boris beizer 3
Data - Flow Testing - BasicsContents
• Synopsis
• Basics• Intro to Data flow, data flow graphs• Motivation & Assumption• Data flow model
• Data Flow Testing Strategies• General strategy• Definitions• Strategies:
• Slicing, Dicing, Data flow, Effectiveness
• Application of DFT, Tools & Effectiveness
U2
ref boris beizer4
Data - Flow Testing - Basics U2
AnomalyAnomaly
Unreasonable processing on dataUnreasonable processing on data
• Use of data object before it is definedUse of data object before it is defined• Defined data object is not usedDefined data object is not used
• Data Flow Testing (DFT) uses Control Flow Graph (CFG) Data Flow Testing (DFT) uses Control Flow Graph (CFG) to explore dataflow anomalies.to explore dataflow anomalies.
• DFT Leads to testing strategies between PDFT Leads to testing strategies between P and P1 / P2 and P1 / P2
ref boris beizer5
Data - Flow Testing - Basics U2
Definition:Definition:
DFT is a family of test strategies based on selecting paths DFT is a family of test strategies based on selecting paths through the program’s control flow in order to explore the through the program’s control flow in order to explore the sequence of events related to the status of data objects.sequence of events related to the status of data objects.
Example:Example:
Pick enough paths to assure that every data item has been Pick enough paths to assure that every data item has been initialized prior to its use, or that all objects have been used for initialized prior to its use, or that all objects have been used for something.something.
ref boris beizer6
Data - Flow Testing - Basics U2
MotivationMotivation
• Confidence in the programConfidence in the program
• Data dominated designData dominated design. . Code migrates to data..
• Source Code for Data DeclarationsSource Code for Data Declarations
• Data flow Machines vs Von Neumann’sData flow Machines vs Von Neumann’s
• Abstract M I M D Abstract M I M D • Language & compiler take care of parallel Language & compiler take care of parallel
computationscomputations
ref boris beizer7
Data - Flow Testing - Basics - Motivation U2
Program Control flow with Von Neumann’s paradigmProgram Control flow with Von Neumann’s paradigm Given m, n, p, q, find eGiven m, n, p, q, find e..
e = (m+n+p+q) * (m+n-p-q)e = (m+n+p+q) * (m+n-p-q)
a := m + na := m + nb := p + qb := p + qc := a + bc := a + bd := a - bd := a - b
e := c * de := c * d
a = n+m
b=p+q
c=a+b
d=a-b
e=c*d
Multiple representations of control flow graphs possible.
ref boris beizer8
Data - Flow Testing - Basics - Motivation U2
Program Flow using Data Flow Machines paradigmProgram Flow using Data Flow Machines paradigm
BEGIN PAR DO
READ m, n, n, p, qEND PARPAR DO
a := m+nb := p+q
END PARPAR DO
c := a+bd := a-b
END PARPAR DO
e := c * dEND PAREND
n m p q
a := m+n b := p+q
c := a+b
The interrelations among the data items remain same.The interrelations among the data items remain same.
d := a-b
e := c * d
ref boris beizer9
Data - Flow Testing - Basics - Motivation U2
• Control flow graphControl flow graph• Multiple representationsMultiple representations
• Data Flow GraphData Flow Graph A spec. for relations among the data objects. A spec. for relations among the data objects.
Covering DFG => Explore all relations under some test.Covering DFG => Explore all relations under some test.
ref boris beizer10
Data - Flow Testing - Basics U2
AssumptionsAssumptions
• Problems in a control flowProblems in a control flow
• Problems with data objectsProblems with data objects
ref boris beizer11
Data - Flow Testing - Basics U2
Data Flow Graphs (DFG)Data Flow Graphs (DFG)
• It is a graph with nodes & directed links
• Test the Von Neumann way - Convert to a CFG
Annotate : program actions (weights)
ref boris beizer12
Data - Flow Testing - Basics U2
Data Object State & UsageData Object State & Usage
Program Actions (d, k, u):Program Actions (d, k, u):
Defined (created)Defined (created) - - explicitly or implicitlyexplicitly or implicitly (d)(d)
Killed (released)Killed (released) - - directly or indirectlydirectly or indirectly (k)(k)
UsedUsed -- (u)(u)
• In a calculation In a calculation -- (c)(c)
• In a predicateIn a predicate -- directly or indirectlydirectly or indirectly (p)(p)
ref boris beizer13
Data - Flow Testing - Basics U2
Data Flow AnomaliesData Flow Anomalies
A Two letter sequence of Actions (d, k, u)A Two letter sequence of Actions (d, k, u)
dddd : : harmless, suspiciousharmless, suspicious
dkdk :: probably a bug.probably a bug.
dudu :: normalnormal
kdkd :: normalnormal
kkkk :: harmless, but probably a bugharmless, but probably a bug
kuku :: a buga bug
udud :: normal. Redefinition.normal. Redefinition.
ukuk :: normalnormal
uuuu :: normalnormal
A
Action
ref boris beizer14
Data - Flow Testing - Basics - Motivation U2
Program Flow using Data Flow Machines paradigmProgram Flow using Data Flow Machines paradigm
BEGIN PAR DO
READ m, n, n, p, qEND PARPAR DO
a := m+nb := p+q
END PARPAR DO
c := a+bd := a-b
END PARPAR DO
e := c * dEND PAREND
n m p q
a := m+n b := p+q
c := a+b
The interrelations among the data items remain same.The interrelations among the data items remain same.
d := a-b
e := c * d
ref boris beizer15
Data - Flow Testing - Basics – Data Flow Anomalies U2
Actions on data objectsActions on data objects
-- no action from no action from START to this pointSTART to this pointFrom this point till the From this point till the EXITEXIT
- d- d normalnormal
- u- u anomalyanomaly - k- k anomalyanomaly
k-k- normalnormal u -u - normalnormal - possibly an anomaly - possibly an anomaly d -d - possibly anomalouspossibly anomalous
ref boris beizer16
Data - Flow Testing - Basics U2
Data Flow Anomaly State graphData Flow Anomaly State graph
• Data Object StateData Object State
• K, D, U, AK, D, U, A
• Processing StepProcessing Step
• k, d, uk, d, u
ref boris beizer17
Data - Flow Testing - Basics U2
Data Flow Anomaly State graphData Flow Anomaly State graph
• Object stateObject state• Unforgiving Data flow state graphUnforgiving Data flow state graph
K
DU Au
d
u
k, u
d, k
d, k, u
d
DefinedUsed
Undefined
Anomalous
ref boris beizer18
Data - Flow Testing - Basics U2
Data Flow Anomaly State graphData Flow Anomaly State graph
Forgiving Data flow state graphForgiving Data flow state graph
K
DU
DD
ud
u
d
d
d
A DD, DK, KU
u
kKU
DK
k
u
u
kd
k
u
k
ref boris beizer19
Data - Flow Testing - Basics U2
Data Flow State GraphsData Flow State Graphs
• Differ in processing of anomaliesDiffer in processing of anomalies
• Choice depends on Choice depends on Application, language, contextApplication, language, context
ref boris beizer20
Data - Flow Testing - Basics U2
Static vs Dynamic Anomaly DetectionStatic vs Dynamic Anomaly Detection
• Static analysis of data flows
• Dynamic analysisIntermediate data valuesIntermediate data values
ref boris beizer21
Data - Flow Testing - Basics U2
Insufficiency of Static Analysis (for Data flow)Insufficiency of Static Analysis (for Data flow)
1. Validation of Dead Variables
2. Validation of pointers in Arrays
3. Validation of pointers for Records & pointers
1. Dynamic addresses for dynamic subroutine calls
2. Identifying False anomaly on an unachievable path
1. Recoverable anomalies & Alternate state graph
2. Concurrency, Interrupts, System Issues
ref boris beizer22
Data - Flow Testing - Basics U2
Data Flow ModelData Flow Model
• Based on CFG
• CFG annotated with program actions
• link weights : dk, dp, du etc..
• Not same as DFG
• For each variable and data object
ref boris beizer23
Data - Flow Testing - Basics : Data Flow Model U2
Procedure to Build:Procedure to Build:
1.1. Entry & Exit nodesEntry & Exit nodes
1.1. Unique node identificationUnique node identification
1.1. Weights on out linkWeights on out link
2.2. Predicated nodesPredicated nodes
3.3. Sequence of linksSequence of links
1.1. JoinJoin
2.2. Concatenate weightsConcatenate weights
3.3. The converseThe converse
ref boris beizer24
Data - Flow Testing - Basics : Data Flow Model U2
Example:Example: a an – 1 – 1Z = b + ---------Z = b + ---------
START a - 1 a - 1INPUT a, b, nZ := 0IF a = 1 THEN Z := 1GOTO DONE1r := 1 c := 1POWER:
c := c * ar := r + 1IF r <= n THEN GO TO POWERZ := (c – 1) / (a – 1)
DONE1:Z := b + Z
END
ref boris beizer25
Data - Flow Testing - Basics – Data Flow model U2
CFG for the ExampleCFG for the Example
1 2
3 4
5 6
Read a,b,nZ := 0 Z := 1 Z := b + Z
Z := (c-1)/(a-1)
a = 1?P1
r < n ?P2
Y
r := 1 c:=1 r := r+1, c:= c*a
Y
ref boris beizer26
Data - Flow Testing - Basics – Data Flow model U2
CFG annotated – Data Flow Model for ZCFG annotated – Data Flow Model for Z
1 2
3 4
5 6d
d or kd cd or ckd
a = 1?P1
r < n ?P2
Y
Y
d or kd
ref boris beizer27
Data - Flow Testing - Basics – Data Flow model U2
CFG annotated – Data Flow Model for cCFG annotated – Data Flow Model for c
1 2
3 4
5 6
-d
ckd or kd
a = 1?P1
r < n ?P2
Y
Y
c-
ref boris beizer28
Data - Flow Testing - Basics – Data Flow model U2
CFG annotated – Data Flow Model for CFG annotated – Data Flow Model for rr
1 2
3 4
5 6
-d
ckd or kd
a = 1?P1
r < n ?P2
Y
Y
p-
ref boris beizer29
Data - Flow Testing - Basics – Data Flow model U2
CFG annotated – Data Flow Model for CFG annotated – Data Flow Model for bb
1 2
3 4
5 6d
a = 1?P1
r < n ?P2
Y
Y
c
ref boris beizer30
Data - Flow Testing - Basics – Data Flow model U2
CFG annotated – Data Flow Model for CFG annotated – Data Flow Model for nn
1 2
3 4
5 6d
a = 1?P1
r < n ?P2
Y
Yp-
ref boris beizer31
Data - Flow Testing - Basics – Data Flow model U2
CFG annotated – Data Flow Model for CFG annotated – Data Flow Model for aa
1 2
3 4
5 6d
a = 1?P1
r < n ?P2
Y
pc-
c
ref boris beizer32
Data - Flow Testing - Basics – Data Flow model U2
A DFM for each variable A DFM for each variable
Single DFM for multiple variablesSingle DFM for multiple variables
Use weights subscripted with variablesUse weights subscripted with variables
ref boris beizer33
Data - Flow Testing – Data Flow Testing Strategies U2
• A structural testing strategy (A structural testing strategy (path testingpath testing))
• Add, data flow strategies Add, data flow strategies with link weightswith link weights
• Test path segments to have a Test path segments to have a ‘d’‘d’ (or (or u, k, du, dku, k, du, dk))
ref boris beizer34
Data - Flow Testing – Data Flow Testing Strategies U2
DEFINITIONSDEFINITIONS
• w.r.t. a variable or data object ‘v’w.r.t. a variable or data object ‘v’• Assume all DF paths are achievableAssume all DF paths are achievable
1.1. Definition-clear path segmentDefinition-clear path segmentno no k, kdk, kd
2.2. Loop-free path segmentLoop-free path segment
2.2. Simple path segmentSimple path segment
3.3. dudu path from node path from node ii to to kk
• definition-clear & simple cdefinition-clear & simple c• definition-clear & loop-free pdefinition-clear & loop-free p
ref boris beizer35
Data - Flow Testing – Data Flow Testing Strategies U2
DFT StrategiesDFT Strategies
1.1. All-du paths (ADUP)All-du paths (ADUP)
2.2. All uses (AU) strategyAll uses (AU) strategy
3.3. All p-uses/some c-uses and All c-uses/some p-usesAll p-uses/some c-uses and All c-uses/some p-uses
1.1. All Definitions StrategyAll Definitions Strategy
1.1. All p-uses, All c-uses StrategyAll p-uses, All c-uses Strategy
Purpose:Purpose:
Test Design, Develop Test CasesTest Design, Develop Test Cases
ref boris beizer36
Data - Flow Testing – Data Flow Testing Strategies U2
1.1. All-du paths (ADUP)All-du paths (ADUP)
• Strongest DFTStrongest DFT
• EveryEvery dudu path for every variable for every definition to every path for every variable for every definition to every use use
2.2. All uses (AU) strategyAll uses (AU) strategy
• At least one At least one definition clear path segment from every definition clear path segment from every definition of every variable to every use of that definition be definition of every variable to every use of that definition be exercised under some test.exercised under some test.
• At least one path segment from every definition to every use that can be reached from that definition.
ref boris beizer37
Data - Flow Testing – Data Flow Testing Strategies U2
3.3. All p-uses/some c-uses and All c-uses/some All p-uses/some c-uses and All c-uses/some p-usesp-uses
• APU + cAPU + c
• Stronger than P2Stronger than P2
• ACU + pACU + p
• Weaker than P2Weaker than P2
ref boris beizer38
Data - Flow Testing – Data Flow Testing Strategies U2
4.4. All Definitions Strategy (AD)All Definitions Strategy (AD)
• Cover every definition by Cover every definition by at least oneat least one p or c p or c
• Weaker than ACU + p and APU + cWeaker than ACU + p and APU + c
ref boris beizer39
Data - Flow Testing – Data Flow Testing Strategies U2
5.5. All-Predicate Uses, All-Computational Uses All-Predicate Uses, All-Computational Uses StrategyStrategy
• APU :APU :
• Include definition-free path for every definition of every Include definition-free path for every definition of every variable from the definition to predicate use.variable from the definition to predicate use.
• ACU :ACU :
• Include for every definition of every variable include at Include for every definition of every variable include at least one definition-free path from the definition to every least one definition-free path from the definition to every computational use.computational use.
ref boris beizer40
Data - Flow Testing – Data Flow Testing Strategies U2
Ordering the strategies Ordering the strategies
All Paths
All du Paths
All-uses Paths (AU)
All-c / some-p (ACU+p)
All c uses (ACU)
All-p / some-c APU+c
All P-uses APU
All Branches P2
All Stmts P1
All Defs AD
ref boris beizer41
Data - Flow Testing – Data Flow Testing Strategies U2
Testing, Maintenance & Debugging in the Data Flow contextTesting, Maintenance & Debugging in the Data Flow context
Slicing:
• A static program A static program sliceslice is a part of a program defined wrt is a part of a program defined wrt a variable ‘a variable ‘vv’ and a statement ‘’ and a statement ‘ss’; It is the set of all ’; It is the set of all statements that could affect the value of ‘statements that could affect the value of ‘vv’ at stmt ‘’ at stmt ‘ss’.’.
Stmt1 var vstmt2Stmt3 var vStmt4 var v
Stmt s var v
ref boris beizer42
Data - Flow Testing – Data Flow Testing Strategies U2
Testing, Maintenance & Debugging in the Data Flow contextTesting, Maintenance & Debugging in the Data Flow context
Dicing:Dicing:
• A program dice is a part of slice in which all stmts. which are known to be correct have been removed.
• Obtained from ‘slice’ by incorporating correctness information from testing / debugging.
ref boris beizer43
Data - Flow Testing – Data Flow Testing Strategies U2
Testing, Maintenance & Debugging in the Data Flow contextTesting, Maintenance & Debugging in the Data Flow context
Debugging:Debugging:
• Select a slice.
• Narrow it to a dice.
• Refine the dice till it’s one faulty stmt.
ref boris beizer44
Data - Flow Testing – Data Flow Testing Strategies U2
Testing, Maintenance & Debugging in the Data Flow contextTesting, Maintenance & Debugging in the Data Flow context
Dynamic Slicing:Dynamic Slicing:
• Refinement of static slicingRefinement of static slicing
• Only achievable paths to the stmt ‘s’ in question are Only achievable paths to the stmt ‘s’ in question are included.included.
Slicing methods bring together testing, maintenance & debugging..
ref boris beizer45
Data - Flow Testing - – Data Flow Testing Strategies U2
Application of DFTApplication of DFT
• Comparison Random Testing, P2, AU - by Ntafos
• AU detects more bugs than
• P2 with more test cases• RT with less # of test cases
• Comparison of P2, AU - by Sneed
• AU detects more bugs with 90% Data Coverage Requirement.
ref boris beizer46
Data - Flow Testing - – Data Flow Testing Strategies U2
Application of DFTApplication of DFT
• Comparison of # test cases for ACU, APU, AU & ADUP
• by Weyuker using ASSET testing system
• Test Cases Normalized. t = a + b * d d = # binary decisions
• At most d+1 Test Cases for P2 loop-free
• # Test Cases / Decision
ADUP > AU > APU > ACU > revised-APU
ref boris beizer47
Data - Flow Testing - – Data Flow Testing Strategies U2
Application of DFTApplication of DFT
Comparison of # test cases for ACU, APU, AU & ADUP by Shimeall & Levenson
Test Cases Normalized. t = a + b * d (d = # binary decisions)
At most d+1 Test Cases for P2 loop-free
# Test Cases / Decision
ADUP ~ ½ APU*
AP ~ AC
ref boris beizer48
Data - Flow Testing - – Data Flow Testing Strategies U2
Application of DFTApplication of DFT
DFT vs P1, P2
• DFT is Effective
• Effort for Covering Path Set ~ Same
• DFT Tracks the Coverage of Variables
• Test Design is similar
ref boris beizer49
Data - Flow Testing - – Data Flow Testing Strategies U2
DFT - TOOLSDFT - TOOLS
• Cost-effective development
• Commercial tools :
• Can possibly do Better than Commercial Tools
• Easier Integration into a Compiler
• Efficient Testing
ref boris beizer50
Data - Flow Testing – Questions from the previous year’s exams U2
1. How is data flow testing (DFT) helpful in fulfilling gaps in path testing?
2. Explain the data flow Graphs (DFG).
3. How can anomaly be detected? Explain different types of data flow anomalies and Data flow Anomaly State Graphs.
4. Write applications of Data Flow Testing.
5. Name and explain Data flow testing strategies.
ref boris beizer51
Data - Flow Testing U-2C
To Unit 4 … Domain Testing