© serg dependable software systems (data-flow testing) dependable software systems topics in...

79
© SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing erial drawn from [Beizer, Mancoridis, Vokolos]

Upload: katherine-crawford

Post on 21-Dec-2015

237 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Dependable Software Systems

Topics in Data-Flow Testing

Material drawn from [Beizer, Mancoridis, Vokolos]

Page 2: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Testing

• Data-flow testing uses the control flowgraph to explore the unreasonable things that can happen to data (i.e., anomalies).

• Consideration of data-flow anomalies leads to test path selection strategies that fill the gaps between complete path testing and branch or statement testing.

Page 3: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Testing (Cont’d)

• Data-flow testing is the name given to a family of test strategies based on selecting paths through the program’s control flow in order to explore sequences of events related to the status of data objects.

• E.g., Pick enough paths to assure that:– Every data object has been initialized prior to its

use.– All defined objects have been used at least once.

Page 4: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data Object Categories

• (d) Defined, Created, Initialized

• (k) Killed, Undefined, Released

• (u) Used: – (c) Used in a calculation– (p) Used in a predicate

Page 5: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

(d) Defined Objects

• An object (e.g., variable) is defined when it:– appears in a data declaration– is assigned a new value– is a file that has been opened– is dynamically allocated– ...

Page 6: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

(k) Killed Objects

• An object is killed when it is:– released (e.g., free) or otherwise made

unavailable (e.g., out of scope)– a loop control variable when the loop exits– a file that has been closed– ...

Page 7: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

(u) Used Objects

• An object is used when it is part of a computation or a predicate.

• A variable is used for a computation (c) when it appears on the RHS (sometimes even the LHS in case of array indices) of an assignment statement.

• A variable is used in a predicate (p) when it appears directly in that predicate.

Page 8: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: Definition and Uses

1. read (x, y);2. z = x + 2;3. if (z < y)4 w = x + 1;

else5. y = y + 1;6. print (x, y, w, z);

What are the definitions and uses for the program below?

Page 9: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: Definition and Uses

1. read (x, y);2. z = x + 2;3. if (z < y)4 w = x + 1;

else5. y = y + 1;6. print (x, y, w, z);

Def C-use P-use

x, y

z x

z, y

w x

y y

x, y, w, z

Page 10: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Anomalies

• A data-flow anomaly is denoted by a two character sequence of actions. E.g.,– ku: Means that an object is killed and then

used.– dd: Means that an object is defined twice

without an intervening usage.

Page 11: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example

• E.g., of a valid (not anomalous) scenario where variable A is a dpd:

A = C + D;if(A > 0) X = 1;else X = -1;A = B + C;

Page 12: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Two Letter Combinations for d k u

• dd: Probably harmless, but suspicious.• dk: Probably a bug.• du: Normal situation.• kd: Normal situation.• kk: Harmless, but probably a bug.• ku: Definitely a bug.• ud: Normal situation (reassignment).• uk: Normal situation.• uu: Normal situation.

Page 13: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Single Letter Situations

• A leading dash means that nothing of interest (d, k, u) occurs prior to the action noted along the entry-exit path of interest.

• A trailing dash means that nothing of interest happens after the point of action until the exit.

Page 14: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Single Letter Situations• -k: Possibly anomalous:

– Killing a variable that does not exist.– Killing a variable that is global.

• -d: Normal situation.• -u: Possibly anomalous, unless variable is global.• k-: Normal situation.• d-: Possibly anomalous, unless variable is global.• u-: Normal situation.

Page 15: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Anomaly State Graph

U

K

AD

k,u,d

u

u

d

k

d

d,k

u,k

state of variable

action

anomalousstate

Page 16: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Anomaly State Graph with Variable Redemption

D U

K

DD

DK

KU

u

u

k

u

d

d

d

dd

d

k

k

k

u

k ku

u

Page 17: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Static vs DynamicAnomaly Detection

• Static Analysis is analysis done on source code without actually executing it.

• E.g., Syntax errors are caught by static analysis.

Page 18: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Static vs DynamicAnomaly Detection (Cont’d)

• Dynamic Analysis is analysis done as a program is executing and is based on intermediate values that result from the program’s execution.

• E.g., A division by 0 error is caught by dynamic analysis.

• If a data-flow anomaly can be detected by static analysis then the anomaly does not concern testing. (Should be handled by the compiler.)

Page 19: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Anomaly Detection Using Compilers

• Compilers are able to detect several data-flow anomalies using static analysis.

• E.g., By forcing declaration before use, a compiler can detect anomalies such as:– -u– ku

• Optimizing compilers are able to detect some dead variables.

Page 20: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Is Static Analysis Sufficient?

• Questions:

• Why isn’t static analysis enough?

• Why is testing required?

• Could a good compiler detect all data-flow anomalies?

• Answer: No. Detecting all data-flow anomalies is provably unsolvable.

Page 21: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Static Analysis Deficiencies

• Current static analysis methods are inadequate for:– Dead Variables: Detecting unreachable

variables is unsolvable in the general case. – Arrays: Dynamically allocated arrays contain

garbage unless they are initialized explicitly. (-u anomalies are possible)

Page 22: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Static Analysis Deficiencies (Cont’d)

– Pointers: Impossible to verify pointer values at compile time.

– False Anomalies: Even an obvious bug (e.g., ku) may not be a bug if the path along which the anomaly exists is unachievable. (Determining whether a path is or is not achievable is unsolvable.)

Page 23: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Modeling

• Data-flow modeling is based on the control flowgraph.

• Each link is annotated with:– symbols (e.g., d, k, u, c, p)– sequences of symbols (e.g., dd, du, ddd)

• that denote the sequence of data operations on that link with respect to the variable of interest.

Page 24: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Control Flowgraph Annotated for X and Y Data Flows

1 3 4 5 6 7

2 8910111213

Z?

END YY U,V? U,V?

U,Z?ELL

SAM

LOOP B(U)?dcc

JOE

Z?

1 INPUT X,Y Z:= X+Y Y:= X-Y3 IF Z>=0 GOTO SAM4 JOE: Z:=Z-15 SAM: Z:=Z+V

U:=06 LOOP

B(U),Q(V):=(Z+V)*U7 IF B(U)=0 GOTO JOE

Z:=Z-18 IF Z=0 GOTO ELL

U:=U+19 UNTIL U=z

B(U-1):=B(U+1)+Q(V-1)10 ELL: B(U+Q(V)):=U+V11 IF U=V GOTO JOE12 IF U>V THEN U:=Z13 YY:Z:=U2 END

Page 25: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Control Flowgraph Annotated for Z Data Flows

1 3 4 5 6 7

2 8910111213

Z?

END YY U,V? U,V? U,Z?

ELL

SAM

LOOP B(U)?d JOE

Z?

p

p cd cd c

cdp

d c p p

p

1 INPUT X,Y Z:= X+Y Y:= X-Y3 IF Z>=0 GOTO SAM4 JOE: Z:=Z-15 SAM: Z:=Z+V

U:=06 LOOP

B(U),Q(V):=(Z+V)*U7 IF B(U)=0 GOTO JOE

Z:=Z-18 IF Z=0 GOTO ELL

U:=U+19 UNTIL U=z

B(U-1):=B(U+1)+Q(V-1)10 ELL: B(U+Q(V)):=U+V11 IF U=V GOTO JOE12 IF U>V THEN U:=Z13 YY:Z:=U2 END

Page 26: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Definition-Clear Path Segments

• A Definition-clear Path Segment (w.r.t. variable X) is a connected sequence of links such that X is defined on the first link and not redefined or killed on any subsequent link of that path segment.

Page 27: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Definition-Clear Path Segments for Variable Z (Cont’d)

1 3 4 5 6 7

2 8910111213

Z?

END YY U,V? U,V?U,Z?

ELL

SAM

LOOP B(), U?d

JOE

Z?

p

p cd cd c

cdp

d c p p

p

Page 28: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Non Definition-Clear Path Segments for Variable Z (Cont’d)

1 3 4 5 6 7

2 8910111213

Z?

END YY U,V? U,V?U,Z?

ELL

SAM

LOOP B(), U?d

JOE

Z?

p

p cd cd c

cdp

d c p p

p

Page 29: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Simple Path Segments

• A Simple Path Segment is a path segment in which at most one node is visited twice.– E.g., (7,4,5,6,7) is simple.

• Therefore, a simple path may or may not be loop-free.

Page 30: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Loop-free Path Segments

• A Loop-free Path Segment is a path segment for which every node is visited at most once. – E.g., (4,5,6,7,8,10) is loop-free.– path (10,11,4,5,6,7,8,10,11,12) is not loop-free

because nodes 10 and 11 are visited twice.

Page 31: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du Path Segments

• A du Path is a path segment such that if the last link has a use of X, then the path is simple and definition clear.

Page 32: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

def-use Associations

• A def-use association is a triple (x, d, u,), where:

x is a variable, d is a node containing a definition of x, u is either a statement or predicate node containing a use of x,

and there is a sub-path in the flow graph from d to u

with no other definition of x between d and u.

Page 33: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: Def-Use Associations

1

3

4 5

read (x, y)

w = x + 1

T F

y = y + 1

6 print (x,y,w,z)

2 z = x + 2

z < y

Some Def-Use Associations:

(x, 1, 2), (x, 1, 4), …

(y, 1, (3,t)), (y, 1, (3,f)), (y, 1, 5), …

(z, 2, (3,t)),...

Page 34: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: Def-Use Associations

What are all the def-use associations for the program below?

read (z)x = 0 y = 0if (z 0){

x = sqrt (z)if (0 x && x 5)

y = f (x) else

y = h (z)}y = g (x, y)print (y)

Page 35: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: Def-Use Associations

read (z)x = 0 y = 0if (z 0){

x = sqrt (z)if (0 x && x

5)y = f (x)

elsey = h (z)

}y = g (x, y)print (y)

def-use associations for variable z.

Page 36: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: Def-Use Associationsread (z)x = 0 y = 0if (z 0){

x = sqrt (z)if (0 x && x

5)

y = f (x) else

y = h (z)}y = g (x, y)print (y)

def-use associations for variable x.

Page 37: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: Def-Use Associationsread (z)x = 0 y = 0if (z 0){

x = sqrt (z)if (0 x && x

5)y = f (x)

elsey = h (z)

}y=g (x, y)print (y)

def-use associations for variable y.

Page 38: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Definition-Clear Paths

• A path (i, n1, ..., nm, j) is called a definition-clear path with respect to x from node i to node j if it contains no definitions of variable x in nodes (n1, ..., nm , j) .

• The family of data flow criteria requires that the test data execute definition-clear paths from each node containing a definition of a variable to specified nodes containing c-use and edges containing p-use of that variable.

Page 39: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Testing Strategies

• All du Paths (ADUP)

• All Uses (AU)

• All p-uses/some c-uses (APU+C)

• All c-uses/some p-uses (ACU+P)

• All Definitions (AD)

• All p-uses (APU)

• All c-uses (ACU)

Page 40: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

All du Paths Strategy (ADUP)

• ADUP is one of the strongest data-flow testing strategies.

• ADUP requires that every du path from every definition of every variable to every use of that definition be exercised under some test All du Paths Strategy (ADUP).

Page 41: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: pow(x,y)/* pow(x,y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout.*/1 void pow (int x, y)2 {3 float z;4 int p;5 if (y < 0)6 p = 0 – y;7 else p = y;8 z = 1.0;9 while (p != 0)10 {11 z = z * x;12 p = p – 1;13 }14 if (y < 0)15 z = 1.0 / z;16 printf(z);17 }

1 5 8 9 1614 17a

b

c

d

e

f

g

h

i

Page 42: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: pow(x,y)du-Path for Variable x

/* pow(x,y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout.*/1 void pow (int x, y)2 {3 float z;4 int p;5 if (y < 0)6 p = 0 – y;7 else p = y;8 z = 1.0;9 while (p != 0)10 {11 z = z * x;12 p = p – 1;13 }14 if (y < 0)15 z = 1.0 / z;16 printf(z);17 }

1 5 8 9 1614 17a

b

c

d

e

f

g

h

i

Page 43: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: pow(x,y)du-Path for Variable x

/* pow(x,y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout.*/1 void pow (int x, y)2 {3 float z;4 int p;5 if (y < 0)6 p = 0 – y;7 else p = y;8 z = 1.0;9 while (p != 0)10 {11 z = z * x;12 p = p – 1;13 }14 if (y < 0)15 z = 1.0 / z;16 printf(z);17 }

1 5 8 9 1614 17a

b

c

d

e

f

g

h

i

Page 44: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: pow(x,y)du-Path for Variable y

/* pow(x,y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout.*/1 void pow (int x, y)2 {3 float z;4 int p;5 if (y < 0)6 p = 0 – y;7 else p = y;8 z = 1.0;9 while (p != 0)10 {11 z = z * x;12 p = p – 1;13 }14 if (y < 0)15 z = 1.0 / z;16 printf(z);17 }

1 5 8 9 1614 17a

b

c

d

e

f

g

h

i

Page 45: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: pow(x,y)du-Path for Variable y

/* pow(x,y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout.*/1 void pow (int x, y)2 {3 float z;4 int p;5 if (y < 0)6 p = 0 – y;7 else p = y;8 z = 1.0;9 while (p != 0)10 {11 z = z * x;12 p = p – 1;13 }14 if (y < 0)15 z = 1.0 / z;16 printf(z);17 }

1 5 8 9 1614 17a

b

c

d

e

f

g

h

i

Page 46: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Example: pow(x,y)du-Path for Variable y

/* pow(x,y) This program computes x to the power of y, where x and y are integers. INPUT: The x and y values. OUTPUT: x raised to the power of y is printed to stdout.*/1 void pow (int x, y)2 {3 float z;4 int p;5 if (y < 0)6 p = 0 – y;7 else p = y;8 z = 1.0;9 while (p != 0)10 {11 z = z * x;12 p = p – 1;13 }14 if (y < 0)15 z = 1.0 / z;16 printf(z);17 }

1 5 8 9 1614 17a

b

c

d

e

f

g

h

i

Page 47: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

/* COUNT This program counts the number of characters and lines in a text file. INPUT: Text File OUTPUT: Number of characters and number of lines.*/1                  main(int argc, char *argv[])2                  {3                  int numChars = 0;4                  int numLines = 0;5                  char chr;6                  FILE *fp = NULL;7                 

Example: Using du-Path Testing to Test Program COUNT

• Consider the following program:

Page 48: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

8                  if (argc < 2)

9                  {

10              printf(“\nUsage: %s <filename>”, argv[0]);

11              return (-1);

12              }

13              fp = fopen(argv[1], “r”);

14              if (fp == NULL)

15              {

16              perror(argv[1]); /* display error message */

17              return (-2);

18              }

Program COUNT (Cont’d)

Page 49: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

19 while (!feof(fp))

20              {

21              chr = getc(fp); /* read character */

22              if (chr == ‘\n’) /* if carriage return */

23              ++numLines;

24              else

25              ++numChars;

26              }

27              printf(“\nNumber of characters = %d”, numChars);

28              printf(“\nNumber of lines = %d”, numLines);

29              }

Program COUNT (Cont’d)

Page 50: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

The Flowgraph for COUNT

1 8 11 17 19 22 24 26 2914 23

• The junction at line 12 and line 18 are not needed because if you are at these lines then you must also be at line 14 and 19 respectively.

Page 51: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for argc

1 8 11 17 19 22 24 26 2914 23d

argc?

p

p

Page 52: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for argc

1 8 11 17 19 22 24 26 2914 23d

argc?

p

p

Page 53: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for argv[]

1 8 11 17 19 22 24 26 2914 23d

argc?

c

c c

fp?

Page 54: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for argv[]

1 8 11 17 19 22 24 26 2914 23d

argc?

c

Page 55: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for numChars

1 8 11 17 19 22 24 26 2914 23d

argc?

c

cd

fp?

fp?

chr?

Page 56: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for numChars

1 8 11 17 19 22 24 26 2914 23d

argc?

c

cd

fp?

fp?

chr?

Page 57: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for numChars

1 8 11 17 19 22 24 26 2914 23d

argc?

c

cd

fp?

fp?

chr?

Page 58: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for numLines

1 8 11 17 19 22 24 26 2914 23d

argc?

c

cd

fp?

fp?

chr?

Page 59: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for numLines

1 8 11 17 19 22 24 26 2914 23d

argc?

c

cd

fp?

fp?

chr?

Page 60: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for numLines

1 8 11 17 19 22 24 26 2914 23d

argc?

c

cd

fp?

fp?

chr?

Page 61: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for chr

1 8 11 17 19 22 24 26 2914 23d

argc?

pd

fp?

fp?

chr?p

Page 62: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for chr

1 8 11 17 19 22 24 26 2914 23d

argc?

pd

fp?

fp?

chr?p

Page 63: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for fp

1 8 11 17 19 22 24 26 2914 23d

argc?

p

d

fp?

fp?

chr?

pcp

p

Page 64: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for fp

1 8 11 17 19 22 24 26 2914 23d

argc?

p

d

fp?

fp?

chr?

pcp

p

Page 65: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

du-Path for fp

1 8 11 17 19 22 24 26 2914 23d

argc?

p

d

fp?

fp?

chr?

pcp

p

Page 66: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

All Uses Strategy (AU)

• AU requires that at least one path from every definition of every variable to every use of that definition be exercised under some test.

• Hence, at least one definition-clear path from every definition of every variable to every use of that definition be exercised under some test.

• Clearly, AU < ADUP.

Page 67: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

All p-uses/Some c-uses Strategy (APU+C)

• APU+C requires that for every variable and every definition of that variable include at least one definition-free path from the definition to every predicate use.

• If there are definitions of the variable that are not covered by the above prescription, then add computational-use test cases to cover every definition.

Page 68: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

All c-uses/Some p-uses Strategy (ACU+P)

• ACU+P requires that for every variable and every definition of that variable include at least one definition-free path from the definition to every computational use.

• If there are definitions of the variable that are not covered by the above prescription, then add predicate-use test cases to cover every definition.

Page 69: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

All Definitions Strategy (AD)

• AD requires that for every variable and every definition of that variable include at least one definition-free path from the definition to a computational or predicate use.

• AD < ACU+P and AD < APU+C.

Page 70: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

All p-uses (APU) All c-uses (ACU)

• APU is the same as APU+C without the C requirement.

• APU < APU+C.

• ACU is the same as ACU+P without the P requirement.

• ACU < ACU+P.

Page 71: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Relationship among DF criteriaALL-PATHS

ALL-DU-PATHS

ALL-USES

ALL-C-USES/SOME-P-USESALL-P-USES/SOME-C-USES

ALL-DEFS

ALL-C-USESALL-P-USES

ALL-EDGES

ALL-NODES

Page 72: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Feasible Data Flow Criteria

• What happens if we eliminate all un-executable associations from consideration?

• If we eliminate all un-executable associations from consideration, then there are significant differences between the Data Flow criteria and the Feasible Data Flow criteria.

• For a large class of “well behaved programs”, the Feasible DF criteria All-p-uses, All-p-uses/some-c-uses, and All-uses bridge the gap between All-edges and All-paths.

• However, for certain programs with anomalies there are tests which satisfy All-p-uses without satisfying All-edges.

Page 73: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

An Example

1

3

4 5

read (x)

read (y)

T F

2 x = sqrt(x)

x < 0

6 y 0

T F

7 8

T F

6

9

• Node 4 is un-executable; y is un-initialized.

• Essentially, only def-use of concern is (x, 2, 3)

• Outcome of node 6 can be either T or F.

• All-p-uses satisfied, but not all-edges.

Page 74: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Effectiveness of Strategies

• Ntafos compared Random, Branch, and All uses testing strategies on 14 Kernighan and Plauger programs.

• Kernighan and Plauger programs are a set of mathematical programs with known bugs that are often used to evaluate test strategies.

• Ntafos conducted two experiments:

Page 75: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Results of 2 of the 14 Ntafos Experiments

Random

Branch

All Uses

StrategyMean Numberof Test Cases

Percentage ofBugs Found

35

3.8

11.3

93.7

91.6

96.3

Random

Branch

All Uses

StrategyMean Numberof Test Cases

Percentage ofBugs Found

100

34

84

79.5

85.5

90.0

Page 76: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Testing Tips

• Resolve all data-flow anomalies.

• Try to do all data-flow operations on a variable within the same routine (i.e., avoid integration problems).

• Use strong typing and user defined types when possible.

Page 77: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Data-Flow Testing Tips (Cont’d)

• Use explicit (rather than implicit) declarations of data when possible.

• Put data declarations at the top of the routine and return data objects at the bottom of the routine.

Page 78: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Summary

• Data are as important as code.

• Define what you consider to be a data-flow anomaly.

• Data-flow testing strategies span the gap between all paths and branch testing.

Page 79: © SERG Dependable Software Systems (Data-Flow Testing) Dependable Software Systems Topics in Data-Flow Testing Material drawn from [Beizer, Mancoridis,

© SERGDependable Software Systems (Data-Flow Testing)Dependable Software Systems (Data-Flow Testing)

Summary

• AU has the best payoff for the money. It seems to be no worse than twice the number of required test cases for branch testing, but the results are much better.

• Path testing with Branch Coverage and Data-flow testing with AU is a very good combination.