Domain testing

Tor Stålhane

Domain testing revisited

We have earlier looked at domain testing as a simple strategy for selecting test cases. We will now take this a step further.

Testers have frequently observed that domain boundaries are particularly fault prone and should therefore be carefully checked.

Predicates

We will assume that all predicates are simple. This implies that they contain only one relational operator. Allowable operators are =, ><, >=, <=, < and >.

Predicates that are not simple can be split into two or more simple predicates.

Path conditions

Each path thorough a system is defined by a path condition – the conjunction of all predicates encountered along this path.

An input traverses a path if and only if the path condition is fulfilled.

The path condition defines a path domain, which is the set of all inputs that cause the path to be executed.

Path domains – 1 A path domain is surrounded by a boundary,

consisting of one or more borders where each border corresponds to a predicate.

A border is• Closed if defined by a predicate containing

=, =< or >=. Closed borders belong to the path domain.

• Open if defined by a predicate containing ><, > or >. Open borders do not belong to the path domain.

Path domains – 2Note the difference between open and

closed domains.

Closed: what is on the line belongs to the domain

Open: what is off the line – below or above – belongs to the domain

X

X

X

Domain error

A domain error occurs if an input traverses the wrong path through the program.

We have no way to know the correct borders and there is no unique correct version of the program.

When domain error occurs along a path, it may be thought of as being caused by one of the given borders being different from the correct one.

Path domains

P1

P2

S1

S5S4

S3

S2 P1 and P2: {S1, S3,S4}

P1 and not P2: {S1, S3, S5}

not P1 and P2: {S2, S3, S4}

not P1 and not P2: {S2, S3, S5}

ON and OFF points – 1 The test strategy is a strategy for selecting

ON and OFF points, defined as follows:• ON point for a

– Closed border lies on the border– Open border lies close to the border and

satisfies the inequality relation• OFF point lies close to the border and on

the open side or – alternatively – does not satisfy the path condition associated with this border

ON and OFF points – 2

The ON and OFF points are used as follows:• For testing a closed border we use

– Two ON points to identify the border– One OFF point, to test that the correct border

does not lie on the open side of the border• For testing an open border the role of the

ON and OFF points are reversed.The strategy can be extended to N-

dimensional space by using N ON points

ON and OFF points – 3

If the border line has V vertices, we will need• One ON point close to each vertex.• One OFF point per vertex at a uniform

distance from the border.

In all cases, it is important that the OFF points are as close as possible to the ON points

Example – two-dimensional space

Correct border

Given border

OFF

ON

ON

Example for an open border: ><, < or >. Border outside the line

Example – two-dimensional space

Given border

Correct border

OFF

ON

ON

Example for an closed border: “=“ predicate. Border on the line

OFF

The problem of sizeThe main problem with this strategy is the

cost. Let us assume we have 20 input variables. Then the suggested strategy would need:

• For each > or <: 20 ON points and one OFF point

• For each = or ><: 20 ON points plus two or three OFF points

Ten predicates would require 210 – 230 test cases.

The problem of precision

The strategy described require the ON points to lie exactly on the border.

For any pair of real numbers, there is always a third real number that lies between them. For a computer, however, this is not the case, due to limited precision.

Thus, there exist borders for which no ON point can be represented in the computer.

A simplified strategyWe will drop the requirement that the border

can be exactly identified. Then we can also drop the requirement that the ON point lies exactly on the border. This remove the precision problem.

In addition we can reduce the number of points by one per border. The only error that will not be detected is if the real border passes between an ON and an OFF point. Thus, these two points need to be close.

Simplified use of On and OFF points

>, < =, ><ON ON

OFF

OFF

OFF

Effectiveness

>, <ON

OFF

L

Assume that • m1 is the smallest real value• m2 is the largest real value

The length of L, called M, isM = m2 – m1 +1

P(not detect) = / M

Code containing array references

Code segment. The array b has 10 elements.i = x + 3;IF b[i] <=5 THEN ,,,ELSE …We need three predicates:• b[x + 3] <= 5 – the original predicate• x + 3 >= 1 – not below lower bound• x + 3 >= 10 – not above upper bound

Non-linear borders – 1

Everything discussed till now has been based on the assumption that the real border is linear. If this is not true, the domain strategy might fail.

In the example on the next slide, the ON points are on the border, OFF point is off the border but nevertheless the given border is wrong.

Non-linear borders – 2

Given border

OFF

ON

ON

Correct border

A simple algorithmWe can apply domain testing as follows:1. Select a test case and run it. This will

cause one path to be traversed.2. Identify the borders of this path and test

them with ON and OFF points3. If the OFF point belongs to a new path

then this path is selected for testing otherwise check another OFF point.

4. Terminate when no new path can be found

Simple algorithm - example

IF x in A THEN S1ELSEIF x in B THEN S2ELSES3

A

BC

12

3

When to use domain testing

Domain testing, as it is described here, requires that we know how the input partition the input space into input domains.

Thus, it is only possible to use it for small chunks of code.

Acknowledgement

This set of slides is based on the paper “A Simplified Domain-Testing Strategy” by B. Jeng and E.J. Weyuker.