chapter_14

Testing Tactics

TCS2411 Software Engineering 1

Software Testing Fundamentals

Testability How easily a computer program can be

tested.What are the characteristics of testability?1.operability 5.Simplicity2.Observability 6.Stability3.Controllability 7.Understandability4.Decompsability


Operability 1. The better the software works, the more efficiently it can be tested.2. The system has few bugs (bugs add analysis and reporting overhead to the test process)3. No bugs block the execution of tests.4. The product evolves in functional stages (allows simultaneous development & testing


Observability in Software Testing:

1.What is seen is what is tested2. Distinct output is generated for each input3. System states and variables are visible or queriable during execution4. Past system states and variables are visible or queriable (eg., transaction logs)5. All factors affecting the output are visible6. Incorrect output is easily identified7. Incorrect input is easily identified8. Internal errors are automatically detected through self-testing mechanism9. Internally errors are automatically reported10. Source code is accessible


Controllability in Software Testing

1. The better the software is controlled, the more the testing can be automated and optimized.2. All possible outputs can be generated through some combination of input in Software Testing3. All code is executable through some combination of input in Software Testing4. Software and hardware states can be controlled directly by testing5. Input and output formats are consistent and structured in Software Testing6. Tests can be conveniently specified, automated, and

reproduced.


Decomposability in Software Testing:

1. By controlling the scope of testing, problems can be isolated quickly, and smarter testing can be performed.

2. The software system is built from independent modules

3. Software modules can be tested independently in Software Testing


Simplicity in Software Testing:

1. The less there is to test, the more quickly it can be tested in Software Testing2. Functional simplicity3. Structural simplicity4. Code simplicity


Stability in Software Testing:

1. The fewer the changes, the fewer the disruptions to testing

2. Changes to the software are infrequent

3. Changes to the software are controlled in Software Testing

4. Changes to the software do not invalidate existing tests in Software Testing

5. The software recovers well from failures in Software Testing


Understandability in Software Testing

1. The more information we have, the smarter we will test2. The design is well understood in Software Testing3. Dependencies between internal external and shared components are well understood.4. Changes to the design are communicated.5. Technical documentation is instantly accessible6. Technical documentation is well organized in Software Testing7. Technical documentation is specific and detailed8. Technical documentation is accurate



Methods

Strategies

white-box

methods

black-box

methods

Software Testing


White Box Testing

Derived from knowledge of program’s structure & implementation

Structural testing - analyse code & use knowledge of the structure of a component to derive test data

Logical paths are tested by providing test cases that exercise specific sets of conditions and/or loops


White Box Testing (Continued)

Thorough white box testing would lead to “100 percent correct programs”

Exhaustive testing are impractical - too many tests!

A limited number of logical paths can be selected and exercised

Important data structures can be probed for validity


White Box Test Cases

Guarantee that all independent paths have been exercised at least once

Exercise all logical decisions on their true and false sides

Execute all loops at their boundaries and within their operational bounds

Exercise internal data structures to ensure their validity


White Box Testing Techniques

Basis path testing Flow graph notation Cyclomatic complexity Derived test cases Graph metrics

Control structure testing Condition testing Data Flow testing Loop testing


Flow Graph Notation

1

2

3

6

7 8

4

5

9

1011

1

2,3

6

7 8

9

10

11

4,5

NodeEdge

R3

R2

R1

R4

Region


Cyclomatic Complexity

Provide quantitative measure for program logical complexity.

Defined number of independent path Any path that introduce one ser of processing

statements or new conditionEg :-

Path 1 : 1-11 Path 2 : 1-2-3-4-5-10-1-11 Path 3 : 1-2-3-6-8-9-10-1-11 Path 4 : 1-2-3-6-7-9-10-1-11


How Is Cyclomatic Complexity Computed?

1. Number of regions The flow graph has 4 regions

2. V(G) = E – N + 2 E : Number of flow graph edges N : Number of flow graph nodes

V(G) = 11 edges – 9 nodes + 2 = 4

3. V(G) = P + 1 P : Number of predicate nodes

V(G) = 3 predicate nodes + 1 = 4


Deriving Test Cases1. Draw flow graph from design/code as foundation.

…i=1;total.input = total.valid=0;sum=0;do while value[i] <> -999 and total.input<100 increment total.input by 1; if value[i] >= minimum AND value[i] <= maximum then increment total.valid by 1; sum = sum + value[i] else skip end if increment i by 1End doIf total.valid > 0 then average = sum / total valid; else average = -999;End if…

1

2

3

4

56

7

8

9

10

1112

13

1

2

3

4

5

6

78

9

10

1112

13


Deriving Test Cases (cont)

2. Determine cyclomatic complexity V(G) = 6 regions V(G) = 17 edges – 13 nodes + 2 = 6 V(G) = 5 predicates nodes + 1 = 6

3. Determine a basis set of linearly independent graph Path 1 = 1-2-10-11-13 Path 2 = 1-2-10-12-13 ….

4. Prepare test cases that will force execution of each path in the basis set.

Data should be chosen so that conditions at the predicate nodes are appropriately set as each path is tested.

Each test case is executed and compared to expected result.


Discussion on White Box Testing

Advantages Find errors on code level Typically based on a very systematic

approach, covering the complete internal module structure

Constraints Does not find missing or additional

functionality Does not really check the interface Difficult for large and complex module

Control structure testing

Condition testing Data Flow testing Loop testing


23

DEF(s)={x/statement S contains definition of x}

USE(s)={x/statement S contains a use of x}

For a statement with S as its statement number

A definition-use chain of variable x is of the form [X,S,S’],where S,S’ are statement no,X is in DEF(S) and USE(S’) and the definition of X in statement S is live at

statement S’.

Loop testing is a white-box testing

technique that focuses exclusively on the validity of loop constructs.


Loop testing is a white-box testing technique that focuses exclusively on the validity of loop constructs.



Black Box Testing

Derived from program specificationFunctional testing of a component of

a systemExamine behaviour through inputs &

the corresponding outputsInput is properly accepted, output is

correctly producedDisregard internal logical structure


Black Box Testing (Continued)

Attempts to find the following errors:Incorrect or missing functionsInterface errorsErrors in data structures or external

database accessPerformance errorsInitialisation and termination errors


Black Box Testing Techniques

Graph Based Testing MethodsEquivalence PartitioningBoundary Value Analysis


Equivalence Partitioning

Divide input domain into classes of dataBased on an evaluation of equivalence classes

for an input conditionGuidelines to define equivalence classes

Range input : One valid and two invalid equivalence Specific value : One valid and two invalid

equivalence A member of a set : One valid and one invalid

equivalence Boolean : One valid and one invalid equivalence

Equivalence Partitioning

In this method ,the input domain data is divided into different equivalence data classes. This method is typically used to reduce the total number of test cases to a finite set of testable test cases, still covering maximum requirements.



Example – Data for Automated Banking Application

The use can access the bank using his personal computer, provide a six digit password, and follow with a series of typed commands that trigger various banking function. During the log on sequence the software supplied for the banking application accepts data in the form:

area code – blank or 3 digit numbersprefix – 3 digit numbers, nor beginning with 0 or 1suffix – 4 digit numberspassword – 6 digits alphanumeric stringscommands – “check”, “deposit”, “bill pay” etc

Input condition

area code : Input condition : Boolean – area code may or may not present Input condition : Range – 200 – 999 with specific exception

prefix : Input condition : Range – specified value > 200 with no 0 digitssuffix : Input condition : Value – 4 digit lengthpassword : Input condition : Boolean – password may or may not present

Input condition : Value – six character stringcommand : Input condition : Set – containing command

Boundary Value Analysis

It’s widely recognized that input values at the extreme ends of input domain cause more errors in system. More application errors occur at the boundaries of input domain. ‘Boundary value analysis’ testing technique is used to identify errors at boundaries rather than finding those exist in center of input domain.



Boundary Value Analysis

Complement equivalence partitioningTest both sides of each boundary Look at output boundaries for test

cases Test min, min-1, max, max+1, typical

valuesExample : 1 <= x <=100

Valid : 1, 2, 99, 100 Invalid : 0 and 101


Discussion on Black Box Testing

Advantages Find missing functionality Independent from code size and

functionality Find some coding errors

Constraints No systematic search for low level errors Specification errors not found


References

“Software Engineering: A Practitioner’s Approach” 5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001

“Software Engineering” by Ian Sommerville, Addison-Wesley, 2001

chapter_14

Documents

software system

software testing1

software modules

smarter testing

scope of testing

selftesting mechanism

path path path path

white box test casesguarantee