1 / 28

CS 425/625 Software Engineering

Verification and Validation

Based on Chapter 19 of the textbook [SE-6] Ian Sommerville,Software Engineering, 6th Ed., Addison-Wesley, 2000 and on theCh19 PowerPoint presentation available at the book’s web-site:www.comp.lancs.ac.uk/computing/resources/IanS/SE6/Slides/index.html

November 10, 2003

2 / 28

Outline

Introduction Testing and Debugging Verification and Validation Planning Software Inspections Program Inspections (forms of Software

Inspections) Automatic Static Analysis

3 / 28

Introduction…..

Verification and validation (V&V): the checking and analysis processes that ensure the software satisfies its specification and meets the needs of the clients who are paying for it

Validation: “Are we building the right product?” Verification: “Are we building the product right?” Verification involves checking the software conforms

with its specification while the more general process of validation ensures the software meets the needs of the clients

V&V is a whole life-cycle process, encompassing requirements reviews, design reviews, code inspections, and program testing

4 / 28

.Introduction.…

V & V techniques:V & V techniques: Software inspectionsSoftware inspections Software testingSoftware testing

Software inspections Are static V&V techniques as they do not require the Are static V&V techniques as they do not require the

software to be executed software to be executed Consist of inspections, automated static analyses, and formal Consist of inspections, automated static analyses, and formal

verifications of source code or system modelsverifications of source code or system models Can only check the correspondence between the software

and its specification Cannot demonstrate the system is operationally useful Cannot check non-functional requirements of the software

5 / 28

..Introduction...

Software testing It is a dynamic V&V technique as it needs an

executable version of the software system The system is executed with test data and its

operational behaviour is assessed Can reveal the presence of errors, not their absence A successful test is a test which discovers one or

more errors The V&V technique for checking non-functional

requirements and for verifying system integration Should be used in conjunction with static techniques

to provide full V&V coverage

6 / 28

…Introduction..

Types of software testing Defect testing

Intended to find inconsistencies between a program and its specification

Tests designed to discover program faults and defects A successful defect test is one which reveals the presence

of defects in a system

Statistical testing Designed for software’s performance and reliability

estimation By running tests that reflect actual user inputs and their

frequency, an estimate of operational reliability can be made

7 / 28

….Introduction.

Static and dynamic V&V [Fig 19.1, Somm00]

Formalspecification

High-leveldesign

Requirementsspecification

Detaileddesign

Program

PrototypeDynamicvalidation

Staticverification

8 / 28

…..Introduction

Verification and validation should establish confidence that the software is fit for purpose

This does not mean the software is completely free of defects; rather, it must be good enough for its intended use

The required level of confidence depends on: Software’s purpose: the level of confidence depends on

how critical the software is to an organisation User expectations: users may have lower expectations

of certain types of software Marketing environment: getting a product to market

early may have higher priority than finding its defects

9 / 28

Testing and debugging.

Defect testing and debugging are distinct processes

V&V is concerned with establishing the existence of defects in a program

Debugging is concerned with locating and repairing these errors

Debugging involves formulating hypotheses about program behaviour then testing these hypotheses to find the errors

10 / 28

.Testing and debugging

The debugging process [Fig. 19.2, Somm00]

Locateerror

Designerror repair

Repairerror

Re-testprogram

Testresults Specification Test

cases

11 / 28

The planning of V&V should start early in the development process

The plan should balance static verification and testing, specify testing standards and procedures, establish checklists for inspections, and define the software test plan

Test planning breaks down V&V into a number of stages, often organized according to the V-model (shown on next page)

The focus is on setting standards and procedures for inspections and testing, not on describing product tests

V & V Planning..

12 / 28

.V&V Planning..V&V Planning.

Requirementsspecification

Systemspecification

Systemdesign

Detaileddesign

Module andunit codeand tess

Sub-systemintegrationtest plan

Systemintegrationtest plan

Acceptancetest plan

ServiceAcceptance

testSystem

integration testSub-system

integration test

The V-model of development [Fig. 19.3, Somm00]

13 / 28

..V&V Planning..V&V Planning

The structure of a software test plan The testing process Requirements traceability Tested items Testing schedule Test recording procedures Hardware and software requirements Constraints

14 / 28

Software Inspections.

Involve people examining software code or models (representations) with the aim of discovering defects

Do not require execution of a system thus can be used throughout the development process

May be applied to any representation of the system: requirements, design, test data, etc.

Very effective technique for discovering errors

15 / 28

.Software Inspections

In software inspections many different defects can be discovered in a single review of the source code or software model

In testing, one defect may mask another hence several executions are required

Software inspections reuse domain and programming knowledge so reviewers are likely to have seen the types of error that commonly occur

Software inspections and software testing are complementary, not competing techniques (see also slides 4 and 5)

16 / 28

Program Inspections…….

Program inspections are a type of software inspections

Consist of formal reviews conducted by teams and intended for program defect detection

Defects may be logical errors, anomalies in the code that might indicate an erroneous condition (e.g., an un-initialized variable), or non-compliances with standards

17 / 28

.Program Inspections……

Inspection pre-conditions: A precise specification must be available Team members must be familiar with the

organisation standards Syntactically correct code must be available An error checklist should be prepared Management must accept that inspection will

increase costs early in the software process Management must not use inspections for staff

appraisal

18 / 28

Inspectionmeeting

Individualpreparation

Overview

Planning

Rework

Follow-up

..Program Inspections…..

The inspection process [Fig. 19.6, Somm00]

19 / 28

…Program Inspections….

Program inspection procedure: The inspection is planned by the moderator (or chairman) The system overview is presented to the inspection team by the

program author (or owner) The code and associated documents are distributed to the

inspectors (inspection team) in advance for individual preparation

The inspection meeting takes place and errors are noted (the inspection also involves a reader and, possibly, a scribe)

During re-work modifications are made by the author to repair discovered errors

Re-inspection may or may not be required, based on moderator’s decision

20 / 28

….Program Inspections…

Inspection teams are Inspection teams are made up of at least 4 members: Author of the code being inspected Inspector who finds errors, omissions and

inconsistencies Reader who reads the code to the team Moderator who chairs the meeting and notes

discovered errors Other roles are chief moderator and scribe

21 / 28

…..Program Inspections..

Inspection checklist A checklist of common errors should be used

during the inspection This error checklist is programming language

dependent The weaker the type-checking of the language,

the larger the checklist Examples of possible errors: initialisation,

constant naming, loop termination, array bounds, etc.

22 / 28

Inspection checks (fault classes) [Fig. 19.7, Somm00]

Fault class Inspection checkData faults Are all program variables initialised before their values

are used?Have all constants been named?Should the lower bound of arrays be 0, 1, or somethingelse? Should the upper bound of arrays be equal to the size ofthe array or Size -1?If character strings are used, is a delimiter explicitlyassigned?

Control faults For each conditional statement, is the condition correct?Is each loop certain to terminate?Are compound statements correctly bracketed?In case statements, are all possible cases accounted for?

Input/output faults Are all input variables used?Are all output variables assigned a value before they areoutput?

Interface faults Do all function and procedure calls have the correctnumber of parameters?Do formal and actual parameter types match? Are the parameters in the right order? If components access shared memory, do they have thesame model of the shared memory structure?

Storage managementfaults

If a linked structure is modified, have all links beencorrectly reassigned?If dynamic storage is used, has space been allocatedcorrectly?Is space explicitly de-allocated after it is no longerrequired?

Exceptionmanagement faults

Have all possible error conditions been taken intoaccount?

23 / 28

………….Program Inspections.Program Inspections

Inspection rates: 500 statements/hour during overview 125 source statement/hour during individual

preparation 90-125 statements/hour can be inspected Inspection is therefore an expensive process Inspecting 500 lines costs about 40 man/hours

effort = £2800

24 / 28

Automated Static Analysis….

Static analyzers are software tools for source code processing

They parse the program text to discover erroneous conditions

Very effective as an aid to inspections; supplement but cannot replace inspections

Particularly valuable for languages such as C that have weak typing (many errors can remain undetected by the compiler)

Less cost-effective for languages such as Java that have strong type checking (many errors can be detected during compilation)

25 / 28

.Automated Static Analysis…

Fault class Static analysis checkData faults Variables used before initialisation

Variables declared but never usedVariables assigned twice but never usedbetween assignmentsPossible array bound violations Undeclared variables

Control faults Unreachable codeUnconditional branches into loops

Input/output faults Variables output twice with no interveningassignment

Interface faults Parameter type mismatchesParameter number mismatchesNon-usage of the results of functionsUncalled functions and procedures

Storage managementfaults

Unassigned pointersPointer arithmetic

Automatic static analysis checks [Fig. 19.8, Somm00]

26 / 28

..Automated Static Analysis..

Stages of static analysis: Control flow analysis. Checks for loops with

multiple exit or entry points, finds unreachable code, etc.

Data use analysis. Detects un-initialised variables, variables written twice without an intervening assignment, variables which are declared but never used, etc.

Interface analysis. Checks the consistency of routine and procedure declarations and their use

27 / 28

…Automated Static Analysis.

Stages of static analysis [cont’d]:

Information flow analysis. Identifies the dependencies of output variables. Does not detect anomalies itself but highlights information for code review (inspection)

Path analysis. Identifies paths through the program and sets out the statements executed in that path. Again, potentially useful in the review process

28 / 28

LINT static analysis

[Fig 19.9, Somm00]

138% more lint_ex.c

#include <stdio.h>printarray (Anarray) int Anarray;{ printf(“%d”,Anarray);}main (){ int Anarray[5]; int i; char c; printarray (Anarray, i, c); printarray (Anarray) ;}

139% cc lint_ex.c140% lint lint_ex.c

lint_ex.c(10): warning: c may be used before setlint_ex.c(10): warning: i may be used before setprintarray: variable # of args. lint_ex.c(4) :: lint_ex.c(10)printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(10)printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(11)printf returns value which is always ignored