static testing

Tester Training Course

Hoang V. Nguyen

[email protected]

startnewday85.blogspot.com

Dept. of Computer Science – FIT - HUA

Tester Training Course

STATIC TESTING

Tester Training Course 2011

implementation

Detailed Design

Global Design

System requirements

User requirements

Unit Testing

Integration Testing

System Testing

Acceptance Testing


• To find defects• Examine the work productfor errors• without executing thecode

• To find defects• Use the product to collectfailures• must execute the code


Early testing principle

• rework cost in average is responsible for ~40% of thetotal software development cost

• engineers spend up to 1/3 of their compiling & testing,relying on these activities to detect defects

Development productivity can be improvedand timescales reduced

Testing costs and time can be reduced


Reductions in lifetime costs

Improved communication results

Gaining understand

Aiding in professional development forteam members


What can be inspected

• anything that is written or typed

What can be tested by static testing

• Requirement specifications

• Design document

• Code

• Schedules

• Test plans, test cases, defect reports


Deviations from standards

Requirements defects

Design defects

Insufficient maintainability

Incorrect interface specifications


is a moderated meeting in whichreviewers list all issues and defectsthey have found in the document andlog them so that they can be addressedby author

commonly inspected work productssuch as SRS, design documents andtest plans


A work product and an inspection team isselected

A moderator is chosen to moderate the meeting

Each inspector prepares for the meeting byreading the work product and noting each defect

Meeting: discussion is focused on each defect,and coming up with a specific resolution

The moderator compiles all of the defectresolutions into a inspection log


Review meeting

Planning Kick off Preparation ReworkFollow

up


Review meeting

Work product and inspection team is selected

Moderator(inspection leader) is chosen

Define entry and exit criteria

Decide which part of work product to review

Assign the roles to the reviewers


up


Review meeting


up

Distributing documents

Explaining the objectives, process, and documents

Checking entry criteria

Note: role assignments, checking rate, the pages to be checked


Review meeting


up

The participants work individually on the work product under review

Can use the related documents, rules and checklists provided

Identify defects, questions and comments and write a report for them

Note: should use checklist

Time

Checking

rate Size


Checklist a list of problems which must be checked

usually in the form of questions

Why• objectively

• based on lessons learned

• as general as possible

can be used by both authors and reviewers

Example checklist for code review• Are comments accurate and meaningful?

• Are all variables declared?


Review meeting


up

The meeting typical consists of: logging, discussion anddecision phase

The meeting is moderated by moderator(inspection leader)

During the logging phase the issues are mentioned page bypage, reviewer by reviewer, and should not discuss

In discussion phase, focused on issues, and classifythem(critical, major and minor)

At the end of the meeting, a decision on the work productunder review has to be made by the participants


Review meeting


up

Based on the defects detected, the author will improve the work product under review step by step

Changes that are made to the document should be easy to identify during follow up


Review meeting


up

The moderator will check that the agreed defects have been addressed

and will gather metrics such as how time was spent on the review, how many defects were found

will also check the exit criteria to ensure that they have been met


is a discussion meeting that focuses onachieving consensus about the technicalcontent

The experts that are needed for a technicalreview

• architects

• chief designers

• key users


Goals

• assess the value of technical concepts and alternativesin the product and project environment

• establish consistency in the use and representation oftechnical concepts

• ensure, at an early stage, that technical concepts areused correctly

• inform participants of the technical content of thedocument


Key characteristics

• is documented

• includes peers and technical experts

• often performed as a peer review with outmanagement participant

• ideally it is lead by a trained moderator, but possiblyalso by technical expert

• Reviewers prepare for review meeting and prepare areview report with a list of findings


The author sends a copy of thework product to selected projectteam members. The teammembers read it, and then writeup defects and comments to sendback to the author


is a simple review

does not produce written logs

can be used as predecessors to inspections

• to reduce the amount of effort involved in theinspection

• to meet entry criteria of inspection process


To present the document to stakeholders bothwithin and outside the software discipline, inorder to gather information regarding the topicunder documentation

To explain (knowledge transfer) and evaluatethe contents of the document

To establish a common understanding of thedocument

To examine and discuss the validity ofproposed solutions and the viability ofalternatives, establishing consensus


A code review is a special kind of inspection inwhich the team examines a sample of code andfixes any defects in it.

In a code review, a defect is a block of code whichdoes not properly implement its requirements,which does not function as the programmerintended, or which is not incorrect but could beimproved

• For example, it could be made more readable or itsperformance could be improved


It’s important to review the code which is most likelyto have defects. This will generally be the mostcomplex, tricky or involved code.

Good candidates for code review include:

• A portion of the software that only one person has theexpertise to maintain

• Code that implements a highly abstract or tricky algorithm

• An object, library or API that is particularly difficult to workwith

• Code written by someone who is inexperienced or has notwritten that kind of code before, or written in an unfamiliarlanguage

• Code which employs a new programming technique

• An area of the code that will be especially catastrophic ifthere are defects


Pair programming is a technique in which twoprogrammers work simultaneously at a singlecomputer and continuously review each others’work

Although many programmers were introduced topair programming as a part of ExtremeProgramming, it is a practice that can be valuablein any development environment.

Pair programming improves the organization byensuring that at least two programmers are ableto maintain any piece of the software


In pair programming, two programmers sit at onecomputer to write code. Generally, one programmer willtake control and write code, while the other watches andadvises

• Some teams have found that pair programming works best forthem if the pairs are constantly rotated; this helps diffuse theshared knowledge throughout the organization. Others prefer topair a more junior person with a more senior for knowledgesharing

The project manager should not try to force pairprogramming on the team; it helps to introduce thechange slowly, and where it will meet the least resistance

• It is difficult to implement pair programming in an organizationwhere the programmers do not share the same nine-to-five (orten-to-six) work schedule.

• Some people do not work well in pairs, and some pairs do notwork well together


Review is a static testing method(without running)

We have many types of review

From informal to formal

How to choose the appropriate level of formality

• The maturity of the development process

• Legal or regulatory requirements


Manager• decides on What is to be reviewed; schedule; closure

Moderator(review leader)• planning the review, moderate review meeting andfollow up after meeting

Author• takes responsibility for fixing any agreed defects

Reviewer• identify and describes defects(with solutions)

Scriber(recorder)

• attends the review meeting and documents all of theissues and defects, problems and open points


Explicitly plan and track review activities


Train participants


Process issues must follow the rules but keep it simple

make the process only as formal as theproject culture or maturity level allows

do not become too theoretical ortoo detailed checklist and roles arerecommended to increasethe effectiveness of

defect identification


Continual Improvement

Process Tools


Cost Benefit


An inspection is lead by the author, whilst a walkthrough is lead by a trained moderator

An inspection has a trained leader, whilst a walkthrough is lead by a trained moderator

A walkthrough is lead by the author, whilst an inspection is lead by trained moderator

What is the main difference between a walkthrough and an inspection?

Question

Authors are not present during inspections, whilst they are during walkthroughs


Reviews cannot be performed on user reuirements specifications

Reviews are the least effective way of testing code

Reviews are unlikely to find faults in test plans

Reviews should be performed on specifications, code and test plans

Which of the following statements about reviews is true?

Question


Walkthrough

Informal Review

Techincal Review

Which review is inexpensive

Question

Inspection


Inspection

Walkthrough

Testing

All of the above

Review is one of the methods of V&V. The other method are

Question


Moderator

Author

Reviewer

In formal review, Rework: fixing defects found typically done by_______

Question

Recorder


Are cheap to perform

Can be performed by the person who wrote the code

Can be performed by inexperienced staff

Enable the code to be tested before the execution enviroment is ready

An important benefit of code inspections is that they:

Question


To identify problems with design

To solve the problems with design

Both A and B

Objective of design review meeting is

Question

None of the above


Formal Reviews

Peer Reviews

Semi Formal Reviews

All of the above

______reviews are often held with just theprogrammer who wrote the code and one or twoother programmers or testers

Question


A walkthrough does not follow a defined process

For a walkthrough individual preparation by the reviewers is optional

A walkthrough requires meeting

Which of the following activities differentiate awalkthrough from a formal review

Question

A walkthrough finds the causes of failures, while formal review finds the failures


It is led by a trained leader, uses formal entry and exit criteria and checklists

It is led by the author of the document to be inspected

It can only be used for reviewing design and code

It is led by the author, uses checklists, and collects data for improvement

What makes an inspection different from otherreview types

Question


A form of automated static testing

• exactly automated review

• check for violations of standards

• check for things which may be fault

often used by developer to automated codereview, and is called Static Code Analysis

also used by designer to review softwaremodels, and is called Analysis of Models


Does the program terminate?

How large can the heap become duringexecution?

What is the possible output?

Some question about PC

• what is value of variable x when it is used

• Can the pointer p be null

• is the variable x initialized before it is read

• what is a lower and upper bound on the value of avariable

• etc


Ensure correctness

• verify behavior

• catch bugs early

Increase efficiency

• resource usage

• compiler optimizations


Static Code Analysis to the rescue


Use an automatic systematic approach

Use formal methods and measures

Without running code => can be applied earlier

Some kinds of defects are hard to find by dynamic testing


Programming(code) standard violations

Security vulnerabilities

Unreachable(dead) code

Undeclared variables

Inconsistent interface

Boundary violations

Syntax violations


The most well-known feature of SCA

Define coding standard

• a set of programming rules

Often configured on IDE

Without such tools, more violations

• the number of rules is usually so large that nobody can remember them all

• some context-sensitive rules that demand reviews of several files are very hard to check by human being

• avoid checking coding standards in review, that will distract them from other defects


Is the study of data flow structure

Data flow structure follows the trail of data items as it is accessed and modified by the code

Typical defects can be found

• referencing a variable with an undefined value

• variables that are never used


Data flow structure

• variable defined where a value is stored into it

• variable used where the stored value is accessed

• variable is undefined before it is defined or when it goes out of scope

y = x+z;

//y is defined; x,z are used

if a>b then read(c);

//a,b are used; c is defined


n := 0;

read (x);

n := 1;

while x > y do

begin

read (y);

write( n*y);

x := x – n;

end;

y is used before it has been defined==> Data flow fault

n is re-defined without being used==> Data flow anomaly


Data structure refers to the organizationof the data itself, independent of theprogram

Provides a lot of information about thedifficulty in writing programs to handle thedata; then is used in designing test cases

Sometimes a program is complexbecause it has a complex data structure,rather than because of complex control ordata flow


Is the study of control flow structure

Control flow structure addresses thesequence in which the instructions areexecuted

Typical defects can be found

• unreachable(dead) code

• infinite loops

• any jumps to undefined labels

• provide the code metrics

• whether code conforms to a flowchart grammar


a := 4;

b := 15;

z := 7;

while b > z do

begin

writeln(z);

z++;

if(a>b) then

b:=a;

end;

unreachable(dead) code


Why

• How big and complex

• Help to decide among several design alternatives

• Help to decide how much risk

Typical code metrics

• LOC – Lines of code

• operands & operators(Halstead’s metrics)

• fan-in & fan-out

• nesting levels

• OO metrics: inheritance tree depth, coupling & cohesion


Is a measure of the complexity of a flowgraph• the code that the flow graph represents

the more complex the flow graph, thegreater the measure

it can most easily be calculated as:

• complexity = number of decisions + 1


1

2 3 5

What is the cyclomatic complexity?


The analysis of batch programs

The reviewing of test plans

The analysis of program code

The use of black box testing

Static analysis is best described as

Question


Possible communications bottlenecks in a program

The rate of change of data values as a program executes

The use of data on paths through the code

Data flow analysis studies

Question

The intrinsic complexity of the code


Same as static testing

Done by the developers

Both A and B

None of the above

Static analysis

Question


Number of independent paths in the basic set of a program

Number of binary decisions + 1

Number bound for the number of test that must be conducted to ensure that all statements have been executed at least one

Cyclomatic complexity is used to calculate

Question

Number of braches and decisions


Type of dynamic testing

Type of static testing

Neither dynamic or static

Performed by the testing team

Code Walkthrough

Question


Compiling code is not a form of static analysis

Static analysis need not be performed before imperative code is executed

Static analysis can find faults that are hard to find with dynamic testing

Which of the following statements is true of static analysis

Question

Extensive static analysis will not needed if white-box testing is to be performed


Inspection

Walkthrough

Technical Review

Formal Review

Peer reviews are also called as:

Question


Ii, iii, iv are correct and i is incorrect

iii, i, iv are correct and ii is incorrect

i, iii, iv, ii are in correct

Success Factors for a review includei. Each review does not have a predefined

objectiveii. Defects found are welcomed and expressed

objectivelyiii. Management supports a good review processiv. There is an emphasis on learning and process

improvement

Question

Ii is correct


Explaining the objective

Fixing defects found typical done by author

Follow up

Individual meeting preparations

The Kick Off phase of a formal review includes the following:

Question


Cheklist

Checkpoint review

Decision table

A series of probing questions about thecompleteness and attributes an applicationsystem is called

Question

Decision tree

static testing

Technology

documenttester training

beenmet tester training

work productunder review

code review

review step

defect meeting

discussion meeting

agreed defects