SOFTWARE INSPECTIONhttp://bzupages.com/

QUALITY: TESTING, INSPECTION, AND ANALYSIS

1. Testing is the most widely used approach to manage software quality

2. Testing and inspection typically account for more resource use than actual design and coding.

3. Testing and inspection cannot find all defects.4. Testing and inspection do not create quality.

Development practices create quality.5. Choices regarding testing and inspection are

influenced by quality metrics visible to management.

6. There are emerging assurance techniques that complement test and inspection

2

Author: M. Muzaffar Hameed Slides available on http://bzupages.com/

SOFTWARE INSPECTIONWhat are software inspections (reviews)? Meetings (real or virtual) during which designs and code are

reviewed bypeople other than the original developer.What are the benefits of inspections?New perspectiveFinding defects may be easier for people who haven't seen the

artifact before and don’t have preconceived ideas about its correctness.Knowledge sharingRegarding designs and specific software artifactsRegarding defect detection practices Find flaws earlyCan dramatically reduce cost of fixing themDuring detail design – even before code is writtenOr code that does not yet have a test harnessOr code in which testing has found flaws but root causes are not

understoodReduce rework and testing effortCan reduce overall development effort

3

Auth

or: M

. Muza

ffar H

am

eed S

lides a

vaila

ble

on

http

://bzu

pages.co

m/

INSPECTION VS TESTING

What attributes are well-handled by inspections but not testing?

“Fuzzy” non-functional properties Maintainability, evolvability, reusabilityOther properties tough to testScalability, efficiencySecurity, integrityRobustness, reliability, exception handlingRequirements, architecture, design documentsCannot “execute” these as a test

4

Auth

or: M

. Muza

ffar H

am

eed S

lides a

vaila

ble

on

http

://bzu

pages.co

m/

KINDS OF INSPECTION1. Inspections / Formal Technical ReviewsParticipation defined by policyDevelopersDesignated key individuals – peers, QA team, Review

Board, etc.Advance preparation by participantsTypically based on checklistsFormal meeting to discuss artifactLed by moderator, not authorDocumented process followedMay be virtual or conferencedFormal follow-up processWritten deliverable from reviewAppraise product

5

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

KINDS OF INSPECTION (CONT.)

2. WalkthroughsNo advance preparationAuthor leads discussion in meetingNo formal follow-upLow cost, valuable for education3. Other review approachesPass-around – preparation part of an inspectionPeer desk check – examination by a single

reviewer (like pair programming)Ad-hoc – informal feedback from a team

member

6

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

TRADEOFF AMONG INSPECTION TECHNIQUES

Formal technical reviews will find more bugsFord Motor: 50% more bugs with formal

processBut they also cost more

7

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

REVIEW ROLESWho are the stakeholders in inspection?1. ModeratorOrganizes reviewKeeps discussion on trackEnsures follow-up happensKey characteristicsGood facilitatorKnowledgeableImpartial and respectedCan hold participants accountable and correct

inappropriate BehaviorSeparate role from Recorder Who captures a log of the inspection process

8

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

REVEIW ROLES (READER)

2. Reader (different from author)Presents materialProvides points of comparison for author and other

team membersDifferences in interpretation provoke discussionReveals ambiguities

If author were to present, others might not mention that their interpretation is different

AlternativeGet comments section by sectionFaster, but does not capture differing perspectives as effectively 9

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

REVEIW ROLES (AUTHOR)

Describes rationale for workNot moderator or readerConflict between objectivity required of moderator/reader and advocacy for the author’s own workOthers raise issues more comfortablyNot recorderTemptation to not write down issues the author disagrees withSignificant benefits to attendingGain insight from others’ perspectivesCan answer questionsCan contribute to discussion based on knowledge of

artifactPotential downside: meeting may be confrontational

10

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

INSPECTION PROCESS

PlanningDetermine objectivesChoose moderatorIdentify inspectors

Good to involve people with connection to artifact

e.g. depends on, interfaces with.Schedule meeting(s)General guideline: 150-200 SLOC/hour, or 3-4 pages/hourPrepare and distribute inspection package

Deliverable, supporting docs, checklistsCross-reference specs, standards

11

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

INSPECTION PROCESS (CONT.)Overview meetingInformal meetingGoal: go over features, assumptions, background, contextOptional stageMay be able to use paper overview or shared context

PreparationInspectors examine deliverable

Defects: cause an error in the productNon-defects: improvements, clarification, style, questions

May want to list typos/spelling/format/style separately and not

discuss during the meetingConformance to standards & specificationOften use checklist

General guidelineprep time ~ meeting time

12

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

INSPECTION PROCESS (CONT.)The MeetingReader describes one segment at a time

Inspectors respond: defects, questions, suggestionsRecorder writes down each defect, suggestion, issueThis is the primary deliverableModerator

Avoid problem solving, inappropriate behavior, lack of participation At conclusion: prepares report with appraisal and data

Outcomes: Appraisal of productAccepted (minor changes, no follow up)Accepted conditionally (minor changes, verification)Reinspect following rework (major changes)Inspection not completed

Outcomes: Input on improving inspection process

Variant: reviewers make comments on electronic bulletin boardCost is lowerLose benefits of direct meeting (face to face, telephone)

Synergy - new bugs found (4%? 25%?)Learning by participantsCommunication about product

13

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

INSPECTION PROCESS (CONT.)Follow-up by authorAuthor addresses each item

Ensure understanding of issueIs it a defect or not? Is it a feature request or requirement change?Fixes defects and makes improvementsUncorrected/unverified defects go into defect tracking system

DeliverablesCorrected work productResponse to each issue and rationale for action

Moderator (or verifier) meets with authorCheck resolution of issuesExamine corrected deliverable

Author checks in code14

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

INSPECTION PROCESS (CONT.)AnalysisCausal analysis

Analyze root causes of defectsMake improvements to development and QA processes

Add issue to checklistChange testing approachDevelop or purchase new static analysis

Measuring effectivenessPercentage of bugs found during inspection

vs. found by other means or afterwards (test, customer)

Measuring efficiency“Defects per hour”Will decrease as your process improves

15

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

MEETINGS: REVIEW GUIDELINESBuild reviews into your scheduleOtherwise unexpected and viewed as intrusionRecognize that reviews can accelerate schedule by reducing other V&V

activitiesKeep review team smallGeneral guidelines: 3-7 participants3 is minimum for formal process to workBelow 3, too few perspectives besides authorAbove 7, work may be slowed by process, schedulingSmaller groups for code, larger groups for other documentsKnowledge is spread around more, more stakeholdersParticular for requirementsFind problems, but don't try to solve themTypically less expensive to address 1-on-1Guideline: halt solution discussion after 1-3 minutesLimit meetings to 2 hours maximumAttention span gets lost beyond thisRequire advance preparationProvides much of the value of a (formal) review 16

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

MEETINGS: CHECKLISTSBenefits Focus on likely sources of error Form quality standard that aids preparers Can bring up issues specific to a productShould be short About seven items

If more, group and do multiple passesFocus Priority issues Issues unlikely to be found other ways Historical problems Issues specific to the documentStart with checklist from well-known source Refine based on experience

17

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

PEOPLE: SOCIAL ASPECTS OF REVIEWSReviews are challenging Authors invest self-worth in product Encourages you to avoid letting others find errorsFor Authors Recognize value of feedback Place value in making code easy to understand Don’t take criticism of code personallyFor reviewers Don’t show off how much better/smarter you are Be sensitive to colleagues

Bad: "you didn't initialize this variable“Good: "I didn't see where this variable was initialized" 18

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/

WHAT TO INSPECT?Requirements, design documentsDifficult to validate in other waysMay have high associated risk

Especially important to get rightCheaper to fix earlier on in process

Many different perspectives are helpfulNeed involvement of multiple stakeholdersCritical or uncertain pieces of codeSecurity-critical codeSafety-critical codeStart inspections at the earliest stages of processCatch mistakes early, when easy to fixAllow rest of system to be built with knowledge gainedSample segments when there is a large body of workConsider what are good “coverage” criteria

19

Auth

or: M

. Muza

ffar H

am

eed S

lides

availa

ble

on h

ttp://b

zupages.co

m/