se 14feb11

44
Software Testing Software Testing Strategies Strategies (Source: Software En gineering, A Practi tioner's Approach, 6 th Edition.)

Upload: rizwan771

Post on 09-Apr-2018

217 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 1/44

Software TestingSoftware TestingStrategiesStrategies

(Source: Software Engineering, A Practitioner's Approach, 6th Edition.)

Page 2: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 2/44

The PresenterThe Presenter

Name:Name: Mobin Al AzimMobin Al Azim

Registration No.Registration No. 14511451--202085202085

Course Code:Course Code: IT 4413IT 4413

Course Title:Course Title: Software EngineeringSoftware Engineering

Program:Program: EMSEMS--ITIT

Quarter:Quarter: Winter 2011Winter 2011

Institute:Institute: Preston UniversityPreston University

Campus:Campus: Islamabad.Islamabad.

Page 3: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 3/44

Software TestingSoftware Testing -- DefinitionDefinition

Testing is the process of exercising aTesting is the process of exercising a

 program with the specific intent of finding program with the specific intent of finding

errors prior to delivery to the end user.errors prior to delivery to the end user.

Page 4: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 4/44

Presentation OverviewPresentation Overview

-- A strategic approach to testing A strategic approach to testing

-- Test strategies for conventional softwareTest strategies for conventional software-- Test strategies for object Test strategies for object--oriented softwareoriented software

-- Validation testing Validation testing

-- System testingSystem testing

-- The art of debuggingThe art of debugging

Page 5: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 5/44

 A Strategic Approach to A Strategic Approach to

TestingTesting

Page 6: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 6/44

General Characteristics of General Characteristics of 

Strategic TestingStrategic Testing

To perform effective testing, a software team should conduct To perform effective testing, a software team should conduct effective formal technical reviewseffective formal technical reviews

Testing begins at the component level and work outward toward theTesting begins at the component level and work outward toward the

integration of the entire computerintegration of the entire computer--based systembased system

Different testing techniques are appropriate at different points inDifferent testing techniques are appropriate at different points intimetime

Testing is conducted by the developer of the software and (for largeTesting is conducted by the developer of the software and (for largeprojects) by an independent test groupprojects) by an independent test group

Testing and debugging are different activities, but debugging must Testing and debugging are different activities, but debugging must be accommodated in any testing strategybe accommodated in any testing strategy

Page 7: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 7/44

 Verification and Validation Verification and Validation

Software testing is part of a broader group of activities calledSoftware testing is part of a broader group of activities calledverification and validation that are involved in software qualityverification and validation that are involved in software qualityassuranceassurance

 Verification (Are the algorithms coded correctly?) Verification (Are the algorithms coded correctly?) The set of activities that ensure that software correctly implements aThe set of activities that ensure that software correctly implements a

specific function or algorithmspecific function or algorithm

 Validation (Does it meet user requirements?) Validation (Does it meet user requirements?)

The set of activities that ensure that the software that has been built isThe set of activities that ensure that the software that has been built is

traceable to customer requirementstraceable to customer requirements

Page 8: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 8/44

Organizing for Software TestingOrganizing for Software Testing

Testing should aim at "breaking" the softwareTesting should aim at "breaking" the software

Common misconceptionsCommon misconceptions The developer of software should do no testing at allThe developer of software should do no testing at all

The software should be tossed over the wall to a strangers The software should be tossed over the wall to a strangers team of testers who will test it unmercifullyteam of testers who will test it unmercifully

The testers get involved with the project only when the testingThe testers get involved with the project only when the testingsteps are about to beginsteps are about to begin

RealityReality The software developer is always responsible for testing theThe software developer is always responsible for testing the

components of the program, ensuring that each performs thecomponents of the program, ensuring that each performs thefunction or exhibits the behavior for which it was designed.function or exhibits the behavior for which it was designed.

Page 9: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 9/44

Page 10: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 10/44

 A Strategy for Testing A Strategy for Testing

Conventional SoftwareConventional Software

Page 11: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 11/44

Levels of Testing forLevels of Testing for

Conventional SoftwareConventional Software Unit testingUnit testing

Concentrates on each component/function of the software asConcentrates on each component/function of the software asimplemented in the source codeimplemented in the source code

Integration testingIntegration testing Focuses on the design and construction of the software architectureFocuses on the design and construction of the software architecture

 Validation testing Validation testing

Requirements are validated against the constructed softwareRequirements are validated against the constructed software

System testingSystem testing

The software and other system elements are tested as a wholeThe software and other system elements are tested as a whole

Page 12: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 12/44

Testing Strategy applied toTesting Strategy applied to

Conventional SoftwareConventional Software Unit testingUnit testing

Exercises specific paths in a component's control structure to ensureExercises specific paths in a component's control structure to ensurecomplete coverage and maximum error detectioncomplete coverage and maximum error detection

Components are then assembled and integratedComponents are then assembled and integrated Integration testingIntegration testing

Focuses on inputs and outputs, and how well the components fit Focuses on inputs and outputs, and how well the components fit together and work togethertogether and work together

 Validation testing Validation testing Provides final assurance that the software meets all functional,Provides final assurance that the software meets all functional,

behavioral, and performance requirementsbehavioral, and performance requirements System testingSystem testing

 Verifies that all system elements (software, hardware, people, Verifies that all system elements (software, hardware, people,databases) mesh properly and that overall system function anddatabases) mesh properly and that overall system function andperformance is achievedperformance is achieved

Page 13: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 13/44

Testing Strategy applied toTesting Strategy applied to

Object Object--Oriented SoftwareOriented Software Must broaden testing to include detections of errors in analysis andMust broaden testing to include detections of errors in analysis and

design modelsdesign models

Unit testing loses some of its meaning and integration testingUnit testing loses some of its meaning and integration testing

changes significantlychanges significantly Use the same philosophy but different approach as in conventionalUse the same philosophy but different approach as in conventional

software testingsoftware testing

Test "in the small" and then work out to testing "in the large"Test "in the small" and then work out to testing "in the large" Testing in the small involves class attributes and operations; the mainTesting in the small involves class attributes and operations; the main

focus is on communication and collaboration within the classfocus is on communication and collaboration within the class

Testing in the large involves a series of regression tests to uncoverTesting in the large involves a series of regression tests to uncovererrors due to communication and collaboration among classeserrors due to communication and collaboration among classes

Finally, the system as a whole is tested to detect errors in fulfillingFinally, the system as a whole is tested to detect errors in fulfillingrequirementsrequirements

Page 14: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 14/44

Criteria for Completion of TestingCriteria for Completion of Testing

There is no definitive answer to this questionThere is no definitive answer to this question

Every time a user executes the software, the program is beingEvery time a user executes the software, the program is beingtestedtested

Sadly, testing usually stops when a project is running out of time,Sadly, testing usually stops when a project is running out of time,money, or bothmoney, or both

One approach is to divide the test results into various severity levelsOne approach is to divide the test results into various severity levels

Then consider testing to be complete when certain levels of errors noThen consider testing to be complete when certain levels of errors nolonger occur or have been repaired or eliminatedlonger occur or have been repaired or eliminated

Page 15: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 15/44

Test Strategies forTest Strategies for

Conventional SoftwareConventional Software

Page 16: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 16/44

Unit TestingUnit Testing

Focuses testing on the function or software moduleFocuses testing on the function or software module

Concentrates on the internal processing logic and data structuresConcentrates on the internal processing logic and data structures

Is simplified when a module is designed with high cohesionIs simplified when a module is designed with high cohesion Reduces the number of test casesReduces the number of test cases  Allows errors to be more easily predicted and uncovered Allows errors to be more easily predicted and uncovered

Concentrates on critical modules and those withConcentrates on critical modules and those with high cyclomatichigh cyclomaticcomplexitycomplexity when testing resources are limitedwhen testing resources are limited

Page 17: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 17/44

Targets for Unit Test CasesTargets for Unit Test Cases

Module interfaceModule interface Ensure that information flows properly into and out of the moduleEnsure that information flows properly into and out of the module

Local data structuresLocal data structures

Ensure that data stored temporarily maintains its integrity during allEnsure that data stored temporarily maintains its integrity during allsteps in an algorithm executionsteps in an algorithm execution

Boundary conditionsBoundary conditions Ensure that the module operates properly at boundary valuesEnsure that the module operates properly at boundary values

established to limit or restrict processingestablished to limit or restrict processing

Independent paths (basis paths)Independent paths (basis paths) Paths are exercised to ensure that all statements in a module havePaths are exercised to ensure that all statements in a module have

been executed at least oncebeen executed at least once Error handling pathsError handling paths

Ensure that the algorithms respond correctly to specific error conditionsEnsure that the algorithms respond correctly to specific error conditions

Page 18: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 18/44

Common Computational ErrorsCommon Computational Errors

in Execution Pathsin Execution Paths

Misunderstood or incorrect arithmetic precedenceMisunderstood or incorrect arithmetic precedence

Mixed mode operations (e.g., int, float, char)Mixed mode operations (e.g., int, float, char) Incorrect initialization of valuesIncorrect initialization of values

Precision inaccuracy and roundPrecision inaccuracy and round--off errorsoff errors

Incorrect symbolic representation of an expression (int vs. float)Incorrect symbolic representation of an expression (int vs. float)

Page 19: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 19/44

Other Errors to UncoverOther Errors to Uncover

Comparison of different data typesComparison of different data types Incorrect logical operators or precedenceIncorrect logical operators or precedence Incorrect comparison of variablesIncorrect comparison of variables

Improper or nonexistent loop terminationImproper or nonexistent loop termination Failure to exit when divergent iteration is encounteredFailure to exit when divergent iteration is encountered Improperly modified loop variablesImproperly modified loop variables Boundary value violationsBoundary value violations

Page 20: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 20/44

Problems in Error HandlingProblems in Error Handling

Error description is unintelligible or ambiguousError description is unintelligible or ambiguous

Error noted does not correspond to error encounteredError noted does not correspond to error encountered

Error condition causes operating system intervention prior to errorError condition causes operating system intervention prior to error

handlinghandling Exception condition processing is incorrect Exception condition processing is incorrect 

Error description does not provide enough information to assist inError description does not provide enough information to assist inthe location of the cause of the errorthe location of the cause of the error

Page 21: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 21/44

Drivers and Stubs Software forDrivers and Stubs Software for

Unit TestingUnit Testing DriverDriver

 A simple main program that accepts test case data, passes such A simple main program that accepts test case data, passes suchdata to the component being tested, and prints the returned resultsdata to the component being tested, and prints the returned results

StubsStubs Serve to replace modules that are subordinate to (called by) theServe to replace modules that are subordinate to (called by) the

component to be testedcomponent to be tested

It uses the modules exact interface, may do minimal dataIt uses the modules exact interface, may do minimal datamanipulation, provides verification of entry, and returns control tomanipulation, provides verification of entry, and returns control tothe module undergoing testingthe module undergoing testing

Drivers and stubs both represent overheadDrivers and stubs both represent overhead Both must be written but that is not delivered with the finalBoth must be written but that is not delivered with the final

Software product.Software product.

Page 22: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 22/44

Integration TestingIntegration Testing

Defined as a systematic technique for constructing the softwareDefined as a systematic technique for constructing the softwarearchitecturearchitecture

 At the same time integration is occurring, conduct tests to uncover At the same time integration is occurring, conduct tests to uncover

errors associated with interfaceserrors associated with interfaces Objective is to take unit tested modules and build a programObjective is to take unit tested modules and build a program

structure based on the prescribed designstructure based on the prescribed design

Two ApproachesTwo Approaches

NonNon--incremental Integration Testingincremental Integration Testing

Incremental Integration TestingIncremental Integration Testing

Page 23: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 23/44

NonNon--incrementalincremental

Integration TestingIntegration Testing

Commonly called the Big Bang approachCommonly called the Big Bang approach

 All components are combined in advance All components are combined in advance

The entire program is tested as a wholeThe entire program is tested as a whole

Many seeminglyMany seemingly--unrelated errors are encounteredunrelated errors are encountered

Correction is difficult because isolation of causes is complicatedCorrection is difficult because isolation of causes is complicated

Once a set of errors are corrected, more errors occur, and testingOnce a set of errors are corrected, more errors occur, and testing

appears to enter an endless loopappears to enter an endless loop

Page 24: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 24/44

Page 25: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 25/44

TopTop--down Integrationdown Integration

Modules are integrated by moving downward through the controlModules are integrated by moving downward through the controlhierarchy, beginning with the main modulehierarchy, beginning with the main module

Subordinate modules are incorporated in either a depthSubordinate modules are incorporated in either a depth--first orfirst orbreadthbreadth--first fashionfirst fashion DF: All modules on a major control path are integratedDF: All modules on a major control path are integrated BF: All modules directly subordinate at each level are integratedBF: All modules directly subordinate at each level are integrated

 Advantages Advantages This approach verifies major control or decision points early in the test This approach verifies major control or decision points early in the test 

processprocess

DisadvantagesDisadvantages Stubs need to be created to substitute for modules that have not beenStubs need to be created to substitute for modules that have not been

built or tested yet; this code is later discardedbuilt or tested yet; this code is later discarded Because stubs are used to replace lower level modules, no significant Because stubs are used to replace lower level modules, no significant 

data flow can occur until much later in the integration/testing processdata flow can occur until much later in the integration/testing process

Page 26: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 26/44

BottomBottom--up Integrationup Integration

Integration and testing starts with the most atomic modules in theIntegration and testing starts with the most atomic modules in thecontrol hierarchycontrol hierarchy

 Advantages Advantages This approach verifies lowThis approach verifies low--level data processing early in the testinglevel data processing early in the testing

processprocess Need for stubs is eliminatedNeed for stubs is eliminated

DisadvantagesDisadvantages Driver modules need to be built to test the lowerDriver modules need to be built to test the lower--level modules; thislevel modules; this

code is later discarded or expanded into a fullcode is later discarded or expanded into a full--featured versionfeatured version Drivers inherently do not contain the complete algorithms that willDrivers inherently do not contain the complete algorithms that will

eventually use the services of the lowereventually use the services of the lower--level modules; consequently,level modules; consequently,testing may be incomplete or more testing may be needed later whentesting may be incomplete or more testing may be needed later whenthe upper level modules are availablethe upper level modules are available

Page 27: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 27/44

Sandwich IntegrationSandwich Integration

Consists of a combination of both topConsists of a combination of both top--down and bottomdown and bottom--upupintegrationintegration

Occurs both at the highest level modules and also at the lowest Occurs both at the highest level modules and also at the lowest level moduleslevel modules

Proceeds using functional groups of modules, with each groupProceeds using functional groups of modules, with each groupcompleted before the next completed before the next  High and lowHigh and low--level modules are grouped based on the control and datalevel modules are grouped based on the control and data

processing they provide for a specific program featureprocessing they provide for a specific program feature Integration within the group progresses in alternating steps betweenIntegration within the group progresses in alternating steps between

the high and low level modules of the groupthe high and low level modules of the group

When integration for a certain functional group is complete, integrationWhen integration for a certain functional group is complete, integrationand testing moves onto the next groupand testing moves onto the next group

Reaps the advantages of both types of integration while minimizingReaps the advantages of both types of integration while minimizingthe need for drivers and stubsthe need for drivers and stubs

Requires a disciplined approach so that integration doesnt tendRequires a disciplined approach so that integration doesnt tendtowards the big bang scenariotowards the big bang scenario

Page 28: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 28/44

Regression TestingRegression Testing

Each new addition or change to baselined software may causeEach new addition or change to baselined software may causeproblems with functions that previously worked flawlesslyproblems with functions that previously worked flawlessly

Regression testing reRegression testing re--executes a small subset of tests that haveexecutes a small subset of tests that havealready been conductedalready been conducted Ensures that changes have not propagated unintended side effectsEnsures that changes have not propagated unintended side effects Helps to ensure that changes do not introduce unintended behavior orHelps to ensure that changes do not introduce unintended behavior or

additional errorsadditional errors May be done manually or through the use of automatedMay be done manually or through the use of automated

capture/playback toolscapture/playback tools

Regression test suite contains three different classes of test casesRegression test suite contains three different classes of test cases

 A representative sample of tests that will exercise all software functions A representative sample of tests that will exercise all software functions  Additional tests that focus on software functions that are likely to be Additional tests that focus on software functions that are likely to be

affected by the changeaffected by the change Tests that focus on the actual software components that have beenTests that focus on the actual software components that have been

changedchanged

Page 29: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 29/44

Smoke TestingSmoke Testing Taken from the world of hardwareTaken from the world of hardware

Power is applied and a technician checks for sparks, smoke, orPower is applied and a technician checks for sparks, smoke, orother dramatic signs of fundamental failureother dramatic signs of fundamental failure

Designed as a pacing mechanism for timeDesigned as a pacing mechanism for time--critical projectscritical projects  Allows the software team to assess its project on a frequent basis Allows the software team to assess its project on a frequent basis

Includes the following activitiesIncludes the following activities The software is compiled and linked into a buildThe software is compiled and linked into a build  A series of breadth tests is designed to expose errors that will keep A series of breadth tests is designed to expose errors that will keep

the build from properly performing its functionthe build from properly performing its function The goal is to uncover show stopper errors that have the highest The goal is to uncover show stopper errors that have the highest 

likelihood of throwing the software project behind schedulelikelihood of throwing the software project behind schedule

The build is integrated with other builds and the entire product isThe build is integrated with other builds and the entire product issmoke tested dailysmoke tested daily

Daily testing gives managers and practitioners a realistic assessment of Daily testing gives managers and practitioners a realistic assessment of the progress of the integration testingthe progress of the integration testing

 After a smoke test is completed, detailed test scripts are executed After a smoke test is completed, detailed test scripts are executed

Page 30: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 30/44

Test Strategies forTest Strategies for

Object Object--Oriented SoftwareOriented Software

Page 31: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 31/44

Test Strategies forTest Strategies for

Object Object--Oriented SoftwareOriented Software With object With object--oriented software, you can no longer test a singleoriented software, you can no longer test a single

operation in isolation (conventional thinking)operation in isolation (conventional thinking) Traditional topTraditional top--down or bottomdown or bottom--up integration testing has littleup integration testing has little

meaningmeaning Class testing for object Class testing for object--oriented software is the equivalent of unit oriented software is the equivalent of unit 

testing for conventional softwaretesting for conventional software Focuses on operations encapsulated by the class and the stateFocuses on operations encapsulated by the class and the state

behavior of the classbehavior of the class

Drivers can be usedDrivers can be used To test operations at the lowest level and for testing whole groups of To test operations at the lowest level and for testing whole groups of 

classesclasses To replace the user interface so that tests of system functionality canTo replace the user interface so that tests of system functionality can

be conducted prior to implementation of the actual interfacebe conducted prior to implementation of the actual interface

Stubs can be usedStubs can be used In situations in which collaboration between classes is required but oneIn situations in which collaboration between classes is required but one

or more of the collaborating classes has not yet been fullyor more of the collaborating classes has not yet been fullyimplementedimplemented

Page 32: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 32/44

 Validation Testing Validation Testing

Page 33: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 33/44

 Alpha and Beta Testing Alpha and Beta Testing

 Alpha testing Alpha testing Conducted at the developers site by end usersConducted at the developers site by end users Software is used in a natural setting with developers watching intentlySoftware is used in a natural setting with developers watching intently Testing is conducted in a controlled environment Testing is conducted in a controlled environment 

Beta testingBeta testing Conducted at endConducted at end--user sitesuser sites Developer is generally not present Developer is generally not present  It serves as a live application of the software in an environment that It serves as a live application of the software in an environment that 

cannot be controlled by the developercannot be controlled by the developer The endThe end--user records all problems that are encountered and reportsuser records all problems that are encountered and reports

these to the developers at regular intervalsthese to the developers at regular intervals  After beta testing is complete, software engineers make software After beta testing is complete, software engineers make software

modifications and prepare for release of the software product to themodifications and prepare for release of the software product to theentire customer baseentire customer base

Page 34: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 34/44

System TestingSystem Testing

Page 35: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 35/44

Different TypesDifferent Types

Recovery testingRecovery testing Tests for recovery from system faultsTests for recovery from system faults Forces the software to fail in a variety of ways and verifies that Forces the software to fail in a variety of ways and verifies that 

recovery is properly performedrecovery is properly performed Tests reinitialization, checkpointing mechanisms, data recovery,Tests reinitialization, checkpointing mechanisms, data recovery,

and restart for correctnessand restart for correctness

Security testingSecurity testing  Verifies that protection mechanisms built into a system will, in fact, Verifies that protection mechanisms built into a system will, in fact,

protect it from improper accessprotect it from improper access

Stress testingStress testing Executes a system in a manner that demands resources inExecutes a system in a manner that demands resources in

abnormal quantity, frequency, or volumeabnormal quantity, frequency, or volume

Performance testingPerformance testing

Tests the runTests the run--time performance of software within the context of time performance of software within the context of an integrated systeman integrated system Often coupled with stress testing and usually requires bothOften coupled with stress testing and usually requires both

hardware and software instrumentationhardware and software instrumentation Can uncover situations that lead to degradation and possibleCan uncover situations that lead to degradation and possible

system failuresystem failure

Page 36: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 36/44

The Art of DebuggingThe Art of Debugging

Page 37: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 37/44

Debugging ProcessDebugging Process

Debugging occurs as a consequence of successful testingDebugging occurs as a consequence of successful testing

It is still very much an art rather than a scienceIt is still very much an art rather than a science

Good debugging ability may be an innate human trait Good debugging ability may be an innate human trait 

Large variances in debugging ability exist Large variances in debugging ability exist 

The debugging process begins with the execution of a test caseThe debugging process begins with the execution of a test case Results are assessed and the difference between expected and actualResults are assessed and the difference between expected and actual

performance is encounteredperformance is encountered

This difference is a symptom of an underlying cause that lies hiddenThis difference is a symptom of an underlying cause that lies hidden

The debugging process attempts to match symptom with cause, therebyThe debugging process attempts to match symptom with cause, therebyleading to error correctionleading to error correction

Page 38: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 38/44

Why is Debugging so Difficult?Why is Debugging so Difficult?

The symptom and the cause may beThe symptom and the cause may be geographically remotegeographically remote

The symptom mayThe symptom may disappear (temporarily)disappear (temporarily) when another errorwhen another erroris correctedis corrected

The symptom may actually be caused byThe symptom may actually be caused by nonerrorsnonerrors (e.g.,(e.g.,roundround--off accuracies)off accuracies)

The symptom may be caused byThe symptom may be caused by human errorhuman error that is not easilythat is not easilytracedtraced

(continued on next slide)

Page 39: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 39/44

Why is Debugging so Difficult?Why is Debugging so Difficult?(continued)(continued)

The symptom may be a result of The symptom may be a result of timing problemstiming problems, rather than, rather thanprocessing problemsprocessing problems

It may beIt may be difficult to accurately reproducedifficult to accurately reproduce input conditions, such asinput conditions, such as

asynchronous realasynchronous real--time informationtime information The symptom may beThe symptom may be intermittent intermittent such as in embedded systemssuch as in embedded systems

involving both hardware and softwareinvolving both hardware and software

The symptom may be due to causes that areThe symptom may be due to causes that are distributeddistributed across aacross anumber of tasks running on different processesnumber of tasks running on different processes

Page 40: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 40/44

Debugging StrategiesDebugging Strategies

Objective of debugging is to find and correct the cause of aObjective of debugging is to find and correct the cause of asoftware errorsoftware error

Bugs are found by a combination of systematic evaluation, intuition,Bugs are found by a combination of systematic evaluation, intuition,

and luckand luck Debugging methods and tools are not a substitute for carefulDebugging methods and tools are not a substitute for careful

evaluation based on a complete design model and clear source codeevaluation based on a complete design model and clear source code

There are three main debugging strategiesThere are three main debugging strategies Brute forceBrute force

BacktrackingBacktracking

Cause eliminationCause elimination

Page 41: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 41/44

Page 42: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 42/44

Strategy #2: BacktrackingStrategy #2: Backtracking

Can be used successfully in small programsCan be used successfully in small programs

The method starts at the location where a symptom has beenThe method starts at the location where a symptom has beenuncovereduncovered

The source code is then traced backward (manually) until theThe source code is then traced backward (manually) until thelocation of the cause is foundlocation of the cause is found

In large programs, the number of potential backward paths mayIn large programs, the number of potential backward paths maybecome unmanageably largebecome unmanageably large

Page 43: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 43/44

Strategy #3: CauseStrategy #3: CauseEliminationElimination

Involves the use of induction or deduction and introduces theInvolves the use of induction or deduction and introduces theconcept of binary partitioningconcept of binary partitioning Induction (specific to general): Prove that a specific starting value isInduction (specific to general): Prove that a specific starting value is

true; then prove the general case is truetrue; then prove the general case is true Deduction (general to specific): Show that a specific conclusion followsDeduction (general to specific): Show that a specific conclusion follows

from a set of general premisesfrom a set of general premises

Data related to the error occurrence are organized to isolateData related to the error occurrence are organized to isolatepotential causespotential causes

 A cause hypothesis is devised, and the aforementioned data are A cause hypothesis is devised, and the aforementioned data areused to prove or disprove the hypothesisused to prove or disprove the hypothesis

 Alternatively, a list of all possible causes is developed, and tests are Alternatively, a list of all possible causes is developed, and tests areconducted to eliminate each causeconducted to eliminate each cause

If initial tests indicate that a particular cause hypothesis showsIf initial tests indicate that a particular cause hypothesis showspromise, data are refined in an attempt to isolate the bugpromise, data are refined in an attempt to isolate the bug

Page 44: SE 14Feb11

8/7/2019 SE 14Feb11

http://slidepdf.com/reader/full/se-14feb11 44/44

Three Questions to ask BeforeThree Questions to ask BeforeCorrecting the ErrorCorrecting the Error

Is the cause of the bug reproduced in another part of the program?Is the cause of the bug reproduced in another part of the program? Similar errors may be occurring in other parts of the programSimilar errors may be occurring in other parts of the program

What next bug might be introduced by the fix that Im about toWhat next bug might be introduced by the fix that Im about tomake?make?

The source code (and even the design) should be studied to assess theThe source code (and even the design) should be studied to assess thecoupling of logic and data structures related to the fixcoupling of logic and data structures related to the fix

What could we have done to prevent this bug in the first place?What could we have done to prevent this bug in the first place? This is the first step toward software quality assuranceThis is the first step toward software quality assurance

By correcting the process as well as the product, the bug will beBy correcting the process as well as the product, the bug will beremoved from the current program and may be eliminated from allremoved from the current program and may be eliminated from all

future programsfuture programs