software testing day1

Software TestingSoftware Testing

Van Thi Kim NganVan Thi Kim Ngan

Aug-2005Aug-2005

AgendaAgenda

• What is test?What is test?• Why is testing necessary?Why is testing necessary?• What is bug?What is bug?• Lifecycle of a bugLifecycle of a bug• Record and Manage bugRecord and Manage bug

What is test?What is test?

• Testing is the process of Testing is the process of exercising or evaluating a exercising or evaluating a system or system component by system or system component by manual or automated means to manual or automated means to verify that it satisfies specified verify that it satisfies specified requirementsrequirements

Some English…Some English…

• Verification : Methods to ensure Verification : Methods to ensure that the system complies with an that the system complies with an organizational standard or organizational standard or processprocess

• Validation: Ensure that the Validation: Ensure that the system operates according to a system operates according to a plan by executing system plan by executing system functionsfunctions

Why is testing Why is testing necessary?necessary?

• Because software is likely to have Because software is likely to have faults faults

• To learn about the reliability of the To learn about the reliability of the softwaresoftware

• Because failures can be very Because failures can be very expensiveexpensive

• To avoid being sued by customersTo avoid being sued by customers• To stay in businessTo stay in business

What is bug?What is bug?

• A fault in a program which A fault in a program which causes the program to perform causes the program to perform in an unintended or in an unintended or unanticipated manner.unanticipated manner.

FailureFailure

• DDeviation of the software from its eviation of the software from its expected delivery or serviceexpected delivery or service

• Failure is an event Failure is an event

FaultFault

• Cause that failure is causedCause that failure is caused• Defect or error inside a programDefect or error inside a program

It is said to be a bug generally

ErrorError

• AA human action that produces an human action that produces an incorrect resultincorrect result

Oversight

Hand omission

Mistake

Relation of failure, fault Relation of failure, fault and errorand error

A person makes

an error ...

… that creates a

fault in thesoftware ...

… that can cause

a failurein operation

““Importance” of bugs (in Importance” of bugs (in $$)$$)

• Frequency (Fre)Frequency (Fre)• Correction cost (Cor)Correction cost (Cor): :

– Typically, significantly more the later Typically, significantly more the later the bug is discoveredthe bug is discovered

• Installation cost (Ins)Installation cost (Ins): : – Cost of installing the fixCost of installing the fix

• Consequences (Con) Consequences (Con)

““Importance” of bugs Importance” of bugs ((Con.Con.))

• Importance = Fre * (Cor + Ins + Importance = Fre * (Cor + Ins + Con)Con)– It’s not obvious that these costs can It’s not obvious that these costs can

simply be added. simply be added. – Another cost item may be “bug Another cost item may be “bug

maintenance”: help desk effort, etc., to maintenance”: help desk effort, etc., to assist users to work around itassist users to work around it

Cost of bugsCost of bugs

Bug LifecycleBug LifecycleLOG DEFECT

Defect status: ERROR

ASSIGN DEFECT

ASSIGNED

CORRECT DEFECT

Defect status: PENDING

Analyse DefectACCEPT DEFECT

ACCEPTED

Retest Defect

CLOSE DEFECT

TESTED

Error

Corrected

Defect status:

Defect status:

Defect status:

Not all bugs are equalNot all bugs are equal

Damage

Cosmetic

Serious

Fatal

Bug Type

Medium

Bug severityBug severity

• Cosmetic bugsCosmetic bugs– A defect that in no way affects the A defect that in no way affects the

performance of the product. It may be a performance of the product. It may be a grammatical error grammatical error

• Medium bugsMedium bugs– This defect doesn’t stop the user from This defect doesn’t stop the user from

proceeding, but causes inconvenience proceeding, but causes inconvenience

Bug severity (Con.)Bug severity (Con.)

• Serious bugsSerious bugs– System cannot work aroundSystem cannot work around

• Fatal bugsFatal bugs– Defects may stop the user from using Defects may stop the user from using

the system further, the system is the system further, the system is crashed crashed

Bug StatusBug Status

• ERRORERROR– The defect is not fixed, or fixed but not The defect is not fixed, or fixed but not

satisfactorily as requiredsatisfactorily as required

• ASSIGNED ASSIGNED – The defect is reviewed and assigned to The defect is reviewed and assigned to

fix it fix it

• PENDING PENDING – The defect is already fixed and waiting The defect is already fixed and waiting

to retestto retest

Bug Status (Con.)Bug Status (Con.)

• TESTEDTESTED– The defect is fixed satisfactorily as The defect is fixed satisfactorily as

required required • ACCEPTEDACCEPTED

– The defect has not been fixed satisfactorily The defect has not been fixed satisfactorily as required, but it’s accepted by as required, but it’s accepted by concession of authority or customer concession of authority or customer

• CANCELLED CANCELLED – It’s not a defect or defect is removed by It’s not a defect or defect is removed by

actions other than bug fixing actions other than bug fixing

Bug Bug classificationclassification

• Requirements and feature bugsRequirements and feature bugs• Structural bugsStructural bugs• Data bugsData bugs• Coding bugsCoding bugs• Interface, integration and system Interface, integration and system

bugsbugs• Test and test design bugsTest and test design bugs

Bug Bug classificationclassification (Con.) (Con.)

• Requirements & Feature BugsRequirements & Feature Bugs– Requirements bugs are the first to Requirements bugs are the first to

invade the system and the last to leaveinvade the system and the last to leave– Specs are often self-contradictory, Specs are often self-contradictory,

incomplete and ambiguousincomplete and ambiguous– Extra features, other gratuitous Extra features, other gratuitous

enhancements tend to be error proneenhancements tend to be error prone– Feature interaction problemsFeature interaction problems

Bug Bug classificationclassification (Con.) (Con.)

• Structural BugsStructural Bugs– Control and sequence bugsControl and sequence bugs– Logic bugsLogic bugs– Processing bugsProcessing bugs– Initialization and dataflow bugsInitialization and dataflow bugs

Bug Bug classificationclassification (Con.) (Con.)

• Structure & Sequence BugsStructure & Sequence Bugs– Unreachable codeUnreachable code– Paths left outPaths left out– Goto problemsGoto problems– SpaghettiSpaghetti code code

Bug Bug classificationclassification (Con.) (Con.)

• Logic BugsLogic Bugs– Misunderstood switch-statement Misunderstood switch-statement

semanticssemantics– Improper Boolean logicImproper Boolean logic– Improper simplification and Improper simplification and

combination of casescombination of cases– Misunderstood expression evaluationMisunderstood expression evaluation

Bug Bug classificationclassification (Con.) (Con.)

• Processing BugsProcessing Bugs– Arithmetic bugsArithmetic bugs– Algebraic bugsAlgebraic bugs– Incorrect conversionIncorrect conversion

Bug Bug classificationclassification (Con.) (Con.)

• Initialization and data bugsInitialization and data bugs– Improper initializationImproper initialization– Using an un-initialized variableUsing an un-initialized variable– Double initializationDouble initialization– Modifying data but not keeping the Modifying data but not keeping the

resultresult– Dangling pointersDangling pointers

Bug Bug classificationclassification (Con.) (Con.)

• Coding BugsCoding Bugs– TyposTypos– Misunderstood language syntax and Misunderstood language syntax and

semanticssemantics– Erroneous commentsErroneous comments

Bug Bug classificationclassification (Con.) (Con.)

• Interface, Integration & System Interface, Integration & System BugsBugs– External interfacesExternal interfaces– Internal interfacesInternal interfaces– Bugs related to hardware, system Bugs related to hardware, system

software, or OS interfacesoftware, or OS interface– Integration bugsIntegration bugs

Bug Bug classificationclassification (Con.) (Con.)

• Tests & Test Design BugsTests & Test Design Bugs– Bugs in the test (“Bugs in the test (“scaffoldingscaffolding”) ”)

softwaresoftware– Bugs in test dataBugs in test data

Bug Tracking ToolsBug Tracking Tools

• Trakium: Issue Tracking SystemTrakium: Issue Tracking System– VVendor: Multitechendor: Multitech– www.trakium.comwww.trakium.com

• Test Director: Provide -Requirements Test Director: Provide -Requirements Management, Test Plan, Test Case, Management, Test Plan, Test Case, and Defects Management and Defects Management – VVendor: Mercuryendor: Mercury– www.mercury.com www.mercury.com

Questions & AnswersQuestions & Answers