material testing training v1.5 hoang a na 2

8/8/2019 Material Testing Training v1.5 Hoang a NA 2

1/47

Test trainingTest training

Version:Version: 1.5 1.5

Author: Author: Hoang A Na Hoang A Na

Aug 2005 Aug 2005

2005 FPT-SOFT 04e-BM/NS/HDCV/FSOFT


2/47

TRAINING MATERIALSFP T-Soft Confidential 2

IntroductionIntroduction

E stimated duration: 1-1.30 hour Purpose: Practical Guide to Software System Testing Targeted attendees: PLs, Testers, SQA


3/47


Obj e ctiv es Obj e ctiv es

Aft er th e course, s tud en ts will clearly und er stand:Te st Proce ssDe f ec t loggi ngSoftwar e Te sting Conc ep ts


4/47


A. Ove rvi ewB. Tes t P roc ess C . Defe ct Tracking

D. Tes t me thodsE. Stag e s of Te stF. Typ e s of Te stG. Te sting Infrastructur e

H. Te st Me asur eme ntsI . Te st We b a ppl ication

Ag e nda Ag e nda


5/47


Be caus e softwar e is l ikely to hav e faults To le arn about th e re liability of th e softwar e To fill th e time be twee n de live ry of the softwar e and th e re le as e dat e To p rove that th e softwar e has no faults

Be caus e t e sting is includ e d in th e p roje ct p lanBe caus e failure s can be ve ry expe nsive To avoid be ing su e d by custom e rsTo stay in busin e ss

A. Ove rvi ew A. Ove rvi ew


6/47


Wh y t e sting?Be caus e soft w ar e is lik e ly to hav e fau lts To le arn a b out th e re lia b ilit y of th e soft w ar eTo fi ll th e ti me b e t wee n d e liv e ry of th e soft w ar eand th e re le as e dat eTo p rov e that th e soft w ar e has no fau ltsBe caus e t e sting is inc lud e d in th e p ro je ct pl anBe caus e fai lur e s can b e ve ry ex p e nsiv eTo avoid b e ing su e d b y custo me rs

To sta y in b usin e ss

A. Ove rvi ew A. Ove rvi ew


7/47


Q A: Q ua lit y Assuranc eQC : Q ua lit y Contro lQ A vs . QC

Th e m ission of Q A is d e fe ct p re ve ntionTh e m ission of QC is d e fe ct d e t e ctionTe sting is a s p e cific me thod of QCPR EVENTI ON I S CHE APER TH AN CU REVe rification and Va lidation

Q A Q A -- QC QC


8/47


Va lidation Va lidation -- Ve rification Ve rification

Va lidationEnsur e that th e s yst em o p e rat e s according to a pl an b y exe cuting s yst em functionsDid we b ui ld th e s yst em right ?

Va lidation : don e b y custo me r ( inc luding re vi ew and acc e p tanc e t e st) .

Ve rificationMe thods to e nsur e that th e s yst em co m pl ie s w ith an organizationa l standard or p roc e ss

Did we b ui ld th e right s yst em?

Ve rification : don e b y p ro je ct t e a m (Re vi ew & Te st)So w hat do we n ee d to do m ost w h e n we t e st ?

QC - Ve rification


9/47


Te sting invo lve s m onitoring and im p roving th e p roc e ss :making sur e that standards and p roce dur e s ar e followe de nsuring that p roble ms ar e found and de alt with .

Te sting invo lve s o p e ration of a s yst em/ a ppl ication und e r contro lle d conditions and e va luating th e re su lts .

Th e contro lle d conditions = nor m a l + a b nor m a l

Cre ativ e ly De stro ying th e P roduct : att e mpt to mak e things go wrong to de t e rmin e if

things happ e n whe n th e y shouldn t or things don t happ e n whe n th e y should

Ch e ck th e q ua lit y of th e PR ODUC T and not th e q ua lit y of th ePR OGR AMMERA Test Cycle is the period in which the Product is tested anddefects are verified. Before the release of a product, it will havegone through numerous Test Cycles to confirm the Quality of theproduct.

Te st conditionsTe st conditions


10/47


Attri b ut e s of an id e a l t e st e r :Ob s e rvation ski ll (W hat you s ee is w hat you t e st)Co mm unication ski ll

Te chnica l ski llSe lf-confid e ntPe rs e ve ranc e, p ati e nc eSke p ticis m

Se lf- m otivating

Ski llsSki lls


11/47


Te sting P rinci pl e sTe sting P rinci pl e s

All t e sts shou ld b e trac e a b le to custo me r req uir eme ntsTe sts shou ld b e pl ann e d long b e for e t e sting b e gins

Te sting shou ld b e gin in th e s m a ll and p rogr e ss to w ard t e sting in th e larg e

Exhaustiv e t e sting is not p ossi b leTo b e m or e e ff e ctiv e, t e sting shou ld b e conduct e d b y an ind e p e nd e nt third p art y

Und e rstanding custo me r n ee dsIde ntify busin e ss nee dIde ntify targ e t custom e rIde ntify usag e be havior and patt e rnsP rioritize ( impact - freq ue ncy)


12/47

TRAINING MATERIALSFP T-Soft Confidential 1 2

Quality Cost includes :S taff trainingReviewTest

CorrectionError Removal Cost :

Requirement 1Design 5

Code 10S ystem test 50Post- Delivery 100

Q ua lit y costQ ua lit y cost


13/47


Source: IBM


14/47


B. Te st P roc e ssB. Te st P roc e ss

1 . Te st t e am2 . Work products of t e st 3. Te st lif e cycle 4 . Te st tools


15/47


1 . Te st Te a m1 . Te st Te a m

Test leader

Tester

Projectleader

Te st e r

SQA

Su pp ort dir e ction

Infrastructur em anag eme nt

Re p ort dir e ction


16/47


In p ut :

Custom e r req uire me nts and Acce ptanc e crit e ria

Softwar e Req uire me nt Spe cification (SRS)

De sign docum e nts

Programs (Modul e s)

Out p ut :Te st docum e nts: Te st plan, Te st cas e s, Te st

script, Te st dataDe f e cts in DMS

Te st re sults

2 . Work P roducts of Te st2 . Work P roducts of Te st


17/47


What t e st e r to do w hi le re vi ewWhat t e st e r to do w hi le re vi ew

Stud y req uir eme nt ( SRS / Sp ec s) :Id e ntif y w hat to b e t e st e d . Is e ve rything

cle ar and d e tai le d? Is e ve rything p ossi b le ? Id e ntif y w hich req uir eme nts cannot b e

t e st e dCr e at e Te st pl an

Re vi ew/ Stud y DDD:

Ve rif y th e consist e nt w ith SRSId e ntif y ke y functionsCr e at e Te st cas e s & t e st data


18/47


Te st cas e s :To cove r th e basic functionaliti e s of th e Product using normal us e r sce nariosTe st Cas e s ar e modifie d during th e proc e ss of th e Product De ve lopm e nt as and whe n ne w f e atur e s ar e

add e d or whe n the ex isting f e atur e s ar e modifie d.De fe ct :

De f e cts ar e sce narios that caus e the Product to throw e rrors or caus e th e Fe atur e not to function as me ntion e d in th e Functional Spe cTe st e rs also nee d to kee p track of the de f e ct as to confirm that th e y hav e bee n fixe d and not face d th e re aft e r => Defect Verification


19/47


The Traditional LifecycleRequirements Design Code TestProblem : testing is being left too late in theprocess high quality cost

Parallel Testing LifecycleOnly test execution needs to wait until after coding.Test planning and test case design can occur in

parallel with development.The act of planning and designing the tests to beapplied would discover bugs

3 . Te sting Life Cycle3 . Te sting Life Cycle


20/47


Test planning Test preparation Test execution Data analysis

Project plan

Customer requirementsand Acceptance criteria

Test plan

Detail design

SRS

Test plan

Test case

Test scrip

Test data

Software/ product

Test case/ test script/ test data

Defect list

Test result


21/47


I m pl eme nt Te st

STA RT

Plan Te st

De sign Te st

Int e gration Te st

FINI SH

Sy st em Te st

Su m u p and re p ort

Eva luat e Te st

What : cre at e a t e st plan to ide ntify and de scrib e the t e sting that will be imple me nt e d and exe cut e dWh e n : req uire me nt analysis phas e Who : Te st Le ad e r

What : ide ntify t e st mod e l and its artifacts: t e st cas e s, t e st proc e dur e s and t e st dataWh e n : de sign phas e Who : Te st Le ad e r & Te st e rs

What : Imple me nt (re cord, ge ne rat e or program) the t e st proc e dur e s that we re de fine d in De sign t e st (t e st scripts)

Wh e n : Coding phas e Who : Te st Le ad e r & Te st e rs

What : Exe cut e all t e st programs/scripts that we re ge ne rat e d in Imple me ntation t e st . Re gre ssion t e st .Wh e n : syst e m t e sting phas e Who : T e st e rs

What : Evaluat e t e st exe cutions Wh e n : Be fore compl e tion of syst e m t e sting phas e Who

: T

e st Le ad e r and T e st e rs

Th e w ho le p roc e ss w ill b ere p e at e d for th e n ex t it e ration

What : Ge ne rat e final t e st

cove rag e summaryWh e n : Be fore de live ring, re port to SQA by the F inal insp e ctionWho : T e st Le ad e r

What : Exe cut e t e st programs/scripts that we re ge ne rat e d in Imple me ntation t e st .Wh e n : int e gration t e sting phas e Who : T e st e rs

Te sting Life -c ycle starts as soon as th e soft w ar e life -c ycle starts


22/47


So me too ls in FPTSoftTe st data manag e me nt: MS Exce lScripting languag e : Rational V isual Te st 6 .0

Rational Te st Studio (imple me nting)De f e ct manag e me nt: DMSGraphic t e st : Virtual Rule r, Eye Dropp e rFunctional Te st : Rational Robot Te st Pe rformanc e t e st : Rational L oad Te st

Tem pl at e s for t e st docu me ntsTe st P lanTe st cas eTe st De s ig n

4 . Te st Too ls4 . Te st Too ls


23/47


24/47


Pro je ct le ad e r co lle cts d e fe ct nu m b e r in DMS to Wee kly Re p ort to sho w -u p th e p rogr e ss and re su lt of t e sting p has e of th e p ro je ct .

Classif y d e fe cts b y:Defect types

S everities

Work products

Quality control activities

Defect origins


25/47


D. Te st me thodsD. Te st me thods

Whit e Box Te stingIs a t e st cas e de sign that us e s th e control structur e of the proc e dural de sign to de rive t e st cas e s

It guarant ee s that all inde pe nd e nt paths within a modul e hav e bee n exe rcise d at le ast onc e Exe rcise all logical de cisions on th e ir tru e an false side sExe cut e all loops t th e ir boundari e s and within th e ir ope rational boundsExe rcise int e rnal data structur e s to assur e th e ir validity

Black Box Te stingFocus e s on functional req uire me nts of th e softwar e De rive se ts of input conditions that will fully exe rcise all functional

req uire me nts for a programIncorr e ct or mission functionsInt e rfac e e rrorsErrors in data structur e s or ext e rnal db acce ssPe rformanc e e rrors


26/47


Whit e b o x : Re ve al probl e ms with the int e rnal structur e of a program

Black b o x : Asse ss how we ll a program mee ts its req uire me nts

I/P O/P


27/47


E. Stag e s of Te st E. Stag e s of Te st V Mod e l V Mod e l


28/47


P ro je ct ov e rvi ewP ro je ct ov e rvi ew

Requirement

FunctionalSpecification

SRS

Design

ADD

DDD Code

Code Review

Unit Test

Functionality

Integration test

System

System Test

Performance Test

Stress Test

Acceptance

Acceptance Test


29/47


2 cat e gori e s :Docu me nt => re vi ewAppl ication => cod e re vi ew, t e st

Unit Te stInt e gration Te stSy st em Te st Acc e p tanc e Te st


30/47


1 . Unit Te st 1 . Unit Te st

Ve rifie s that th e compon e nt/modul e functions prop e rlyDone in a controll e d e nvironm e nt Normally whit e box orie nt e d

What to t e st :Data structur e is examin e dBoundary conditions ar e t e st e d to e nsur e that th e modul e op e rat e s prop e rly for input and output dataAll inde pe nd e nt paths ar e exe rcise d

Error handling paths ar e t e st e dUse of stubs (modul e s to be calle d) and drive rs (modul e s that calls)


31/47


2 . Int e gration Te st2 . Int e gration Te st

Compar e with ADD: Che ck that modul e s int e grat e with one anoth e r corr e ctlyProble ms may occur du e to:

Data lost across int e rfac e One modul e can hav e an adv e rse aff e ct on anoth e rSub functions may not produc e the de sire d major functionsIndividual acc e ptabl e impr e cision but unacc e ptabl e if magnifi e dGlobal data structur e s pre se nts probl e ms

Int e gration t e st: FunctionalityInt e rfac e


32/47


3 . Sy st em Te st3 . Sy st em Te st

Co m p ar e w ith SRS : Che ck that th e whole syst e m works as expe ct e dSourc e s of Soft w ar e Fau lts

not a ll fau lts ar e s ee nfau lts m a y occur in diff e re nt p has e s

p rogra mme rs cannot b e ex p e ct e d to s p ot d e sign fau ltsfau lts m a y b e add e d w h e n chang e s ar e m ad e to corr e ct oth e r fau ltsm aint e nanc e fau ltsdocu me ntation fau lts

Proc e ssPe rfor m anc e Te sting (Load t e st , Str e ss t e st , Vo lu me t e st)Te st Sta b ilit y, Co m p ati b ilit yInsta llation Te sting


33/47


4 . Acc e p tanc e Te st 4 . Acc e p tanc e Te st

Re su ltsit is the custom e r s way to ve rify that what was want e d is what is built uncov e rs mor e than req uire me nts discr e panci e sallows th e custom e rs to de t e rmin e what th e y re ally want, whe the r sp e cifie d in th e docum e nt or not .ne w probl e ms may aris e rapid prototypingchang e s may not only me an improp e r de finition of req uire me nt, but also be caus e custom e rs may de cide that th e probl e m is chang e d and a diff e re nt solution is nee de d

Goa l:N o defect during a ccept a nce test


34/47


F. Typ e s of Te stingF. Typ e s of Te sting

F unctional testingInterface testingPerformance testing : Test performances requirementspecifications (response time)

Data & Data integrity testingS ecurity & Access control testingLoad testingS tress testing

Volume testingRegression testingInstallation testing

F. Typ e s of Te stingF. Typ e s of Te sting


35/47


1. Test environments2. Test documentation3. Configuration management and change control

G. Te sting Infrastructur eG. Te sting Infrastructur e


36/47


The test environment includes:the system or components under testtest documentation, describing the tests to be

performed and recording resultstest cases, test data

Test Environment IndependenceIt is very impor tant to m a inta in ind epe nd e nc e be twee n the tes t e nvironme nt and the pro duct ion e nvironme nt.

1 . Te st Environ me nt1 . Te st Environ me nt


37/47


Test plan:Pur p os e : to organiz e t e sting activiti e sDe fin e s :

Sco p e of t e st : stag e s and t yp e s of t e st

Risks m a y aff e ct t e sting Req uir eme nts to t e st : w hat w ill b e t e st e d

Te sting strat e g y: ho w w ill b e t e st e d

Te st crit e ria ( start , sto p , sus p e nsion , co m pl e tion)

Too ls us e d to t e stRe sourc e and e nviron me nt of t e st

De liv e ra b le s of t e st

2 . Te st docu me nts2 . Te st docu me nts


38/47


Test case: build test cases from use cases- Te st cas e s p e cifications :

Basic flo wAlt e rnat e flo w s

- Te st p roc e dur e s :P re -condition for t e st this cas et e st actions tak e n b y th e actor Te st in p ut : in p ut data (v a l id and in v a l id data)Exp e ct e d re su l t

2 . Te st docu me nts 2 . Te st docu me nts ( cont .)( cont .)

Input 1 => Expected Output 1




39/47


Te st cas e :

Nor m a l cas e s

Ab nor m a l cas e s Boundar y va lu e s


40/47


Test dataTest result: report test coverage

Tes t ca se IDO utput

R es ult (pa ss, f a ile d , ) by comp a ring with tes t ca se expe cte d output.Tes t Cover ag e/ S ucc ess Tes t r at e

Test status:

P a ss: tes t done s ucc essf ullyF a ile d : tes t done and res ult d iffere nt from expe cte dN /A: tes t cann o t be done ye t

2 . Te st docu me nts 2 . Te st docu me nts ( cont .)( cont .)


41/47


3 . CM and Chang e Contro l3 . CM and Chang e Contro l

CM: tester knows where to get source codes for testChange requirements:

update TP: type of testing, time of testupdate TC: add or modify test casesaccording to CRstrack the change in Modification historyre-test basing on updated TCregression test


42/47


Measurements:y Defects in DM Sy Defect rate

Weighted defects/ project size (in UCP)y

Test coverage: number of test cases executedy S uccessful Test coverage: number of passed

TCs/ Total executed TCsMetrics to assess individual tester performance:

y Test effectivenessWeighted defects/ Test effort

y LeakageWeighted defects found after release/ project size

H. Te st Me asur eme ntsH. Te st Me asur eme nts


43/47


I . Te st We b a ppl icationI . Te st We b a ppl ication

What to t e st :

All co m p on e nts of a We b a ppl ication on b oth th e clie nt

and s e rv e r sid e

Va lidation or Functiona l Te stingHTML Va lidation

Link Te sting

Load and str e ss Te sting

Se curit y Te stingRe gr e ssion Te sting


44/47


Pro b lem : I m p ossi b le to t e st a ll p ossi b le d e p e nd e nci e s and e ve rything that cou ld go w rong w ith this sit e .

I m p ossi b le to t e st vari e d targ e t audi e nc e and pl atfor m s

that a we b a ppl ications addr e ss e s

Too ls us e d for load t e st , vo lu me t e st

So lution :De t e rm in e w h e re to focus t e sting e fforts w ithin b udg e t

and sch e du le constraints

Ana ly s e risks and s e t p rioriti e s b as e d on risk ana ly sis

I . Te st We b a ppl icationI . Te st We b a ppl icationI . Te st We b a ppl icationI . Te st We b a ppl ication


45/47


Risk ana lysis :Which functiona lit y is m ost critica l to its p ur p os e ?

Which function is m ost lik e ly to b e th e m ost fr eq u e nt ly

us e d or th e s eq u e nc e of st e p s to b e m ost fr eq u e nt ly p e rfor me d?

What t yp e of ar e as cou ld p ossi b ly caus e th e m ost

co m pl aints or b ad p u b licit y?

What p arts of th e a ppl ication p os e th e m a x im u m s e curit y risk?

I . Te st We b a ppl icationI . Te st We b a ppl icationI . Te st We b a ppl icationI . Te st We b a ppl ication


46/47


Re sourc e s & re fe re nc e sRe sourc e s & re fe re nc e s

Re fe re nc ePractical Guide to Software System Testing Test Guideline

Lists of common defects: We b, Java, C /C++ . VB,.Net TestTraining for beginer.doc


47/47

Q uestions and Answers Q uestions and Answers

material testing training v1.5 hoang a na 2

Documents