to requirements and beyond

To Requirements and Beyond

To Requirements and Beyond

#

1

Keynote Outline

Lets break the ice!

Why do you need good requirements?

How can you better structure and what for?

Some tips and tricks

A TRH! Oh great!...erh remind mewhat for exactly?

Pilot, prioritize, communicate, follow-up

Yes, but Im an Analyst, not a TM or a TA

Thats all folks!

#

2

Whos Who?Mais que diable allait-il faire dans cette galre? - Molire

#

A Schizophrenic View on Me, Myself and I, Im 20 !

Olivier DenooIm Belgian

#

4

Keynote Outline

Lets break the ice!







Thats all folks!

#

5

Functional Defect

Project : LouvoisLate payments, wrong salary calculationThe French Ministry was unable to fix the software issueFamilies are appalled346 Mio Loss no usageFrench Servicemen hit by a payroll software issue

#

6

Security Defect

Orange TelCo blamed Privacy rules over hacking

#

7

User Satisfaction issue

Windows 8 Even Vista was better!

#

8

User / Customer IssueTechnical IssueFunctional IssueOw! Lets go fishing!ProductionMaintenancePoor Non-Functional TestingPoor Code qualityPoor Technical architectureBug - DefectFailure - IssueInsufficient involvementPoor business processesCommunicationNeedsMarketPoor RequirementsPoor Functional ArchitecturePoor functional TestingInsufficient involvementPoor Business ProcessesCommunicationNeedsMarket

Poor Non-Functional TestingPoor Code qualityPoor Functional Testing

ProductionMaintenancePoor Technical ArchitecturePoor RequirementsPoor Functional Architecture

CMMI for DEV

#

Keynote Outline

Lets break the ice!







Thats all folks!

#

10

Language & PatternsA matter of

#

How Do We (testers) speak?Constraints:TimeCostsResources

Requirements:CoverageTest depth

CoverageDepthCosts, ResourcesTime

#

Managing Your Test Process29/04/2016

And What About the Others?

#

Managing Your Test Process29/04/2016Who knows the cost of a defect in their company?

Understand them

Speak their language

Fulfil their needs

Put yourself into their shoes

#

Well only test 1 week - vs. Well make sure testing will fit within one week14

Keynote Outline

Lets break the ice!







Thats all folks!

#

15

Use Patternsand make TESTABLE requirements:FUNCTIONALAs ROLE I need ACTION in order to OBJECTIVE + context qualifyersAs user I need to access my personal file in order to complete it with the necessary info

SYSTEMWhen TRIGGER the system NAME must ACTION/PERMISSION to WHOM/WHAT in order to OBJECTIVE + qualifyersWhen book is available, BorrowBook must allow subscribers to borrow if subscription paid and limit not reached

#

MeasuringSTART

#

Often (very often)Absence of referencesFew measures and metrics (or none)Unknown coverageUnclear efficiencyNo capitalizationDecisions = estimations

Management cannot evaluate properly

#

What Would we Want to Know?Whats left to do?Am I on track?What can I improve?What is MY quality?Do defects get solved?What did I learn?

Not measuring = Not knowing

#

No clear specificationWhat is really needed?Basic or advanced functionality?Usability?Performance?Security?When do we have enough tests?How critical is the functionality?Deep or shallow test?How long to design, execute the tests?What should we test first?How can we draw a conclusion from the tests?Difficulties Encountered

#

Managing Your Test Process29/04/2016Story line:The vast majority of tests we do are requirements based.

But when you really start to design tests, you will soon find out that requirements are of poor quality or not existing at all.

Next to that, you will also be confronted with questions such as how deep you need to test, how long it will take to design and exexute the test cases, priorities, etc.

The TRH technique has been specifically designed to address these burning questions. The next slide shows how we can improve your grip on the search box function.

Keynote Outline

Lets break the ice!







Thats all folks!

#

21

A TRH???Logic : a good tool that is sold to us with no user manual - Pierre Vron

#

Hierarchical tree view of application requirements

Business process thinkingHow to reach business goals (notion of benefit)

Process-oriented Tracing down from process to functions

Quality referenceStructuring with a TRHtest requirement 1test requirement 1.1test requirement 1.2test requirement 1.2.1test requirement 1.2.2test requirement 1.2.3test requirement 1.3test requirement 1.4test requirement 1.3.1test requirement 1.3.2

#

The Fundamental Test ProcessTest planning and controlTest analysis and designTest implementationand executionEvaluating exit criteriaand reportingTest closure activities

TRH used throughoutthese stages

#

Managing Your Test Process29/04/2016Speakers notes:In this section, we are using the fundamental test process, as it is taught in the ISEB course. We prefer this model because of our inclination towards international standards. You can easily substitute the I-model that has been used by ps_testware in the past. Both models are very alike; the fundamental test process has one additional step (Test Closure Activities) but the TRH is not used during that step.

V

UnittestingIntegrationtestingSystemtestingAcceptancetestingRequirementsSpecificationFunctionalSpecification GlobaldesignDetaileddesignCodeBusinessObjectives

The TRH and the V-modelTest requirementBusiness requirementSystem requirement

#

Managing Your Test Process29/04/2016

The Problem with RequirementsRequirements form the basis for testing

Requirements are not always testableOften imprecise, ambiguousOften incompleteCan be contradicting or totally wrongSometimes, requirements simply dont exist

Challengesmake imprecise, incomplete requirements testableIdentify wrong, missing requirements

#

Managing Your Test Process29/04/2016Speakers notes:This slides recaps what we found out in section 1 and announces what we are going to learn next.

How would you test a train ticket application?A Small Example

#

Managing Your Test Process29/04/2016Speakers notes:This training is build around a small testing assignment that runs as a read thread through the presentation and binds the concepts together.

We chose to test the search box on www.pstestware.com because(1) it does not require any additional software to be installed,(2) it does not require a lot of upfront reading or any specialized knowledge,(3) it is a functionality that we are all familiar with and we have certain expectations about it,(4) there is no specification whatsoever.

Ideally, the trainees should have internet access, so they can explore the search box in case they never really thought about it. If internet access is not an option, they will have to rely on memory or their knowledge of web search functions in general.

Allow trainees 10 to 15 minutes to determine what they want to test and write it down on paper. Hand out flip chart paper if you have any that speeds up presentation of the results. Let them work in small groups (ideally 2, 3 people).

If they ask for specification, tell them there are none and that they should give it their best shot.

Constructing the Hierarchy

HOW?

Lets bring the Money InSell ticketsExchange ticketsRefund tickets

#

Managing Your Test Process29/04/2016Story line:So, decomposition is the main method to create a TRH,And asking How ? is the main method for decomposition.

In our exanple, asking How do we search on the web site? yields the answerWell, the user specifies a query, the system executes the query and produces the result, the user scans the list, clicks on an hyperlink, ,,,,

This leads to the decomposition shown here.

Speakers note:Next slide shows additional levels of decomposition

Constructing the Hierarchy

HOW?

Lets bring the Money InSell ticketsExchange ticketsRefund ticketsInternationalNationalFull priceOff peakReduced - YoungReduced - SrReduced LoyaltyHOW?

#

Managing Your Test Process29/04/2016Story line:Here we show the effect of asking How ? several time in succession.He additional arrows show the result on questions such as How can visitors specify a query?, How can we use with the result list, ?, etc.

Continuously asking this kind of silly becomes silly. Image that you have these one after the other for an 1 or 1,5 hour.

In addition to that, the users may find these simple questions hard to answer. In that case, you must go fishing for the answer, for instance by using simple test specification techniques:In the upper branch, we used benchmarking (comparing to a similar system, the competition, a de facto standard, ) to come up with possible query options. Your (business) clients will find it easier to pick from the list you are drawing up, than to name these top of their heads.On the lower branch, we used a (kind of) boundary value analysis: what needs to happen when there is 0,1, many resuts.

Asking How, CreativelyUse alternative formulations:What needs to happen?Are there any related problems?What are you likely to have done previously/next?What needs to happen when this fails?How does the user know this?How could things be screwed up?

Use basic test specification techniques to stimulate responses.Manage things CRUD (create/read/update/delete)Equivalence partitioningBoundariesNegative cases

Make sure sub-requirements cover parent 100%

#

Managing Your Test Process29/04/2016Story line:As already pointed out, systematically asking How ? is silly.And you may not always give your audience sufficient clues. Here are some alternative formulations.

Avoid leading questions, though. For instance, during workshops (discussed later) you must not provide your answer in interrogative form, expecting the user to simply say Yes.. The alternative we provide here (and there are many others) make people think about a certain aspect, but they are nevertheless formulated in a neutral way.

Establish Business Context

Lets bring the Money InSell ticketsExchange ticketsRefund ticketsInternationalNationalFull priceOff peakReduced - YoungReduced - SrReduced Loyalty

WHY?

WHY?

WHY?

#

Managing Your Test Process29/04/2016Story line:

Glossary: CMS = Content Management System

We return to our running sample.This is still the same TRH, but the branches are laid out in a more familiar fashion.

In section 2, we looked at completeness and formulated test requirements such as Verify that the search engine examines articles, categories, news items.By symmetry, we could also formulate Verify that the search engine examines keywords.

But what are keywords, Somebody told us that keywords are one of the four entities in the content management system (CMS) behind the web site.Now, I can see articles, I can see news items and I can see categories (after someone told me that these are the menu items). But where are the keywords? And more importantly: why would I care??

To answer that question, we must take a step back and find out what we are trying to achieve here.We could step back and decide we test keywords because we are testing the search function and keywords just happen to be part of what needs to be searched.Thats not a strong answer. Because why should we test the search function? Because it happens to be on web site. I surfed lots of web sites without ever using their search function. In fact, I dont ever remember they have such a function.

The, why are we testing all this? Why did ps_testware foresee such a function? The answer is: because we want to inform visitors about who we are and what we do, and the search function helps them to find the stuff that is of interest to them before they loose their interest.

Speakers notes:At this point, transition to the next slide where the tree is extended. Story line continues on that slide.

Asking Why, UnaggressivelyThe question why? by itself can be quite intimidating (or even look foolish)

Use alternative formulations:What is the purpose of that?Can you explain why you need it to do that?Whats the relation to your business?What business problem is this solving?What was the thinking behind that?What is the underlying reason for that?What would happen if that feature was not available?

#

Managing Your Test Process29/04/2016Story line:Repeatedly asking why? is ridiculous and you will quickly start to resemble a policemen interrogating a suspect. So make sure to have some alternatives up your sleeve to vary your questioning. Also keep in mind that customers may be reluctant to reveal where the money is because it makes them look greedy

How to Get There?Previous Knowledge & ExperienceReading and retro-analysisFace-to-face meetingsWorkshops

NEVER ASSUME !!!- Loyalty is not about calculating & redeeming points It is about COMMUNICATION

#

Keynote Outline

Lets break the ice!




A TRH! Oh great!...erh remind me what for exactly?



Thats all folks!

#

34

A TRH What for?PrioritiseCommunicatePilotEstimate

Test strategy backbonetest requirement 1test requirement 1.1test requirement 1.2test requirement 1.2.1test requirement 1.2.2test requirement 1.2.3test requirement 1.3test requirement 1.4test requirement 1.3.1test requirement 1.3.2

1112233

#

Test Implementation and ExecutionTRHTestdesignspecTestdesignspecTestprocedurespecTestcasespecTestprocedurespecTestPlan

bugsBaselineplanning

Progressreport

#

TRH and Test Design

TEST SPECIFICATION ID: XYZ

PURPOSE: Verify that search functionexamines articles.

APPROACH REFINEMENTS:

#

TRH and ProcessTest procedureScenarioData

Validationpoints

Holistic

Physical DesignWHAT ?

Test procedureScenarioData

Validationpoints

Holistic

Test case1Test case2Test case3Test case5Test case4

Test Requirements HierarchyWHY / WHAT?test requirement 1test requirement 1.1test requirement 1.2test requirement 1.2.1test requirement 1.2.2test requirement 1.2.3test requirement 1.3test requirement 1.4test requirement 1.3.1test requirement 1.3.2

Logical DesignWHAT?

#

Whos the Pilot?Damn its the TRH!Nb testable requirements (leaves)Identified / Developed / written% Coverage(s)% ExecutedProgress (t vs. t-1 vs. planned)Risk / Impact (before and after execution)Planning / Effort

#

In PracticePlanning - Execution100 requirements to test100 persons*days budgetWriting / Execution = 60/40

Quality50% requirements coverage50% passed productGrey zones (50 shades)?

#

Software ToolsWho Cares?There are (too) many of themFrom Excel to Doors via TestLink & othersIntegrated in suites (HP, IBM) or notOpenSource or Commercial

Most (at least the known ones) are good

Tools are NOT the issue

#

Provided they AllowReal-time share and followInformation and reporting (collect and process)DecisionsAnticipation

Adapt the tool to YOUR organisation

#

Word Processor in Outline Mode

#

Managing Your Test Process29/04/2016Speakers notes:Purpose here is to let your audience exercise the use of MS Word as a test requirements capturing tool during workshops.

Hand out a USB stick containing the sample file (trh-pstestware-search.doc) let them all install it and let them do the following:Collapse and expand sections-- Click on the + or signs in front of sectionsPromote and demote requirements-- Use TAB or SHIFT+TAB while positioned at the beginning of a requirement to promote single requirement)-- Use TAB or SHIFT+TAB after selecting a requirement plus all its sub-requirements-- Grab a + or sign and drag it horizontally-- Note how the numbering is automatically adjusted.Restructure the TRH-- Grab a + or sign and drag it vertically to move requirements around.

Mind-Mapping Software

#

Managing Your Test Process29/04/2016Speakers notes:Similar exercise, for those who have a mind mapper installed on their machine.

Spreadsheets

#

Managing Your Test Process29/04/2016Speakers note:There is no added value in letting them exercise Excel they should know this. I am merely including this to demonstrate how you can add extra columns in Excel for all kinds of attributes such as BP and TC scores, pass/fail indication, etc.

but How to Use them MattersThey must be robust The in-house requirements tracker

fit to your needs and organisation

and should allow to share and pilotCustomer / supplier encode twice

#

RequirementsTotal Nb & Testable >> coverageRepartition / severityRepartition / phase / versionRepartition / priorityNb incorrect or maintained requirementsNb scenarios / RequirementNb unused requirementsNb added requirementsNb changed requirements

#

Test Scenarii Test DesignsTotal & Testable (Nb)Rpartition / severityRepartition / phase or versionRepartition / priorityNb scenarii reviewed and/or incorrectNb scenarii / RequirementNb scenarii unusedNb scenarii addedMeasurements detailsFine but complex vs.coarse but blurry estimationsDiffers from phase to phase depending on goals

#

DefectsTotal NbRepartition / severityRepartition / priorityRepartition / statusRepartition / responsible Nb found / unit timeNb false positiveNb unsolvedNb found next phase

#

Adjust your Strategy

You are HERE

TimeScopeEXPECTED DeadlineEXTENDED Deadline100%0%50%1

32

66%Productivity

#

Different Mitigation OptionsOption 2Limit Scope (shrink till you drop)Quality dropIncreased riskLet it slip (the whoosh sound of dealines)Late market presenceExtra costsOption 3Increase resources (Stakhanov Taylor)Extra costs2 women + 4,5 mths = NO baby

#

51

5111/11/2014

One ExampleMitigating options 2 (slippage) and 3 (increase ressources)Use what you already know to predict and planRisk oriented strategySanity checkFlexible resourcesPlanning by roles and capabilityUse existing resources Identify and manage activities Test new functionalityRe-test (bug fix)Regression test (re-validation)/Multiply PC (automation)

Execute based on risk / priorityMost important firstMost important = deeper and longer

BA

#

52

5211/11/2014

Estimation Model

Estimated (found)

FixedPlanning

Exported(next phase)

Inherited(previous phase)

TEST Phase n

#

Test Campaigns

1234567

Go / No Go

SP 2.1SP 2.2SP 2.3 ?

Production?

Production?h1h2h3Inherited from SP1.x h1h2hi

hid2d3Manual Tests (n)Manual tests (n+1) + automated (n)

! Performance ! Availability ! Stability27/0302/0521/05Inheritedfrom SP2.0d10 ?

?

?Correction > c1 Correction > c2

SC 21/0318/048 ?2/04

di = Nb estimated defcts / phaseci = Nb fixed defects in the same phasehi = Nb inherited defectshihi

#

54

5411/11/2014

And if you HaveTIMEToday (status time)Consumed vs, attended resourcesForecast based upon experience

Mean time (write / fix / execute)Mean time (code / analyse / fix)Application up timeApplication performance

MTBE, MTBF Good stop criterionif enough maturity

#

Ressources metricsTime span / person (per profile)# Hrs / test activity

PreparationPlanningDesignEnvironmentExecutionMaintenanceFixing testsReportingOverhead

expenditure

#

Work Product Trend

#

Do NOT Overperform

#

58

Keynote Outline

Lets break the ice!







Thats all folks!

#

59

Yes, SureHoweverYou are also accountable for QualityYou can set-up and follow what everyone understands Added valueBusiness meansYou bring the missing link to life

#

I Can Do Everything Im An Expert

#

Keynote Outline

Lets break the ice!







Thats all folks!

#

62

#

63

Test Weather ReportTest Weather ReportDate8/27/02Sprint2Release3# days15/2315# TestersEffectiveCum. % planned resources for SP2.3Execution test designs5.558%Check printed documents3Ad hoc tests563%Quotation (Product Management)0100%Re-test SP1 + SP2 ad hoc defects0100%Traduction0100%Total resources coverage Release 313.562%Availability - Lost timeAvailability of the application that day100.0%week 1/5week 2/5week 3/5week 4/5week 5/5Average availability for SP2.385.7%80.1%78.6%95.6%85.7%Test execution (2849 data sets)# data sets foreseen that date170Found defects# data sets tested that date905% data set coverage SP2.3 - against planned ds53%OKNOKNot testableIntermediateTotal# data sets executed SP2.3214818502330651672148% data sets executed SP2.386.1%10.8%0.0%3.0%100%% data set coverage SP2.3 - against total ds75.4%Docs waiting for check# SP2 defects foreseen to be re-tested during SP2.3445# SP2 defects re-tested that date22OKNOKNot testable# SP2 defects re-tested while executing ds SP2.3211184261211Cumul # SP2 defects re-tested while executing ds SP2.387.2%12.3%0.5%100%% re-test defects coverage47%Average datasets/p/d foreseen for SP2.319.9*Average test design (15 ds/p/d) and check document (29 ds/p/d)Average datasets/p/d effective that day16.4Average datasets/p/d effective in SP2.317.3Re-test SP2 AD HOC defects# SP2 ad hoc defects foreseen to be re-tested during SP2.3106# SP2 ad hoc defects foreseen that date0NOK +OK +# SP2 ad hoc defects re-tested that date0OKNOKNew defectNew defectNot testable# SP2 ad hoc defects re-tested in SP2.399811600299Cumul # SP2 ad hoc defects re-tested in SP2.381.8%16.2%0.0%0.0%2.0%100%% SP2 ad hoc defects re-test coverage93%Average defects re-tested/p/d foreseen for SP2.310.0Average defects re-tested/p/d effective that day-Average datasets/p/d effective in SP2.3-# Defects# encoded defects that day18UrgentHighMediumLow# active defects in Defect Tracker for SP2.31022051247102# active defects in Defect Tracker4112925711015411OpenIT examPend. TESTSub-totalActiveWholeIT Factory65021864112419Ready TLFPend. TLFUser ExamTest Factory229726325ClosedDeferredArchivedNot active defects4544715072008Test exam00Remarks :Today :Finalised retest of fiches linked to "marques-modles" has been doneApplication has been better regarding response time but a lot of instabilities (T000): see daily urgent defects sheet for details.Program for tomorrow :Same as today

&L&"Arial,Gras"&F&C&D&RPage &P/&N

Resources8/27/02EffectiveForeseenReasonsInterneBusines team00Test Team66ExterneProcess Managers11Superviseurs externes22Gestionnaires externes44Coordination Informatique (ADJ)0.51Product Managers00Service Traduction00Total13.514.0

&F&L&BAxa Royale-Belge Confidential&B&C&D&RPage &P

Fitness ProgramFitness ProgramPHOENIX8-Apr9-Apr10-Apr11-Apr12-Apr15-Apr16-Apr17-Apr18-Apr19-Apr22-Apr23-Apr24-Apr25-Apr26-Apr29-Apr30-Apr2-May3-May4-Jun5-Jun6-Jun7-Jun10-Jun11-Jun12-Jun13-Jun14-Jun17-Jun18-Jun19-Jun20-Jun21-Jun24-Jun25-Jun26-Jun27-Jun28-Jun1-Jul2-Jul3-Jul4-Jul5-Jul5-Aug6-Aug7-Aug8-Aug9-Aug12-Aug13-Aug14-Aug19-Aug20-Aug21-Aug22-Aug23-Aug26-Aug27-Aug28-Aug29-Aug30-Aug2-Sep3-Sep4-Sep5-Sep6-Sepv1.4RELEASE 2,1Retest dfautsRELEASE 2,2RELEASE 2,3AvailabilityAvailability this day39%82%66%92%92%83%81%99%94%99%88%99%57%88%92%61%84%68%84%85%73%84%100%89%30%87%100%95%99%100%79%57%95%----------91%92%24%100%93%64%89%82%86%100%100%100%92%71%100%Overall SP239%61%63%70%74%76%76%79%81%83%83%85%82%83%83%31-Dec82%81%81%82%81%81%82%82%80%81%81%82%82%83%83%82%82%82%82%82%82%82%82%82%82%82%82%83%83%81%82%82%82%82%82%82%82%83%83%83%83%83%0%0%0%0%0%0%0%0%Planned CoverageDS foreseen this day (intial planning)20725420723319112122215210221730733410110010100--102170136136240249.5266.5275208292--------------1481201209090454545207222222207222221236236236137-----Foreseen Cumulated DS (intial planning 04/06/01)20746166890110921213143515871689190622132547264827482849284928491022724085447841033.5130015751783207520752075207520752075207520752075207520752075207520752075148268388478568613658703910113213541561178320042240247627122849-----Theoretical coverage7%16%23%32%38%43%50%56%59%67%78%89%93%96%100%100%100%4%10%14%19%28%36%46%55%63%73%73%73%73%73%73%73%73%73%73%73%73%73%73%73%5%9%14%17%20%22%23%25%32%40%48%55%63%70%79%87%95%100%-----Actual CoverageDS executed this day8118416615935569712698203210263325168140150137--82106123126142131133215145155126230859155147050250383210106612245867245613923032927222930016290Executed Cumulated DS8126543159062568177890410021205141516782003217123112461259882188311437579710843105812031358148417141799189019452092209221422167216722052237224722576618823331939143649753676610951367159618962058214800000000Actual coverage3%9%15%21%22%24%27%32%35%42%50%59%70%76%81%86%91%3%7%11%15%20%25%30%37%42%48%52%60%63%66%68%73%73%75%76%76%77%79%79%79%2%7%8%11%14%15%17%19%27%38%48%56%67%72%75%0%0%0%0%0%0%0%0%Daily Delta12670417415665125264149771-224-68-39-150-1372064131098118.5133.56063137--------------82-2754180-166-23-107-50-22-785914600000000Cumulated Delta126196237311467532657683687701798869645577538388251208497107205323.5457517580717591361276185130-17-17-67-92-92-130-162-172-1828280DefectsEncoded this day771139232424162022553954432824295119311115150191618065764766675201210170050031117138231452391912118Cumulated SP2 / by release71425648711113515117119324828734138441243646551653557688398981171331511511572142783544204955155275375545545545595595591324415462859910410610911813714915016800000000Actual CoverageForeseen to be retested this day (intial planning)002020202015202024--------------3030302010------------------Cumulated re-tested002040608095115135159159159159159159159159159159159159159159159306090110120------------------Theoretical Coverage0%0%13%25%38%50%60%72%85%100%100%100%100%100%100%100%100%100%100%100%100%100%100%100%25%50%75%92%100%------------------Re-tested this daySP1 + SP2 ad hoc60022131219582053224381300124750026192511191393520000000000000392646020000000000Cumulated re-tested565656586073851041621821872192432812942942944184930026457081100113122157159159159159159159159159159159159159159159396511111111311311311311311311311311311311300000000Actual Coverage11%11%11%12%12%15%17%21%33%37%38%44%49%57%60%60%60%85%100%0%0%16%28%44%51%63%71%77%99%100%100%100%100%100%100%100%100%100%100%100%100%100%100%33%54%93%93%94%94%94%94%94%94%94%94%94%94%94%0%0%0%0%0%0%0%0%Remaining today :751751751751751751751751751751751751821862849829860-928848842848808799762713727-703694705723785751697710-648-627634607-544547524522519--47747447746544741340841100000000- to be corrected by IT306306306306306306306306306306306306400426467493501-5361008488106106102127139-140214281356426519526536-528-487484486-1524444955--7375778267737386- to be re-tested445445445445445445445445445445445445421436382336359-392748758760702693660586588-563480424367359232171174-120-140150121-529523480473464--404399400383380340335325

&L&"Arial,Gras"&F&C&D&RPage &P/&NZE1:reviewed dataZE1:independent from the releaseZF7:re-testZE1:re-testZE1:Delta = 90 due to automation re-check effort & wrong user ID.ZE1:recalculated for every release = init delta: 50ZE1:Open + IT Exam + Pending TestZE1:Ready for TLF + Pending TLF + User ExamZF7:57 defects have been encoded during Sanity CheckZF7:Small error in bulletin meteo. Was 15 and not 13.ZF7:7defecst encoded after Bulletin mteo during the evening and not counted the following dayZF7:10 dfauts encods lors du Sanity Check

Fitness Program GraphicFitness Program


Fitness Program Graphic661481882682333883194783915684366134976585367037669101095113213671354159615611896178320582004214822400247602712028490000000000

Executed Cumulated DSDS foreseen this day (intial planning)Data set foreseen / executed

0.9140.920.24410.9290.64285714290.89285714290.82142857140.85635359121110.9230.71413749637497374983750137502375033750437505

AvailabilityAvailability of the application

393065601119011111011312011301130113011301130113011301130113011300000000000000000

Re-tested this daySP1 + SP2 ad hocForeseen to be retested this day (intial planning)Retest defects SP1 foreseen / executed











0.9140.920.24410.9290.64285714290.89285714290.82142857140.85635359121110.9230.71413749637497374983750137502375033750437505


393065601119011111011312011301130113011301130113011301130113011300000000000000000


Test Weather ReportTest Weather ReportDate27/08Sprint2Release3# days15/2315# TestersEffectiveCum. % planned resources for SP2.3Execution test designs5.558%Check printed documents3Ad hoc tests563%Quotation (Product Management)0100%Re-test SP1 + SP2 ad hoc defects0100%Traduction0100%Total resources coverage Release 313.562%Availability - Lost timeAvailability of the application that day100.0%week 1/5week 2/5week 3/5week 4/5week 5/5Average availability for SP2.385.7%80.1%78.6%95.6%85.7%Test execution (2849 data sets)# data sets foreseen that date170Found defects# data sets tested that date905% data set coverage SP2.3 - against planned ds53%OKNOKNot testableIntermediateTotal# data sets executed SP2.3214818502330651672148% data sets executed SP2.386.1%10.8%0.0%3.0%100%% data set coverage SP2.3 - against total ds75.4%Docs waiting for check# SP2 defects foreseen to be re-tested during SP2.3445# SP2 defects re-tested that date22OKNOKNot testable# SP2 defects re-tested while executing ds SP2.3211184261211Cumul # SP2 defects re-tested while executing ds SP2.387.2%12.3%0.5%100%% re-test defects coverage47%Average datasets/p/d foreseen for SP2.319.9*Average test design (15 ds/p/d) and check document (29 ds/p/d)Average datasets/p/d effective that day16.4Average datasets/p/d effective in SP2.317.3Re-test SP2 AD HOC defects# SP2 ad hoc defects foreseen to be re-tested during SP2.3106# SP2 ad hoc defects foreseen that date0NOK +OK +# SP2 ad hoc defects re-tested that date0OKNOKNew defectNew defectNot testable# SP2 ad hoc defects re-tested in SP2.399811600299Cumul # SP2 ad hoc defects re-tested in SP2.381.8%16.2%0.0%0.0%2.0%100%% SP2 ad hoc defects re-test coverage93%Average defects re-tested/p/d foreseen for SP2.310.0Average defects re-tested/p/d effective that day-Average datasets/p/d effective in SP2.3-# Defects# encoded defects that day18UrgentHighMediumLow# active defects in Defect Tracker for SP2.31022051247102# active defects in Defect Tracker4112925711015411OpenIT examPend. TESTSub-totalActiveWholeIT Factory65021864112419Ready TLFPend. TLFUser ExamTest Factory229726325ClosedDeferredArchivedNot active defects4544715072008Test exam00Remarks :Today :Finalised retest of fiches linked to "marques-modles" has been doneApplication has been better regarding response time but a lot of instabilities (T000): see daily urgent defects sheet for details.Program for tomorrow :Same as today


Resources27/08EffectiveForeseenReasonsInterneBusines team00Test Team66ExterneProcess Managers11Superviseurs externes22Gestionnaires externes44Coordination Informatique (ADJ)0.51Product Managers00Service Traduction00Total13.514.0








0.9140.920.24410.9290.64285714290.89285714290.82142857140.85635359121110.9230.71413749637497374983750137502375033750437505


393065601119011111011312011301130113011301130113011301130113011300000000000000000


Test Weather ReportTest Weather ReportDate8/27/02Sprint2Release3# days15/2315# TestersEffectiveCum. % planned resources for SP2.3Execution test designs5.558%Check printed documents3Ad hoc tests563%Quotation (Product Management)0100%Re-test SP1 + SP2 ad hoc defects0100%Traduction0100%Total resources coverage Release 313.562%Availability - Lost timeAvailability of the application that day100.0%week 1/5week 2/5week 3/5week 4/5week 5/5Average availability for SP2.385.7%80.1%78.6%95.6%85.7%Test execution (2849 data sets)170Found defects905% data set coverage SP2.3 - against planned ds53%OKNOKNot testableIntermediateTotal# data sets executed SP2.3214818502330651672148% data sets executed SP2.386.1%10.8%0.0%3.0%100%% data set coverage SP2.3 - against total ds75.4%Docs waiting for check44522OKNOKNot testable211184261211Cumul # SP2 defects re-tested while executing ds SP2.387.2%12.3%0.5%100%% re-test defects coverage47%19.9*Average test design (15 ds/p/d) and check document (29 ds/p/d)16.417.3Re-test SP2 AD HOC defects1060NOK +OK +0OKNOKNew defectNew defectNot testable# SP2 ad hoc defects re-tested in SP2.399811600299Cumul # SP2 ad hoc defects re-tested in SP2.381.8%16.2%0.0%0.0%2.0%100%% SP2 ad hoc defects re-test coverage93%10.0--# Defects# encoded defects that day18UrgentHighMediumLow# active defects in Defect Tracker for SP2.310220512471024112925711015411OpenIT examPend. TESTSub-totalActiveWholeIT Factory65021864112419Ready TLFPend. TLFUser ExamTest Factory229726325ClosedDeferredArchivedNot active defects4544715072008Test exam00Remarks :Today :Finalised retest of fiches linked to "marques-modles" has been doneApplication has been better regarding response time but a lot of instabilities (T000): see daily urgent defects sheet for details.Program for tomorrow :Same as today




Fitness ProgramFitness ProgramPHOENIX8-Apr9-Apr10-Apr11-Apr12-Apr15-Apr16-Apr17-Apr18-Apr19-Apr22-Apr23-Apr24-Apr25-Apr26-Apr29-Apr30-Apr2-May3-May4-Jun5-Jun6-Jun7-Jun10-Jun11-Jun12-Jun13-Jun14-Jun17-Jun18-Jun19-Jun20-Jun21-Jun24-Jun25-Jun26-Jun27-Jun28-Jun1-Jul2-Jul3-Jul4-Jul5-Jul5-Aug6-Aug7-Aug8-Aug9-Aug12-Aug13-Aug14-Aug19-Aug20-Aug21-Aug22-Aug23-Aug26-Aug27-Aug28-Aug29-Aug30-Aug2-Sep3-Sep4-Sep5-Sep6-Sepv1.4RELEASE 2,1Retest dfautsRELEASE 2,2RELEASE 2,3AvailabilityAvailability this day39%82%66%92%92%83%81%99%94%99%88%99%57%88%92%61%84%68%84%85%73%84%100%89%30%87%100%95%99%100%79%57%95%----------91%92%24%100%93%64%89%82%86%100%100%100%92%71%100%Overall SP239%61%63%70%74%76%76%79%81%83%83%85%82%83%83%31-Dec82%81%81%82%81%81%82%82%80%81%81%82%82%83%83%82%82%82%82%82%82%82%82%82%82%82%82%83%83%81%82%82%82%82%82%82%82%83%83%83%83%83%0%0%0%0%0%0%0%0%Planned Coverage20725420723319112122215210221730733410110010100--102170136136240249.5266.5275208292--------------1481201209090454545207222222207222221236236236137-----Foreseen Cumulated DS (intial planning 04/06/01)20746166890110921213143515871689190622132547264827482849284928491022724085447841033.5130015751783207520752075207520752075207520752075207520752075207520752075148268388478568613658703910113213541561178320042240247627122849-----Theoretical coverage7%16%23%32%38%43%50%56%59%67%78%89%93%96%100%100%100%4%10%14%19%28%36%46%55%63%73%73%73%73%73%73%73%73%73%73%73%73%73%73%73%5%9%14%17%20%22%23%25%32%40%48%55%63%70%79%87%95%100%-----Actual CoverageDS executed this day8118416615935569712698203210263325168140150137--82106123126142131133215145155126230859155147050250383210106612245867245613923032927222930016290Executed Cumulated DS8126543159062568177890410021205141516782003217123112461259882188311437579710843105812031358148417141799189019452092209221422167216722052237224722576618823331939143649753676610951367159618962058214800000000Actual coverage3%9%15%21%22%24%27%32%35%42%50%59%70%76%81%86%91%3%7%11%15%20%25%30%37%42%48%52%60%63%66%68%73%73%75%76%76%77%79%79%79%2%7%8%11%14%15%17%19%27%38%48%56%67%72%75%0%0%0%0%0%0%0%0%Daily Delta12670417415665125264149771-224-68-39-150-1372064131098118.5133.56063137--------------82-2754180-166-23-107-50-22-785914600000000Cumulated Delta126196237311467532657683687701798869645577538388251208497107205323.5457517580717591361276185130-17-17-67-92-92-130-162-172-1828280DefectsEncoded this day771139232424162022553954432824295119311115150191618065764766675201210170050031117138231452391912118Cumulated SP2 / by release71425648711113515117119324828734138441243646551653557688398981171331511511572142783544204955155275375545545545595595591324415462859910410610911813714915016800000000Actual CoverageForeseen to be retested this day (intial planning)002020202015202024--------------3030302010------------------Cumulated re-tested002040608095115135159159159159159159159159159159159159159159159306090110120------------------Theoretical Coverage0%0%13%25%38%50%60%72%85%100%100%100%100%100%100%100%100%100%100%100%100%100%100%100%25%50%75%92%100%------------------Re-tested this daySP1 + SP2 ad hoc60022131219582053224381300124750026192511191393520000000000000392646020000000000Cumulated re-tested565656586073851041621821872192432812942942944184930026457081100113122157159159159159159159159159159159159159159159396511111111311311311311311311311311311311300000000Actual Coverage11%11%11%12%12%15%17%21%33%37%38%44%49%57%60%60%60%85%100%0%0%16%28%44%51%63%71%77%99%100%100%100%100%100%100%100%100%100%100%100%100%100%100%33%54%93%93%94%94%94%94%94%94%94%94%94%94%94%0%0%0%0%0%0%0%0%Remaining today :751751751751751751751751751751751751821862849829860-928848842848808799762713727-703694705723785751697710-648-627634607-544547524522519--47747447746544741340841100000000- to be corrected by IT306306306306306306306306306306306306400426467493501-5361008488106106102127139-140214281356426519526536-528-487484486-1524444955--7375778267737386- to be re-tested445445445445445445445445445445445445421436382336359-392748758760702693660586588-563480424367359232171174-120-140150121-529523480473464--404399400383380340335325






0.9140.920.24410.9290.64285714290.89285714290.82142857140.85635359121110.9230.71413749637497374983750137502375033750437505


393065601119011111011312011301130113011301130113011301130113011300000000000000000












0.9140.920.24410.9290.64285714290.89285714290.82142857140.85635359121110.9230.71413749637497374983750137502375033750437505


393065601119011111011312011301130113011301130113011301130113011300000000000000000


to requirements and beyond

Education