asee tmap brief -070911
Post on 04-Jun-2018
237 Views
Preview:
TRANSCRIPT
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 1/36
1
Test Management Approach (TMap)
Chris Hampton, PMP
Practice Manager, Software Control & Testing
Sogeti USA
September 2007
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 3/36
3
Agenda:
• The State of, and Business Case for, Software
Quality• What is testing?
• Sogeti‘s Test Management Approach® (Tmap®)
• Sogeti‘s Test Process Improvement®
(TPI®
)Methodology
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 4/36
4
Poor State of Software Delivery
“Software failure costs the American
economy $59.5 billion annually.” Source: NIST (2002)
30% of projects cancelled early
54% of projects over budget
66% of projects rated unsuccessful
90% of projects are delivered late
Source: Standish 2003
4
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 5/36
5
The Business Case For QA
• Compliance
– Basel II, Sarbanes Oxley, HIPAA, etc…
– Reduced liability from defective software and defectiveprocess
• Cost Savings
– Quality processes now reduce support costs later
– Lost revenue• Satisfaction
– Higher User satisfaction
– Improved business alignment
• Growth
– Better quality and measurements allows developmentoptimization
– Increase ability to deliver
5
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 6/36
6
Where Quality Problems Originate
Source: James Martin
Code
Other
Design
Requirements
56%
7%
10%
27%
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 7/36
7November 7, 2013
How Quality Provides Tremendous Savings
UnitTest
Production
Code
COST TOREPAIRDEFECT
Planning
AnalysisSystemDesign$140
UnitTest
$1000
Code
$14000
Time cost and resource effort increaseexponentially later in the lifecycle
Source: Cutter Consortium/Forrester/Sogeti
IntegrationTest
$2500
System
Test$4500
CertificationTest
$7000
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 8/36
8
What is testing?
Testing is a process aimed at:
• Finding defects in a controlled manner
• Detecting the level of quality of the test object
• Demonstrating the gap between specifications and the
actual product
• Demonstrating that the end product functions as called
for in the requirements
8
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 9/36
9
Testing is not . . . .
Testing is not: • Implementation
• Acceptance
• Defect repair
• A phase after development – although testing includes aphase after development
• Training on a new system
9
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 10/36
10
Why do we test ?
• To mitigate business risks that have been
identified: – Validate software quality
– Verify integration of business processes
– Reduce time-to-market
– Competitive purposes
• To ensure business usability of the software:
– Release low-error/known-error software
– Move high quality software (meets or exceeds qualityexpectations) into production
– Demonstrate the usefulness of new technology
10
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 11/36
11
Testing of information systems
• Application Software
• Hardware
• System Software
• Procedures
• Documentation
• Functionality
• Continuity
• Performance
• Usability
• Interoperability (between
different applications andsystems)
What do we test?
11
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 12/36
12
Structured Software Testing
• Testing everything is impractical
– Funding limitations
– Limited time and resources
– Diminishing value after a certain point
• Isn‘t there a more effective way?
• Yes! Structured software testing: a risk-based, quality-centric approach to testing
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 13/36
13
Test Management Approach (TMap®)
• Sogeti‘s methodology for structured
testing of software products.
• TMap® has evolved to be thestandard for software testing inEurope and quickly gaining traction in
the US. – It is being used by more than three
hundred Dutch, Belgium and Germanorganizations.
– Industry adoption includes :
Financial Services
Insurance
Government
Consumer Electronics
Telecommunications
Medical systems
13
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 14/36
14
4 Essentials of TMap®
• Business-driven test management
(BDTM) approach• Structured test process
• Complete tool box
• Adaptive test method
BDTM puts focus on risk-based testing—onlytesting what has impacts to the organization.
14
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 15/36
15
Business Driven Test Management
• Choices must be made in what
is tested and how thoroughly.• This is an economic decision
based on:
– The risks that an organization
thinks it will incur
– The availability of time andresources
– The results the customer
wishes to achieve.
• The choices based on risks,result, time and cost constitutes
the basis for the BDTM
approach.
15
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 16/36
16
The BDTM Process
1. Formulate the test goals.
2. Determine the risk class for eachcombination of characteristic andobject part.
3. Determining the depth of testingrequired.
• Based on risk/resource trade off4. Develop the test estimate and test
plan.
5. Allocate the appropriate testtechniques.
6. Throughout the test process, thetest manager provides thestakeholders with insight into andcontrol options over test processand test object.
16
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 17/36
17
Risk Definition
• Testing should be based on the mitigation of risk and
validation of expected quality defined by the businessrequirements
• Definition of risk
– A ―risk‖ is a chance of a failure occurring, related to
expected damage to the project (should the failure occur)• Risk Categories
– Risks identified in different categories such as businessrisks, project risks, product risks, process risks
• Risk Ranking
– Risk are ranked in criticality relative to one another, byinstituting a method of risk ranking (or ―risk rating‖)
17
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 18/36
18
How Risk Ranking Works
1. Assemble a team of people representing various rolesfrom the project
– PMs, Business Leads/SMEs, Test Leads, Tech Leads
2. Create an initial list of risks
3. Team assigns a numeric value to each risk = theprobability of occurrence of each risk
4. Team assigns a second value to each risk, representingthe impact on the project/organization
5. Multiply the two values together (the probability ofoccurrence X the impact)
6. The result is a relative value for each risk
7. Order the risks by their relative risk values – ―risk rating‖ or ―risk ranking‖
– Ranking helps manage the most critical risks, especiallythose falling in the middle tier of the ranking
8. Periodically update the list of risks
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 19/36
19
Risks – quality characteristics matrix
HighPriority
LowPriority
Q u a l i t y
C h a r a c t e r i s
t i c s
MediumPriority
Low High
Risks
Medium
High Priority/Low Risk
Low Priority/High Risk
Low Priority/Low Risk
High Priority/High Risk
Medium Priority/Low Risk
Medium Priority/Medium Risk
Medium Priority/High Risk
High Priority/Medium Risk
Low Priority/Medium Risk
Multi-Level Matrix
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 20/36
20
Ten steps for creating a test case
Step 1
Step 2
Step 3
Step 4
Step 5
Step 10
Step 9
Step 8
Step 7
Step 6
Identify sub-systems / sub-processes
Identify risks
Rank risks
Agree on relevant quality characteristics
Rank relevant quality characteristicsCreate subsystem / quality characteristic pairings
Create test scenarios for each subsystem
Associate test scenarios with risks
Select suitable test techniques
CREATE TEST CASES
Facilitated
Sessions
FUNCTIONAL AREA
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 21/36
21
Structured Test Process
• Master Test Plan, managing the total test process
– Acceptance and System Tests – Developmental Test
– Supporting Processes
• Lifecycle Model
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 22/36
22
Master Test Plan
• Developed by the test managerin partnership with the
appropriate stakeholders
• Jointly define:
– What will be tested for eachtest level,
– When it will be tested
– To what degree ofthoroughness.
• This plan constitutes the basisfor the test plans for the
separate test levels.
– Unit Testing
– Systems Integration Testing
– Acceptance Testing
– Developmental Testing
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 23/36
23
Life Cycle Model
• The life cycle model is a generic model.
– Can be applied to all test levels and test types
– Used in parallel with the life cycle models for systemdevelopment.
• Test activities are divided across seven phases:
– Planning
– Control
– Setting up and maintaining infrastructure
– Preparation,
– Specification
– Execution
– Completion
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 24/36
24
TMap - Structured testing lifecycle
P&C
PS
EC
PREPARATION-Testability review
-Allocate techniques-Requirements review-Data cycle test-Process cycle test-Other tests-Specify test cases-Create/assessinfrastructure-Test specification-Data flow test
EXECUTION-Pretest
-(Re)test-Checking-Assessing-Debugger -Record and playback-Monitoring-Preserve testware-Evaluate process
PLANNING & CONTROL-Inspection and study -Test strategy-Develop test strategy -Test estimation
oRisk analysis -ReportsoTest Estimation (TPA) -Management tools-Setup organization -Budgeting tools
-Prepare test plan -Defect management-Management and control tool
SPECIFICATION-Create test scripts-Create infrastructure
COMPLETION-Preserve testware forfuture use-Evaluate process
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 25/36
25
Complete Tool Kit
• Techniques
– Many techniques can be used
in the test process. A testtechnique is a combination ofactions to produce a testproduct in a universal manner.
• TMap provides techniques forthe following:
Test estimation
Defect management
Creating metrics
Product risk analysis Test design
Product evaluation.
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 26/36
26
Adaptive Test Method
• Complete And Flexible
• Works with any SDLC
– Iterative
– RAD
– Agile
– Waterfall
– RUP• Works with any technology
– Mainframe
– Web
– ERP
– SOA
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 28/36
28
TMap® – Differentiators
• Business-Driven Test Management
• Risk-based test strategy• Efficient testing—coverage without
overlap
• Testing off of critical path as muchas possible
• Criteria and metrics aboutproduction readiness
• Management of testing to projecttimelines
• Compliments industry tools
Proven methodology applied in hundreds ofcompanies internationally over the past 14 years!
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 29/36
29
Benefits of Structure Testing
• Defects are found early, costing less time/money to reachproduction delivering a higher quality product
• Required quality of the various test objects is tested forand validated, by focusing on testing for quality as a riskmitigation strategy
• By keeping a larger percentage of the testing
process/effort off the critical path, faster time-to-marketresults
• Structured software testing is more cost-effective andefficient than non-structured testing approaches
• Sound test coverage is provided, without the need to
overlap phases
• Establishes a test organization that is prepared andefficient
• Delivers a repeatable process
29
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 30/36
30
Test Process Improvement (TPI )
• The Sogeti TPI® Model was developed to facilitate astepwise improvement of the testing process.
– Developed from practical experience from 30+ years ofSogeti work in testing.
– Offers a frame of reference to determine the strengths andweaknesses of your current testing process.
– Covers 20 key areas that need specific improvement to
achieve a well define testing process.• Key TPI Areas:
Test StrategyLife-Cycle ModelMoment of InvolvementEstimating and Planning
Test Specification TechniquesStatic Test TechniquesMetricsTest AutomationTest EnvironmentOffice Environment
Commitment and MotivationTest Functions and TrainingScope ofMethodology CommunicationReportingDefect ManagementTestware ManagementTest Process ManagementEvaluation and Low-LevelTesting
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 31/36
31
Typical TPI Process Steps
1. Interview key people
2. Describe the ‗As-Is‘ process including Strengths &Weaknesses
3. Describe the ‗To-Be‘ process
4. Document process improvement actions
5. Define expected benefits in terms of Objectives6. Plan for Implementation of the Improvement Actions
(short-term and long-term)
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 32/36
32
TPI - Test Maturity Matrix
Key area / Scale 0 1 2 3 4 5 6 7 8 9 1
0
1
1
1
2
1
3
1 Test strategy A B C D
2 Life-cycle model A B
3 Moment of involvement A B C D
4 Estimating and planning A B
5 Test specification techniques A B
6 Static test techniques A B
7 Metrics A B C D
8 Test automation A B C
9 Test environment A B C
10 Office environment A
11 Commitment and motivation A B C
12 Test functions and training A B C
13 Scope of methodology A B C
14 Communication A B C
15 Reporting A B C D
16 Defect management A B C
17 Testware management A B C D
18 Test process management A B C
19 Evaluation A B
20 Low-level testing A B C
Increasing Maturity
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 33/36
33
Resources
• Web Sites
– Sogeti USA Web Site: www.us.sogeti.com
– TMap Web Site: http://eng.tmap.net/Home/
• TMap® certification (Netherlands)
– Foundation
– TMap
®
professional Advanced/Expert – TMap ® management Advanced/Expert
• Books
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 34/36
34
Sogeti SCT Services
• Managed Testing Services
(MTS)• Quality Management
Services
• Test Automation Services
• Testing Professional Services• SAP Testing Services
• 3000+ people worldwide in
SCT space• 30+ yrs experience
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 35/36
35
Sogeti’s Geographical Presence
A workforce of over 18,000 staff spread over 200 locations
USA
France
Spain
Sweden
Belux Germany
Switzerland
NLIreland
India
UK Denmark
8/14/2019 Asee Tmap Brief -070911
http://slidepdf.com/reader/full/asee-tmap-brief-070911 36/36
36
Thank you!
Chris Hampton, PMP
Practice Manager, Software Control & Testing
Sogeti USA
Chris.Hampton@us.sogeti.com
Cell: (214) 549-7613
Sogeti Office: (972) 892-3400
top related