capture - asqf.de · outsourced testing can offer a positive business case for the customer...
TRANSCRIPT
Capture Consulting and IT services
Mar 12, 2013
Profile
Business and IT competencies
Near-shore operating model
60+ skilled professionals
Deloitte Technology Fast 50 Central Europe 2012 Award (10th in CEE)
Project managers
Business analysts
Consultants
Process engineers
IT architects
Application developers
Test engineers
System administrators
Services
Energy
IT Governance
Business Applications
Software Testing
Business Consulting
SMART metering consultancy
Data measurement and analyzes
Energy management software products
Project Portfolio Management
IT governance processes and tools
Project methodology
Custom application development
IT development and support outsource
IT architecture consultancy
Test design and execution outsourcing
Test management
Test automation and tool implementation
CRM capabilities and solution implementation
Process management (BPR, Lean)
Forrester Research
Products
Business Consulting
Customer Centricity – Maturity model Forrester Research – Analysis and Research Balanced Scorecard
Energy
ARGUS – Electricity and Gas data analysis and management system POMS – Portfolio Management System Process Analysis and Optimization System
IT Governance
CA Clarity Project Portfolio Management Capture Taskfit Capture Projectfit CA Product Vision CA Idea Vision CA Service Desk
Business Applications
Corporate Credit Scoring System (23rd hour) ETL – High-volume data Management Framework Deltim CRM LYNX SNA
Software Testing
Capture Kolibri Test automation Framework HP Application Lifecycle Management HP Quality Center HP LoadRunner
Customers & Partners
5
Near-shore Test Outsourcing
Management
• Managed services, KPIs • Risk sharing options • Service level agreements
Resources
• Located in Budapest & Miskolc • Internal team & External Partners • Software tools & test environments
Methodology
• Own methodology • Based on ISTQB & ISO/IEC 29119 • Standard processes
References
Software development on-site in Germany and outsourced in
Budapest & Miskolc, Hungary
Outsourced test design & execution for product development
and implementation in 10 aviation software products
Test automation and maintenance using both tools provided
by the Customer and tools developed by Capture
Test consulting and project management
Implementation of two Capture products: anomaly
detection and energy portfolio management
Iterative (cyclic) development methodology to get
closer to agile
Testing and test automation of Argus and PoMS using
Capture Kolibri test automation framework
Our key benefits
Quality &
Efficiency
Improving customer
experience
with state-of-the-art quality assurance methodology and
technology
Cost effective quality
with a scalable,
professional team on a near-shore location,
without a cultural gap Easy-to-use business
model
with managed testing services and flexible
Service Level Agreement schemes
Test outsourcing via Testing-as-a-Service
ASQF FG Software-Test, Erlangen
12 March 2013
10/50
Why have we chosen this topic?
Importance of software is
increasing in every industry
Customers are demanding better and
better quality
Testing is inevitable for
good customer experience
New ways of outsourcing are emerging day
by day
11/50
Agenda
Business value of test outsourcing
Maturity of test outsourcing
Is testing-as-a-service really out there?
Standard services or frame services
Testing-as-a-service on the Customer side
Testing-as-a-service on the Service Provider side
Case study: implementation of Testing-as-a-Service
Business value of test outsourcing
13/50
Business value of testing
€
more testing
benefit of quality
cost of testing
14/50
Elements of the value of testing
Benefits of quality
Customer experience
Product reputation
Cost of maintenance
Costs of testing
Technology
Team
Efficiency
15/50
Can anybody quantify the benefits?
Customer experience
Net Promoter
Score
Product reputation
Voice of the
Customer
Costs of maintenance
business case of testing
16/50
Business case of testing
Requirement Design Code Unit testing UAT Production
1x 3-6x 10x
15-40x
30-70x
40-1000x
Source: HP, IBM, SEI
17/50
Costs can be easily meaured
Technology
investments
licences
Team
wages & salaries
trainings
travel & living
Efficiency
management
test planning & design
allocation & dedication
knowledge of software
knowledge of testing
experience
18/50
So what we were talking about
Business value of testing
benefits of quality
customer experience
product reputation
cost of maintenance
costs of testing
technology team efficiency
19/50
Business value of testing
benefits of quality
customer experience
product reputation
cost of maintenance
costs of testing
technology team efficiency
Is it worth to outsource?
? ? ?
20/50
€
more testing
benefit of quality
cost of testing
Where is the value?
Maturity of test outsourcing
22/50
Is it really maturing?
do-it-yourself
body- leasing
fixed price
managed test service
testing-as- a-service
Source: Forrester Research, Inc.
23/50
State of the Union
Source: Forrester Research, Inc.; Tholons
Already outsourced
To outsource next year
Considering outsourcing
Not to outsource
Is Testing-as-a-Sevice really out there?
25/50
Testing-as-a-Service is…
Source: Forrester Research, Inc.
an evolutionary step in
managed test services
a model of
managed outcomes
a consumption-based
cooperation approach
special in level of
automation and analytics
26/50
Many questions have to answered
Risk handling
responsibility
contracting
preparation
Control
distance
culture
processes
Technology
access
availability
data management
27/50
Is Testing-as-a-Service really out there?
Testing-as-a-Service is still developing
The Customer can customize the service
The Customer has to be well prepared
Testing-as-a-Service can be used the best in specific areas
Standard services or frame services
29/50
Standard equals TaaS? Frame equals MTS?
pre-defined inputs/outputs
standard prices, pay-as-you-go
specific basic services
well prepared customer
service descriptions
prices subject to negotiation
general high-level services
common preparation
Testing-as-a-Service on Customer side
31/50
A lot to do
Results instead
of resources Consumption- based pricing
Service level agreements
Methods of
controls
32/50
What is the result of testing?
Date Day Plan Tested Pass Fail A B C D Sum A% B% C% D%
2013.02.21 1 115 0 0 0 0 0 0,0% 0 0 0 0 0 0% 0% 0% 0%
2013.02.22 2 231 0 0 0 0 0 0,0% 0 0 0 0 0 0% 0% 0% 0%
2013.02.25 3 346 75 8% 54 6% 21 28% 0 0 15 8 23 0% 0% 65% 35%
2013.02.26 4 461 188 20% 137 15% 51 27% 2 0 34 19 55 4% 0% 62% 35%
2013.02.27 5 576 338 37% 243 26% 95 28% 2 3 56 35 96 2% 3% 58% 36%
2013.02.28 6 692 475 52% 339 37% 136 29% 2 8 76 39 125 2% 6% 61% 31%
2013.03.01 7 807 586 64% 423 46% 163 28% 2 12 87 42 143 1% 8% 61% 29%
2013.03.04 8 922 666 72% 480 52% 186 28% 2 16 101 45 164 1% 10% 62% 27%
2013.03.05 9 922 827 90% 604 66% 223 27% 2 21 119 44 186 1% 11% 64% 24%
2013.03.06 10 922 905 98% 677 73% 228 25% 2 22 120 44 188 1% 12% 64% 23%
0
100
200
300
400
500
600
700
800
900
1000
1 2 3 4 5 6 7 8 9 10
PlanPassFailTested
0
20
40
60
80
100
120
140
160
180
200
1 2 3 4 5 6 7 8 9 10
D
C
B
A
33/50
To control is to govern
Source: Forrester Research, Inc.
34/50
Quality
test coverage
number of bugs
bug rejection
reoccurring bugs
Progress
actual/plan executed test cases
actual/plan duration
test/development duration ratio
Efficiency
ext/int bug ratio
actual/plan effort
test/development effort ratio
To control is to measure
Testing-as-a-Service on Service Provider side
36/50
Even more to do
Standard
processes
Resource
management
Responsibility,
professionalism
Continuous
improvement
Step A Step B
Step C Step D
Step E
37/50
Invest in processes
International standards
ISTQB
ISO/IEC 25000
ISO/IEC 29119
Tools
test management
test automation
Customer/Provider
Cooperation setups
Provider Customer
Software Vendor Provider
Customer
38/50
Invest more in resource management
Source: CA Clarity
39/50
And invest continuously in improvement
automation cloud mobile
Case study: implementation of Testing-as-a-Service
41/50
Background
do-it-yourself
body- leasing
fixed price
managed test service
testing-as- a-service
AG
Customer
Customer
KFT
Customer
42/50
Business trigger
Solution
Trigger of the market
Goal of AG
Expectation for KFT
price
quality
price
quality
price/quality is too high
price reduction & quality increase
cost reduction
new structure and new idea
new idea: Testing-as-a-Service
responsibility, resource control
price
quality
43/50
The Project
Was not a
project
Was no idea
what is TaaS
Implementing
MTS instead
No support
from AG
44/50
Delivered
Test
Service
Catalog
Project Analysis Toolset
Business Case
Template
Service Level
Agreement
Transfer of
Products
do-it-yourself
body- leasing
fixed price
managed test service
testing-as- a-service
45/50
Test Service Catalog
Compound test services
Feature test
Regression test
Release test
User acceptance test
Life-cycle test
Basic test services
Test management
Test design
Test execution
Test environment
Test lab
Value added test services
Test consulting
Test automation
Test training
46/50
changing the commercial conditions
analysis of current status of testing
changing the commercial conditions
test optimizations
Transferring a product to TaaS
47/50
Lessons learnt
• KFT should have been prepared
The Service Provider have to be ready
• AG should have demand services
A Customer is needed
• bottom-up approach costs a lot
Buy-in from the Management is necessary
Conclusion
49/50
What we were talking about?
Outsourced testing can offer a positive business case for the Customer
Testing-as-a-Service offerings are on the market but still developing
The possible Customer has to consider risks and ways of control
The possible Customer has to be also well prepared for outsourcing
For more information please contact:
Capture IT Solutions and Consulting Plc. E-mail: [email protected]
www.capture.hu
János GÁLIK