04 kalyan rao konda applabs
TRANSCRIPT
-
8/12/2019 04 Kalyan Rao Konda Applabs
1/24
Generating BusinessValue through TCoE
-
8/12/2019 04 Kalyan Rao Konda Applabs
2/24
AppLabs Confidential 1
Testing Organization ModelsEvolutionary approach to building a TCOE is pragmatic. Slowly growing a Test COE yieldsopportunities to introduce mechanisms to overcome the pains of centralization.
-
8/12/2019 04 Kalyan Rao Konda Applabs
3/24
AppLabs Confidential 2
TCoE services summary
Characteristics:
Common approach Ability to support frequent releases Common methodology and deliverables Manage key set of few metrics Centralisation of tasks / skills / functions
Benefits:
Consistent quality High performance Faster time to market Flexibility to support business needs Lower costs
-
8/12/2019 04 Kalyan Rao Konda Applabs
4/24
AppLabs Confidential 3
Introduction TCOE
TCOE
Tools & AssetsProcess &TestMethodology
People &Organisation
Leverage a dedicated group of skilled testresources
Utilise offshore resources
Ensures appropriate skills are available
People & Organisation
Standardised approach and deliverableswith greater delivery consistency
Rapid adoption of best practice
Clear process accountability / responsibility
Disciplined test management
Process & Test Methodology
Effective leveraging of tools and automation
Assets made available to projects
Maintenance of multiple environments
Tools & Assets
Lower cost Consistent quality
Flexibility to supportbusiness needs
Control
Benefits
Higher quality
High performance
Speed to market
Benefits
Lower cost
Consistent quality
Speed
BenefitsCharacteristics:
Common Approach Ability to support frequent releases
Common methodology and deliverables
Management to key set of Few metrics
Centralisation of key testing tasks / skills /functions
-
8/12/2019 04 Kalyan Rao Konda Applabs
5/24
AppLabs Confidential 4
AppLabs SCORE methodology
AppLabs proven proprietary methodology Maps processes and metrics to organization business goals Focuses on building organization wide test platform Core + Flexi resource model Cost goes down exponentially with each addition of product into TCoE
-
8/12/2019 04 Kalyan Rao Konda Applabs
6/24
-
8/12/2019 04 Kalyan Rao Konda Applabs
7/24
AppLabs Confidential 6
TCoE Framework
AppLabs Framework for Assessment
Clients Vision & Goals
AppLabs SCOREMethodology
Test Process Standardization Index Process Artifacts Creation / Modification
Plan Test Metrics & Measurement Roadmap Test Process Deployment Plan
TCoE Organization Model Resource Skill Matrix Capacity Planning and Utilization Report Proposed Knowledge Management
Framework
Infrastructure Inventory Report
Test Environments and Utilization Report Test Infrastructure Optimization Plan
Test Tool Inventory and Utilization Report Recommended Test Frameworks ROI Analysis of Tools Implemented Proposed Tool Optimization Plan
Understand Information gathered 360 degree Analysis Actionable Recommendations Establish baseline for Measuring
Improvements
Process Review
Infrastructure Review
Tools Review
Test Teams Review
Understand Analyze Recommend
Test Maturity Rating Test Spend Analysis Report
-
8/12/2019 04 Kalyan Rao Konda Applabs
8/24
AppLabs Confidential 7
Centralization of Testing Services
Business / Product / Technology based frameworks
Resource management Types, skills and knowledge management
Process methodologiesReviews Review expertise Review techniques Review process Review database
Test management Test case Defect Test data Roles & Responsibilities
Risk management Plan Budget Reviews Prioritization
Measurement & Reporting Metrics framework MIS Escalation SLA
Testing infrastructure Inventory, scheduling and operational processes
Test engineering tools
Core testing
Functional andregression
Test strategy &approach
Methodology Templates
authoring & review Test suite selection
Certification(Compatibility)
Compatibilitymatrix
Acceptancecriteria
Performance
Test types Load, stress,etc
Tools Capacity
planning
Security
Testapproach
Analysis Tools Vulnerability
assessment Scorecard
Consulting
Test processreview
Baseline Audit Process
Improvement
Tools andAutomation
Test strategy Approach &
methodology Framework Tool feasibility Coverage
-
8/12/2019 04 Kalyan Rao Konda Applabs
9/24
-
8/12/2019 04 Kalyan Rao Konda Applabs
10/24
-
8/12/2019 04 Kalyan Rao Konda Applabs
11/24
AppLabs Confidential 10
Business value of testing AppLabs has successfully demonstrated Return on Investment (ROI) for several customers in categories mentioned below. Metrics mentioned under each category are just a sample to showcase cost advantage, qualityimprovement and time-to-market
Test infrastructure Reduction in test tool licensing
costs Hardware infrastructure utilization Reduction in hardware servers Test data reusability
Test processes
Test processes standardization Process compliance Test maturity level
Test Engineering Defects in production Defect containment Reduction in test cycle duration Test case authoring productivity Test case execution productivity Test coverage
Effort to find a defect Test Automation
Test script authoring productivity Automation ROI
-
8/12/2019 04 Kalyan Rao Konda Applabs
12/24
-
8/12/2019 04 Kalyan Rao Konda Applabs
13/24
Leading InsuranceProvider in US
-
8/12/2019 04 Kalyan Rao Konda Applabs
14/24
AppLabs Confidential 13
ROI / Efficiency GainsTest Authoring - Cost SavingsKey Initiatives:
Used All-Pairs techniques Participated in business requirements/review
meetings Reverse traceability to avoid redundant test
cases 2-Level formal review before Sign-off from
Business team (reduction in rework)
Test Execution - Cost SavesKey Initiatives::
Mapping defects to test objectives
Participation of test teams in early SDLC
cycles to gain application knowledge
Automating 40% of regression suite
Early Defect detection AFT/SIT/UAT
Releases
Releases
-
8/12/2019 04 Kalyan Rao Konda Applabs
15/24
AppLabs Confidential 14
Post Production Root Cause Distribution
Analyzed the root causes for all the warranties and built awarenessacross all the groups to drive reduction in warranties
Root Cause for each production defect has been applied as
lessons learnt in the respective teams
-
8/12/2019 04 Kalyan Rao Konda Applabs
16/24
AppLabs Confidential 15
Defect detection percentage has decreased for Functional, Regression and Warranty phase
Early defect detection ensured adherence to release schedules
Early Defect Detection
Integration System Regression Warranty
2007 2008 2009
-
8/12/2019 04 Kalyan Rao Konda Applabs
17/24
AppLabs Confidential 16
Cost Saving$1.34m
50%
Cost Saving$1.99m
75%Baseline
Onsite/Offshore ratio Initial onsite: offshore = 80% Reduced onsite: offshore to 38%
Currently it is 15%
Key Initiatives:
Constant review and override of
onsite/offshore ratio
Maximized offshore capability using
effective governance
Onsite/Offshore rotation model
ROI / Efficiency Gains - Summary
-
8/12/2019 04 Kalyan Rao Konda Applabs
18/24
AppLabs Confidential 17
Executive dashboard
-
8/12/2019 04 Kalyan Rao Konda Applabs
19/24
AppLabs Confidential 18
Oct 07 Agent dashboard (functional, automation, regression)Aug 08 Agent commissioning system (functional)
Sep 08 Sales mgmt (functional, regression, maintenance, compatibility, integration)
Sep 08 Contact management system (functional, regression, automation)
Nov 08 Policy processing system (automation QTP/Certify )
Dec 08 Helpdesk application (functional, regression, automation)
Jan 09 Fast Quote 2 (rate validation)
Apr 09 Fast Quote 3 (rate validation)
Mar 09 Service management system (vulnerabil ity)
Phase I Discovery(Oct 07 Mar 08)
Phase II - Steady State & Transformation (Apr 08 Mar 10)
Phase III - Transformation(Apr10 )
Acquired knowledge onbusiness processes
Understood technology andarchitecture of applications
Reviewed current testingprocesses
Devised testing strategies Defined qualitative and
quantitative goals Institutionalize Insurance
certification program
Common practice and methodology for testing Revamp testing and implementation life cycle for current
applications Enforce discipline in SDLC by adherence to defined milestones Increase the rigor in usage of automation tools Focused groups for various test events Rollout and implementation of QA practices for other applications Rollout of QA practices and delivery of QA services for other
Lines of Business and other divisions Develop and roll out infrastructure optimization and non
functional testing services
Continuous improvement Annual review of core team Quarterly review of flexi team
Multi LinesPersonal Lines
Dedicated Team size 240 (15 onsite +225 offshore)
Dec 09 FAB Regression
Aug 09 Claims (functional, automation, performance)
Journey
Feb 09 Service management system (integration)
-
8/12/2019 04 Kalyan Rao Konda Applabs
20/24
Largest e-LearningPlayer
-
8/12/2019 04 Kalyan Rao Konda Applabs
21/24
AppLabs Confidential 20
Largest e-Learning Player Set up(Nov03 -Oct04)
Transform(Nov04 - Dec06)
Deliver (Jan07 - onwards)
Benchmarking Test scope Testing effort Test approach
Standardization of QA processes
Basic reporting functions for Defect life-cycle Resolution management Test case authoring / execution
Established basic communication model
Methodology
Adopted agile practices for QA
Quarterly audits and reviews
Introduced quality metrics
Service delivery performance
Monitored effort and schedule Resource utilization
Scope change Monitored defect detection and defect removal efficiency
Built local test lab Isolation server and QA environment
Trusted partner for QA
Manage the entire quality function of
customer with minimal supervision
Integrated AppLabs into client
business process and release
management
Nov 2003 Manual (Functional, Compatibility, Multi Lingual, Integration)
Nov 2004 Test Automation (Functional, Integration)
Oct 2005 Performance
Dec 2006 Certification
Jan 2008 Engineering (L2 & L3)
Team size 150Oct 2009 Professional Services
-
8/12/2019 04 Kalyan Rao Konda Applabs
22/24
AppLabs Confidential 21
Defect Detection Cost Savings Effort to find defect reduced by 4 times
for core release
Effort to find defect reduced by 30% for
maintenance releases
Cost savings of $2.58 Million for core
releases
Cost savings of $191 K for maintenance
releases
Cost to find a defect
Drivers
Enhanced product knowledge through AppLabs Knowledge Management Framework
Better test coverage by using Bi-directional requirements traceability matrix
Formalized Statistical analysis and impact analysis to find vulnerable test areas
Defect Taxonomies used to drive testing effort
-
8/12/2019 04 Kalyan Rao Konda Applabs
23/24
AppLabs Confidential 22
Test Engineering Efficiencies
Better product quality & decrease in test cyclehours due to early defect detection
Volume of defects found during sprints is50% higher
Reduction in overall testing efforts by 20%(appr. 10,000 hrs)
Effort variance reduced YOY due to better TestPlanning
Defects acceptance ratio improved by 7%because of defect triage and SME knowledge
Cost saving due to test engineering efficiencies is$152 K
Defects in Sprints
Effort Variance
D e f e c t s
Defects Acceptance Ratio
-
8/12/2019 04 Kalyan Rao Konda Applabs
24/24