shift left principle – key enabler for first time right · shift left principle – key enabler...
TRANSCRIPT
10 June 2013 TCS Public
Shift Left Principle – Key enabler for First Time Right
Dr Ramesh K Tumuluru
Structure
1 Context Setting
2 Shift Left – What is it?
10 June 2013- 2 -
3 How to make it Work ?
4 Case Study
310 June 2013
Context Setting
Triple Constraint
Large
Programs
10 June 2013
Iron Triangle / Triple Constraint
Effort
Slippage
Test Coverage
Gap
Inadequate
Requirements
Delivery
Slippage
Poor Quality
Deliverables
Common Symptoms
10 June 2013 5- 5 -
SlippageRequirements
Infrastructure
Issues
Many CRs
Cost
Overruns
Performance
Issues
Some metrics
10 June 2013 6
…. The bigger they are the harder they fail………. McKinsey Quarterly Report
Standish Group Report
Factors Contributing to Failure
10 June 2013
Standish Group Report
…. Unfortunately there are no SILVER BULLETS…..
Complexity
“…. No one thing seems to cause the difficulty – any particular paw
can be pulled away. But the accumulation of simultaneous and
interacting factors brings slower and slower motion…”
10 June 2013
The Mythical Man-Month
Frederick P Brooks
910 June 2013
Shift Left Principle - ?
Shift Left Principle
SHIFT LEFT IS
MUCH MORE THAN
10 June 2013
A PROCESS Or A TOOL
IT IS ALL ABOUT MINDSET SHIFT
Perception about a Tester
General Perception
Pedantic
Skeptical
Nit-picker
Shift Left approach:
Mentor
Advisor
10 June 2013 11
Does not Add much Value
Delays Go Live
Constructive
Business face
Adds Value
Test Strategy
Test Planning
Test Design
Requirements Review
Define Test Strategy
Create Test Scenarios
Create Test Cases / Scripts
Te
st
Ma
na
ge
me
nt
RequirementAnalysis
Design
Development
De
ve
lop
me
nt
Ma
na
ge
me
nt
Define Detailed Test Plan
Software Development Life Cycle (SDLC)
10 June 2013
Test Design
Test Execution
Test Completion
Create Test Data
Run Test Cases
Raise Defects
Re Test fixes
Test Summary Report
Defect Summary Report
Te
st
Ma
na
ge
me
nt
Development
Testing
Production
De
ve
lop
me
nt
Ma
na
ge
me
nt
Waterfall Model
• Generally the testing team will NOT be part of requirements phase
• After the requirements phase is signed off, the test lead or test manager
will pick up the document and produce Test Strategy document
• The testing team is ramped up much later in the cycle
• They independently validate requirements and perhaps identify gaps,
which might be too late in the life cycle and therefore make assumptions
How does this model work in reality
10 June 2013
• The testing & development teams will work independently especially the
test design phase and build phase respectively, with their own
interpretation of requirement
• After the requirements are provided by Business or Users, they come
back only during UAT, which might be almost one year later
• Gaps in the requirements found during testing
• Test coverage is inadequate
• Multiple defects and multiple CRs raised during user acceptance testing
• Due to timeline pressure most of the CRs have to be completed within
the original schedule
• Work extra hours ……so on and so forth…..
Outcome of this model
10 June 2013
• Work extra hours ……so on and so forth…..
Inadequate understanding
of requirements
Test Coverage
Acceptance from Client
Fixes
Go Live Date
Test Execution timelines
Outcome of this model
10 June 2013 15
Additional Test Cases
Defects in UAT
Availability of Controlled
Environment
New CRs
timelines
Resources & Effort
…. Problem becomes bigger and bigger…..
What is Shift Left principle
Conventional approach:
Assurance team would get involved in Testing phase.
Shift Left approach:
Assurance team would get involved at an
early stage i.e. from Requirement
gathering itself and would continue to
get involved in design and development,
unit testing, system testing, integration
10 June 2013
unit testing, system testing, integration
testing, UAT etc, until ‘go-live’ in to
production
…. Validation & Verification…..
What is Shift Left principle
Conventional approach:
How should I test the application / system once it is delivered
Or
How should I release a defect free software to UAT
Shift Left approach: How to engage both business and developers
throughout the SDLC ?
How to enable requirements review process
EFFECTIVE
How to enable EFFECTIVENESS in test design
phase
How to enable the developer to UNDERSTAND
10 June 2013
How to enable the developer to UNDERSTAND
end to end requirement
How to enable developer to do COMPREHENSIVE
test coverage
How to ALIGN business testing with internal
testing
How to enable effective CODE REVIEW
Shift left is all about MINDSET Shift
What is Shift Left principle
Test StrategyRequirement Analysis
Design Test Plan
SHIFT LEFT
10 June 2013
Development Test Design
Test ExecutionUnit Testing
The lesser the Total Internal Refection Angle the greater the ALLIGNMENT The lesser the Total Internal Refection Angle the greater the ALLIGNMENT
TIME TO MKT
QUALITY
COST
Requirements Review
Joint Test Coverage Matrix
Early Involvement of User
Shift Left Principle
10 June 2013
COST
Unit Test
Reviews
Design &
Code Reviews
Early Interface
Testing
Enables high First Time Right in client’s testing
Shift Left Principle
Shift Left principle enables close collaboration with Business,
SHIFT LEFT IS ALL ABOUT MINDSET SHIFT
10 June 2013
Developers and Testers DURING THE EARLY STAGES of
development life cycle to ensure that we are building the RIGHT thing
and also building the thing RIGHT.
2110 June 2013
What do we need to make Shift Left happen?
7 S model of Shift Left
Structure
Systems
Strategy
Shared
Values
Mind Set
10 June 2013- 22 -
StyleStrategy
StaffSkills
Mind Set is the corner stone of Shift Left Principle
Test Design Process 7 S model
1 Strategy
� While constructing the test strategy there should be fundamental mindset change in terms of
how can the testing add value prior to system testing phase . We need to come with up a
different set of metrics to measure the alignment
2 Structure
� The Test Delivery Manager should be part of core management team and should have a say in
the program and it should be an independent team
10 June 2013
3 Systems
� The processes should be modified to accommodate workshops with internal and external
stakeholders after the requirements phase, design reviews and code reviews
� Test design phase templates should be simplified to accommodate changes in requirements
4 Skill
� The testing team should understand the overall solution and should have business knowledge
� The testing team should have technical skills for Unit Testing review, and to do design reviews
� The testers should be techno-functional
2410 June 2013
Case Study
Program
Outsourced
Multiple Geographical
Locations
First Time interaction
Multi Services
Scope was very BIG
Product Driven
Solution
10 June 2013
SYSTEM
Requirements Evolved No baseline
business processes
Lean Organization
Different Time Zones
RegimentedContractual Obligations
interaction with 3rd Party
systems
Urgency to Go Live
Hardship
• Functionality 52 FSDs
• % of T Ss Used 50+
• % of TCs Used 40+
• Total Effort X person months
• Total Defects in UAT Huge
First Release
10 June 2013
• Total Defects in UAT Huge
Shift Left Principle practiced
Worked on
• How to enable requirements review effective √
• How to enable test design phase simple and effective √
• How to enable to developer to understand the end to end requirement √
• How to enable developer to do comprehensive test coverage √
10 June 2013
• How to enable developer to do comprehensive test coverage √
• How to align business testing with internal testing √
• What should be the measurement criteria √
• How to carry out early integration testing √
Shift Left Impact
Scenario – Improved Business Confidence resulting in less UAT effort
GL & MI requirements
Improved Test Coverage
High DefectContainment
More Time to review SIT coverage
10 June 2013
High First Time Right in
UAT
IncreasedBusiness
Confidence
Containment
Less UATEffort
coverage
One of the feedback loops after the intervention which changed the behavior
Outcome
• Functionality 52 FSDs
• % of T Ss Usede 50+
• % of TCs Used 40+
• Total Effort X person months
• Total Defects in UAT Huge
• Functionality 29
• Test Scenarios Used 98%
• Test Cases Used 99%
• Total Design Effort 40% (of
previous release)
• Only 20% of Code related Defects
First Release Second Release
10 June 2013
Effort Saved = > 60 % of design effort
Time Saved = > 50% of design schedule
Quality = UAT First Time Right increased to 85%
• Total Defects in UAT Huge
Qualitative Benefit -> Accountability, Satisfaction, Self Organized teams
3010 June 2013
Wrap Up
Wrap Up
Large transformational programs are intrinsically COMPLEX.
The success rate is NOT high
In dealing with complex systems in general, it is helpful to consider several
DIFFERENT perspectives of the complex system.
10 June 2013
One of the enablers for achieving iron triangle is
SHIFT LEFT principle
The shift left principle enables VALIDATION & VERIFICATION
Shift Left is all about MINDSET SHIFT
Shift Left Principle
10 June 2013
Testing team to continuously engage business and development
Shift Left Principle
Business
Requirements
Development
Test Design
Analysis
10 June 2013
Development
Testing
Support
Should we look at S model in place of V ?
I would like to thank
Mrs. VIJAYA DEEPTI, VP, Head of Insurance, TCS FS
Acknowledgement
10 June 2013 34
for encouraging and allowing me to think & and apply these
principles & models to very critical & complex programs
Thank YouThank You