John Fodeh - Spend Wisely, Test Well

Download John Fodeh - Spend Wisely, Test Well

Post on 15-Jul-2015

45 views

Category:

Software

0 download

Embed Size (px)

TRANSCRIPT

INSERT PRESENTATION TITLE HERE

SPEND WISELY, TEST WELLMaking a Financial Case for TestingHP - Global Testing PracticeJohn Fodehjohn.fodeh@hp.com 2009 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice 11Introductions

22Originally presented as a day tutorial by Susan Herrick, EDS US at StarEAST and StarWEST in 2008 & 2009About You (and Me)Briefly introduce yourself by telling us:Your name

Your testing role

How long youve been in the testing arena

One thing youd like to get out of this Workshop33This is a workshop, so dont get to comfortable, you are expected to participate, lets start with some details????About This SessionAfter youve completed this workshop, you should be able to:Address and even turn around the prevailing view of testing among business stakeholdersThrough the effective use of industry data:Establish and/or reinforce your credibilitySubstantiate your testing recommendations and strategiesDemonstrate to business stakeholders in terms they can understand and accept the relative cost of testing and not testing for a specific project

Credits: Ian Howles & Susan Herrick!

44AdministriviaThe duration of this workshop is 1.5 hours

There will be no breaks

Ask questions whenever they occur to you

55Session uses a simple waterfall approach but is applicable to all lifecycles models.Its All About Communication

66Remember, testing is a service organization.

Our mission is to provide information about quality and risk to decision makers within our organization.TestingOur Ideal Organization ChartThe Remainder of the OrganizationForgetting the Organizational ContextFrom The Nine Forgettings by Lee Copeland77why we exist and who we serve

Testing does not create quality. All we do is look for it.

Forgetting the Organizational ContextTesting should not be the Ship / Dont Ship decision maker. That is not our stewardship.88upside down and/punishment mechanisms Wally Im going to write me a minivan

The Business SpectrumMass-market packaged softwareMarket-dominating softwareNiche softwareHigh consumer toleranceCustom softwareMarket-entry softwareCompetitive softwareLow consumer toleranceRegulatory complianceCritical safety requirementsDefect ToleranceHighLow

Perfection is the enemy of the good- Voltaire99Cost of QualityQuality PerspectiveC Quality = C Conformance + C Non-Conformance

WHEREC Quality = Cost of Quality C Conformance = Cost of ConformanceCost of preventionCost of detectionCost of internal failure

C Non-Conformance = Cost of Non-ConformanceCost of external failureOther related costs

1010Cost of QualityBusiness PerspectivePV C Quality = NV

WHEREPV = Potential Value, e.g., ALL sources of revenue associated with a software product, annualized cost reduction that results from productivity improvementsNV = Net Value, e.g., actual revenue for the software product, actual annualized cost reduction

AND

PV= PM or ROI (% + or -)NV

WHEREPM = Profit Margin ROI = Return on Investment

1111Its All Greek to Me!The language of Testing focuses on: Quality Defects Test coverage RiskThe language of Business focuses on: Scope Time Money Risk (and, oh, yes, Quality)

1212Bridging the Communication GapThe decision-makers will tell you they care about quality and customer satisfaction but not if they believe that achieving either will adversely affect their bottom lineTalking about defect discovery alone isnt enoughTesting has to be able to:Demonstrate the financial consequences of testing involvement or lack thereof in a projectSubstantiate the financial consequences with reliable data

1313For example. Ive selected Profit Margin and/or ROI because they are commonly used terms.However, YOU NEED TO HAVE A SOUND UNDERSTANDING OF WHATEVER LANGUAGE YOUR COMPANY WORKS IN. DO THEY USE INTERNAL RATE OF RETURN, PAYBACK PERIOD, TIME TO VALUE? SOMETIMES BUSINESS LEADERS DON'T ALWAYS SYNC UP TO THE VALUE LANGUAGE OF THE COMPANY. IF CAPITAL IS INVOLVED, YOU NEED TO UNDERSTAND THE PROCESS YOUR FINANCE DEPARTMENT USES TO APPROVE THE BUDGET AND GET IT INTO THE LANGUAGE THEY SPEAK.Business case assumptions must be thoughtful and clearly supported in terms a CFO will understand, Show Me the MoneyTesting requires financial investment in:

Resources, tools, equipment

Time to develop, test, manage

Escaped defects

Rework and retesting

Business stakeholders legitimately expect a healthy return on that investment

1414Defining the Communication ChallengeLets quickly discuss some views of testing that:

You hear from decision-makers in your organization and/or on your projects

You would like to change

1515Prevailing View vs. Desired ViewTesting is a back-end project phase

Testing is easy and anyone cando it

Significant rework is inevitable

Testing provides advice that can be selectively ignored

Testing creates bottlenecks and increases the risk of schedule overruns

Testing costs too muchTesting is an integral aspect of all phases in the product life cycle

Effective and efficient testing requires skilled testing professionals

Focusing on early defect discovery and prevention can significantly reduce rework

Testing provides information that quantitatively specifies outstanding risk, application readiness and potential cost ignore it at your peril!

Testing early and testing often can actually reduce the risk of schedule overruns and accelerate time-to-market

Not testing costs way too much in rework (pre- and post-release), failures, and production support

1616Exercise:Turning a Problem into a SolutionThe following exercise is derived from a tutorial developed by Steve Bender of The Quality Connection (www.qualityconn.com):

Effective Methods to Develop Creative Approaches to Todays Challenges

The exercise requires you to:Define a problemDefine a solution that is:Stated in positive termsUnder your controlIntegratedTestable

1717ExerciseStep 1 - Defining the ProblemTake 5 minutes to answer these questions. Whats the problem?How long has this problem been a problem?Why does this problem exist?How are you and/or your team limited by this problem?What is the worst thing this problem has caused?

1818ExerciseStep 2 - Defining the SolutionTake 5 minutes to answer these questions. What do you want? Be as specific as possible.How will you know when you get it?Where, when and with whom do you want it?When you get it, what else will improve?What resources do you already have that will help you get it?What is the first step to take to get it?

1919Possibly in groups, depends on the audience size, could be a collaborative exercise?Defining the ProblemAn ExampleWhats the problem?Perception that testing costs more than its worthHow long has this problem been a problem?For as long as I can remember (15+ years)Why does this problem exist?No real proof to the contraryTesting world does not speak the language of businessHow are you and/or your team limited by this problem?Inadequate allocation of time, money, resourcesMarginal influence over allocation decisionsLack of senior management supportWhat is the worst thing this problem has caused?5-10% of project effort allocated to testingUnnecessary schedule and cost over-runsDefective software in production

2020Example from Susan HerrickDefining the SolutionAn ExampleWhat do you want? Be as specific as possible. Real, tangible proof that the decision-makers get itHow will you know when you get it?Commitment through actionProjects - Upfront testing involvement in planning, scheduling, impact analysisSenior management Testing Policy and StrategyWhere, when and with whom do you want it?Senior management and project purse string holdersWhen you get it, what else will improve?Testing involvement and influenceProduct quality, cost, speed to marketWhat resources do you already have that will help you get it?Industry dataAccess to audienceWhat is the first step to take to get it?Pull together the dataFilter and home in on compelling proof

2121Communication PrincipleThe meaning and merit of the communication is in the response it elicits

2222The Proof is in the Data

2323First a couple of quotes:

Demming

CarrolIm from Missouri show meIn God we trust all others bring data

- W. Edward Deming

2424A Common Modus OperandiIf you dont know where you are going, any road will get you there - Lewis Carroll

2525An Alternative PerspectiveIf you dont know where youre going, how willyou know when youve arrived?

OR

If you dont know what testing or not testing is really going to cost you, how can you know if youve spent wisely?

2626Show Me the ValueTwo axioms of management are:What gets measured gets doneWhat gets valued gets funded

So, it's important to remember that testing delivers value that we can and should measure- Rex Black

2727Note on ROI:In todays economic climate, the word investment is probably anathema to a lot of organizations. An alternative to the term ROI could be Return on Expenses or, even better, Returned Value.

From Hung Q, Nguyen Chairman and CEO, LogiGear Corp.Addressing the ChallengeFocus on the financial impact of defect resolution through testing activities throughout the product life cycleUse the following industry data to substantiate claims about the financial impact of:Defect Insertion RateAmplification FactorDefect Removal EffectivenessTime to Resolve a DefectCost of Defect ResolutionRework RateAddress the assumption that automation solves the problemDemonstrate the cost of schedu