the value-adding tester

Download The Value-Adding Tester

Post on 06-May-2015




1 download

Embed Size (px)


A presentation about how testers add value to a project or organization, and the costs and value-detractors associated with testing.


  • 1.The Value-Adding Tester How a Tester Adds Value to an Organization or Project12013-12-13PA1Confidential

2. Introduction The Value-Adding Tester adds value to the project or organization The Value-Adding Tester understand and reduces costs The Value-Adding Tester does not detract value To be a value-adding tester it is required to continuously evaluate value and cost How does a tester add and detract value, and what are the costs?22013-12-13PA1Confidential 3. How to Add Value - OverviewMandatory Tests32013-12-13PA1Fixed DefectsInformation to StakeholdersDevelopment SupportConfidential 4. Fixed Defects One of the major ways a tester can add value is by finding defects in the product which are actually fixed at some point during the product life cycle The value of a defect found can be quantified by estimating how the defect would impact sales, and if the customer returns the product because of the defect42013-12-13PA1Confidential 5. Information to Stakeholders Another way to generate value for a tester is to provide stakeholders with information and material for tollgates, milestones and decision points The value is harder to quantify but can be divided into two parts: Helping stakeholders taking informed decisions Allowing stakeholders to feel more confident in their decisions52013-12-13PA1Confidential 6. Mandatory Tests In many cases different governing bodies require that certain criteria are met for a product to be allowed to be sold Meeting these criteria often require mandatory tests that must be executed according to strict requirements Specific customers can also have mandatory requirements that must be tested to be able to sell to those customers The value a tester provides by executing these tests can be quantified by comparing how much it would cost to outsource this to an accredited outsourcing partner62013-12-13PA1Confidential 7. Development Support A tester can also provide value by increasing the efficiency of the development team This can be done by providing a robust, easy-to-use test framework, or by creating and maintaining automated integration and regression test suites which the developers can use The tester can also offer support by helping with unit test plans and strategies Yet another example could be pushing for and educating in testability, which can also drive efficiency72013-12-13PA1Confidential 8. How to Detract Value - OverviewLow Quality Reports82013-12-13PA1Irrelevant DefectsIrrelevant InformationInefficient Test ToolsConfidential 9. Low Quality Reports Testers should not create defect reports that Are hard to understand because of bad language Are duplicates of already existing defect reports Lack information that is critical to understand the defect Lack critical attachments such as log files or screen shots Etc. This will lead to increased report handling and analysis effort for developers If testers create reports which cannot be understood due to improper structure, language, formatting, etc., this also costs additional analysis effort for stakeholders92013-12-13PA1Confidential 10. Irrelevant Defects When a tester reports a defect this has to be analyzed, prioritized and handled in different ways If the defect is not fixed, and the information stored in the defect is not used by stakeholders, submitting the defect actually detracts value instead of adding Many irrelevant defects could however together provide valuable data by revealing trends or problem areas The value detraction can be reduced by handling less relevant defects differently than higher priority issues102013-12-13PA1Confidential 11. Irrelevant Information When testers provide reports to stakeholders, the information presented must be beneficial and help the stakeholders to take required decisions Presenting data that is not relevant or even misleading is not only not valuable, but can be very costly Burying key information in a mountain of data A report that states 99 % Pass Rate as the main conclusion, when the remaining 1% represents critical quality issues in the product, will not help stakeholders make the correct decisions 112013-12-13PA1Confidential 12. Inefficient Test Tools The cost that testers impose on stakeholders with inefficient tools When testers use tools that impact others, such as developers, project managers, or other stakeholders, in a negative way, this detracts value One example could be inefficient test frameworks or automation tools which cause developers to write inefficient tests Another example could be a reporting tool which is difficult for stakeholders to extract reports from, or the tool generates reports which take time to analyze and understand122013-12-13PA1Confidential 13. Costs - OverviewTest Design & ExecutionTest PlanningTest Tools & Frameworks132013-12-13PA1Test ReportingAdministrative OverheadConfidential 14. Test Design & Execution Designing, executing, maintaining and porting tests Execution Effort Exploratory TestScripted TestAutomated TestScripted TestExploratory Test Design Cost Automated Test142013-12-13PA1Confidential 15. Test Planning Planning test activities and creating corresponding test artifacts such as test plans Setting scope for different test activities Risk analysis as impact to scope selection Probability of failure Technical risk analysis Impact of failure Business risk analysis152013-12-13PA1Confidential 16. Test Reporting Creating test reports Test result metrics Qualitative summaries/product stories Creating defect reports Analysis of automated test results162013-12-13PA1Confidential 17. Test Tools & Frameworks Creating new tools and test frameworks Cost of buying test tools and frameworks Integrating new tools Maintaining tools and frameworks172013-12-13PA1Confidential 18. Administrative Overhead How much time does the tester actually spent on design, analysis, execution, tools and reporting, and how much time does the tester loose to administrative overhead? Getting the right software artifacts for test, and understand what to actually test Coordinating between testers, developers, and organizations Non-value adding meetings, etc.182013-12-13PA1Confidential 19. Net Value of a TesterNet Value192013-12-13PA1ValueCostValue DetractionConfidential 20. How do we increase value and decrease cost and value detraction? We want to maximize value gain, and minimize costs and value detraction How we do this depends heavily on context However we can still give some general guidelines Of course there are many other ways to reduce costs and add value, but these are some suggestions202013-12-13PA1Confidential 21. How do we increase value and decrease cost and value detraction? Evaluate test artifacts and remove the non-value adding ones 10-minute Test Plan dont create extensive test plans that no one actually uses or updates Very detailed scripted test cases cost more than they add value most of the time Test Strategy documents that no one reads or uses should not be created at all However the discussions which lead to the creation of the documents are still valuable and important to have212013-12-13PA1Confidential 22. How do we increase value and decrease cost and value detraction? Evaluate tools among all different stakeholders to secure that they are efficient Even if a tool is easy to use for some purposes, it could impose large costs for other stakeholders to use A test administration tool that is easy to record data in, may be very costly to generate reports from Evaluate the actual value gain of the automated test framework Dont just calculate the execution effort saved Many different costs: Design, Analysis, Maintenance, Porting Actual gains: Defects Found, Mandatory Tests, Information and Support to Developers222013-12-13PA1Confidential 23. How do we increase value and decrease cost and value detraction? Secure that the information and defects testers report is what stakeholders actually need Defects must be fixed Information must be used Make sure that testers actually work with testing, and not spend their time on everything else Let testers list what they spend their time on, and try to reduce administrative overhead and let the testers actually work with test232013-12-13PA1Confidential 24. How do we increase value and decrease cost and value detraction? Proper risk analysis to support scope selection Risk-based testing Should be used when the reduction in cost for test execution is larger than the increase in cost for test planning that the risk analysis adds Efficiency gain of risk-based testing is very dependant on the complexity of the system under test Show the cost that the system complexity adds to testing this can drive actions to reduce the complexity of the system242013-12-13PA1Confidential 25. Conclusion The Value-Adding Tester continuously evaluates what value is added, what value is detracted, and the costs for providing this value There are many ways to add value, but there are also many ways to detract value and either way there is an associated cost Understanding the values and costs is critical to become efficient Understanding must be supported by actual metrics, and not only gut feelings252013-12-13PA1Confidential