7/27/2019 IEC_Validation_and_Testing_ITIL_v3_Cheat_Sheet.pdf

1/2

ITIL V3SERVICE TRANSITION SERVICE VALIDATION AND TESTING

INFRASTRUCTURE EXECUTIVE COUNCIL

DEFINITION

A service providers method forensuring that a new or changed service meets customer requirements and verifying that it is ready for productionsupport by IT operations.

VALUE TO BUSINESS:

An effective Service Validation and Testing process ensures:1. Reduction in business disruption from incident introduced via release2. Improved resource allocation through alignment of testing with areas of risk and business valueHIGH-LEVEL PROCESS FLOW:

Stakeholders: Test Manager Business Stakeholders Demand Manager Service Level Manager Infrastructure Manager Application Manager

Test Manager Test Manager Service Level Manager Demand Manager Release Manager

Focus On: Elicit customers expectations Allocate testing resources Schedule delivery and acceptance stages Prepare and baseline the test environment Create a testing model that measures service

efficiency and effectiveness against

customer-relevant KPIs

Perform all automated or manual tests Document all test results Compare test results to projections Reset the test environment

Identify areas of improvement (if any)

KEY CONCEPTS:

Test:A validation that all release components, tools and processes required for deployment, migration and back out are acceptable, and onlythe components meeting stringent quality criteria are deployed into the live productive environment.

Test Environment:A controlled environment used to test release components Test Strategy:A master plan for quality assurance outlining all testing processes and resources within a broader project plan Test Model:A more detailed test plan and script indicating how release components will be tested Service Design Package: Specifications about how customer requirements for a business service will be fulfilled through a combination of

supporting services

Validation: Confirmation that a new or changed service is complete, accurate, reliable, and matches its design specificationsKEY PERFORMANCE INDICATORS (KPI):

Acceptance Test Failure Rate:Percentage of all release components that fail the acceptance test

Total Identified Errors Per Release: Total number of errors identified during testing New Incidents Per Release:Number of incidents caused due to the new release of a service Error Resolution Time: Time taken to fix errors identified during testing and resubmit the release

COMMON PITFALLS:

Test process do not use results of a business risk assessment (including performance, lost business, security, etc) which results indevelopment of services which do not meet customer's expectation

Change management process does not have synergic relationship with service validation and testing process leading to changes in serviceswhich do not meet business requirements. This leads to redundant and/or less effective services

EvaluateTestPlan

7/27/2019 IEC_Validation_and_Testing_ITIL_v3_Cheat_Sheet.pdf

2/2

ITIL V3SERVICE TRANSITION SERVICE VALIDATION AND TESTING

INFRASTRUCTURE EXECUTIVE COUNCIL

IECRESEARCH:The Infrastructure Executive Council has identified the following core elements of effective validation and testing:

Engage the infrastructure team at the pre-coding stage Measure projects for alignment with target architecture and performance across all key phases Offer prescriptive and actionable infrastructure feedback to applications as a service during design and coding Winnow the number of stakeholders who need to be deeply involved in production acceptance by asking project stakeholders to complete a

checklist identifying the areas of greatest risk

CASE IN POINT:THE PROGRESSIVE GROUP -PULL FORWARDLIFECYCLE PERFORMANCE

Although SLAs are captured and documented early in a development cycle, they are generally set aside until the beginning of t he testing phase.

Moreover, project timelines often threaten to compress the testing cycle, resulting in user-facing defects and disruptions. To resolve these issues,Progressive created a strategy to test alignment with service level requirements across earlier stages of the development cycle. This strategy

comprises of two distinct components:1. An Early-Action Checklist: Progressive captures service level requirements early and builds out built out an entire process devoted to

optimizing performance, provides to developers as a set of guidelines for making decisions about nonfunctional requirements needed to

satisfy performance. Developers can then follow the checklist and typically perform activities during design that happen much later, suchas logging, hooks to infrastructure instrumentation, or capacity scripts.

Performance Optimization Guidelines for Developers

Service Level Agreement Availability RequirementsDeveloper Activities Checklist

Reliability

(Proactive Measures)

Supportability

(Improving Reactive Measures)

Performance Requirements

Response Time Requirements

Normally expressed as a rangeExample: 23 seconds

Throughput RequirementsNormally expressed as transactions per second

Example: 25 transactions per second

Completion Times

Error/

Exception

Handling

All exception-handling codeconforms to the ExceptionHandling standards

described in the PolicyPro

Development StandardsDocument.

Sufficient failure test caseshave been created and

executed to exercise every

catch block.

Availability RequirementsService Hours

A description of the hours the service is required to be

availableExample: 08:0018:00Monday to Friday

Availability RequirementsTarget availability levels within the agreed service hours

Normally expressed as percentagee.g., 99.5%

Reliability Requirements

Maximum number of service breaks that can be tolerated per

period e.g., four per month, OR as Mean-Time-Between-

Failure (MTBF)

FaultTolerance

Application failover logic, retrytimers, retry counters, and retry

logic have been agreed upon andimplemented for external and

internal dependencies.

2. Performance Test Foundation: Using guidance from service level agreements, the company identifies specific activities developersshould perform. For example, if a developer seeks to meet an appropriate throughput for a business transaction as defined by its duration,frequency, and execution time, Progressives recommendation to that developer is to execute representative transactions and then enter the

throughput information into its transaction monitoring system.

Harmonizing application and infrastructure designs greatly improves Progressives test acceptance rate while also reducing the cost of supporting a

new application across its early lifecycle, when conventional applications may go through intensive performance tuning.

Access the ful l story onl ine:The Progressive Group - Pull Forward Lifecycle PerformanceOTHERIECRESOURCES

Capital One: Collaborative Performance Optimization Schlumbergers Network Application Qualification Process Texas Instruments: Process Control-Focused ERP Performance Management

https://aec.executiveboard.com/Members/ResearchAndTools/Abstract.aspx?cid=47315677https://aec.executiveboard.com/Members/ResearchAndTools/Abstract.aspx?cid=47315677https://aec.executiveboard.com/Members/ResearchAndTools/Abstract.aspx?cid=47315677http://__dopostback%28%27ctl00%24ctl00%24main%24main%24browsesearch%24browsesearchrepeater%24ctl00%24titleabstractlink%27%2C%27%27%29/http://__dopostback%28%27ctl00%24ctl00%24main%24main%24browsesearch%24browsesearchrepeater%24ctl00%24titleabstractlink%27%2C%27%27%29/http://__dopostback%28%27ctl00%24ctl00%24main%24main%24browsesearch%24browsesearchrepeater%24ctl00%24titleabstractlink%27%2C%27%27%29/http://__dopostback%28%27ctl00%24ctl00%24main%24main%24browsesearch%24browsesearchrepeater%24ctl00%24titleabstractlink%27%2C%27%27%29/http://__dopostback%28%27ctl00%24ctl00%24main%24main%24browsesearch%24browsesearchrepeater%24ctl00%24titleabstractlink%27%2C%27%27%29/http://__dopostback%28%27ctl00%24ctl00%24main%24main%24browsesearch%24browsesearchrepeater%24ctl00%24titleabstractlink%27%2C%27%27%29/http://__dopostback%28%27ctl00%24ctl00%24main%24main%24browsesearch%24browsesearchrepeater%24ctl00%24titleabstractlink%27%2C%27%27%29/http://__dopostback%28%27ctl00%24ctl00%24main%24main%24browsesearch%24browsesearchrepeater%24ctl00%24titleabstractlink%27%2C%27%27%29/https://aec.executiveboard.com/Members/ResearchAndTools/Abstract.aspx?cid=47315677