building a performance system for it.pdf

8/11/2019 Building a performance System for IT.pdf

http://slidepdf.com/reader/full/building-a-performance-system-for-itpdf 1/16

White paper

BUILDING APERFORMANCESYSTEM FOR IT



Table of contents

Executive summary 3Key requirements for an IT performance system 4The anatomy of a performance system 4

Governance 4

Delivery 4Measurement 4Building a performance-oriented culture 5The HP IT Performance Suite 5The HP IT Executive Scorecard: a system of measures 6Establishing a common data model 7Building and using your system of measures 8Key takeaways 8Suggested next steps 8

Appendix: key performance indicators 9Customers 9IT value 10Future orientation 11Operational excellence 12



Executive summary

Every leader in today’s modern enterprise should have aperformance system—a means to transparently measureand manage progress against the goals they are given.

With $5 trillion per year invested in enterprise IT, it is essentialthat each IT leader has a digitized system that enables themto manage and improve the key performance indicators (KPIs)that matter to the organization.

This white paper documents the requirements of an effectiveIT performance system and introduces a set of associatedmeasures and best practices that are both aspirational andpractical. In addition, the paper outlines a frameworkof key performance indicators to help IT perform betterand presents specific KPIs to consider for leaders withindifferent IT domains.



Narrative

Governance and alignmentCustomer Sat. Future Orientation IT Value Operational

Excellence

Delivery - people, processes and tools

Measurement and relationships

Narrative

IT staff Backup

Data ModelPerformanceData

Testing Server Automaton

Etc. .. Processes

Figure 1. Elements of an IT performance system.

Key requirements for anIT performance systemTo be effective, an IT performance system must meetthree requirements:1. Comprehensive. Your CIO and IT leadership team

collectively require oversight and control across allof the people, processes, and systems in the ITdomain, from infrastructure to information, applications,and security, and from planning to development andoperations. You cannot manage what you cannotmeasure, so comprehensive visibility into the performanceof the entire IT investment portfolio is required for success.

2. Connected. To make informed decisions, IT leadersneed a dynamic performance system comprised of ITmanagement and security tools that not only automatetheir discrete functions, but also deliver insight byfeeding digital updates to their KPIs in a timelymanner. IT leaders should be able to view and sharetheir KPIs in a series of connected and cascadingscorecards so that each leader’s KPIs support theirmanager’s up to the CIO level.

3. Flexible. An IT performance system needs to providethe flexibility to enable your IT leaders to set andevolve the KPIs that map to their unique priorities,create scorecards that align with their organizational

structures, and select the underlying IT managementtools that automate and run inside their diverse andheterogeneous operating environments, withoutcreating vendor lock-in.

The Balanced Scorecard: Translating Strategy into Action. Robert SKaplan, David P. Norton, 1996. ISBN 978-0875846514

https://www.isaca.org/ http://www.itsmfi.org/ http://www3.opengroup.org/ https://www.issa.org/ http://www.edrm.net/

The anatomy of a performance system An IT performance system consists of three keyfunctional layers: governance, delivery, and measure-ment, as indicated in Figure 1.

GovernanceThe process of direction and evaluation of IT from a busioutcomes perspective. This is also where performanceobjectives are agreed and documented, establishing abalanced scorecard comprised of key performance

indicators which are set and tracked for variance from plaBy providing a written as well as visual history of progagainst goals, a scorecard enables your stakeholders tomaintain the “story behind the numbers,” or narrativeThis is especially important when performance goals not being met. In these situations, a documented narratiserves as a valuable record that can help isolate root-cause information and/or historical context that wouldotherwise not show up in a purely data-driven dashboar

DeliveryThis is where performance is improved, sustained, orreduced. The delivery aspect of a performance systemencompasses the people, processes, and technologyinvolved in the day-to-day delivery of new and existinservices. There are three critical subsets inside everyorganization:

• People—including employees, contractors, andvendor partners.

• Processes—typically a hybrid of organization-specifbest practices, industry-standard frameworks (such COBIT, ITIL, TOGAF, ISSA, EDRM), and enterprisspecific processes and cultural norms, which can bedocumented and/or formalized within an ITperformance system.

• Technology—the software systems used by IT leadeand their teams to automate, monitor, and measurespecific IT functions. Typically tools vary by role orfunction. A chief information officer (CIO) will likefocus on high-level tools for strategy, planning, andgovernance, including tools for financial planningand analysis, portfolio and asset management, andworkforce management. A vice president (VP) ofapplications and his or her organization will likely utools for application lifecycle management, includinquality assurance, project and portfolio management,and business service management. The VP of operatiochief information security officer (CISO), and VP ofinformation and their teams will similarly use specializtools for ensuring the success of their functions.

MeasurementThis is the functional layer where performance metricsare created and performance data is stored. Continualimprovement also requires that IT establish a commondata model which enables the consolidation of factsfrom disparate sources. To allow for exception-basedmanagement and continual improvement, these measurments should be made consistent such that they can becompared across the processes, systems, and functionalteams outlined above.

http://www.isaca.org/

http://www.itsmfi.org/

http://www3.opengroup.org/

http://www.issa.org/

http://www.edrm.net/

http://www.edrm.net/

http://www.issa.org/

http://www3.opengroup.org/

http://www.itsmfi.org/

http://www.isaca.org/



Figure 3. Four key reporting areas of an IT Executive Scorecard.

Commondata

modelIT valueOperational

excellence

Futureorientation

Customers

The HP IT Executive Scorecard:a system of measuresCritical to delivering better performance is the abilityto set, cascade, and measure KPIs such that they becomepart of the day-to-day behaviors of your IT organization.HP research has identified 170 KPIs associated withworld-class IT service spanning all key IT processesand organizations. Once set, each KPI can then becascaded through your organization to drive alignmentto the performance initiatives of the entire team,effectively establishing a common language to describeIT performance.

The IT Executive Scorecard automates the collection,presentation, and tracking of a growing number of KPIsthat populate a system of scorecards that can cascadeup and down the IT leadership team. You can composeyour own performance system from a KPI library or startwith one of the persona-specific editions HP has createdfor the CIO and key IT function leaders, and then extendit as needed.

Each HP IT Performance Suite edition features the HPExecutive Scorecard software bundled with a set of softwtools that improve domain-specific performance andautomatically populate the KPIs with up-to-date statuThis approach is designed to provide rapid benefitsrealization in terms of measuring and improving yourperformance along dimensions of cost, quality, innovatiand your organization’s readiness to address future demand

The data is organized into the four categories: IT valucustomers, operational excellence (including risk and

security), and future orientation, all of which are drivefrom HP’s common data model based on the HPUniversal Service Model architecture (Figure 3).

The IT value element of an IT balanced scorecard seeto capture and categorize business and IT alignmentat the highest level, primarily around areas such asalignment with business strategy, the stewardship of thoverall IT spend, and improved return on investment aa function of value versus cost.

The customers element of the IT balanced scorecardtracks improvement or declines in metrics that impactcustomer satisfaction, such as service-level at tainmenas well as qualitative metrics, such as customer survey

results. Highly correlated to customer satisfaction areservice availability and the duration, mean-time-to-re(MTTR), and number of open incidents associated wia service.

The operational excellence element of the IT balancedscore card seeks to track and maximize the throughpuof planned work, such as new projects. This elementfocuses on streamlining and automating processes forunplanned or reactive work, such as help desk incidenas well as metrics that reflect the quality and effectiveof the broader delivery process, such as measure ofrework, escalations, and work backlog. Here also, yotrack security, risk, and compliance metrics.

Finally, to enable your IT leaders to better respond tochange, the IT balanced scorecard tracks the futureorientation of the IT function. In addition to trackinginformation about the age and complexity of major ITsystems, the CIO edition also tracks measures related human capital, such as the relative contribution of theexternal versus internal labor required to respond toenterprise IT demands, which provides a measurableproxy of internal readiness.



Figure 4. Delivering KPI-based Scorecards on top of an Open Data Model

Finance Configuration Businessservices

Application

LifecycleManagement

InformationLifecycleManagement

Risk andSecurity

OperationsManagement

Qualitative

Executive scorecard

Administration

KPI library KPI engine Collaboration

Scorecards, KPIs/metrics, dashboard pages

Deployment

Configuration

Security Data integration, consolidation, stewardship

Open data model

Customer AssetProject ContractChange

IncidentCost ServiceSLA

Vendor

XLS

HP, along with a number of Fortune 500 companies,has also led the creation and publication of an open,industry-standards-based model for configurationmeta-data (refer: open data model). This data modelallows you to formalize, specify, and query therelationship between business processes, applicationsinfrastructure, and service providers in a consistentmanner using industry standards, such as TQL and XM

Further information on the open data model can befound in the references section of this document.

Establishing a common data model A comprehensive and connected IT performance systemrequires a consistent data model that enables consolidation,sharing and comparison of trusted metrics which form abasis for continuous improvement. When establishing acommon data model, it is important to use a comprehensivemodel that is independent of any particular sourcesystem and open to any and all source systems, so thatthe entirety of IT can be covered, including today’spublic Cloud service providers, and so that sourcechanges do not affect performance history as seen bythe governance user.

The primary consideration when importing measurementdata is enabling consistency across data sources. Inorder to support both quantitative- and qualitative-baseddata, the HP IT Performance Suite ETL architecture allowsfor data to be imported and normalized across bothtraditional SQL databases and flat files, such as spread-sheets or tabular data.



Building and using your systemof measuresHaving established a functioning data model and structureda balanced scorecard, the next step is to determine whatKPIs to track at what level of the IT organization.

While there are no hard rules, there are a number ofbest practices to consider as you structure your ownsystem of measures and scorecards:

1. Less is more. A few, well selected KPIs on eachleader and individual contributor’s scorecard is morelikely to be successfully executed than a comprehensivebut overwhelming one.

2. IT is a team sport. Scorecards and associated KPIsshould cascade clearly across the IT leadership team(see Figure 5). Many scorecards reinforce silos withinIT by creating isolated KPIs for each functional teamin the delivery chain. While only one functional teamshould be accountable for attaining specific KPIs, ITleaders should ensure at least one of the cascadedgoals is heavily weighted toward the end-to-endattainment of enterprise outcomes.

3. You get what you inspect, not what you expect. Atransparent rewards and measurement system canhave a powerful effect on driving behavior. Wherepossible, ensure that metrics are balanced such thatbehaviors reflect the broader objectives of the

organization.4. Focus on benchmarks and continuous improvement.

Benchmark and continuously improve—equally importantto defining what KPIs to measure is establishing thebenchmarks and goals. KPIs that are unrealisticreduce the IT team’s confidence of achieving them—equally, KPIs that are too easy to achieve maytranslate into sufficient effort put into improvement.By constantly testing and measuring, comparinginside the organization and outside against industrypeers, you can set benchmarks to drive continuousimprovement in a meaningful manner.

ProjectManagement

Office

Office ofthe CIO

CIO

Business Analyst

VP ApplicationDelivery

Quality Assurance

NetworkManager

Service DeskManager

VP Infrastructure& Operations

Chief Information

Security Officer

Development

Figure 5. An IT Performance System provides scorecards that cascadeto multiple levels of IT leadership. Key takeaways

In order to always perform better, CIOs and IT executineed a digitized and automated system that continuoumeasures, communicates, and improves IT performanby instrumenting the entire IT function in a comprehenmanner. To gain timely insights and provide fact-baseevidence of how IT supports the business, it is essentithat such a performance system dynamically updatesKPIs and is connected to every element of the IT landsc

The IT performance system should provide the flexibfor IT to move fast, leverage existing capabilities, operin a heterogeneous landscape, and exploit new technologyand opportunities to drive better business outcomes.

The HP IT Performance Suite not only meets these corequirements—comprehensive, connected, and flexiblbut also offers your IT leaders rapid deployment editifor KPIs and scorecards to accelerate their time to insiand promote greater confidence.

Suggested next stepsIf you are interested in establishing an IT performancsystem, you can select from a range of HP workshopsincluding:

• IT Performance Management Workshop that acceleratalignment around an IT end-state vision required todeliver the desired enterprise outcomes in the contexof the current shifts in IT production and consumptimodels (four-hour workshop onsite, with follow-up

• Value Discovery Workshop that develops a roadmapfor key transformational IT initiatives, including peprocess, technology, and related KPIs (approximatelfour days onsite)

• Readiness Workshop that assists in building outprocess-and-tool specific implementation plans forbetter business outcomes (one to two days onsite)

If you are interested in sharing and improving best-practiwith your peers, join the Discover PerformanceCommunity:www.hp.com/go/discoverperformance/reg

http://www.hp.com/go/discoverperformance/reg

http://www.hp.com/go/discoverperformance/reg



Table 1. Customers

Solution Domain Optimization Opportunity/KPI CIO VP of Operations VP of Applications VP of InformationManagement

Chief SecurityOfficer

Applications % deviation from planned hours worked x x x

Applications % of healthy projects x x x

Applications % of project tasks on time x x x

Applications % of projects on time x x x

Applications % of projects with unresolved urgent issues x x x

Applications Customer satisfaction via survey (sometimes inlieu of timeliness and/or budget KPIs) x x x

InformationManagement Improve user access to information x

InformationManagement Time to respond to e-discovery x

Operations # unplanned outages per month x x

Operations % of IT budget spent on maintainingexisting systems x x

Operations % of met SLAs x x

Operations % of met SLOs for IT process Activities x x

Operations % of met SLAs x x

Operations % of met SLOs for IT process Activities x x

Operations % of outages/total SLA uptime over time x x

Operations % of satisfied customers x xOperations % of service availability x x

Operations % of service performance not met x x

Operations Availability & performance SLAs of missioncritical applications x x

Operations Avg. outage duration per incident x x

Operations Customer satisfaction for help desk x

Operations Hours monthly downtime x x

Operations Mean time to implement changes x

Operations Mean time to repair incidents x

Operations Mean time to resolve incidents & alerts x x

Operations MTBF of business services x x

Operations MTTR of business service incidents x x

Appendix: key performance indicatorsHP research has identified a number of KPIs frequentlyassociated with measuring performance at leading ITorganizations across the domains of customers, IT value,operational excellence, and future orientation. The actualnumber and specific KPIs implemented by customers willdiffer based on the performance framework they designas well as their unique culture and performance initiatives.

Note: We have not provided targets and best practicelevels for KPIs, as they vary by industry and need to betailored to each customer’s circumstances. Also, in somecases, there are multiple ways to measure a KPI (such ascustomer satisfaction) and this paper does not prescribea single formula for measuring a KPI. HP and our partneras well as third parties, offer workshops and other programto help customers select KPIs, determine appropriatemeasurement criteria, and apply best practice benchmarks

Personas



Table 2. IT value



Applications % of change in projects cost x x x

Applications % of projects - cost reduction x x x

Applications % of projects associated withbusiness objectives x x x

Applications % of projects budget at risk x x x

Applications Innovation - hours worked on new vs.maintenance x x

Applications On budget variance x x x

Financials % of actual vs planned projects cost x

Financials % of capex vs opex spending x

Financials % of change in business service cost x x

Financials % of IT POR vs. total revenue x x

Financials % variance of actual vs. planned costs x x

Financials Avg cost of IT—delivery per customer x x

Financials Business service cost reduction x x

Financials Innovation delivery x x x

InformationManagement % of data/information stored with 3rd parties x

InformationManagement

% of information management processesthat are automated x


Customer satisfaction with informationmanagement processes x


Past budget vs. current budget fordata/information stored in cloud x

InformationManagement Fully loaded cost per terabyte x

InformationManagement ROI analysis x

Operations % of assets—cost reduction x x x

Operations % of software license in use x x x

Operations Availability & performance SLAs of missioncritical applications, especially cloud providers(How close am I to breaching a contract?)

x

Operations Cost of monitoring per application x

Operations % of IT budget spending on opex x x

Operations Revenue loss per outage x x

Operations The revenue impact of a businessservice outage x

Personas



Table 3. Future orientation



Applications % of employee utilization rate x x

Applications % of FTE x x x x x

Applications % of project effort done by external resources x x x x x

Applications % of project effort done by external staf f x x x x x

Applications Employee retention/morale x x


Litigation readiness to support in-house,automated, repeatable e-discovery process x

Operations % of employees exceeding leadershipcompetency model x x x x x

Operations % of satisfied employees x x x x x

Operations Employee turnover x x x x x

Personas



Table 4. Operational excellence



Applications Avg delivery time for new products or services x x x

Applications Avg project initiation time x x x

Applications Effort variance x

Applications Nimbleness—Mean duration from change

request to productionx x

Applications On time delivery variance x x x

Applications Quality (defect leakage, normalized by % perfunction point or hours worked) x

Applications Resource utilization x x

Applications Time-to-market of new products or services x x x

InformationManagement # of data leakages/losses x

InformationManagement # of past platforms vs. current # of platforms x


# of sanctions or penalties for non-compliancewith discovery or regulatory request x

InformationManagement % improvement of staff productivity andprocess improvements x


% of information assets governed by properaccess controls x


% of legacy data classified as expiredand disposed of x

InformationManagement % of records classified with retention policy x x

InformationManagement % of storage mapped to tiered storage profiles x x


% of tests/simulations passed measuringimpact on business services through policynon-compliance

x

InformationManagement % of user information/data backed up underretention compliance policies x

InformationManagement % virtual server/storage implementation x

InformationManagement Active user processes x

InformationManagement % of business records with retention policy x

InformationManagement Compliance audit x

InformationManagement Compliance/regulatory mandates supported x

Information

Management

Control storage capacity with deduplication,

compression and storage tieringx


Dispositions scheduled, completed by numberand storage size x


Hours worked/time spent searching forinformation x


Implement policy based information archivingstrategy x

InformationManagement Ingestion volumes x

Personas



Table 4. Operational excellence—continued



InformationManagement Number of Data sources x

InformationManagement Past storage cost vs current storage cost x

InformationManagement Pursue/implement cloud services strategy xInformationManagement

Current and tested backup and recoveryarchitecture x


Retention schedules, by number andstorage size x

InformationManagement Tier 1 storage growth x

InformationManagement Total records/documents/objects stored x


Upgrade disaster recovery and businesscontinuity capabilities x


Headcount ratio—storage capacity to oneadmin x x

InformationManagement Mean time to identify storage bottleneck x x

InformationManagement Storage capacity reclaimed x x

Operations % events processed without human intervention x

Operations % of assets in maintenance x x

Operations % of assets returned to supplier x x

Operations % of changes resulting in outage x x

Operations % of devices compliant with corp policies x

Operations % of devices compliant with regulatory policy x

Operations % of emergency changes x x

Operations % of escalated Incidents x x

Operations % of FCR x x

Operations % of incident aging x x

Operations % of interactions in backlog x x

Operations % of network service availability x

Operations % of outages due to changes x

Operations % of problems resolved within therequired time period x

Operations % of reopened Incidents x x

Operations % of SLA expirations x x

Operations % of SLAs coverage x x

Operations % of unauthorized implemented changes x x

Operations % of unplanned changes x x

Operations % of urgent changes x x

Operations % problems with RC x

Operations % servers under management x

Operations % storage utilization x x

Operations Application (custom app) provisioning (hrs.) x

Personas






Operations Availability of network devices x

Operations Average age of hardware assets x

Operations Average time to procure x x

Operations Time needed to achieve, percentage correct x

Operations Change, configuration management Accuracy x

Operations Closure duration x x

Operations Database provisioning (hrs.) x x

Operations Ensuring optimal connectivity x

Operations Headcount ratio—client devices to one admin x

Operations Headcount ratio—databases to one DBA x x

Operations Headcount ratio—network devices toone admin x

Operations Headcount ratio—servers to one admin x

Operations Infrastructure utilization (server, storage) x

Operations Mean time between failures x

Operations Mean time to resolution for network incidents x

Operations Mean time to restore services x

Operations Mean time to resolution for network incidents x

Operations Network efficiency/latency x

Operations Network provisioning (hrs.) x

Operations Number of decommissioned servers x

Operations % of IT budget spending on opex x x

Operations Physical server provisioning (hrs.) x

Operations Problem queue rate x

Operations Storage provisioning (hrs). x x

Operations Virtual server provisioning (hrs.) x

Operations VM to server ratio x

Operations Changes to backlog size x

Security Number of security and compliance(corporate, government, industry) violations x

Security Security and compliance (corporate,government, industry) x

Security Simulation/change & configuration impact onsecurity and compliance x

Security Status of organizational controls for eachcompliance area x x x x x

Operations Status of compliance to internal policies x x x x x

Operations Device coverage (%)—devices protected bysecurity controls x x

Operations Number of security incidents open/closed,response time x

Operations Patch latency (avg missing patches/machine,days old) x x

Operations Status of on-going “posture” improvementefforts as aligned to priorities and objectives x

OperationsPlatform compliance (0-10)—ports open,default passwords/policies, etc. mapped toCIS controls

x x

Personas






Operations Regulatory controls—top out-of-compliancecontrols x

Operations Level of risk by asset class (quarterly) x

Operations Accepted loss expectancy (annual, by assetclass,$ revenue)

x

Operations Safeguard implementation status x

Operations Total accepted cumulative risk (dynamicmeasure) x

Operations Data loss prevention controls x x x

Operations Status of security project activities andremediation/cases (time/budget) x

Personas



Share with colleaguesGet connectedwww.hp.com/go/getconnected

Get the insider view on tech trends, alerts, andHP solutions for better business outcomes

© Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warrantiesfor HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should beconstrued as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

http://www.hp.com/go/getconnected

http://www.hp.com/go/getconnected

building a performance system for it.pdf

Documents