developing performance measures
DESCRIPTION
Developing Performance Measures. Purpose of Workshop. To assist you in developing Performance measures Measurement methods To discuss Institutional Effectiveness and changes in reporting requirements. First Validate Your Mission, Goals, and Objectives. Why?. Necessary and Sufficient. - PowerPoint PPT PresentationTRANSCRIPT
Developing Performance Measures
December 13, 2000 OEAS--IE Training Workshop 2
Purpose of Workshop
To assist you in developing – Performance measures
– Measurement methods
To discuss Institutional Effectiveness and changes in reporting requirements
December 13, 2000 OEAS--IE Training Workshop 3
First Validate Your Mission, Goals, and Objectives
O b je ctive 1 O b je ctive 2
G o a l 1
O b je ctive 1 O b je ctive 2
G o a l 2
O b je ctive 1 O b je ctive 2
G o a l 3
U n it M iis io n
Why?
How? Necessary and Sufficient
December 13, 2000 OEAS--IE Training Workshop 4
Performance Measure
A means of objectively assessing programs, products, activities, or services
Should be related to your mission and goals Should indicate how you will measure your
objective Should indicate when your objective will be
measured Should indicate who will do the
measurement
December 13, 2000 OEAS--IE Training Workshop 5
Structure of a Performance Measure Statement
The (description of the measure) based on (approach used to develop the measure)– The (number of students attending workshops)
based on (annual attendance records)– The (satisfaction level of students with the given
service) based on (an annual survey of all graduating seniors conducted by OEAS)
– The (application processing time for applications) based on (the time from receipt of an application in the Admissions Mail room to the key-in of the admission decision as recorded in PeopleSoft)
December 13, 2000 OEAS--IE Training Workshop 6
Two Approaches to Generating Objectives and Performance Measures
First Approach Describe mission Describe vision List goals List objectives Determine at least one
performance measure for each objective
Second Approach Describe mission Describe vision List goals Brainstorm
performance measures Select “preferred”
performance measures Set targets which
become the objectives
December 13, 2000 OEAS--IE Training Workshop 7
Generating Objectives then Performance Measures Example: (Admissions)
– Goal: To provide timely response to undergraduate applicants
– Objective: To reduce the application processing time to less than 14 days for 95% of applications
– Performance measure: The application processing time for applications based on the time from receipt of an application in the Admissions Mail room to the key-in of the admission decision as recorded in PeopleSoft
December 13, 2000 OEAS--IE Training Workshop 8
Generating Performance Measures then Objectives Example: (Admissions)
– Goal: To provide timely response to undergraduate applicants
– Related performance measures• The number of days to process application based
on…
• The average number of applications processed per file manager per day based on…
• The student satisfaction with the timeliness of the admission decision based on…
December 13, 2000 OEAS--IE Training Workshop 9
Evaluating the Quality of Your Performance Measures Does each measure relate to the associated
mission, goal, and objective? Is each measure important to management? Is it possible to collect accurate and reliable data
for each performance measure? Taken together, do the measures accurately reflect
the key results of the program, activity or service? Is there more than one measure for each goal or
objective?
December 13, 2000 OEAS--IE Training Workshop 10
Generating Performance Measures then Objectives Goal: To provide timely response to
undergraduate applicants– Objective 1: to increase the productivity of the
file managers by 10%• PM: Average number of applications processed per
file manager per day
– Objective 2: To reduce the application processing time to less than 14 days for 95% of applications
• PM: Number of days to process each application
December 13, 2000 OEAS--IE Training Workshop 11
Two Ways of Measuring Performance Direct measures of performance
– Time– Error rates– Compliance– Cost– Number of outputs per input– Standardized tests
Indirect measures of performance– Perceived time– Perceived efficiency– Perceived quality
December 13, 2000 OEAS--IE Training Workshop 12
Example: To reduce the application processing time to less than 14 days for 95% of applications
Direct measure: The application processing time for applications based on the time from receipt of an application in the Admissions Mail room to the key-in of the admission decision as recorded in PeopleSoft
Indirect measure: The perceived time based on survey of a random sample of 500 applicants
December 13, 2000 OEAS--IE Training Workshop 13
Categories of Performance Measures Input measures (e.g., staff time, materials,
equipment, resources) are useful in showing resources or effort used to provide services; however does not show effectiveness– You may be a spending a lot of effort doing the wrong
things Output measures (e.g., number of products
produced or services provided) are useful in defining program or service; however, does not reveal quality or efficiency– You may be producing or providing a lot of the wrong
things inefficiently or with poor quality
December 13, 2000 OEAS--IE Training Workshop 14
Categories of Performance Measures Outcome measures (e.g., score on standardized test,
distance from proposed targets) are useful in showing the impact or benefit of the program or service
Efficiency measures (e.g., cost per unit of output, outputs per unit of input, outputs per unit time) are useful in showing productivity and cost effectiveness
Quality measures (e.g., reliability, accuracy, courtesy, competence, responsiveness) are useful in measuring the effectiveness in meeting customer expectations
– Lack of quality can be measured (e.g., error rates)
December 13, 2000 OEAS--IE Training Workshop 15
Example Goal: To provide timely response to undergraduate applicants
Input measure: The number of file managers Output measure: The average number of applications
processed per day Outcome measure: The application processing time for
applications based on the time from receipt of an application in the Admissions Mail room to the key-in of the admission decision as recorded in PeopleSoft
Efficiency measure: The average number of applications processed per file manager per day
Quality measure: The satisfaction of applicants with the application processing time as measured by a survey
December 13, 2000 OEAS--IE Training Workshop 16
Another Example: Developing Performance Measures The mission of the XYZ Foundation is to
promote the social welfare of persons resident or located in the Orlando Metropolitan Region by all appropriate means, including relief of the poor, care of the sick and aged, care and nurture of children, elimination of vermin…
December 13, 2000 OEAS--IE Training Workshop 17
Example Goal, Objective, and Performance Measure Goal: To minimize the number of rats in
the City of Orlando Objective: To reduce the number of rats to
ten per city block in the next year Performance measure: The number of rats
per city block as measured by an extensive search of a random sample of 10 city blocks
December 13, 2000 OEAS--IE Training Workshop 18
PMs Relate to Goal: To minimize the number of rats in the City of Orlando The dollars spent per rat killed based on Budget Office and
exterminator records Percent increase in the number of traps set this year based on
city records The satisfaction of the residents with the extermination
program based on survey of a random sample of residents The dollars spent on extermination based on the Budget
Office records The number of rats remaining per city block obtained by
conducting an extensive search of a random sample of the city blocks in Orlando.
The approximate number of rats killed during the year per city block based on the exterminator reports
December 13, 2000 OEAS--IE Training Workshop 19
Evaluating the Quality of Your Performance Measures Does each measure clearly relate to the associated
mission, goal, and/or objective? Is each measure important to management? Is it possible to collect accurate and reliable data for
each performance measure? Taken together, do the measures accurately reflect
the key results of the program, activity or service? Is there more than one measure for each goal and/or
objective? Are your measures primarily outcome, efficiency or
quality measures?
December 13, 2000 OEAS--IE Training Workshop 20
Evaluate: Related, Important, Measurable, Comprehensive, and Preferred Type
MINIMIZE THE NUMBER OF RATS IN ORLANDO The dollars spent per rat killed based on Budget Office and
exterminator records Percent increase in the number of traps set this year based
on city records The satisfaction of the residents with the extermination
program based on survey of a random sample of residents The dollars spent on extermination based on the Budget
Office records The number of rats remaining per city block obtained by
conducting an extensive search of a random sample of the city blocks in Orlando.
The approximate number of rats killed during the year per city block based on the exterminator reports
December 13, 2000 OEAS--IE Training Workshop 21
Some Resulting Objectives Supporting the GoalMINIMIZE THE NUMBER OF RATS IN
ORLANDO
To improve the cost effectiveness by 10% by Fall 2001
To obtain high to very high satisfaction rating of 90% of city residents
To reduce the number of rats to ten per city block in the next year
December 13, 2000 OEAS--IE Training Workshop 22
Activity: Develop Performance Measures for each Goal Try to develop an input, output, outcome,
efficiency, and quality performance measure for each goal
Identify whether each is a direct or indirect measure– Can you identify a direct measure for the ones
that are currently indirect?– Can you identify an indirect measure for the
ones that are currently direct?
December 13, 2000 OEAS--IE Training Workshop 23
Develop Objectives
Evaluate quality of performance measure– Related– Important– Measurable– Comprehensive– Preferred Type (outcome, efficiency, quality)
Develop objectives