local government and regeneration committee seminar: 10 th september 2012

Taking Benchmarking and Performance Measurement Forward

Martin Walker - Assistant Director Best Value & Scrutiny Improvement Group - Audit Scotland

Local Government and Regeneration Committee Seminar: 10th September 2012

2Taking Benchmarking & Performance Measurement Forward

Proposed areas for discussion

• Benchmarking and performance measurement – an auditors perspective

• Key messages in audit reports

• Making performance measures and benchmarking work – common issues and practical considerations

10 Sept 2012


Benchmarking and performance measurement – an auditors perspective

• Audit work is evidence based – performance measures and benchmarking provides a source

• Used in all areas of audit work• Annual audit• BV audits• National performance audits• SPIs• CPP audits• Shared risk assessment

• We recognise that performance is political

10 Sept 2012



• The types of performance measures we might look for

• The comparisons we look for:• Performance against targets• Performance over time• Performance compared to others

• Combinations of measures e.g. VFM

10 Sept 2012

Input Measures

Process Measures

Output measures

Outcome Measures



• Most importantly we are looking for what the measures tell us about :

• Response/ activity – has an organisation acted on what the benchmarking is telling them?

• Impact – what difference has the response had on service performance and outcomes?

10 Sept 2012


Common issues in audit reports

• Performance information/ performance management frameworks still developing and variable

• Between councils• Within councils• Lack of comparative data to inform scrutiny• Link between performance information, activity and

impact

10 Sept 2012


Also positive examples

10 Sept 2012


Our audit work tells us that it’s important ..…

• To select measures which reflect priorities

• To ensure there are a range of measures (inputs, process, outputs, outcomes) and use them in combination

• To remember that performance measures are for performance management i.e. taking action

• Scrutiny and accountability• Improvement

• Be wary of the quest for the ‘perfect measure’

10 Sept 2012


Observations

• Benchmarking is a means to an end not and end in itself

• Benchmarking performance measures identifies• Issues• Questions

• Benchmarking can also provide:• Solutions• Answers

• Other types of benchmarking can help with this (strategic, functional, process)

• Benchmarking tends to work best when there is an infrastructure in place

10 Sept 2012

10Taking Benchmarking Performance Measurement and Forward

A scenario

10 Sept 2012

people consider what data is available that could be compared agonise over who to benchmark with – in a quest for benchmarking partners

who are “like them” spend a great deal of time and effort collecting the data collate data in many spread-sheets with benchmarking partners identify discrepancies have the “apples with oranges” discussion and agree to go away to revise the

data in line with agreed standards return to find there are still differences report back to their own Council ( though not always ), noting the differences but

stressing that “they’re not the same as us, so it’s not comparing like with like” find the whole exercise time consuming and without major benefits and remain

cautious about future exercises


Some points of reference

10 Sept 2012

12

Questions & Discussion

Taking Benchmarking & Performance Measurement Forward 10 Sept 2012

Contact details:

Martin WalkerAssistant Director– Audit Scotland

Phone: 0131 625 1813E-mail: [email protected]

local government and regeneration committee seminar: 10 th september 2012

Documents

service performance

otherscombinations of

range of measures inputs

auditors perspectiveaudit

auditors perspectivemost

response activity

practical considerations10

regeneration committee