![Page 1: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/1.jpg)
‘Pathfinder’: Maximising Results for Communities
Dr Roger Waite (Pathfinder Project / NZ Treasury)
io.ssc.govt.nz/pathfinder
![Page 2: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/2.jpg)
Meta-messages
• All NZ Depts: enhanced outcome focus by 2003• Outcomes: ultimate goal (Holy Grail?) • Pathfinder (collective action, individual solutions)
• Outcomes driving strategy and output mix• Evidence-based approach to enhancing outcomes /
achieving Millenium Development Goals (MDGs)• Maximising outcomes from development proposals
(prioritise, assess & assist roles for development banks?)
An Incremental Approach to Capability Building
![Page 3: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/3.jpg)
Focus on Outcomes
PATHFINDER
Strategic Plan;Statementof Intent
Value-for-Money
TOOLBOX; DECISIONMAKING SYSTEMS
STRATEGY,CAPABILITY,
PERFORMANCE
TRADE-OFFS,TRANSFERS(Within Vote)
(Next FY)
Goal: Build NZ Capability at Agency, Sector and ‘Joined Up’ Levels
![Page 4: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/4.jpg)
Why Pathfinder?
After 10+ yrs, output funding has delivered
Quantum gains requires modified approach
Keep what works (output mgmt), but …
Complete public sector management modelLink inputs (through outputs) to outcomes
Meet requirements of legislation, parliament, etc
Ensure we deliver to stakeholders, taxpayers
Start: Limited capability & management tools
![Page 5: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/5.jpg)
8 Agencies, Supported by CentreBuilding capability to enhance outcomesGoal: Better results for NZ communitiesAssumes performance feedback (outcomes) is a key factor in further improving performanceDefine outcomes; Prioritise; Measure; Manage– Performance of whole agencies / business units– Maximise outcomes from the output mix / $– Accentuate the positive; eliminate the negative
Outcomes / MDGs core to strategy, prioritising funding (macro) and proposals, assessing impact
Is the approach helpful in implementing Milennium Development Goals?
![Page 6: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/6.jpg)
PATHFINDERBetter Results, Stronger Communities
![Page 7: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/7.jpg)
Three Basic Approachesto Improving Results
• Top Down (Pathfinder emphasis / SOI / MDG focus?)
• Hierarchical (evolutionary direction for Pathfinder)
• Bottom Up, practitioner driven (opportunity)
SOI: Statement of Intent – a forward looking strategic plan deriving outputs from outcomes, and identifying capability
needs
![Page 8: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/8.jpg)
Pathfinder ‘Building Blocks’1. Define & measure ‘mission critical’ outcomes
(using ‘state’ or ‘situation’ outcome indicators)
2. Map causal logic linking outcomes to outputs3. Assess impact of interventions4. Assess cost-effectiveness 5. Management to maximise outcomes6. Benchmarking with outcomes (business units / nations)
7. Focus strategic / annual plans on improving o/c8. Redesign planning & operations to maximise o/c
(incl. feedback & continuous improvement)
9. Improving outcomes across agencies / sectors
Building Blocks being documented on io.ssc.govt.nz/pathfinder
![Page 9: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/9.jpg)
First Steps towards Real Results Mission, Measures, Management (M3)
• From aspirational outcomes (‘safer communities’)
• To the ‘Vital Few’ – focus on business actions• Tight Outcome Definitions (‘% re-offending in 24 months’)
• Outcome Indicators (‘28% re-imprisonment rate for Maori’)
• Impact Measures (‘12% reduction in reimprisonment rate’)
• Outcomes as Inputs into Management Systems
Delivery-based measures can substitute for Impact Measures where local factors can be managed (e.g. drugs work, but delivery factors critical)
![Page 10: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/10.jpg)
Levers for Improving Outcomes
Status quo
outcome (client /service groups)
Risk, Need &
Likely Change (individuals / cases)
Improvement
in outcome (‘treated’ groups)
Situation Report
Benchmarking
Crude targeting
Priority setting
Demand analysis
Risk managemnt
Continue?
Modify? Grow?
Kill?
Measure Applications
ImpactAssessment
Assessment /AllocationDecisions
State /Environment
Indicators
Outcome Type
![Page 11: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/11.jpg)
Screening for Risk: ValidationSocial Costs (Justice + Community)
Risk of Imprisonment (decile)
Societal
Cost
Top (Squares): Community-based Offenders
Bottom (Diamonds): Prison Inmates
$0
$100,000
$200,000
$300,000
$400,000
$500,000
$600,000
$700,000
0 0.2 0.4 0.6 0.8 1
![Page 12: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/12.jpg)
Results Chains (After OAG Canada)Simple tools for reviewing funding proposals?
Inputs
Activities
Outputs
Immediate Results
Intermediate Outcomes
End/Final Outcomes
Outputs
Activities
Inputs
End/Final Outcomes
Intermediate Outcomes
Immediate Results
O2
A2
I2
O3
A3
I3
O4
A4
I4
IO
IR
Etc
Etc
Etc
Technical Efficiency Allocative Efficiency
![Page 13: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/13.jpg)
Core Outcomesof AgencyDefined
Agency OutcomeIndicators
ImpactMeasures
Risk-basedTargeting
Tools
StrategicPriorities &
Planning
Benchmarking& Best Practice
MaximisingBenefits from
Intervention Mix
BusinessProcess Design
ManagementApplications
Linked OutcomeMeasures
After a business model used by the Department of Corrections
MANAGEMENT SYSTEMS & CAPABILITY
![Page 14: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/14.jpg)
Identify CoreOutcomes
Define in Measurable
Terms
Measure as Outcome Indicators
Identify AreasFor Change
Define / RefineIntervention
Logic
IdentifyIntervention
Options
PrioritiseIntervention
Options
Design & Deliver
Interventions
ImpactMeasurementFramework
Cost Effectiveness
AnalysisOptimising Performance
Strategy Formulation
Ex Ante
![Page 15: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/15.jpg)
Focus on Systems, Not Measures(lessons from the NZ Experience)
• Build management, not o/c measurement, systems• End game is outcomes – but output results important• Conceptual approaches give ‘one-off’ gains …
• Significant technical challenge to measure impact (management stuff is easy (?) with good measures)
• Ongoing gains in outcomes needs impact feedback?• Risk-based targeting an important lever for gains
• Stretching the capability of first world nation(s) • Capability development for 3rd world required?• Outcome-based allocative tools for the funder
![Page 16: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/16.jpg)
‘Pathfinder’: Maximising Results for Communities
Dr Roger Waite (Pathfinder Project / NZ Treasury)
io.ssc.govt.nz/pathfinder
![Page 17: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/17.jpg)
Success is more a function of consistent Success is more a function of consistent common sense than it is of genius -common sense than it is of genius -
Success is not a place at which one Success is not a place at which one arrivesarrives,, but the spirit with which one but the spirit with which one undertakes and continues the journeyundertakes and continues the journey
Most advanced NZ agencies have several things in common: Most advanced NZ agencies have several things in common: Committed senior management; Long-term focus; Clear Committed senior management; Long-term focus; Clear
outcomes; Strong analytical capability & MIS in support; outcomes; Strong analytical capability & MIS in support; Incremental development to vision rather than ‘big bang’Incremental development to vision rather than ‘big bang’
Will require cultural & capability changes of all managers; Will require cultural & capability changes of all managers; Outcomes subversive – managers will not like all answers; Outcomes subversive – managers will not like all answers;
No quick fixes – requires effort over multiple yearsNo quick fixes – requires effort over multiple years
![Page 18: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/18.jpg)
Office of the
Controller and Auditor-GeneralTe Mana Arotake
![Page 19: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/19.jpg)
Robust Impact Measures: Challenging
Structured Trial
Clear, EnforcedInterventionCriteria andThresholds
RetrospectiveDesign of
Structured Trial
RandomisedDesign
Non-randomisedDesign
RegressionDiscontinuity
Impact typicallymeasured in absolute
terms. Substantialcorrection for external
factors
Output / quality measures good surrogates where outcomes not strongly influenced by local factors (e.g. Drugs; Engineering)
![Page 20: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/20.jpg)
Accountability: High Demands
Management (SOI)• Logical story of how
outputs enhance outcomes
• Selective use of indicator and impact measures to ‘tell story’ & set priorities
• Some discretion on external reporting
Accountability• Robust model linking
outputs to outcomes
• Precise, timely and comprehensive impact measures for each output
• Mandated external reporting of outcomes
In a ‘perfect world’ managers would have all the information required by an outcome-based accountability regime
![Page 21: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/21.jpg)
Outputs and Outcomes
After: HM Treasury (UK)
Value for Money
Better healthMore treatmentsMore nursesMore money
External influences
Budget Inputs Output Outcome
Economy Efficiency Effectiveness
![Page 22: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/22.jpg)
Which Outcome?
F ew er V ic tim s(Ju s tice )
P erce ived S a fe ty(Ju s tice )
R eso lved
Tim es from C h arg esF iled to P rosecu tion
(C ou rts )
C ou rt H ou rsb y C rim e Typ e
(C ou rts )
P rop ortion C on vic t ion sO vertu rn ed on A p p ea l
(C ou rts )
P rosecu ted C on vic ted
C rim es S o lved(P o lice )
[P ro po rtio n; T im eliness]
R ed u cedR e-o ffen d in g
(C orrec tion s , C Y F ,P o lice D ivers ion s )
L ess R ep orted C rim e(Ju s tice / P o lice )
CO M M UNITY S AFE TY(F rom C rim e)
Example
![Page 23: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/23.jpg)
Rehabilitation QuotientsDetermined on NZ Offender
Groups / Population
BEST USE OF REHABILITATIVE FUNDING
Full Cost (Per Head) ofIntervention as Delivered in
New Zealand
Mean Future Cost perNZ Offender within thePrimary Target Group
Risk(s) of Reconviction
Cost, if Reconvicted
Cost-Benefit Ratio(Individual Interventions)
Rank Individual Interventionson the Basis of Cost-Benefit
Determine Budget for 'Core'Rehabilitative Interventions
Determine Mix of CoreProgrammes to be Delivered
in the Next Year
Determine Mix of CoreProgrammes to be Delivered
in the Next Year
Input Government andDepartmental Priorities
RI / RQ Framework
[A] CORE PROGRAMMESAND INTERVENTIONS
[B] PILOT / EVALUATIONOF NEW INTERVENTIONS
Cost and Benefit Datafrom Successful Pilots
Cost and Benefit Datafrom Successful Pilots
![Page 24: Meta-messages All NZ Depts: enhanced outcome focus by 2003 Outcomes: ultimate goal (Holy Grail?) Pathfinder (collective action, individual solutions)](https://reader036.vdocuments.site/reader036/viewer/2022062518/56649e7c5503460f94b7da1f/html5/thumbnails/24.jpg)
Managing for Better Outcomes• Broad focus (strategy, priorities, output mix, business systems)
• Decision making, better outcomes (evaluation just a tool)
• Measure outcomes (different outcome measures & groups)
• Work with (& test) clear intervention logic• Integrate with resource allocation systems• Focus business strategy on improving outcomes• Focus business processes on improving outcomes• Improve outcomes over multiple business cycles, and
across business units / agencies / sectors
Goal: maximise social benefit from finite resources