Results Based Accountability BasicsA Half Day PresentationStandard Training Slides Sponsored by the Ministry of Social Development

The Fiscal Policy Studies InstituteSanta Fe, New Mexico

Websites

raguide.orgresultsaccountability.com

Book - DVD Orderssheapita.co.nzamazon.comresultsleadership.org

Results Based Accountability

How could RBA add value to you?

3

SIMPLE STEPS

COMMON SENSE

PLAIN LANGUAGE

MINIMUM PAPER

TALK TO ACTION!

Results Based Accountabilityis made up of two parts:

Performance Accountabilityabout the wellbeing of

CLIENT GROUPS/CUSTOMERS

For Teams - Providers – Programmes - Agencies – Service Systems

E.g. Clients of Services, Collectives, Ministries or the Health System

Population Accountabilityabout the wellbeing of

WHOLE POPULATIONS

For Communities – Cities – Districts – Countries

E.g. All Rangatahi/Youth in Te Tai Tokerau, All Migrants in Nelson

5

The Language TrapToo many terms. Too few definitions. Too little discipline

Benchmark

Target

Indicator Goal

Result

Objective

Outcome

Measure

Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic

Lewis Carroll Center for Language DisordersMeasurable urgent systemic indicatorsCore qualitative strategic objectivesYour made up jargon here

Results Based Accountability

COMMON LANGUAGE

COMMON SENSE

COMMON GROUND

Definitions• RESULT / OUTCOME– A condition of wellbeing for children, adults, families or communities

All Tamariki in Hamilton are Born Healthy, Safe Roads, Nurturing Whānau/Families, A Prosperous Economy

• INDICATOR / BENCHMARK– A measure which helps quantify the achievement of a result.

Rate of low-birth weight babies, Rate of road crashes, Rate of child abuse and neglect, Unemployment rate

• PERFORMANCE MEASURE– A measure of whether a programme, agency or service system is

working. Three types1. How much did we do?2. How well did we do it?3. Is anyone better off? = Client Results / Outcomes

Popu

latio

nPe

rfor

man

ce

Translation Guide/Rosetta StoneNot the Language Police

Ideas

1. A condition of well-being for children, adults, families & communities

2.

3.

etc.

Group 1 Group 2 Group 3 etc.

RESULT OUTCOME GOAL

TRANSLATION

Back to the Idea

From Ends to Means

ENDS

MEANS

RESULT / OUTCOME

INDICATOR / BENCHMARK

PERFORMANCEMEASURE

Client /Customer result = EndsService delivery = Means

From Talk to Action

10

Popu

latio

nPe

rfor

man

ce

Results – Indicators – Performance Measures inMaori, Fijian, Tuvaluan

1. A Safe Community

2. Percentage of Total Recorded Offences

3. Average Police response time

4. An Educated Workforce

5. Installing street lights to make people feel safe

6. People have living wage jobs and income

7. % of people with living wage jobs and income

8. % of participants in job training programme who get living wage jobs

Result, Indicator, Strategy or Performance Measure?

Result

Indicator

Perf Measure

Result

Strategy

Result

Indicator

Perf Measure

Key RBA concepts• 2 key types of accountability and language discipline:

– Population accountability - results / outcomes and indicators

– Performance accountability - performance measures

• 3 types of performance measures:– How much did we do? – How well did we do it? – Is anyone better off?

• 7 questions from ends to means:– baselines and turning the curve – to make life better for our

families / whānau, children / tamariki, and communities.

13

Population AccountabilityFor whole populations in a geographic area

Mark Friedman (author)www.resultsaccountability.comwww.raguide.org

14

The 7 Population Accountability Questions1. What are the quality of life conditions we want for the children, adults and

families who live in our community? (Population & Results)2. What would these conditions look like if we could see them? (Experience)3. How can we measure these conditions? (Population Indicators)4. How are we doing on the most important of these measures? (Baseline Data

and Story)5. Who are the partners that have a role to play in doing better? (Partners)6. What works to do better including no- cost and low-cost ideas? (What

works)7. What do we propose to do? (Action Plan)

15

16

Christchurch City Community Outcomes

• A safe city • A city of inclusive and diverse communities• A city of people who value and protect the natural

environment• A well governed city • A prosperous city • A healthy city • A city of recreation, fun and creativity • A city of lifelong learning • A city that is attractive and well designed

17

Kotahitanga Whānau Ora Collective

• All whaanau in Te Puuaha ki Manuka(Greater South Auckland) are ...– Mana Ora: Healthy and safe;– Mana Motuhake: Economically secure;– Mana Tangata: Culturally confident;– Mana Rangatiratanga: Knowledgeable and skilled;– Mana Whānau: Connected, engaged and

entrepreneurs

Acknowledgement: Kotahitanga Collective Members: Turuki Healthcare Trust, Papakura Marae, Huakina Development Trust and Te Kaha O Te Rangatahi Trust, South Auckland, New Zealand.

positive statements - positive focus

Aranui Community Trust

Acknowledgement: Aranui Community Trust Inc Society (www.actis.org.nz)

Implementing RBA Aranui Result areas and indicators

A community full of knowledge and learning Indicator:% of students with NCEA level 1 @year 11% primary school students performing at national

average for literacy and numeracy

A community that is spiritually and socially strongIndicator:% of police callouts for family violence

Result Area 1:

Result Area 2:

A great physical environmentIndicator:

Expenditure on repairs and maintenance to city property in the Aranui Burwood Pegasus area

% of $ R & M that is due to damage

Result Area 3:

Result Area 4:

People who know and fit in AranuiIndicator:

% of people who offer to participate in local events

A community that is healthyIndicator:

% of Aranui residents presenting at Accident and Emergency with no trauma needs/ concerns

# of total acute inpatient admissions

Result Area 5:

Results for Children, Families and CommunitiesA Working List of Population Results

Healthy Births Healthy Children and Adults Children Ready for School Children Succeeding in School Young People Staying Out of Trouble Stable Families Families with Adequate Income Safe and Supportive Communities

24

Georgia Policy Council for Children and Families

RESULTS Healthy children

Children ready for school

Children succeeding in school

Strong families

Self-sufficient families

25

Tip for Drafting Population Accountability Results

All ______ in ______ are __________

Insert your Population

Insert your Geographic area

Insert your Condition of Wellbeing

Families Tauranga Economically Secure

26

Examples of Means not Ends

1. COLLABORATION

2. SYSTEMS REFORM

3. SERVICE INTEGRATION

4. DEVOLUTION

5. FUNDING POOLS

27

Leaking Roof(Results thinking in everyday life)

Experience:

Measure:

Story behind the baseline (causes):

Partners:

What Works:

Action Plan:

Inches of Water

? Fixed

Not OK

Turning the Curve

28

Action Plan: #2

3 criteria for choosing Indicators

Communication Power

Proxy Power

Data Power

Does the indicator communicate to a broad range of audiences?

Does the indicator say something of central importance about the result?

Does the indicator bring along the data HERD?

Quality data available on a timely basis.

29

Choosing IndicatorsWorksheetOutcome or Result_______________________

Candidate IndicatorsCommunication

PowerProxyPower

DataPower

H M L

H

Measure 1

Measure 2

Measure 3

Measure 4

Measure 5

Measure 6

Measure 7

Measure 8

HData

Development

Agenda

Safe Community

H M L H M L

H H

H L

Three Part Indicator List for each Result

Part 1: Primary Indicators

Part 2: Secondary Indicators

Part 3: Data Development Agenda

• 3 to 5 “Headline” Indicators• What this result “means” to the community• Meets the Public Square Test

• Everything else that’s any good (Nothing is wasted.)

• Used later in the Story behind the Curve

• New data• Data in need of repair (quality, timeliness etc.)

31

What do we mean by a baseline?

Baselines have two parts: history and forecast

H

M

L

History Forecast

Turning the CurvePoint to Point

OK?

32

New Zealand

KruidenbuurtTilburg, Netherlands

Portsmouth, UK

Country

Neighbourhood

City

Indicator Reports

MADD

34

Key RBA concepts• 2 key types of accountability:

– Population – results / outcomes and indictors– Performance – performance measures

• 3 types of performance measures:– How Much Did We Do?– How Well Did We Do It?– Is Anyone Better Off?

• 7 questions from ends to means – baselines and turning the curve – to make life better for our

families / whānau, children / tamariki, and communities.

35

QUICK EXERCISE

Population Accountability

All ______ in ______ are __________

Insert your Population

Insert your Geographic area

Insert your Condition of Wellbeing

How would you experience this outcome? What would be different?

How would you measure success? What Indicator would you use?

Tip for Drafting Population Accountability Results

Performance AccountabilityFor clients of programmes, agencies, teams and service systems

Mark Friedman (author)www.resultsaccountability.comwww.raguide.org

38

Results Based Accountabilityis made up of two parts:

Performance Accountabilityabout the wellbeing of

CLIENT GROUPS/CUSTOMERS

For Teams - Providers – Programmes - Agencies – Service Systems

E.g. Clients of Services, Collectives, Ministries or the Health System

Population Accountabilityabout the wellbeing of

WHOLE POPULATIONS

For Communities – Cities – Districts – Countries

E.g. All Rangatahi/Youth in Te Tai Tokerau, All Migrants in Nelson

39

The 7 Performance Accountability Questions1. Who are our clients? (Client Group/Customers)2. How can we measure if our clients are better off? (Client/Customer

Result / Outcome)3. How can we measure if we are delivering services well? (Quality

Measures)4. How are we doing on the most important of these measures? (Baseline

Data and Story)5. Who are the partners that have a role to play in doing better? (Partners)6. What works to do better including no-cost and low cost ideas? (Common

sense ideas & research where available)1. What do we propose to do? (Action Plan)

40

Performance AccountabilityGetting from talk to action

Client Group/Customers

“All performance measures

that have ever existed

for any programme

in the history of the universe

involve answering two sets of

interlocking questions.”

HowMuchdid we do?

( # )

Quantity

Performance MeasuresQuality

HowWell

did we do it?

(%)

Performance Measures

EffortHow hard did we try?

EffectAnyone better off?

Performance Measures

HowMuch

HowWell

Effort

Effect

How much service did we deliver?

How welldid we

deliver it?

How much change / effect

did we produce?

What quality of change / effect

did we produce?

Quantity Quality

Effe

ct

Effo

rt

O

utpu

t

In

put

Performance Measures

How much did we do? How well did we do it?

Is anyone better off?

# of young people (clients) receiving job training / mentoring

services

% clients who complete the job training /

mentoring programme

# of clients who move off a working age benefit and into

employment (at 6 months and at 12

months)

% of clients who move off a working age benefit and into

employment (at 6 months and at 12

months)

Social Services Example

E

ffect

Ef

fort

Quantity Quality

How much did we do?

Education example

How well did we do it?

Is anyone better off?

Number ofstudents

Student-teacherratio

Number ofgraduates

Percent ofgraduates

Quantity Quality

E

ffect

Ef

fort

How much did we do?

Drug/Alcohol Treatment ProgrammeHow well did we do it?

Is anyone better off?

Number ofpersonstreated

Percent ofstaff withtraining/

certification

Number of clientsoff of alcohol & drugs - at exit - 12 months after exit

Percent of clientsoff of alcohol & drugs - at exit - 12 months after exit

Quantity Quality

E

ffect

Ef

fort

How much did we do? How well did we do it?

Is anyone better off?

Quantity Quality

Number ofstudents

Student-teacherratio

Number of secondary school

students whograduate on timeand enter Uni or

employment after graduation

Percent of secondary school

students whograduate on timeand enter Uni or

employment after graduation

Education example # 2

E

ffect

Ef

fort

How much did we do?

Primary Care Practice

How well did we do it?

Number ofchildren aged 0-2 enrolled

Percent ofchildren who did

not attend

Quantity

#children

aged 8 monthsimmunised

(in the practice)

%children

aged 8 monthsimmunised

(in the practice)

Effe

ct

Effo

rtQuality

Is anyone better off?

How much did we do?

Not all performance measures are created equal

How well did we do it?

Is anyone better off?

LeastImportant

Quantity Quality

MostImportant

Least

Most

AlsoVery Important

Effe

ct

Effo

rt

How much did we do?

The matter of control

How well did we do it?

Is anyone better off?

Quantity Quality

LeastControl

PARTNERSHIPS

MostControl

Effe

ct

Effo

rt

Separating the Wheat from the ChaffTypes of performance measures found in each quadrant

How much did we do? How well did we do it?

Is anyone better off?

# Clients/customers served

# Activities (by type of activity)

% Common measurese.g. client staff ratio, workload ratio, staffturnover rate, staff morale, % staff fully trained, % clients seen in their own language,worker safety, unit cost

% Skills / Knowledge (e.g. parenting skills)

#

% Attitude / Opinion (e.g. toward drugs)

#

% Behavior (e.g. School attendance)

#

% Circumstance (e.g. working, in stable housing)

#

% Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Point in Time vs. Point to Point

Improvement

The matter of use

1. The first purpose of performance measurement is to improve performance.

2. Avoid the performance measurement equals punishment trap.

● Create a healthy organisational environment

● Start small

● Build bottom-up and top-down simultaneously

Key RBA concepts• 2 key types of accountability and language discipline:

– Population accountability - results / outcomes and indicators

– Performance accountability - performance measures

• 3 types of performance measures:– How much did we do? – How well did we do it? – Is anyone better off?

• 7 questions from ends to means:– baselines and turning the curve – to make life better for our

families / whānau, children / tamariki, and communities.

57

QUICK EXERCISEPerformance Accountability

How much did we do?

Performance Measures for my …(insert the name of your Programme or Service here)

How well did we do it?

Is anyone better off?

# Clients/customers served

% client satisfaction with xxx

% Skills / Knowledge (e.g. parenting skills)

# /

% Attitude / Opinion (e.g. toward drugs)

# /

% Behavior (e.g. School attendance)

# /

% Circumstance (e.g. working, in stable housing)

# /

Choose one

What would you put in here?

Who are your clients?

How Population & Performance Accountabilities Fit Together

60

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE

POPULATION ACCOUNTABILITY

Result: Healthy Safe Young People Youth crime rates

CLIENT

RESULTS/OUTCOMES

# young people on programme

% meeting weekly with

mentor

# reoffending % reoffending

PERFORMANCE ACCOUNTABILITY

POPULATION

RESULT

Mentoring Programme for Young Offenders

Every timeyou make a

presentation,

use atwo-partapproach

Result: to which you contribute to most directly.

Indicators:

Story:

Partners:

What would it take?:

Your Role: as part of a larger strategy.

Population Accountability

Programme:Performance measures:

Story:

Partners:

Action plan to get better:

Performance Accountability

Your Role

Result: to which you contribute to most directly.

Indicators:

Story:

Partners:

What would it take?:

Your Role: within the larger strategy.

Population Accountability

Programme:Performance measures:

Story:

Partners:

Action plan to get better:

Performance Accountability

Your Role

Every timeyou make a

presentation,

use atwo-partapproach

Different kinds of progress1. Data

a. Population indicators:

Reporting on curves turned: % increase or decrease of the graphed data (e.g. the baseline).

b. Performance measures: Client group progress and improved service delivery:

How much did we do?How well did we do it?Is anyone better off? E.g. Skills/Knowledge, Attitude/Opinion, Behaviour Change,

Circumstance Change

2. Accomplishments Other positive activities accomplished, not included above.

3. Stories Real stories that sit behind the statistics that show how individuals are better off e.g. case studies, vignettes, social media clips.

64

What’s next?A Basic Action Plan for Results Based Accountability

TRACK 1: POPULATION ACCOUNTABILITY

• Establish results• Establish indicators, baselines and charts on the wall• Create a result card• Set tables (action groups) to turn curves

TRACK 2: PERFORMANCE ACCOUNTABILITY• Performance measures, and charts on the wall for

programmes, agencies and service systems

• Use 7 Questions manager by manager, and programme by programme, in management, budgeting and strategic planning

IN CLOSING

67

Kia ora / thank you!

WEBSITES:

www.raguide.orgwww.resultsaccountability.com

BOOK /DVD ORDERS:

www.sheapita.co.nzwww.trafford.comwww.amazon.com

68