introduction to results-based accountability™ · achieving “collective impact” with...

FPSI/RLG 1

Introduction to

Results-Based Accountability™

Early Childhood Colorado Partnership

Deitre Epps, Partner/ Senior Consultant

Results Leadership Group

www.resultsleadership.org

MEASURABLE RESULTS

FOR CLIENTS AND COMMUNITIES

FPSI/RLG 2

Percent of Children in Maryland Entering School Fully Ready

2002 2012

49%

82%

March 2013

Children Entering School Ready to LearnThe 2012-2013 Maryland School Readiness Report

Maryland

Maryland DemographicsFacts About Young Children

Population DataMD Department of Planning, 2010

Estimated Children Age 4 72,898

Children <5 (age 0-4) 364,488

School EnrollmentMSDE, School Year 2011-2012

PreK Students 28,850 Kindergarten Gender

Full-Day Program 29.3% Male 51.7%

Half-Day Program 70.7% Female 48.3%

Kindergarten Students 64,726 Kindergarten Ethnicity

American Indian 0.4%

Kindergarteners by Subgroup Asian 5.9%

Children with Disabilities 8.2% African American 32.7%

English Language Learners 14.9% Native Hawaiian/Pacific Islander 0.2%

Free/Reduced Priced Meals 44.7% White 41.0%

Hispanic 15.5%

Two or More Races 4.3%

Source: Maryland State Department of Education

• 82% of children entering

kindergarten in school year

2012-2013 are fully school-

ready, up from 49% in 2001-

2002 – a 33-point or 67%

improvement in overall school

readiness.

• Maryland continues to exhibit an

overall upward trend and

maintains high readiness levels

even with the one-point decline

from 2011-2012.

Striking Overall Improvements

MMSR, 2012-2013

Note: A “Trendline” denotes the overall upward progression of Maryland’s full school readiness levels.

Source: Maryland State Department of Education

• 76% of Baltimore City’s

kindergarten students are fully

school-ready in 2013-2014, a

48-point readiness gain from

2001-2002.

• The City’s kindergarteners are

within 7 points of their Maryland

peers (83% fully ready) and

show greater long-term

improvements (48 points

compared with a 34 point

improvement in the state).

Continuing Our Strong Progress

MMSR, 2013-2014

Achieving “Collective Impact” with

Results-Based Accountability™By Deitre EppsPartner & Senior Consultant, Results Leadership Group

Five Conditions of

Collective Impact

A common agenda

Shared measurement systems

Mutually reinforcing activities

Continuous communication

A backbone support organization

Kania, John, and Mark Kramer. “Collective Impact.”

Stanford Social Innovation Review (2011)

Two Key Principles of Results Based Accountability

1. Starting with ends, working backwards to means

2. Data-driven,

transparent decision making

Distinguishing “Ends”

FPSI/RLG 10

about the well-being of

CLIENT POPULATIONS

about the well-being of

WHOLE POPULATIONS

Population Accountability

Performance Accountability

For Communities – Cities – Counties – States - Nations

For Programs – Agencies – and Service Systems

FPSI/RLG 11

DEFINITIONS

RESULT

INDICATOR

PERFORMANCE MEASURE

Young Children Are Ready for School

Babies Born Healthy

Healthy People

Clean Environment

% children fully ready to learn at kindergarten

rate of low-birth weight babies,

obesity rate,

air quality index

1. How much did we do?

2. How well did we do it?

3. Is anyone better off?

A condition of well-being for children, adults, families or communities.

A measure which helps quantify the achievement of a result.

A measure of how well a program, agency or service system is working.

= Customer Results

PO

PU

LA

TIO

N

AC

CO

UN

TA

BIL

ITY

PE

RF

OR

MA

NC

E

AC

CO

UN

TA

BIL

ITY

Three types:

Language

Discipline[ ]

FPSI/RLG12

THE LANGUAGE TRAP

Benchmark

Target

Indicator Goal

Result

Objective

Outcome

Measure

ModifiersMeasurable Core

Urgent Qualitative

Priority Programmatic

Targeted Performance

Incremental Strategic

Systemic

Lewis Carroll Center for Language Disorders

Too many terms. Too few definitions. Too little discipline.

FPSI/RLG 13

Translation Guide/Rosetta Stone

Not the Language Police

Ideas

1. A condition of

well-being for

children, adults,

families &

communities

2.

3.

etc.

Group 1 Group 2 Group 3

etc.

RESULT OUTCOME GOAL

TRANSLATION

Back to the Idea

MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES

14

Why Distinguish Population from

Performance Accountability?

ENDS

MEANS

RESULT

&

INDICATORS

A STRATEGY

&

PERFORMANCE MEASURESClient result

Service

delivery

Ends

Means


FPSI/RLG

15

Children are Valued, Healthy and Thriving

Community-Wide Strategy/Partners

Program Performance Measures

(baselines)

EN

DM

EA

NS

Client results END

? X

Population

Accountability


1. Doing the

right things?

2. Doing those things right?


Results-Based Budgeting

The purpose of

Results Based Budgeting

is not how to spend

the money we have.

It’s how to get the results we want.

RLG/FPSI

16


Population

Accountability:

Common Agenda and Shared

Measurement System

FPSI/RLG 17MEASURABLE RESULTS FOR CLIENTS AND COMMUNITIES

Population

Accountability



Overarching Vision: All children are valued, healthy and thriving

• Children live in safe, stable and supportive families and communities

• Families are engaged as leaders in their child’s healthy development and education

• Children’s health and development are on track

• Children of all races and income levels are successful learners


Local Examples:

Population Results

• Children are Valued, Healthy and Thriving

• Children are Ready for Kindergarten

• Children have health and well-being, including oral and mental health

• Children Enter School Ready to Learn

• All Children are healthy and ready for school


FPSI/RLG 21

Selecting

Indicators


FPSI/RLG 22

Right data for useful feedback?

Early Childhood Colorado

Overarching vision:

All children are valued, healthy and thriving

Results:

• Young children reach their developmental potential and are ready to succeed in school and in life

• Environments that impact children are safe, stable and supportive

• Adults are knowledgeable, responsive and interact effectively with and on behalf of children

• Localities and the state attain economic and social benefits by prioritizing children and families


FPSI/RLG 24

Criteria for

Selecting Indicators

Communication Power

Proxy Power

Data Power

Does the indicator communicate to a broad range of audiences?

Does the indicator say something of central importance about the result?

Does the indicator bring along the data “herd”?

Quality data available on a timely basis.


FPSI/RLG 25

Choosing Indicators

Worksheet

Result_______________________

Candidate IndicatorsCommunication

Power

Proxy

Power

Data

Power

H M LMeasure 1

Measure 2

Measure 3

Measure 4

Measure 5

Measure 6

Measure 7

Measure 8

H

Data

Development

Agenda

H M L H M L

H L

H H H


Turn the Curve

Thinking™:

Continuous Communication led

by a Backbone Organization


FPSI/RLG 27

Story behind the baseline

Partners (with a role to play in turning the curve)

What Works

Strategic Action Plan (with Budget)

Result: All Children are Valued, Healthy and Thriving

Indicator

BaselineHow are

we doing?

Why?

Help?

Options?

Propose

to do?

Turn-the-Curve Thinking™: Talk to Action


FPSI/RLG 28

The Matter of Baselines

Baselines have two parts: history and forecast

History Forecast

Turning the Curve

OK?

Return* on

Investment

* The “ROI” is not financial, it is Results


FPSI/RLG 29

The Story Behind the Baseline

Root Causes (ask “Why?” five times)

Share and discuss the positive and

negative causes and information

Prioritize – which are the most

important to address to “turn the curve”

of the baseline?

Research agenda (i.e. what research exist? What

are the knowledge gaps?)


FPSI/RLG 30

Force Field Analysis

Factors Contributing?

Factors Restricting?


FPSI/RLG 31

Indicator

Baseline

Result:


Story behind the baseline Research AgendaWhy?


FPSI/RLG 32

Indicator

Baseline


Result:

Research Agenda

Turn-the-Curve Thinking™ Talk to Action

Partners (with a role to play in turning the curve)Help?


FPSI/RLG 33

Partners

Who are partners who may have

a role to play in turning the

curve?

Does the story behind the curve

suggest any new partners?


FPSI/RLG 34

Indicator

Baseline


Partners (with a role to play in turning

the curve)

Result:

Research Agenda


What Works Research AgendaOptions?


FPSI/RLG 35

What Works

Options for actions to “turn the curve”?

Research-based (e.g. kangaroo care)

Low-cost/no-cost?

Off-the-wall ideas?

Research agenda?


FPSI/RLG 36

What Works

Options for actions to “turn the

curve”?

Research-based?

Low-cost/no-cost options?

Innovative, off-the-wall ideas?

Do we have a research agenda?


FPSI/RLG 37

Indicator Baseline



What Works

Result:

Research Agenda

Research Agenda


Criteria: Leverage; Feasible; Specific; Values

StrategyPropose

to do?


FPSI/RLG 38

Strategy

Recommendations

Leverage: will turn the curve of the

baseline?

Feasible (a.k.a. “reach)?

Specific: who, what, when, where,

how?

Consistent with community values?


FPSI/RLG 39

Step-by-step Turn the Curve Process

Please select a recorder and timekeeper

Represent yourself and one other person/agency. Name

your two roles (5 min.)

How are we doing (5 min.)

What is the story behind the curve? Remember to start

with the positives and ask why 5 times. Prioritize the

root causes. (15 minutes)

Complete a forecast to 2016. Is it okay? (5 min.)

Who are partners for prioritized causes? (5 min)

What strategies work to improve the data? (15 min.)

What strategies do you propose to do? (10 min.)


FPSI/RLG 40

Measure

Baseline

Story Behind the Baseline (prioritized)

---------------------------

---------------------------

Strategy Plan -with partners (pass criteria) 3 strategies (including one “no cost/

low-cost idea and one innovative idea)

Turn-the-Curve Report


Strategy Selection

(Rating: High, Med, Low) •

.

41

Specificity Leverage Values Feasibility

Strategy 3

Strategy 3

Possible Headline Indicators

Rated High in communication, proxy and data power

% of children demonstrating school readiness

% children 2-8 who are overweight

% parents with concerns about their child’s emotions,

concentration, behavior or ability to get along with others

Sample Strategies

Rated High in specificity, leverage, values and feasibility

Expand the taxpayer funded model for increasing access to

high quality preschool by engaging voters and legislators by

county (e.g. Denver Preschool Program, Right Start)

Ensure that all licensed child care providers are rated at levels

3-5 (by 2020)

Use Help Me Grow Implementation Guide (with stakeholder

recommendations) to implement Help Me Grow as a resource

for families for their child’s social, emotional and development

needs

(Overweight)Implement four sugar sweetened beverage policy

changes in community organiztions

FPSI/RLG 44



What Works

Strategic Action Plan (with Budget)


How are

we doing?

Why?

Help?

Options?

Propose

to do?



Indicator

A Results-based ECCP

• Population Accountability: shared accountability for measurable state and local results

• Performance Accountability: measurable quality improvement for programs, agencies and services systems

• ECCP Partner Accountability: performance accountability with measures of success for the statewide network


FPSI/RLG 46

Story behind the baselineParents with concerns about their child may not know where/how to access

services

There are multiple points of entry for receiving services (different organizations

and agencies)

What Works

Integrate developmental, social and

emotional screening practices


% children demonstrating

school readiness

BaselineHow are

we doing?

Why?

Help?

Options?

Propose

to do?



FPSI/RLG 47

ECCP,

Region work,

City,

Neighborhood …

E.g., Early

Childhood

Service System

Agency/

Org

“System”with

Community with a

its clients

its clients

whole population

ECCP: Population and Performance Accountability


Performance

Accountability

Mutually Reinforcing

Activities


Performance Measures

49

1. How much did we do?

Who are our customers and what services

do we provide to them?

2. How well did we do it?

How well do we provide those services?

3. Is anyone better off?

What is the desired impact of those services

for/on our customers (a.k.a. “customer

results”)?

Impact

50

Who are our clients?

What services do we provide

to those clients?

What is the desired

impact of those

services on those

clients?

How are we doing at

achieving that desired

impact?

Selecting

Performance

Measures


FPSI/RLG 52

Use performance measures to ask the right

question.

FPSI/RLG 53

How

much did

we do?

The Three Kinds of Program Performance Measures

How well

did we do

it?

Is anyone

better off?

Quantity QualityE

ffect

E

ffo

rt

# %


FPSI/RLG 54

Selecting Headline Performance Measures

How much did we do?

# Clients/customers served

# Activities

(by type of activity)


FPSI/RLG 55

How

much did

we do?

Developing / Sorting Program Performance Measures

How well

did we do

it?

Is anyone

better off?

Quantity QualityE

ffect

E

ffo

rt

# %


RBA Guide

pages 8-9

56


How much did we do?How well did we do it?

% Common measures

e.g. workload ratio, staff

turnover rate, % staff fully

trained, unit cost

% Activity-specific measures

e.g. % timely intakes, %

accreditation standards met


TACS

Timelines

Attendance

Customer Service

Completion

Standards

RLG/FSPI

57


How well did we do it?

Is anyone better off?

#/% Skills / Knowledge(e.g. cognitive, social, physical)

#/% Attitude

(e.g. toward language, parenting)

#/% Behavior(e.g. reading to child at home)

#/% Circumstances(e.g. child care, transportation)

How much did we do?


BACKS

Behavior

Attitude

Circumstances

Knowledge

Skills

FPSI/RLG 58

How much did we do?

Not All Performance Measures Are Created Equal

How well did we do it?


LeastImportant

Quantity QualityE

ffect

Eff

ort 2nd Most

Important

3rd MostImportant

MostImportant

Least

Most


How much did we

do?

The Matter of Control

How well did we do

it?


Quantity Quality

Eff

ec

t

Eff

ort

LeastControl

PARTNERSHIPS

MostControl


FPSI/RLG 60

Criteria for

Selecting Headline


Communication Power

Proxy Power

Data Power

Does the measure communicate to a broad range of audiences?

Does the measure say something of central importance about the program ?

Does the measure bring along the data HERD?

Quality data available on a timely basis.


FPSI/RLG 61

Choosing Headline


Program: _______________________

Candidate


Communication

Power

Proxy

Power

Data

Power

H M LMeasure 1

Measure 2

Measure 3

Measure 4

Measure 5

Measure 6

Measure 7

Measure 8

H

Data

Development

Agenda

H M L H M L

H L

H H H


FPSI/RLG 62



What Works

Action Plan (with Budget)

Program

Performance

Measure

Baseline

How are

we doing?

Why?

Help?

Options?

Propose

to do?



A Schematic for Results-Based

Planning and Accountability

(See Results Scorecard for web-based version)

RLG/FPSI 63

1. All Children are Healthy, Valued and Thriving

2.Headline Indicators (with baselines)

3.Story Behind the Baselines

4. Early Childhood Colorado Partnership

Programs

Program A

Program B

Program C

Local __________

Federal __________

Businesses _________

Civic __________

Non-profits _________

Program A


Story Behind the

Baselines

• Action Plan & Budget


Implementation

Considerations


Every time

you talk about

of results, use

a two-part

approach

Result: Children are Valued, Healthy and Thriving

Indicators:

Story:

Partners:

What would it take?:

Roles: Each partner role is identified (as part of a state-wide strategy)

Population Accountability

Program:

Performance measures:

Story:

Partners:

Action plan to get better:



FPSI/RLG 66

RBA Meeting AGENDA

1. New data

2. New stories behind the curve

3. New partners with a role to play

4. New information on what works.

5. Changes to action plan/ budget

6. Adjourn


Management, Budgeting & Strategic Planning

Management

Monthly or quarterly performance

assessment and action planning

using the Turn the Curve process

Use the Turn the Curve process at all

levels of implementation in the


RLG/FPSI 67

Budgeting

Use the Performance

Report format for

budget submissions to

present current

performance and what

will be done next year to

improve.

Budget priorities

informed by the

Framework strategies

and programs

Strategic

Planning

Population Level

Results & Indicators,

comprehensive strategy

among and all

stakeholders

ECCP

The Framework

detailed with multi-year

strategies

Management, Budgeting & Strategic Planning

Management

Monthly or quarterly performance

assessment and action planning

using the Turn the Curve process

Use the Turn the Curve process at all levels of implementation in the


RLG/FPSI 68

Budgeting

Use the Performance

Report format for

budget submissions to

present current

performance and what

will be done next year to

improve.

Budget priorities

informed by the

Framework strategies

and programs

Strategic

Planning

Population Level Results &

Indicators, comprehensive

strategy among all

stakeholders

ECCP

The Framework

detailed with multi-year

strategies

FPSI/RLG 69

Thank You!

Feel free to email questions to:

Deitre Epps

410-262-3470

[email protected]

Facebook: The Results Based Accountability

Group

Twitter: @Deitre_Epps

www.resultsleadership.org

mailto:[email protected]

RLG 70

Mark Friedman, Founder of the Fiscal Policy Studies Institute and author of:

Trying Hard Is Not Good Enough (Trafford, 2005)

www.resultsaccountability.com; www.raguide.org

Kania, John, and Mark Kramer. “Collective Impact.” Stanford Social Innovation

Review (2011)

http://ssireview.org/articles/entry/collective_impact/


Acknowledgement

Portions of these materials draw upon the work of:

http://www.resultsaccountability.com/

http://www.raguide.org/

http://ssireview.org/articles/entry/collective_impact/

introduction to results-based accountability™ · achieving “collective impact” with...

Documents