Governance and social service delivery: An agenda for research

and action

Buenos AiresSeptember 29, 2009

Ariel FiszbeinChief Economist, Human Development

World Bank

The challenge of achieving human development results

•More inputs ≠ better outcomes•Outcomes = f (inputs; technology, behaviors; context)

•Complex systems: outcomes influenced by decisions of many actors

Outline

1. Conceptual framework

2. Reform strategies

3. Indicators and measurement tools

4. An agenda for research and action

1. Conceptual framework

Governance of service delivery

•Governance: rules under which actors interact

•Can be seen as set of principal-agent relations defined by the:▫Incentives facing agents▫Accountability mechanisms available

to principals

Client Power

Voice/

Politic

s

Compact/Policies

Politicians & Policymakers

ServiceProvidersCitizens/Users

6

A set of principal-agent relations…

..characterized by … (Dixit 2002)

•Moral Hazard: Principal cannot attribute outcomes to (often unobserved) actions of agent –e.g. role of teacher effort in learning outcomes

•Adverse Selection: Agents have private information on their ‘type’ –e.g. information on real costs at the health clinic level

•Costly Verification: Outcomes are not easily observable –e.g. quality of medical care/teaching

Public policy as a function of provider & user/citizen behavior (Le Grand 1997, 2003)

•Provider motivation: Knights or Knaves?▫Knaves: self-interested, extrinsic motivation▫Knights: altruistic, intrinsic motivation

•Agency: Capacity for action and choice among citizens/service users▫Pawns: Passive▫Queens: Active

Ag

en

cy

Motivation

Queens

Pawns

Knights Knaves

MarketQuasi-Market

“Mom &

Dad”

Improve agency (information, client power)

Improve provider incentives (Pay-for-performance,

autonomy)

2. Reform Strategies

Reform Intended Goals Examples

I. Improve citizen/user access to information

•Influence policy•Enable informed user decisions•Make providers accountable

•Right-to-information•Information campaigns•Score cards

II. Link provider pay to performance

•Focus teachers on improving student learning• Focus providers on quantity & quality of care

•Teacher bonuses linked to student performance•Health providers paid according outputs and outcomes

Reform Strategies

I. Improving access to information • Are better informed

citizens able to influence decisions by policy makers?

• Can information empower users to: (1) make better decisions as consumers and (2) demand better service from providers?

Voice/

Politic

s

Politicians & Policymakers

Citizens

Client Pow

er

Providers

Users

Are better informed citizens able to influence decisions by policy makers?

•Providing information is low cost intervention

•When is information the binding constraint?

•Is political system permeable?

Examples

• Newspaper campaign in Uganda

• Recurso Peru

Newspaper Campaign in Uganda

– In 1995 in Uganda, only 13 percent of non-wage recurrent spending on primary education reached primary schools.

– From 1996-2001, amounts and dates of grant to local governments published in local and national newspapers.

Voice/

Politic

sPoliticians &

Policymakers

Citizens

Schools in Uganda received more of what they were due, post-intervention:

Source: Reinikka and Svensson (2001), Reinikka and Svensson (2003a)

IMPACT ON LEAKAGE:A school close to a newspaper outlet experienced 25 percentage points less leakage compared to a school one standard-deviation (30 km) further away from a newspaper outlet.

IMPACT ON LEARNING:A one standard deviation reduction (30 km) in distance led to a 0.09 standard deviation increase in average test scores.

Recurso – Peru • In 2005 Peru instituted a series

of interventions to increase transparency and accountability in the social sectors.

• However, despite an increase in stakeholder participation, quality of services remained stagnant. Is information a binding constraint?

•Recurso produced instruments (including videos distributed by television stations and media) designed to educate stakeholders about benchmarks in education and nutrition.

•Universal testing introduced for 2nd grade students, with results being provided back to parents starting in 2008

•Nutrition standards established and provided to communities.

Voice/

Politic

s

Politicians & Policymakers

Citizens

Can information empower citizens as users?

• In decentralized settings where state has low capacity, community monitoring could be essential

• Can community monitoring be promoted at low cost?

• When there is choice, can information influence demand for services?

Examples

• Community monitoring of health services in Uganda

• Community information campaign for education in India

Citizen report cards for health in Uganda• 2001 all public health services become free-of-charge.

Dispensaries are controlled at the district level.• What’s the effect of providing information on service

delivery status relative to other providers and gov’t standards to users?

Impact of Citizen Report Cards: •.17 z-scores increase in weight of infants and 33% reduction in infant mortality.

• Utilization of services was 16% higher in treatment facilities than controls.

Impact of Community Information Campaigns:

(Three Indian States: Karnataka , MP, UP)

•India decentralized school management to local level and established ‘village education committees’

•Can information in the hands of communities help in making schools better and improve learning outcomes?

Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.

School Community Information Campaigns

Campaigns to inform the community about its oversight roles in public schools & the services to which children are entitled. 8-9 meetings in a village over 2 months

Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.

Client P

ower

Providers

Users

Impact of Information Campaign - On Educational Inputs & Student Learning:

REGION Teacher Attendan

ce

Teacher

Activity

% Receiv

ed Unifor

m

Participation rate of parents in school inspections

Scholarships

Students received

Improvement in

Reading

UP 11 % Unchanged

18 % (girls)

.11 33 %(general caste)

27 %(Grade 3)

MP Unchanged 30% 14 % (general caste)

.02 26 %(SC/ SC)

14 %(Grade 3)

Karnataka

Unchanged Unchanged

N/A Unchanged N/A 15 %(Grade 4)

Pandey, Priyanka et al. Community Participation in Public Schools: Impact of Information Campaigns in three Indian States. South Asia Human Development. Presentation. World Bank. 2007.

Strategy IILinking pay to provider

performance

1) Can paying health providers for results instead of inputs result in improved quality and quantity of health care?

2) Will linking teacher pay to student performance increase achievement?

Compac

t/Polic

ies

Politicians & Policymakers

Service Providers

Results-based financing (RBF) ≈ Pay-for-performance (P4P)

Provision of payment for the

attainment of well-defined

results

Transfer of money or material goods conditional on taking a

measureable action or achieving a predetermined performance target

(CGD, 2009)

DonorCentral governmentLocal governmentPrivate insurer

$Recipients of careHealth care providersFacilities / NGOsCentral governmentLocal governments

RBF takes many forms…

Payers Payees

Can the introduction of pay for performance for patient services impact the quality and performance of hospitals/health centers?

• Impact on equity of services?

• Will quality be affected by workers focusing on quantity of services provided?

• Can facilities maintain an adequate amount of autonomy?

• Issues of autonomy for facilities.

Impact Evaluations:

RBF in Argentina’s Health Sector: Plan Nacer

• PN (launched 2002) introduces RBF for maternal and child health care. Expanded to all provinces in 2006.

• Performance agreements made between Nation and Provincial Ministries of Health; & between provinces and providers;

• Capitation payments provide incentives for providers to expand coverage;

NACION

CO

NV

EN

I O

MA

RC

O

PROVINCIA

CO

MPR

OM

I SO

DE

GE

ST I

ON

EFECTORES

INSCRIPCIÓN Validación final de

Padrones Nacionales

Identificación e Inscripción

Administración Padrón Provincial

Identificación e Inscripción

NOMENCLADOR Diseño Valoración Brinda Prestaciones y las

factura

FINANCIAMIENTO

Paga Cápita: 60% por inscripción 40% por

cumplimiento de resultados sanitarios

Administración Cuenta Única

Paga prestaciones

Utiliza los fondos en: Recursos humanos Equipamiento Infraestructura Insumos

AUDITORIA

Padrón de Beneficiarios

Trazadoras Prestaciones

Prestaciones Registro en Historia

Clínica

•Accountability ensured through concurrent audits;

•Social accountability through communication campaigns, beneficiary satisfaction surveys.

•How does the introduction of RBF impact health indicators for pregnant women and children from 0 to 6 years of age?

•Does the program impact the coverage and quality of primary care for pregnant women and children from 0 to 6 years of age?

• What is the impact on service delivery quality? And, how can the incentive structure be maximized to promote improved service delivery and health outcomes?

Impact Evaluation of Plan Nacer

Other eligible Children

- Questions about the type, quality and satisfaction of services received andname of the health care agent providing care

Mother

Last Child Born (aged between 1 and 12

months)

Managers

Doctors

Exit Poll

- Information on the newly-born child shall be provided by the mother or personin charge of nursing the child

House

hold

Surv

ey

- Household usual socio-demographic indicators. History of pregnancies andchildbirths

Speci

fic

Medic

al R

eco

rds

- Questions related to three-four hypothetical visits to learn about good practice protocols.

Medical records recovery at the PHC Facilities

- Measuring encephalic perimeter

- Measuring height

- Measuring weight

Specific measures taken in the field (to the mother

and the latest born child)

- Weight and height

- Contraceptive methods

- Last visit of the child, PHCC he attended, name of the health agent

- Anemia test (Hemo cue)

- Child's APGAR index

- General features of the center (staff, infrastructure, equipment, inputs, resources, expenditures and sources of funds)

- Demographic and Labor and professional qualifications

- Questions related to their attendance to the center

- Socio-economic questions

- In PHCCs, number of consultations per type of patient (age, gender) and reason, per month during the last year.

- Reconstruction of medical care received

Serv

ice P

rovid

ers

- Mother's aerobic capacity; contraceptive methods; practices of hygiene and healthy behavior

- Information on the latest pregnancy and childbirth

- Weight and height of the mother and the child, child's APGAR index; mentalhealth

- Checklist all the contents of the medical records

Plan Nacer Questionnaire 2008

Advance from Rwanda Evaluation: Gertler, Basinga et al

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

InstitutionalDelivery

Tetanus Preventive 0-23months

Preventive 24-59 months

2006

08- no PBF

08 - PBF

RwandaMonetary incentives to health center

conditioned on 14 maternal and child health outputs

Payments allocated at discretion of management

Can merit pay for teachers improve student achievement?

Concerns

• Unintended consequences (e.g. “teaching to the test”, cheating)

• How will it affect teacher motivation?

• Attribution?• Political feasibility?

Examples

•Teacher Incentives in Pernambuco, Brazil

•Teacher Pay in Andra Pradesh, India

Teacher Pay in India – The Context• Very low levels of learning in India

-~60% of children aged 6-14 in India cannot read a simple paragraph, though 95% enrolled in school (PRATHAM, 2008)

• Large inefficiencies in delivery of education -25% teachers absent, less than half are teaching -Over 90% of non-capital spending goes to teacher salaries

-Higher ‘levels’ of pay not associated with better performance

-Strong unions, almost impossible to fireCan linking pay to performance improve school quality?Identification of the causal impact of teacher performance pay is a central limitation

Teacher Pay Incentives in Andra Pradesh (Muralidharan and Sundararaman, 2008)

State introduces three policies:• Group and individual pay incentives based on

average improvement in test scores of all students• Block grants to schools (extra teachers, more inputs)• Assessments conducted by an independent NGO

Impact evaluation results (two rounds):• Incentives increase student performance by 0.22 SD• 1st year both incentives work well. 2nd year individual

incentives work better• Additional inputs improve scores only by 0.08 SD• Teachers liked the program

Teacher Incentives: Pernambuco, BrazilBackground•School attendance increased. Quality still low. •Weak pay incentives (flat salary scale determined by tenure)•Low teacher motivation and high absenteeism•Poorly qualified teachers (do not pass student exams).•Pernambuco one of the lowest•performing states

Teacher Incentives - PernambucoNon-linear performance pay system (July 2008):

•School goals for achievement in Math and Portuguese by grade• Staff reaching at least 50% of the goal get bonus (amount varies with salary & % of goal achieved)

Impact evaluation examining teachers responses:

• Strength of incentives (measured by the targets set by an education quality index)?• Do teachers that come close to the bonus get encouraged or discouraged?• Sorting of teachers across schools?

3. Indicators and Measurement Tools

A measurement framework

Measuring Governance Performance

Many of these indicators can be collected through Quantitative Service Delivery Surveys….

• The facility is used as the unit of analysis▫ Could be complemented with a household/users survey

• QSDS collect quantitative information about▫ Physical infrastructure▫ Staff characteristics▫ Income and expenditures▫ Governance and management▫ Characteristics/Quality of service provision▫ Outcomes

Measuring quality of medical care – Medical Vignettes

Getting at Clinical Quality

Knowledge or “Competence”

of Doctors

Behavior – Actual

Performance – of Doctors

0

.1.2

.3.4

% W

ho a

sked

the

rele

vant

que

stio

n

Private MBBS Private, No MBBS Public

...And What They DoWhat They Know

% Asked (DCO) % Asked (Vignettes)

What Doctors Know… and What They Do – Evidence from India

Das and Hammer 2006

Stallings Classroom Observation Instrument

Trained enumerators take 1 minute “snapshot” of class every 6 minutes. Repeat on 2 successive days. Revisit every two months. Being utilized in the Pernambuco evaluation.

Measuring quality of teaching?

Percent of time officially allocated to schooling; when a teacher is present; and spent in teaching and learning activities

Beyond absenteeism: Effective supply of teaching

Sources: Egypt, Yemen and Lebanon from Lane and Millot (2002); Tunisia, Pernambuco, Morocco and Ghana from Abhadzi, Millot and Prouty (2006); Cambodia from Benveniste, Marshall and Caridad Araujo (2008); and Laos from Benveniste, Marshall and Santibanez (2007).

Percentage of Time Use in 4th Grade (common for all countries)

Country Discipline %

Interactive Learning

%

Passive Learning

%

Organizational Management

(non-teaching)

%

Pernambuco (Brazil)

1.75 52.89 18.61 25.50

Ghana 1.44 52.50 12.50 35.00

Morocco 4.20 62.85 20.03 17.78

Tunisia .88 61.70 26.32 11.98

Abadzi, Millot and Prouty (2006)

4. An Agenda for Research and Action

Where are the opportunities and what are the main challenges?

•Measurement: Inter and intra-national benchmarking

•Experimentation and evaluation•International platforms for information

exchange on reform strategies•Entry points for reform: matching reform

to politics

END

Absence rates among teachers and health workers

Note: Surveys were all fielded in 2002 or 2003. Sources: Chaudhury et al (2006) except for PNG, World Bank (2004) and Zambia, Das et al (2005).

0

10

20

30

40

50

Bangladesh Ecuador India Indonesia Papua NewGuinea

Peru Zambia Uganda

Primary schools Primary health facilities

Disbursed public spending on school grants that actually reach schools

Percent

GNI per capita (2000)

GNI per capita PPP

(2000)

Ghana 1997/98 51 330 1880

Kenya 2004 (secondary school bursary funds)

78 250 810

Madagascar 2002 88 2050 4610

Peru 2001 (utilities) 70 / 97 670 2280

PNG (2001/2002) 72 / 93 280 510

Tanzania 2002-2003 62 270 1250

Uganda 1991-1995/2001 <20 / 80

Zambia 2001-2007 (discretion/rule) 24 / 90 320 740Ye and Canagarajah (2002) for Ghana; Republic of Kenya (2005) for Kenya; Francken (2003) for Madagascar; Instituto Apoyo and World Bank (2002) for Peru; World (Bank 2004)

for PNG; MOF, Government of Tanzania (2005) for Tanzania; Reinikka and Svensson (2005) for Uganda; Das et al. (2002) for Zambia.

Percent of school grants that actually reach schools

Disbursed public spending on school grants and funds allocated for health facilities that actually reached intended recipient/facility

EducationPercent

Health Percent

Ghana 1997/98 51 20

Kenya 2004 20 62

Madagascar 2002 88 --

Peru 2001 (utilities) 70 / 97 --

PNG (2001/2002) 72 / 93 --

Tanzania 2002-2003 62 59

Uganda 1991-1995/2001 <20 / 80 --

Zambia 2001-2007 (discretion/rule) 24 / 90 --

Chad 2003 (regional/local health centers)

-- 27/ <1

Ye and Canagarajah (2002) for Ghana; Republic of Kenya (2005) for Kenya; Francken (2003) for Madagascar; Instituto Apoyo and World Bank (2002) for Peru; World (Bank 2004) for PNG; MOF, Government of Tanzania (2005) for Tanzania; Reinikka and Svensson (2005) for Uganda; Das et al. (2002) for Zambia; Wane for Chad (2004).