Second Cross-country Workshop of theAfrica Programs for Educationand AIDS Impact EvaluationDakar, December 2008

What do we know about

school bursaries?Targeting evidence from Kenya

Katie ConnAfrica Impact Evaluation Initiative (AIM), World Bank

School bursaries are an excellent example of the need for impact evaluation

Potential questions that can be answered with an impact evaluation: What amount of bursary is necessary to

increase enrollment? What is impact of schooling on a student’s

future earnings or health outcomes? What is the best way to target bursaries?

What do we need to know?

Bursary Targeting

Merit bursaries Relatively easy to assign Disproportionately benefit wealthier students

Bursaries targeting the poor Need-based bursaries more equitable Difficult to target the poorest of the poor Evidence from the Africa Program for

Education Impact Evaluation on bursaries: Kenya

Kenyan Context

June 2007: Ministry of Education officials met with technical experts (Pascaline Dupas and the Poverty Action Lab) in Abuja, Nigeria, and decided to evaluate targeting of bursaries

January 2008: Government announced secondary school subsidy of $160 per student Intended to cover the cost of day school

April 2008: Still, 20% of eligible students in Western Kenya did not enroll in 2008 Most cited insufficient means

November 2007 – May 2008 : Targeting Exercise

Kenya’s traditional secondary school bursary targeting system

Current system of allocating bursaries Proportionate amount allocated to political

constituencies (districts, more or less) They distribute applications and bursary

amounts as they wish Anecdotes: Not wanting to favor a single

child, committees spread funding across so many children that it fails to be useful

Further concern: favoritism

The Reform: a new form of targeting

Ministry of Education two-part form: Class 8 teacher Form & Student Guardian Form

Participatory Meetings with School Community Participatory rural appraisals with School

Management Committee & teachers only PRAs with SMC, teachers, and parents of class 8

students(Facilitated by locally based agency: IPA)

Exhaustive household survey (Gold standard? ) Implemented by IPA

The evaluation

Using Poverty Maps select 36 poorest “locations” in Western Kenya

Randomly select 1 school from each “location.” Randomly split them into 2 groups

Then, with this information in hand, checked to see who actually went on to go to school in 2008

Group Household Survey

Standard 8 Form PRA Ranking Session Participants Ex-Post Data:

MoE/ CDF Bursary Recipient Info. & Enrollment Status

1 Yes Yes SMC + Teachers Yes

2 Yes Yes SMC + Teachers + Parents Yes

Results (preliminary)

Neither the MoE form measuring assets nor the detailed household survey did a good job of predicting who would not attend without a scholarship Need more than assets to identify the

needy!

The PRAs with parents were very poor at predicting enrollment.

Results (preliminary)

Ranked as most needy according to…

MoE parent form (assets)

PRA (SMC + teachers)

PRA (SMC + teachers + parents)

Household survey (assets)

% enrolled in school in 2008

78% 72% 82% 74%

Ranked among TOP 5 MOST NEEDY STUDENTS according to…

MoE parent form (assets)

PRA (SMC + teachers)

PRA (SMC + teachers + parents)

Household survey (assets)

% enrolled in school in 2008

76% 66% 79% 73%

Results (preliminary)

Concrete recommendations for MoE forms

Difficult to distinguish among the poor: Needs better measures of lower income

Certain assets were owned by (almost) no households and so could be removed

Questions where teachers ranking students could be improved

What about the traditional method?

Remember the Constituency Committees disbursing funds as they please?

Anecdotes: Spread to everyone In fact: only 4% of students received

41% of government bursaries were given to students ranked in the less needy group by the PRA methods

Timeline First disbursements: May (2nd term) September 2008: 4 of 9 committees had not

disbursed funds!

Final Thoughts/ Questions What kind of results will we get from other

measures of income/ poverty (apart from assets)?

Did a student’s rank in class influence his/her neediness ranking?

What are some other methods of student identification that could be used in PRA poverty targeting meetings?

Are these results externally valid? How would this work in your country/ community?

Cautionary note from Cambodia:Am I even studying the right thing?Students in Grade 6 apply for a

scholarship for Grade 7: mix of need and merit

Scholarships of $45 and $60

Scholarships led to 20% higher attendance 15% higher enrolment Filmer & Schady

2008

Learning? No effect!

Schedule of Activities

----2007---- ----------------------------------2008----------------------------------

Activity/ Month 11 12 1 2 3 4 5 6 7 8

Fine-tuning of study design with MoE

Selection of 36 pilot schools with MoE

Pre-testing of PRA protocol

MoE Standard 8 Forms distributed to schools and collected by IPA

PRA Targeting meetings (Methods 2 and 3)

Pre-testing of household surveys

Household surveys with sampled standard 8 parents

School visits to collect data on KCPE results, admissions/ enrollment status of former Std 8 students

Visits to constituencies to collect data on CDF & MoE bursary recipients

Data cleaning and analysis

Completion Report

All IPA activities on hold due to post-election violence in Kenya from J an-Feb of 2008.