how do vouchers work? evidence from colombia eric bettinger, case western u michael kremer, harvard...
TRANSCRIPT
![Page 1: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/1.jpg)
How do Vouchers Work?Evidence from Colombia
Eric Bettinger, Case Western U
Michael Kremer, Harvard
Juan Saavedra, Harvard
7 June 2007World Bank
![Page 2: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/2.jpg)
Educational Vouchers
• Controversial and Often Debated Educational Reform• Large-Scale Voucher Programs
– Chile & Colombia– US Sites (Milwaukee, Cleveland, Florida, Ohio)– Other Voucher-Like Programs
• Private Programs in US• Regulated Voucher Programs Internationally (e.g. Sweden, Japan,
The Netherlands)
• Motivation for Programs Differ Substantially Across Sites– Competition and Opportunity– Overcrowding
![Page 3: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/3.jpg)
Evidence on Educational Vouchers
• International Evidence– Colombia– Chile
• US Evidence– New York City Scholarship Program– Milwaukee Voucher Program– Other Private/Public Programs
![Page 4: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/4.jpg)
Welfare Implications of Vouchers
• Costs/Benefits to Students Directly Receiving the Voucher– Cost of Voucher– Change in Educational Outcomes
• Costs/Benefits to Students not Receiving the Voucher– Change in Peers– Competition– Change in Resources
![Page 5: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/5.jpg)
Peer Effects and Vouchers
• Hsieh and Urquiola (2006)– Chilean Voucher Program– Key Finding was that Aggregate Outcomes Did Not
Change– Voucher Recipients Experienced a Positive Peer
Effect– Other “Left Behind” Students Experienced a Negative
Effect• Epple and Romano (1998)
– Model of Educational Vouchers– Peer Effects Need Not Be Zero Sum but Some
Students are Worse Off
![Page 6: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/6.jpg)
Other Peer Effect Literature
• Sacerdote (2001)
• Duncan, Boisjoly, Kremer, Levy and Eccles (2005)
• Zimmerman (2003)
![Page 7: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/7.jpg)
Simple Model with Peer Effects
• Consider the Following Educational Production Function
• Yi = Educational Outcome Student i• Xi = Socioeconomic Characteristics of Student i• = Average Characteristic of School• P = Program Participation
![Page 8: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/8.jpg)
Model (cont.)
• Positive Voucher Effects but No Peer Effects
β1=0, β2>0
• Positive Voucher Effects are Solely Peer Effects
β1>0, β2=0
![Page 9: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/9.jpg)
Voucher Comparisons
• Because of Randomization, Difference in Average Outcome of Voucher Lottery Winners and Losers Measures the Effect:
• Reported Effects in Angrist et. al. (2002, 2006)
21 )( loserwinner XX
![Page 10: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/10.jpg)
Social Effects
• Depends on Values of β1 and β2
• If β1=0, β2>0, then social effect is β2
• If β1>0, β2=0, then no social effect in simple linear-in-means peer effect model
21 )( loserwinner XX
![Page 11: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/11.jpg)
How does One Untangle Peer and Voucher Effects?
• Ideal Experiment:– Randomly Assign Voucher Students to Group
with “Worse” Peers– Measure Whether Voucher Effect is Present
When Peer Effect is Bad– If Voucher Effects are from Peer Effects, then
Voucher Effects Should Disappear or Even Be Negative
• Colombia PACES Program May Have Such an Experiment
![Page 12: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/12.jpg)
Colombia’s PACES Program
• Objectives– Increase secondary school enrollment for poor
• Rules– Student must be entering 6th grade and under 15 years old– Students must provide evidence that they live in poor
neighborhood (strata 1 or 2)– Renewable through graduation unless student is retained in a
grade– Vouchers awarded by lottery if demand exceeds supply– Covered about 60% of fees
• Scope– 216 Municipalities have Participated– Over 125,000 secondary students have received support
![Page 13: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/13.jpg)
Previous Work on Colombia
• Angrist, Bettinger, Bloom, King, Kremer (AER 2002)• Surveyed Voucher Applicants from Bogotá 1995 Lottery• Compared Voucher Lottery Winners and Losers• Effects after Three Years• Key Findings on Voucher Recipients:
– Increased Usage of Private Schools– Higher Educational Attainment– No Difference in Drop-out Rates– Less Grade Repetition– Higher Test Scores– Less Incidence of Teen-age Employment
![Page 14: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/14.jpg)
More Previous Work on Colombia
• Angrist, Bettinger, Kremer (AER 2006)• Tracked Applicants to Bogotá 1995 Lottery
Using Administrative Records• Compared Voucher Lottery Winners and Losers• Effects after Six to Seven Years• Key Findings on Voucher Recipients:
– Increased Likelihood of Taking College Entrance Exam
– Improved Exam Performance in Math and Language
![Page 15: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/15.jpg)
Other Details on Colombia’s PACES
• Students Had to be Accepted at a Private School PRIOR to the Voucher Lottery– Some Students Applied to Vocational Schools– Vocational Schools May Attract “Worse” Peers
• The Voucher was Portable in Theory but Not Practice– 1998 MIT Survey of Voucher Applicants Shows that
Voucher Winners Did Not Retain Voucher When Transferring Schools
![Page 16: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/16.jpg)
Patterns of Attendance
School Applied To
Vocational
School Attended Three Years after Voucher Lottery
Winner
Vocational .5962
Non-Vocational .2692
Dropout .1346
![Page 17: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/17.jpg)
Patterns of Attendance
School Applied To
Vocational
School Attended Three Years after Voucher Lottery
Winner Loser
Vocational .5962 .4264
Non-Vocational .2692 .4264
Dropout .1346 .1473
![Page 18: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/18.jpg)
Patterns of Attendance
School Applied To
Vocational Non-Vocational
School Attended Three Years after Voucher Lottery
Winner Loser Winner Loser
Vocational .5962 .4264 .0369 .0626
Non-Vocational .2692 .4264 .7995 .7517
Dropout .1346 .1473 .1590 .1790
![Page 19: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/19.jpg)
Voucher Winners and Vocational Schools
• Voucher Winners who Applied to Vocational Schools Stay in Vocational Schools
• Lack of Portability Creates Rigidity– Voucher Winners Stay Even if School is
Suboptimal
• Did Vocational Schools Have “Worse” Peers?
![Page 20: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/20.jpg)
Were Vocational Schools Worse?
• WB Funded School Survey in 2006
• Attempted to Contact 300 Schools– Chose Schools with Most Voucher Applicants
Attending– Schools Represented 85 Percent of Voucher
Applicants
• Survey Gathered Significant Data on Peer and School Quality
![Page 21: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/21.jpg)
Comparing Peer Quality
• Estimate for Sample of Students Who Applied to Vocational Schools Prior to Lottery
Wi = α + γVi +πZi+ ui
Wi = School characteristic of student i
Vi = Student i is voucher winner
Zi = Other Student Characteristics
• Key issue will be low sample size. We only know schools of attendance for about 200 voucher students who had wanted to attend vocational school.– Pooling Estimates
![Page 22: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/22.jpg)
Diffs in Peer Quality Vocational School Applicants (1)
Losers' Mean
(2) Vouher Diff w/o Covars
(3) Voucher Diff W/ Covars
A. Academic Indicators of Peer Quality
Mean ICFES Score 46.50 (3.188)
-.1518 (.4234)
-.1863 (.4552)
Proportion of Entering Class who later take the ICFES
.7591 (.4225)
-.0429 (.0749)
-.0699 (.0852)
Proportion of Students Enrolling in College
.3141 (.3158)
-.0736* (.0421)
-.0897** (.0448)
Percentage of Students Who Dropped Out in Senior Yr
.0339 (.0339)
.0118* (.0063)
.0080 (.0065)
![Page 23: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/23.jpg)
More Diffs in Peer Quality Vocational School Applicants (1)
Losers' Mean
(2) Vouher Diff w/o Covars
(3) Voucher Diff W/ Covars
B. Other Indicators of Peer Quality
Has Job Training Program .1517 (.3597)
.0894** (.042)
.0931** (.0448)
Has Tutoring for Disadvantaged Students
.2809 (.0451)
.0718 (.0557)
.0648 (.0569)
Fees 40752.6 (47212.4)
-8778.2 (5959.8)
-9497.9 (6151.0)
Specialized Facilities Available
.9367 (.3101)
-.0034 (.0157)
-.0015 (.0167)
![Page 24: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/24.jpg)
How to Increase Power?
• Average Effect Sizes– Normalize Measures
• Standard Deviation Units• Monotonicity
– Estimate Effect Sizes Simultaneously• Fully Interacted Model• Similar to Seemingly Unrelated Regression
– Combine Coefficients• Kolmogorov-Smirnov
– If there is no Effect of a Program, the Family of T-Statistics Should Behave Like Normal Distribution
– Doesn’t Account for Correlation Between Outcomes
![Page 25: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/25.jpg)
Average Effect Sizes
Category of School Characteristics
Difference by Voucher Status (std. dev units)
Standard Error
Number of Measures
Student College Entrance & Graduation Behavior
-.1671* .0942 8
Student Dropout Behavior .1188 .2272 3 Existence of Remedial or
Vocational Programs .2672** .1160 6
Total Fees Across All Categories
-.3870** .1757 2
School Appearance -.1589 .1329 4 School Facilities .1330** .0552 22 Teacher Characteristics -.4260** .1315 6
![Page 26: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/26.jpg)
Family of T-Statistics Vocational Voucher Applicants
0.1
.2.3
De
nsity
-4 -2 0 2 4t
PDF for Vocational Schools
PDF of Standard Normal
![Page 27: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/27.jpg)
Family of T-Statistics Non-Vocational Voucher Applicants
0.1
.2.3
.4D
en
sity
-4 -2 0 2 4t
PDF for Non-Vocational Schools
PDF of Standard Normal
![Page 28: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/28.jpg)
Key Results on Peer Quality
• Among Voucher Applicants to Vocational Schools, Winners’ Peers Are– Less Likely to Attend College– Less Likely to Graduate– More Likely to Attend Remedial Programs– Attend Schools with Lower Fees
• Voucher Winners’ Schools Also Appear to Have– Less Qualified Teachers– More Facilities on Campus
• Same Differences are not Present Among Applicants to Non-Voucher Schools
![Page 29: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/29.jpg)
Are Voucher Effects Still Present?Vocational Non-vocational Dependent Variable
(1) Losers' Means
(2) Regression-Adjusted Voucher Diff
(3) Losers' Means
(4) Regression-Adjusted Voucher Diff
A. Probability of Taking ICFES ID Match
.2875 (.4528)
.0586** (.0175)
ID & City Match
.2725 (.4454)
.0577** (.0174)
ID & Name Match
.2117 (.4087)
.0336** (.0164)
N 1200 2612 B. Performance Outcomes on the ICFES Math Score cond'l on
taking
42.39 (4.762)
.3094 (.3559)
Reading Score cond'l on taking
47.19 (5.450)
.3427 (.3962)
Math Score (Upper Bound Estimate)
-- 1.5270** (.3302)
Reading Score (Upper Bound Estimate)
-- 1.6949** (.3648)
N 319 874
![Page 30: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/30.jpg)
Are Voucher Effects Still Present?Vocational Non-vocational Dependent Variable
(1) Losers' Means
(2) Regression-Adjusted Voucher Diff
(3) Losers' Means
(4) Regression-Adjusted Voucher Diff
A. Probability of Taking ICFES ID Match .2548
(.4364) .0607* (.0300)
.2875 (.4528)
.0586** (.0175)
ID & City Match .2520 (.4348)
.0494* (.0299)
.2725 (.4454)
.0577** (.0174)
ID & Name Match .1884 (.3915)
.0536* (.0281)
.2117 (.4087)
.0336** (.0164)
N 361 810 1200 2612 B. Performance Outcomes on the ICFES Math Score cond'l on
taking 41.46 (4.865)
.7661 (.6370)
42.39 (4.762)
.3094 (.3559)
Reading Score cond'l on taking
45.71 (5.951)
2.060** (.7804)
47.19 (5.450)
.3427 (.3962)
Math Score (Upper Bound Estimate)
-- 2.507** (.6156)
-- 1.5270** (.3302)
Reading Score (Upper Bound Estimate)
-- 4.364** (.7376)
-- 1.6949** (.3648)
N 87 256 319 874
![Page 31: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/31.jpg)
Key Conclusions
• Among Students Who Originally Applied to Vocational Schools, Voucher Winners . . .– Attended Schools with Worse Observable
Characteristics– Had Higher Likelihood of Taking the ICFES
College Entrance Exam– Had Higher Test Scores on ICFES Exam
• Peer Effects cannot Explain Voucher Effects
![Page 32: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/32.jpg)
Voucher Mechanisms?
• Incentives– Vouchers were Renewable Conditional on Passing– Emerging Literature on Incentives (e.g. Kremer, Miguel, and
Thornton 2005)
• Attending School that Students Value More– Vocational Schools Increasingly Focused on Emerging Service
Industry– Private Sector Has Adjusted Quickly to Changes in Labor Market– Differences in Apprenticeship Opportunities– Industrial versus Commercial Education Curricula
• Limited Commercial Education Slots in Public Sector• Private Commercial Schools Retain Students
![Page 33: How do Vouchers Work? Evidence from Colombia Eric Bettinger, Case Western U Michael Kremer, Harvard Juan Saavedra, Harvard 7 June 2007 World Bank](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649d215503460f949f65a1/html5/thumbnails/33.jpg)
In Conclusion
• Peer Effects are Important in Education• Disentangling Voucher and Peer Effects is Difficult
– Need Unique Experiment where Vouchers Assign Students to “Worse” Peers
– Applicants to Vocational Schools (PRIOR) to Colombian Voucher Lottery
• Vocational Voucher Winners Attend Schools with “Worse” Peers
• Vocational Voucher Winners Have Higher Academic Achievement Than Voucher Lottery Losers
• Peer Effects May Not Explain the Voucher Effects• Other Mechanisms are Plausible