![Page 1: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/1.jpg)
Impact Evaluations and Social Innovation in Europe
Bratislava, 15 December, 2011Joost de Laat (PhD)Human Development EconomicsEurope and Central Asia The World BankComments: [email protected]
![Page 2: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/2.jpg)
PROGRESS 15 December 2011 Deadline
![Page 3: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/3.jpg)
Inputs Activities Outputs Impacts on Outcomes
Finance
State budget
European Social Fund
Human resources
Min. of Labor
Public employment offices
Private/public training providers
Project preparation activities (4 months)
Identify 2000 long-term (at least 2 years) unemployed people interested in training program
Identify 10 private / public training providers
Design monitoring database
Poject implementation activities (1 year).
Offer training vouchers to 2000 unemployed
Training institutes provide 3 months skills trainings to participating unemployed
Enter data on training participation and job placement into database
Improved labor market skills of current long-term unemployed
Improved employment rates of current long-term unemployed
Improved wage rates of current long-term unemployed
Reduced poverty of current long-term unemployed
Greater education enrolment among children of current long-term enrolment
Improved health outcomes among children of current long-term unemployed.
Project preparation outputs
2000 long term unemployed identified
Contracts signed with 10 private/public training providers
Monitoring database in place
Project implementation
2000 vouchers allocated
Est. 1400 long term unemployed accept voucher and enlist in training
Est. 1000 long term unemployed complete training
Est. 600 long term unemployed find jobs following training
Database on participants
![Page 4: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/4.jpg)
Outline
What?
Who?
How?
Ethics?
Why?
Impact Evaluations
?
![Page 5: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/5.jpg)
What?
Isolates causal impact on beneficiary outcomes
Globally hundreds of randomized impact evaluations• Canadian self-sufficiency welfare program• Danish employment programs• Turkish employment program• India remedial education program• Kenya school deworming program• Mexican conditional cash transfer program
(PROGRESA)• United States youth development programs
Different from:e.g. evaluation measuring whether social assistance is reaching the poorest households
![Page 6: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/6.jpg)
Who?
Often coalitions of:• Governments • International organizations• Academics• NGOs• Private sector companies
Examples:• Poverty Action Lab (academic)• Mathematica Policy Research (private)• Development Innovations Ventures (USAID)• International Initiative for Impact Eval. (3ie)• WB Development IMpact Evaluation (DIME)
![Page 7: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/7.jpg)
7Ex 1: Development Innovation Venture (USAID)
![Page 8: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/8.jpg)
Ex 2: International Initiative for Impact Evaluation (3ie)
![Page 9: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/9.jpg)
9Ex 3: MIT’s Poverty Action Lab (JPAL)
![Page 10: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/10.jpg)
10Ex 4: WB Development IMpact Evaluation (DIME)
![Page 11: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/11.jpg)
11Make publicly available training materials in partnership with other WB groups (e.g. Spanish Impact Evaluation Fund)
![Page 12: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/12.jpg)
12
Location Date Countries Attending Participants Project Teams
El Cairo, Egypt January 13-17, 2008 12 164 17
Managua, Nicaragua March 3-7, 2008 11 104 15
Madrid, Spain June 23-27, 2008 1 184 9
Manila, Philippines December 1-5, 2008 6 137 16
Lima, Peru January 26-30, 2009 9 184 18
Amman, Jordan March 8-12, 2009 9 206 17
Beijing, China July 20-24, 2009 1 212 12
Sarajevo, Bosnia September 21-25, 2009 17 115 12Cape Town, South Africa December 7-11, 2009 14 106 12
Kathmandu, Nepal February 22-26, 2010 6 118 15
Total 86 1,530 143Organize trainings on impact evaluations in partnership with others (e.g. Spanish Impact Evaluation Fund)
![Page 13: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/13.jpg)
Impact Evaluation Clusters
• Conditional Cash Transfers• Early Childhood Development• Education Service Delivery• HIV/AIDS Treatment and Prevention• Local Development• Malaria Control• Pay-for-Performance in Health• Rural Roads• Rural Electrification• Urban Upgrading• ALMP and Youth Employment
Help coordinate impact evaluations portfolio
![Page 14: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/14.jpg)
How to carry one out?
Basic Elements
• Comparison group that is identical at start of program
• Prospective: evaluation needs to be built into design from start
• Randomized evaluations generally most rigorous• Example: randomize phase-in (who goes
first?)• Qualitative information – helps program
design and understanding of the 'why'
![Page 15: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/15.jpg)
Ethics?
Implementation considerations• Most programs cannot reach all:
randomization provides each potential beneficiary fair chance of receiving program (early)
• Review by ethical review boards
Broader considerations• Important welfare implications of not
spending resources effectively
• Is the program very beneficial? If we know the answer, there is no need for an IE
![Page 16: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/16.jpg)
Why?
EU2020 Targets (selected)• 75% of the 20-64 year-olds to be employed • Reducing school drop-out rates below 10% • At least 20 million fewer people in or at risk of
poverty and social exclusion
Policy Options Are Many• Different ALMPs, trainings, pension rules,
incentives for men taking on more home care etc. etc.
• For each policy options, also different intensities, ways of delivery…
![Page 17: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/17.jpg)
Why?
Selective Use of Impact evaluations• Help provide answers to program
effectiveness and design in EU2020 areas facing some of the greatest and most difficult social challenges
But impact evaluations can also• Build public support for proven programs• Encourage program designers (govts, ngos,
etc.) to focus more on program results• Provide incentive to academia to focus
energies on most pressing social issues like Roma inclusion!
![Page 18: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/18.jpg)
Why? Help Encourage Social Innovation
+
![Page 19: Impact Evaluations and Social Innovation in Europe Bratislava, 15 December, 2011 Joost de Laat (PhD) Human Development Economics Europe and Central Asia](https://reader035.vdocuments.site/reader035/viewer/2022062518/56649edb5503460f94bebb31/html5/thumbnails/19.jpg)
Thank you for your attention!