armenias millennium challenge account: assessing impacts ken fortson, mpr ester hakobyan, mca anahit...
TRANSCRIPT
Armenia’s Millennium Challenge Account: Assessing Impacts
Armenia’s Millennium Challenge Account: Assessing Impacts
Ken Fortson, MPREster Hakobyan, MCA Anahit Petrosyan, MCAAnu Rangarajan, MPR Rebecca Tunstall, MCC
Ken Fortson, MPREster Hakobyan, MCA Anahit Petrosyan, MCAAnu Rangarajan, MPR Rebecca Tunstall, MCC
MCA-Armenia ProgramMCA-Armenia Program
Compact was signed on March 27, 2006
Total amount is $235.6 million
Implementation started on September 29, 2006
Duration is five years
Compact was signed on March 27, 2006
Total amount is $235.6 million
Implementation started on September 29, 2006
Duration is five years
MCA-Armenia Program (continued)MCA-Armenia Program (continued)
Overall Goal: Create conditions for sustainable economic growth to reduce rural poverty
Overall Objective: Rural development through investment in (1) Rural Roads to improve access to economic and social infrastructure, and (2) Irrigation infrastructure to increase agricultural productivity
Overall Impact: It is expected that by 2011 rural poverty will decline by additional 6 percentage points.
Overall Goal: Create conditions for sustainable economic growth to reduce rural poverty
Overall Objective: Rural development through investment in (1) Rural Roads to improve access to economic and social infrastructure, and (2) Irrigation infrastructure to increase agricultural productivity
Overall Impact: It is expected that by 2011 rural poverty will decline by additional 6 percentage points.
ProjectsProjects
Rural Roads Rehabilitation: – Improve up to 943 km of rural roads
Irrigation Infrastructure Rehabilitation:– Improve access to and efficiency of irrigation
systems
Rural Roads Rehabilitation: – Improve up to 943 km of rural roads
Irrigation Infrastructure Rehabilitation:– Improve access to and efficiency of irrigation
systems
Projects (continued)Projects (continued)
Water-to-Market Activities– Improve the profitability
of WUA members– Institutional
strengthening of Water User Associations
Water-to-Market Activities– Improve the profitability
of WUA members– Institutional
strengthening of Water User Associations
How is the MCA-Armenia Program Different?
How is the MCA-Armenia Program Different?
Country ownership of the program– Developed by the country– Implemented by the country
Economic analysis focuses on integrated approach to solve rural/agricultural issues
New rigorous approach to evaluate the program – Project level impact evaluations
Country ownership of the program– Developed by the country– Implemented by the country
Economic analysis focuses on integrated approach to solve rural/agricultural issues
New rigorous approach to evaluate the program – Project level impact evaluations
Overview of Presentation Overview of Presentation
I. Water-to-Market Project (Anahit)
II. Associated Impact Evaluation (Ken)
I. Water-to-Market Project (Anahit)
II. Associated Impact Evaluation (Ken)
Water-to-Market Activity Objectives Water-to-Market Activity Objectives
Increase agricultural efficiency and profitability– Training – Technical assistance – Credit
Maintain long-run sustainability of the irrigation infrastructure through increased water usage fee collection
The direct impacts of the Water-to-Market activities are employment generation in rural areas and increased incomes for farms and rural businesses
Increase agricultural efficiency and profitability– Training – Technical assistance – Credit
Maintain long-run sustainability of the irrigation infrastructure through increased water usage fee collection
The direct impacts of the Water-to-Market activities are employment generation in rural areas and increased incomes for farms and rural businesses
Introduction of New On-Farm Water Management Technologies
Introduction of New On-Farm Water Management Technologies
Objective: Improve farmers’ skills in water management and access to farm-level equipment to enhance the efficiency of water usage– Train 60,000 farmers in water management
Expected Result: Improved productivity and increased income from agriculture by adoption of water saving technologies
Objective: Improve farmers’ skills in water management and access to farm-level equipment to enhance the efficiency of water usage– Train 60,000 farmers in water management
Expected Result: Improved productivity and increased income from agriculture by adoption of water saving technologies
Transition to Higher Value AgricultureTransition to Higher Value Agriculture
Objective: Support transition to more profitable agricultural production through– Train 30,000 farmers in high-value agriculture– Crop substitution – Increased cropping intensity– More productive livestock activities– Introduction of new technologies, techniques, and higher
yield generating inputs
Expected Results: Improved productivity and increased income from agriculture by increasing the commercial value of farm outputs
Objective: Support transition to more profitable agricultural production through– Train 30,000 farmers in high-value agriculture– Crop substitution – Increased cropping intensity– More productive livestock activities– Introduction of new technologies, techniques, and higher
yield generating inputs
Expected Results: Improved productivity and increased income from agriculture by increasing the commercial value of farm outputs
Key Research QuestionsKey Research Questions
Did the program affect agricultural practices of Armenian farmers?– Irrigation– Higher-value crops
Did the program affect agricultural productivity?
Did the program improve household well-being?– Increase income– Reduce poverty
Did the program affect agricultural practices of Armenian farmers?– Irrigation– Higher-value crops
Did the program affect agricultural productivity?
Did the program improve household well-being?– Increase income– Reduce poverty
Random Assignment: The “Gold Standard”
Random Assignment: The “Gold Standard”
Randomly assign eligible participants into two groups – Program group: Receives the program– Control group: Does not receive the program for
some period
Program and control groups are the same on average, except one group has access to program – Any observed differences over time can be attributed
to the program
Randomly assign eligible participants into two groups – Program group: Receives the program– Control group: Does not receive the program for
some period
Program and control groups are the same on average, except one group has access to program – Any observed differences over time can be attributed
to the program
Treatment and Control GroupsTreatment and Control Groups
Randomly assign when training will start in each village, with three groups:– Compact Year 2 (Treatment)– Compact Years 3 and 4– Compact Year 5 (Control)
Random selection is a fair way to determine timing of training
Compare Compact Year 2 villages to Compact Year 5 villages
Randomly assign when training will start in each village, with three groups:– Compact Year 2 (Treatment)– Compact Years 3 and 4– Compact Year 5 (Control)
Random selection is a fair way to determine timing of training
Compare Compact Year 2 villages to Compact Year 5 villages
Selection of VillagesSelection of Villages
Total of 277 village clusters will be assigned:– Year 2: 120 clusters– Years 3 and 4: 77 clusters– Year 5: 80 clusters
Some villages not included in the selection – Pilot phase villages– Villages that do not have adequate water and
would not benefit from training yet (eligible later)
Total of 277 village clusters will be assigned:– Year 2: 120 clusters– Years 3 and 4: 77 clusters– Year 5: 80 clusters
Some villages not included in the selection – Pilot phase villages– Villages that do not have adequate water and
would not benefit from training yet (eligible later)
Selection of Villages (continued)Selection of Villages (continued)
Random assignment within Water User Associations– Equitable– Ensures balance
Selection conducted in public in August 2007– Transparency
Random assignment within Water User Associations– Equitable– Ensures balance
Selection conducted in public in August 2007– Transparency
Data SourcesData Sources
Farming Practices Survey conducted for this evaluation– Interview sample of WUA members– Fielded each year, Fall 2007 through Fall
2010
WUA databases, compiled by Irrigation PIU– Information on all WUA members– Source of outcome data at end of follow-up
period
Farming Practices Survey conducted for this evaluation– Interview sample of WUA members– Fielded each year, Fall 2007 through Fall
2010
WUA databases, compiled by Irrigation PIU– Information on all WUA members– Source of outcome data at end of follow-up
period
Evaluation TimelineEvaluation Timeline
Fall 2007: Baseline Survey– Prior to training in targeted villages
Fall 2008 and Fall 2009: Follow-up Surveys
Summer 2009: Preliminary Evaluation – Focus on short-term outcomes
Fall 2010: Final Follow-up Survey
Summer 2011: Final Impact Evaluation– Include all outcome measures
Fall 2007: Baseline Survey– Prior to training in targeted villages
Fall 2008 and Fall 2009: Follow-up Surveys
Summer 2009: Preliminary Evaluation – Focus on short-term outcomes
Fall 2010: Final Follow-up Survey
Summer 2011: Final Impact Evaluation– Include all outcome measures
Outcome MeasuresOutcome Measures
Adoption of agricultural techniques
Agricultural productivity– Quantity and quality– Revenue and profits
Household well-being– Consumption– Non-farm employment– Income– Poverty
Changes in these outcomes for Year 2 villages compared to Year 5 villages– Difference in means is the impact
Adoption of agricultural techniques
Agricultural productivity– Quantity and quality– Revenue and profits
Household well-being– Consumption– Non-farm employment– Income– Poverty
Changes in these outcomes for Year 2 villages compared to Year 5 villages– Difference in means is the impact
Findings will Inform Future Investments Findings will Inform Future Investments
Provides essential information when deciding whether to expand project activities
Identifies what conditions or characteristics affect program success
Provides better data for economic analysis on similar programs
Contributes lessons for all future development programs
Provides essential information when deciding whether to expand project activities
Identifies what conditions or characteristics affect program success
Provides better data for economic analysis on similar programs
Contributes lessons for all future development programs
Additional MaterialsAdditional Materials
Post-Harvest, Processing and Marketing
Post-Harvest, Processing and Marketing
Objectives: – Introduce and expand post-harvest operations that best
preserve the quality of agriculture products and add value to production
– Improve access to reliable information on market conditions and opportunities
– Ensure compliance with food safety and quality standards
Expected Results: By the end of the program 300 agribusinesses directly and 15,000 farmers indirectly will benefit from these activities.
Objectives: – Introduce and expand post-harvest operations that best
preserve the quality of agriculture products and add value to production
– Improve access to reliable information on market conditions and opportunities
– Ensure compliance with food safety and quality standards
Expected Results: By the end of the program 300 agribusinesses directly and 15,000 farmers indirectly will benefit from these activities.
Improved Access to CreditImproved Access to Credit
Objective: Support the increase of affordable, longer-term credit to Water-to-Market beneficiaries by:– Developing the capacity of credit providers to lend
efficiently in the agriculture sector – Developing the capacity of Water-to-Market
beneficiaries to access and use credit effectively
Objective: Support the increase of affordable, longer-term credit to Water-to-Market beneficiaries by:– Developing the capacity of credit providers to lend
efficiently in the agriculture sector – Developing the capacity of Water-to-Market
beneficiaries to access and use credit effectively
Training and Adoption Targets Training and Adoption Targets
On-Farm Water Management Training 60,000 farmers
Higher Value Agriculture Training 30,000 farmers
Post-harvest Enterprises Technical Assistance
300 enterprises/ 15,000 farmers
Bank Loans provided to project beneficiaries and related businesses
$8.5 million in loans
Adoption of improved farm water management
38,350 farmers
Hectares Converted to High Value 7,845 hectares
Impact EvaluationImpact Evaluation
Estimating impacts involves comparing:– Outcomes with the program– Outcomes if there were no program (counterfactual)
Counterfactual: What participants would have experienced if there were no program– True counterfactual is not directly observed
Goal of impact study is to identify a comparison group to approximate the counterfactual
Estimating impacts involves comparing:– Outcomes with the program– Outcomes if there were no program (counterfactual)
Counterfactual: What participants would have experienced if there were no program– True counterfactual is not directly observed
Goal of impact study is to identify a comparison group to approximate the counterfactual
Importance of the Counterfactual:An Illustrative Example
Importance of the Counterfactual:An Illustrative Example
Rural poverty has decreased in recent years, prior to MCA-Armenia programs
In this case, we want to see how much more poverty declined because of the program
Rural poverty has decreased in recent years, prior to MCA-Armenia programs
In this case, we want to see how much more poverty declined because of the program
True Impacts May Be Smaller Than Observed Changes
True Impacts May Be Smaller Than Observed Changes
Poverty With and Without MCA
2005 2006 2007 2008 2009 2010
Po
ve
rty
Ra
te
Impact
A
B
C