document of the world bank filedocument of the world bank. report no: icr2260 . implementation...

Document of The World Bank

Report No: ICR2260

IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-57955, TF-90807, TF-93074, TF-070672)

ON

GRANTS

IN THE AMOUNT OF US$32.76 million

TO THE

REPUBLIC OF INDONESIA

FOR A

SUPPORT FOR POOR AND DISADVANTAGED AREAS PROJECT IN ACEH AND NIAS

December 20, 2012

Social Development – Indonesia Sustainable Development Department East Asia and Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

CURRENCY EQUIVALENTS

(Exchange Rate Effective December 2012)

Currency Unit = Indonesian Rupiah (Rp) Rp 1,000 = US$0.1044

US$1.00 = Rp 9,600

FISCAL YEAR January 1 – December 31

ACAP AG APL AusAID

Anti-Corruption Action Plan Attorney General Adaptable Program Loan Australian Agency for International Development (AusAID)

Bappeda Bappenas BRA-KDP BB BCR BOS BRR

Badan Perencanaan Pembangunan Daerah (Region Development Planning Agency) Badan Perencana Pembangunan Nasional (National Development Planning Agency) Community-based Reintegration Assistance for Conflict Victims Bank Budget Benefit Cost Ratio Bantuan Operational Sekolah (School Operation Assistance) Badan Rehabilitasi dan Rekonstruksi (Reconstruction and Rehabilitation Agency for NAD and Nias)

CDD CH CPDA CPS

Community Driven Development Complaints Handling Consolidating Peaceful Development in Aceh Country Partnership Strategy

DFID DIPA DMC DO DPRD

Department for International Development Daftar Isian Pelaksanaan Anggaran (Budget-Funded Activities Proposal Lists) District Management Consultant Development Objective Dewan Perwakilan Rakyat Daerah (Regional Legislative Assembly)

EACIF EAPFM EAPPR EASHD EASFP EASID EASIS EGA EGI

World Bank Office: Jakarta Office Financial Management Unit, Jakarta Procurement Unit, Jakarta Human Development Sector Department, Jakarta Financial and Private Sector Unit, Jakarta Social Development Unit, Jakarta Sustainable Development Unit, Jakarta Economic Governance in Aceh Economic Governance Index

iii

EIRR ET

Economic Internal Rate of Return Extended Term

FGD FM Forda UKM

Focus Group Discussion Financial Management Forum Daerah Usaha Kecil dan Menengah (Small and Medium Scale Enterprises Association)

GA GAM GDS GoI

Grant Agreement Gerakan Aceh Merdeka (Free Aceh Movement ) Governance and Decentralization Survey Government of Indonesia

ICM ICR IDP IP

Implementation Completion Memorandum Implementation Completion and Results Report Internally Displaced People Implementation Progress

JSDF Japan Social Development Fund Kab. KDP KPDT KPI KPK

Kabupaten (District) Kecamatan Development Project Kementerian Pembangunan Daerah Tertinggal (Ministry for Disadvantaged Areas, MinDA) Key Performance Indicator Komisi Pemberantasan Korupsi (Corruption Eradication Commission)

LG LGSP

Local Government Local Government Support Project, USAID

M&E MCLE MDF MinDA MIS MOHA MSF MTR

Monitoring and Evaluation Mediation and Community Legal Empowerment Multi Donor Fund for Aceh and Nias Ministry for Disadvantaged Areas Management Information System Ministry of Home Affairs Multi-Stakeholder Forums Mid-term Review

NGO NMC NPV

Non-Government Organization National Management Consultants Net present value

O&M OSS

Operation and Maintenance One Stop Shop

P2DTK PAD PATTIRO PCR PDO Perda PIU PMC

Percepatan Pembangunan Daerah Tertinggal dan Khusus (Support for Poor and Disadvantaged Areas Project - SPADA) Project Appraisal Document Pusat Telaah dan Informasi Regional (Center for Regional Information and Studies) Project Completion Report Project Development Objectives Peraturan Daerah (Regional Regulations) Project Implementation Unit Provincial Management Consultant

iv

PMU PNPM PSD PSF

Project Management Unit Program Nasional Pemberdayaan Masyarakat (National Program for Community Empowerment) Private Sector Development PNPM Support Facility

QEA QSA

Quality at Entry Quality of Supervision

RIA RP RTI

Regulatory Impact Assessment Rupiah (official currency of Indonesia) Research Triangle Institute

Satker SME SP2D SPADA

Satuan Kerja (central work unit) Small and medium scale enterprise Surat Perintah Pencairan Dana (Funds Disbursement Order) Support for Poor and Disadvantaged Areas Project

TA TF TOR TPK TTL

Technical Assistance Trust Fund Term of Reference Tim Pengelola Kegiatan (District Sub-Project Management Team) Task Team Leader

U.K UNDP UPKD USAID USD

United Kingdom United Nations Development Programme Unit Pelaksana Kegiatan Dinas (Technical Agency Sub Project Implementing Unit) United States Agency for International Development United States Dollar

WLE Women's Legal Empowerment

Vice President: Pamela Cox

Country Director: Stefan G. Koeberle Sector Manager: Jan Weetjens

Project Team Leader: Natasha Hayward ICR Team Leader: Hans Antlov

v

COUNTRY

INDONESIA SUPPORT FOR POOR AND DISADVANTAGED AREAS PROJECT

IN ACEH AND NIAS

CONTENTS

Data Sheet A. Basic Information……………………………………………………………….vii B. Key Dates………………………………………………………………………..vii C. Ratings Summary………………………………………………………………..vii D. Sector and Theme Codes………………………………………………………..viii E. Bank Staff……………………………………………………………………….viii F. Results Framework Analysis……………………………………...………………ix G. Ratings of Project Performance in ISRs….……………………………………..xiv H. Restructuring………………………………………………………………….....xiv I. Disbursement Graph………………………...……………………………………xiv

1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes .......................................................................................... 15 4. Assessment of Risk to Development Outcome ......................................................... 19 5. Assessment of Bank and Recipient Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 22 7. Comments on Issues Raised by Recipient/Implementing Agencies/Partners .......... 23 Annex 1. Project Costs and Financing .......................................................................... 25 Annex 2A. Outputs by Component ............................................................................... 26 Annex 2B. Summary of Linked Trust Funds and Supporting Operations .................... 27 Annex 3. Economic and Financial Analysis ................................................................. 35 Annex 4. Grant Preparation and Implementation Support/Supervision Processes ....... 36 Annex 5. Beneficiary Survey Results ........................................................................... 38 Annex 6. Stakeholder Workshop Report and Results ................................................... 41 Annex 7. Summary of Recipient’s Project Completion Report .................................... 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 48 Annex 9. List of Supporting Documents ...................................................................... 51

MAP…………………………………………………………………………………….52

vi

A. Basic Information

Country: Indonesia Project Name: ID SPADA in Aceh and Nias

Project ID: P097605 L/C/TF Number(s): TF-57955,TF-90807,TF-93074, TF-070672

ICR Date: 02/08/2012 ICR Type: Core ICR

Lending Instrument: APL Grantee: REPUBLIC OF INDONESIA

Original Total Commitment:

USD 32.11M Disbursed Amount: USD 25.41M

Revised Amount: USD 32.76M Environmental Category: B Implementing Agencies: Ministry for Disadvantaged Areas (MinDA), Agency for Rehabilitation and Reconstruction (BRR), National Development Planning Agency (Bappenas) Cofinanciers and Other External Partners: The projects and trust funds reported on are the three projects funded by the Multi Donor Fund for Aceh and Nias (MDF): TF-57955 (the main MinDA project) and TF-90807 and TF-93074 (two smaller NGO executed trust funds) and the DFID-funded TF-070672. Since the DFID TF was mistakenly mapped against the nationwide SPADA project and only recently remapped, the system-generated figures are not identical to those in SAP and CC". B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 03/23/2006 Effectiveness: 01/02/2007 05/22/2007 Appraisal: 06/23/2006 Restructuring(s): N/A Approval: 01/25/2007 Mid-term Review: N/A 02/16/2009 Closing: 06/30/2010 12/31/2011 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Grantee Performance: Moderately Satisfactory

vii

C.2 Detailed Ratings of Bank and Recipient Performance (by ICR) Bank Ratings Recipient Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory

Quality of Supervision: Satisfactory Implementing

Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Recipient

Performance: Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) General education sector 30 30 Health 30 30 Other social services 40 40

Theme Code (as % of total Bank financing) Infrastructure services for private sector development 33 33 Other social development 67 67 E. Bank Staff

Positions At ICR At Approval Vice President: Pamela Cox James W. Adams Country Director: Stefan G. Koeberle Andrew D. Steer Sector Manager: Jan Weetjens Cyprian Fisiy Project Team Leader: Natasha Hayward Scott Guggenheim ICR Team Leader: Hans Antlov ICR Primary Author: Heinz Unger

viii

F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objectives of the project are to strengthen governance, promote growth and improve service delivery in participating provinces and participating districts, and support the recovery of affected areas through: (a) strengthening of local participation in development planning; (b) promotion of private investment and job creation; and, (c) an increase in the utilization of effective education, health and dispute resolution services. Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Improved access to cost-effective, quality infrastructure (as measured by Economic Internal Rate of Return, EIRR) through participatory planning processes

Value quantitative or Qualitative)

0 EIRR >20% EIRR 34%

Date achieved 30-Nov-2007 30-June-2010 31-Dec-2011

Comments (incl. % achievement)

23 infrastructure projects were analyzed in 2011, with results similar to past PNPM surveys. A full EIRR study was therefore not launched. All SPADA activities used participatory processes in planning and implementation.

Indicator 2 : % decrease in children aged 7-12 not attending primary school in Aceh and Nias Value quantitative or Qualitative)

in Aceh: 0 in Nias: 0

in Aceh: >5% in Nias: >11%

in Aceh: 57% In Nias: 12%



Based on secondary data from the Ministry of Education. The large increase of student enrollment is related to the substantial number of new schools built by various post-tsunami recovery projects, and cannot only be attributed to SPADA.

Indicator 3 : % decrease in children aged 13-15 not attending Junior secondary school in Aceh and in Nias

Value quantitative or Qualitative)

in Aceh: 0 in Nias: 0

in Aceh: 13% in Nias: 29%

In Aceh: 14% In Nias: 32%

Date achieved 30-Nov-2007 30-June-2010 31-Dec-2011 Comments (incl. % achievement)

Based on review from the secondary data collected by PIU MinDA from the Ministry of Education.

ix

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : % of District Grants disbursed for each project cycle Value (quantitative or Qualitative)

0 >75% 84%


After a slow start, disbursement picked up in 2009.

Indicator 2 : % District using participatory planning processes for budgeting and financing of development activities

Value (quantitative or Qualitative)

0 >80% 100%


100% of districts now use GoI's participatory planning and budgeting process (Musrenbang)

Indicator 3 : % district infrastructure classed as "satisfactory" to "excellent" Value (quantitative or Qualitative)

0 >70% 75%


The remaining 23% were rated as “good” and 2% as “unsatisfactory”, as reported in the Recipient’s PCR. The ratings were made by field teams, facilitators, consultants, MinDA staff and NMC.

Indicator 4 : % of Villages with active participation in school management committees Value (quantitative or Qualitative)

0 >40% 100%


All targeted 1,921 villages have school-based management committees

Indicator 5 : No. of village teachers that have received teacher training Value (quantitative or Qualitative)

0 >500 9,551


There are indications – based on anecdotal evidence from school principals – that graduation rates in some schools have increased significantly as a result of the teacher training program.

Indicator 6 : % of respondents satisfied with the level of education services provided through

x

the project. Value (quantitative or Qualitative)

0 >65% 91%


In 2012 all project prim. & second. schools in 7 randomly selected districts in Aceh and Nias were assessed by interviewing students, parents, school committees and teachers.

Indicator 7 : % of respondents satisfied with the level of health services provided through the project


0 >65% 87.5%


In 2012 all project health care centers in 7 randomly selected districts in Aceh and Nias were assessed by interviewing health workers and communities.

Indicator 8 : No. of district business forum established and functioning Value (quantitative or Qualitative)

0 19 6



Established in 5 districts through Economic Governance for Aceh (EGA) program. 1 district replicated EGA with local funds. Business forum not established in other 13 districts due to procurement delays and cancelled contracts.

Indicator 9 : % increase in new business registrations Value (quantitative or Qualitative)

0 >5% 94%


In the 5 districts where the EGA program was active, new business registrations increased between 2008 and 2010 by 94%. This can largely be attributed to the peaceful development of Aceh.

Indicator 10 : No. of districts adopting at least three recommendations for improvements in their regulatory environment


0 13 12


7 districts assisted by LGSP (TF-90807) adopted recommendations for health, education, transparency, planning, public services management & tech review of projects. 5 districts improved business environment through EGA program.

Indicator 11 : No. of districts where community legal advisers, sub district facilitators and village paralegals are appointed, trained and are providing legal assistance to communities

Value (quantitative

0 >3 5

xi

or Qualitative) Date achieved 30-Nov-2007 30-June-2010 31-Dec-2011


With DFID-funds (TF-91675), 50 community facilitators & paralegals were trained & provided legal education and assistance in 67 villages in 5 districts, handling 269 cases.

Indicator 12 : No. of NGO staff and formal justice officials (judges, police and prosecutors) trained to undertake judicial services and legal education


0 >70 125



Staff of religious courts, police, religious affairs, civil registries and district organizations were trained to conduct legal education campaigns. The project issued modules to increase awareness of women's legal rights.

Indicator 13 : No. of persons benefiting from legal aid assistance through the project Value (quantitative or Qualitative)

0 >500 2,300+


WLE trained paralegals in 5 districts trained to assist women's groups in 23 sub-districts & 67 villages (about 2,300 women). Formal legal support provided in 44 cases & 85 formal legal dialogues were conducted with community members.

Indicator 14 : No. of project facilitators, NGO staff, local government officials and youth trained in conflict mediation, leadership and civic education


0 >300 813



408 PNPM and SPADA facilitators, and an additional 405 village leaders including village heads and youth representatives were trained in conflict mediation, leadership, and civics by October 2009. Targets were achieved by collaborating with PNPM.

Indicator 15 : No. of vulnerable women, IDPs, and ex combatants provided technical assistance and financial support to rebuild their lives.


0 >2000 233,115


This value reflects that a range of projects – BRA-KDP-SPADA –collaborated to channel $24million in funds to 1,724 villages in Aceh after the tsunami and peace process. This was not anticipated at the time of target setting.

Indicator 16 : No. of districts that received training in proper financial management and procurement


0 19 19

xii


Training in financial management was delivered to all districts from July to August 2010.

Indicator 17 : % of Government and consultant teams at all levels recruited and functioning Value (quantitative or Qualitative)

0 >80% 95%


After a slow start, teams were largely in place from early 2008, with a total of 135 staff. Only the Private Sector Development specialists were not recruited, due to late procurement.

Indicator 18 : % of complaints resolved Value (quantitative or Qualitative)

0 >60% 100%


A total of 290 complaints cases have been finalized and settled.

Indicator 19 : Beneficiaries as % of total population. Value (quantitative or Qualitative)

0 >30% 29%


There were 1,364,561 beneficiaries in Aceh, constituting 29% of the total population of Aceh in 2011

Indicator 20: No of impact surveys completed Value (quantitative or Qualitative)

0 2 1


No base-line survey was carried out, only a final impact study.

Indicator 21: % of primary schools in the districts adopting school based management Value (quantitative or Qualitative)

0 >40% 100%


School based management is compulsory in Indonesia since 2007.

xiii

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 06/24/2008 Moderately Satisfactory Moderately Unsatisfactory 2.50

2 08/29/2008 Moderately Satisfactory Moderately Satisfactory 2.50

3 01/13/2010 Moderately Satisfactory Moderately Unsatisfactory 15.42

4 07/30/2010 Moderately Satisfactory Moderately Satisfactory 16.35 5 05/18/2011 Moderately Satisfactory Moderately Satisfactory 19.45

H. Restructuring (if any) Not Applicable

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal Summary. This ICR provides an account of the Support for Poor and Disadvantaged Areas Project (SPADA) Aceh-Nias project: a grant-funded operation designed on an emergency basis in response to the Asian tsunami of December 26, 2004. At the time of the tsunami, the Bank was already engaged in preparation of the SPADA National project, intended to target a number of disadvantaged provinces across Indonesia with a community-driven approach to local development that integrated participatory approaches into development planning at the district level. In the aftermath of the Aceh disaster, and at the request of the Government of Indonesia and the concerned local governments, the Bank team expedited preparation of a standalone SPADA operation for the province of Aceh and the island of Nias (in the province of North Sumatra), intending to complement the CDD approach in the Kecamatan Development Project by linking community demand with district planning. This expanded the geographic scope for SPADA implementation within the two disaster-affected areas and was financed by a range of grants from donors interested in more participatory and socially aware approaches to reconstruction and recovery. Background. A particularly striking aspect of Indonesian development is the regional variation. Rural provinces, such as the eastern islands, Tanah Papua, and parts of Sumatra show poverty rates exceeding 50% of the rural population, and substantial areas within the eastern islands still experience poverty and food insecurity. In addition to high poverty and the difficult geography, problems in the off-Java areas have been exacerbated by occasional outbursts of violence and civic conflict, particularly after the fall of the New Order government in 1998. Soon after this political transition, Indonesia also embarked on an ambitious decentralization program through Laws 22 and 25/1999 that became effective in 2001. This shifted many government functions from the national level to the country's 430 local governments. District heads, formerly appointed by Jakarta, were elected through direct elections, local government spending rose significantly, and development services, such as health and education, were transferred to the districts. Due to the heightened needs in disadvantaged, post-conflict and post-disaster areas - including damaged or destroyed infrastructure, disrupted essential services, large numbers of internally displaced people, and lack of services for conflict resolution - and with enduring risks of social disorder, there was a clear need for a framework and an operation that could address both poverty and governance in the newly decentralizing environment. The World Bank was already supporting community-level participatory processes through the series of Kecamatan Development Projects (KDP), a successful community-driven development (CDD) project which had made significant investments in community planning and building village assets since 1998. The Bank also recognized that community-based approaches in disadvantaged and conflict-affected regions had to be complemented with supporting the new role of district governments in arbitrating and deploying authority. Therefore, a separate project was prepared that would combine the

2

comparative advantages of a CDD approach – participatory planning and investments in communities – with support for development planning. A number of projects for local government strengthening were being developed by the Bank at the time, and there were requests from the GoI that the CDD approach used by KDP should be applied for better integration with regular government operations. This was to be achieved under the nationwide “Support for Poor and Disadvantaged Areas Project (SPADA)” (see Report No. ICR00002259), approved by the Bank in June 2005, and which started implementation later that year by the Ministry for Disadvantaged Areas (MinDA). This provided a mechanism through which participatory and conflict-sensitive planning procedures could be incorporated into government decision-making in 40 disadvantaged districts. This national project included twelve of the poorest districts of Aceh, selected based on a set of criteria established by MinDA. The tsunami and earthquake disasters. The earthquakes and tsunamis that struck Indonesia and several other countries in the Indian Ocean region on December 26, 2004, resulted in one of the worst natural disasters in recorded human history. Indonesia bore the brunt of the disaster. In Aceh and North Sumatra, the disaster left more than 230,000 people dead or missing (including 5,000 civil servants), caused an additional 700,000 to become homeless, washed away hundreds of communities, collapsed many local governments, and devastated normal social, economic and political life in the affected areas. At the time of the disaster, there was also an independence movement active in Aceh that was in serious conflict with the Indonesian government, and also affected the general population, especially in more remote communities. Following the tsunami and the destruction in its wake, a peace agreement – the Helsinki Accords – was negotiated and signed by the GoI and the Free Aceh Movement in August 2005, and the armed conflict ended, facilitating reconstruction and recovery. Bank and Government actions. At the time of the tsunami, SPADA was under preparation and it was intended that Aceh would be covered, in part, under the national program. However, following the tsunami and the subsequent establishment of the Rehabilitation and Reconstruction Agency1(BRR), the government accelerated launch preparation for the SPADA project in Aceh, and subsequently proposed that it be expanded to cover all 17 districts in Aceh, and the 2 districts in the neighboring island of Nias. The Government also asked that SPADA’s programs for post-conflict reintegration and development be substantially strengthened for Aceh by including damage and vulnerability assessments, support for local justice sector reform, and practical training to help local governments mediate conflict successfully. Rather than incorporate these additional efforts into what at the time was an un-tested nationwide SPADA project, the GOI requested the World Bank-managed Multi Donor Trust for Aceh and Nias (MDF) to provide a grant for a special free-standing SPADA project in Aceh and Nias with the aim to rebuild public services such as infrastructure, health and education, building trust in government and consolidating the peace efforts. A 2006 evaluation of SPADA by MDF

1 BRR was charged with coordinating and implementing rehabilitation and reconstruction activities in the Province of Aceh and on the island of Nias (in the Province of North Sumatra).

3

firmly established it as an appropriate and relevant mechanism to meet the post-disaster challenges in Aceh and Nias. The grant agreement for the standalone operation made specific reference to the nationwide SPADA project that was financed by a loan and credit from the Bank. SPADA’s design fitted well with BRR’s reconstruction strategy, which was tasked to be the executing agency for the new project and receive the overall budget. The project was implemented through MinDA; they provided all technical assistance and implementation support through the project, using the same structure and staffing as for the nationwide SPADA project. Bappenas was the project management unit (PMU) for both SPADA projects.

1.2 Original Project Development Objectives (PDO) and Key Indicators The objectives of the project are to strengthen governance, promote growth and improve service delivery in participating provinces and participating districts, and support the recovery of affected areas through: (a) strengthening of local participation in development planning; (b) promotion of private investment and job creation; and, (c) an increase in the utilization of effective education, health and dispute resolution services. 2 The key PDO indicators are (i) improved access to cost-effective, quality infrastructure (EIRR>20%) through participatory planning processes; (ii) a decrease in non-attendance of primary school by children aged 7 – 12, by more than 5% in Aceh, and more than 11% in Nias; and (iii) a decrease in non-attendance of junior secondary school by children aged 13 – 15, by more than 13% in Aceh and by more than 29% in Nias. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDO and key indicators were not revised. However, at the time of project Negotiations, two PDO indicators from the PAD regarding assisted births and immunization coverage were dropped, since it was felt that the project alone would not be responsible for achieving these targets, even as the health component of the project was retained, but mainly for infrastructure and training. In addition, three intermediary (minor) indicators in the PAD and GA have not been regularly reported on, although they are captured in this ICR. One indicator in the PAD and GA (“percentage of M&E and study findings used in the Project decision-making process”) was mistakenly not included in the M&E framework and cannot be reliably tracked, and, as this ICR describes, the project did not deliver comprehensively on its planned M&E studies.

1.4 Main Beneficiaries The main beneficiaries of the project were people living in Aceh and Nias that were affected by natural disasters (tsunami and earthquakes), conflict, poverty, illiteracy and

2 Note that in many documents, including ISRs, this is simplified to reference only the first half, dropping the “through” section. However, the PAD, GA and MTR are consistent in using the longer formulation. The shorter formulation is identical to that of the nationwide SPADA project with the addition of “support the recovery of affected areas”.

4

poor quality education, disease and inadequate medical facilities, ineffective local planning processes, communal conflicts, and limited, or lack of, access to justice. Additional beneficiaries were small business people who were supported by the private sector development (PSD) initiative. Also, local government staff (at the district and sub-district levels), education and health professionals, and legal staff, benefited from training and improved working conditions. Additional beneficiaries were internally displaced persons (IDPs) and ex-combatants.

1.5 Original Components The project consisted of the four components of the Government Executed grant (TF-57955), which are described below. Additional investments included the two NGO-executed trust funds (TF-90807 and TF-93074), funded by MDF and linked to the project. An additional umbrella trust fund, TF070672, financed by the United Kingdom’s Department for International Development (DFID) and administered by the World Bank supported eight additional recipient managed ‘child’ trust funds3. All these separate trust funded activities were carried out and implemented by various national and international NGOs in support of the main project objective. While not included in the PAD financing plan for the original project and while each TF had its own grant agreement, these trust funds are justifiably covered in this ICR and the assessment of the project’s PDO. Fuller details of their activities and outputs are reported under “Trust fund support” in section 2.2 and in Annex 2a. These trust funds also have GRM/ICM filed that may be further referenced. Component A: District Grants (US$ 12.76 million). The purpose of the district block grants was to support larger-scale technical proposals – emerging from the village and sub-district planning process – that contributed to reconstruction, cooperation, and development. For the first two years of the project, the maximum size of each proposal was US$50,000. The component covered all 17 districts in Aceh (but excluding two municipalities) and the two districts in Nias (in 2009 split into five). District grant amounts were set at an aggregate of about US$300,000 (equivalent) in the project’s first year. This would allow CDD operations to be linked with the newly empowered districts, especially the planning boards and technical service agencies. The main difference from the nationwide SPADA was the absence in SPADA Aceh-Nias of sub-district block grants for tertiary village infrastructure. The reason for this was that KDP/PNPM operated in all Aceh-Nias sub-districts, providing block grants; outside of Aceh-Nias, sub-districts were divided between SPADA and KDP. Component B: Capacity Development/Planning Grants (US$ 9.22 million).4 Activities financed under this component were a combination of training, practical exercises, professional technical support, and the development of learning networks. In addition, funds were provided for operating costs arising under the project on account of

3 Its legal name was “Strengthening Post-Conflict Reintegration Aspects of the Support for Poor and Disadvantaged Areas Project (SPADA) for Aceh”. 4 This sum is for the main SPADA project trust fund (TF-57955), EGA (TF-90807), LGSP (TF-93074) and seven DFID trust funds (TF-070672).

5

local contractual support staff salaries other than salaries and benefits of the Recipient’s full time employees. This was aimed to support the Recipient’s contribution towards the Project Implementation Unit at MinDA and BRR. In addition, under this component, SPADA partnered with existing donor programs, such as the United States Agency for International Development’s (USAID) Local Government Support Project (LGSP), providing support to district governments and the communities through health and education diagnostics, support for civil society forums, training of district legislators in public expenditure analysis and formulation, and improved planning and budgeting. This component supported the Aceh provincial government, particularly for monitoring progress of improving local investment climates, funded separately by TF093074, implemented by The Asia Foundation. This component also included several activities specifically related to post-conflict reintegration: an alternative dispute resolution program consisting of a program of facilitated mediation and court referrals for peaceful resolution of conflicts; and a provincial and district working group that involved representatives from the police, the Attorney General’s office, the MOHA and legal NGOs. There were also smaller pilot programs to develop community-based reconciliation activities, youth employment and other incentives to promote nonviolent dialogue; pilot work on community-based land mapping for dispute resolution was launched in the project’s second and third operational year. These activities were funded under the DFID grant (TF070672). Component C: Implementation Support (US$ 9.83 million)5. Three implementation support packages were included in this component. Firstly, technical assistance for the three different levels of government as follows: (i) a national management team, (ii) an oversight and monitoring unit in each province, and (iii) a multi-sector support team in each participating district. Linked to this management structure was a specialist procurement consultant to work with the procurement committee to manage procurement, mainly during the project’s startup period. Secondly, this component financed specialized consultants to carry out baseline inventories, investment climate surveys, and needs assessments; these were intended to deliver educational and health services and to provide training and operational support for an alternative dispute-resolution program. And lastly, the component included programs for information dissemination, including radio and public awareness programs to address issues related to reconciliation, funded through the DFID grant. Component D: Monitoring, Evaluation, and Studies (US$0.91 million).6 The core of this component was a package of quantitative and qualitative baseline and intermediate surveys and case analyses that could track impacts due to project interventions. US$0.2 million was budgeted for independent audits that would begin in the project’s second year.

5 This sum is for the main SPADA project trust fund (TF-57955) and two DFID children trust funds (TF-070672). 6 This sum is for the main SPADA project trust fund (TF-57955) and one DFID child trust fund (TF-070672).

6

1.6 Revised Components There was no revision of the project components.

1.7 Other significant changes On January 8, 2009, in anticipation of the end of BRR’s mandate on April 16, 2009 and with MDF’ agreement, the Bank approved the Ministry of Disadvantaged Areas (MinDA) as the full executing and implementing agency for this project. MinDA assumed all commitments and responsibilities of BRR. On April 8, 2010, with agreement by the MDF, the Bank approved an extension of the Grant closing date by 18 months to December 31, 2011 to allow completion of sub-project implementation. This was formalized through an amendment to the Grant Agreement.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry Project preparation. Preparations for the nationwide SPADA project begun in Aceh in June 2005 – six months after the tsunami had struck – and included mapping of the socio-economic conditions, assessment of social tensions, human resources and local government capacity. At BRR’s request and with MDF funding, the separate SPADA Aceh-Nias project extended coverage to all seventeen rural districts in Aceh and the two districts on Nias. With coverage of post-conflict, post-disaster and disadvantaged districts, this was a project of a different magnitude than the original SPADA and the intervention generated much interest among development partners to contribute, either through MDF or separately, to the project. At the time the project was designed, there was still relatively limited experience in the Bank of working in post-conflict and politically fragile areas: in fact, the preparation and implementation of SPADA – and its challenges – contributed to the development of the operational policy on post-conflict projects. Implementation was thus characterized by improvisation and “sailing while building the ship”, both by MinDA and the Bank. Furthermore, there were some naivety in terms of how the project would interact with local government planning, and a limited understanding of the challenges posed by a deeply entrenched bureaucracy and rent-seeking political economy. Neither the implementing agency nor the Bank was able to provide a strong framework for the type of local government reforms that the project intended to bring about, beyond training and increased ownership. It was therefore decided that, rather than launching an entirely new program component for local government capacity strengthening, SPADA Aceh-Nias would collaborate with the already active and successful Local Governance Support Program (LGSP) funded by USAID and executed by RTI International. The LGSP was expanded to ten districts in Aceh, and was funded by an additional US$2.53 million grant from the MDF. A third separate trust fund – the Economic Governance in Aceh project

7

(EGA) implemented by The Asia Foundation – supported the local business climate by streamlining local government business regulations and strengthening businesses’ capacity to advocate for reforms. Project design and the decision to have MinDA as project implementation unit was in line with the national SPADA project that had been approved a year earlier but was only just starting operations. Some experiences from the SPADA appraisal and early implementation was taken into account, but SPADA Aceh-Nias implementation arrangements had to be adapted, first to the post-disaster and post-conflict situation in Aceh and Nias, and secondly to the key role the BRR played during the early stages of reconstruction in Aceh and Nias. Until the expiration of BRR’s mandate, all funds flowed through BRR and they transferred block grants to local governments and TA funds to MinDA that built on the nationwide SPADA structure and staffing. Risk identification and mitigation. Project design and appraisal had identified several risks as “High” and proposed mitigation strategies and actions as follows: (i) Institutional weakness was a major risk because MinDA was newly established and lacked experience. Furthermore, as a state ministry without line agencies, MinDA was dependent on local government agencies, whose post-disaster capacity was limited. Moreover, all funds had to be channeled through BRR, and MinDA was responsible for technical assistance contracts only. GOI agreed that, should implementation performance be unsatisfactory, the project would be moved to the Ministry of Home Affairs (MOHA) - the implementing agency for the KDP projects. (ii) In 2006, peace was still fragile in Aceh after 30 years of armed conflict. Expectations for a ‘peace dividend’ were high, while government’s ability to deliver was low. To address this risk, DFID was providing SPADA with a parallel grant to specifically support post-conflict reintegration of ex-combatants in support of the PDO. (iii) There was a high risk of corruption because of the large number of ex-combatants and weak local governments who received large reconstruction and development budgets. The project was provided with extra Bank oversight and supervision from its Aceh office to ensure full implementation of the anti-corruption action plan (ACAP). (iv) Financial management was rated as high risk, particularly how grant beneficiaries would use and account for the funds, and whether the funds were used for the intended purposes. Based on KDP experience, a comprehensive action plan was drawn up to mitigate the FM risks; it included special TA to the government’s project management unit (PMU) and Project Implementing Unit (PIU) and to the local implementation teams, independent monitoring by NGOs and journalists, internal controls and external audits. (v) Some district governments in Aceh had been wiped out almost completely and were not functioning anymore having lost many of their staff, their offices and their records. They had to be re-built and developed over the medium term which entailed high risks. The main mitigation action was the support of the LGSP and EGA projects and additional technical assistance and implementation support for the affected districts.

8

2.2 Implementation Project launch. The organizational arrangements for implementation consisted of a coordination committee formed by BRR, a PMU at Bappenas and the project implementation unit (PIU) in MinDA. At the local levels, district planning committees and project management units (Satker) were established. After BRR’s mandate expired in 2009, the coordinating role was taken over by the provincial and district coordination teams hosted at the provincial planning board (Bappeda). Although most district Project Management Units (Satker), district management and technical teams had been established by November 2007 (a year after effectiveness), government processes such as procurement and disbursement were seriously delayed in the project’s initial years, including the critical appointments of the District Management Consultants (DMC) and the Provincial Management Consultants (PMC) that were only awarded in early 2009. However, while waiting for project mobilization, MinDA used loan funds from the national SPADA project (P078070) to fund project activities in Aceh and Nias, including the DMC and PMC. To further assist implementation, the technical assistance by the national SPADA National Management Consultants (NMC) was amended in 2009 to also cover technical assistance to Aceh Nias. Furthermore, having identified the pitfalls of delay fairly early on, and in order to mitigate against a vacuum in technical assistance to project implementation, in 2007, MinDA and the Bank established joint task teams based in Aceh to assist project implementation, each consisting of three consultants. The Bank also conducted procurement training for the GoI procurement committee and MinDA PIU in 2007 and 2008. In an attempt to accelerate the mobilization of consultants, the Bank also offered assistance with simplifying and streamlining procurement processes. In 2008, Crown Agents was contracted to provide procurement and FM training to Aceh district coordination teams. The initial difficulties and delays were due to a number of reasons: (i) BRR’s work overload and lack of experience with CDD projects; (ii) national-level problems with procurement due to inconsistency between World Bank and GoI procurement guidelines, (iii) late issuance of central government budget that caused poor quality planning; and (iv) difficulties with attracting qualified consultants willing to work at government-set salary rates in a setting saturated by international agencies. Progress of implementation and MTR. As a result of the early challenges described above, in June 2008 and again, in January 2010, the project’s implementation progress (IP) was rated as “moderately unsatisfactory”. Specifically, financial management (FM), project management, procurement, and M&E were rated as “unsatisfactory”. In early 2008, the Bank requested that (i) coordination between project actors be increased; (ii) the consultants’ poor performance be improved; and (iii) the procurement processes be accelerated, especially for the NMC, PMC and DMC as well as other technical assistance such as procurement and financial management, and implementation assistance for Private Sector Development and for Mediation and Community Legal Empowerment. During the same period, GOI had already taken some corrective action by holding weekly coordination meetings between Bappenas, MinDA and BRR. But procurement delays continued to impede progress throughout 2008, and supervision by consultants was

9

deemed to be ineffective due in part to poor training and insufficient remuneration and operational costs. Disbursement was only at 36 percent by 2009, and a mid-term review (MTR) was conducted in February-March 2009, jointly with the MTR for the nationwide SPADA project. Some 16 problems and corresponding actions were identified and agreed, mainly relating to procurement, financial management, and general project monitoring and management. The Bank also suggested a closing date extension to complete the 2009 sub-projects subject to good implementation progress. During the subsequent review mission in early 2010, the Bank found that – due to MinDAs’ efforts – 13 of the 15 agreed actions had been taken and project performance had improved, especially the completion of sub-projects, fiduciary control and dealing with fraud and mismanagement cases. Due to limited human resources and low capacity, implementation in the remote island of Nias posed special challenges. In 2007 and 2008, disbursement was at 30% of the block grants, and in 2009 there were no disbursements at all7. In 2010 the Bank held meetings in Medan and Jakarta in an effort to get some sub-projects implemented, but administrative and political problems persisted. There was no formal restructuring at the time of the MTR. Instead, a series of actions and changes were agreed with MinDA, including (i) extending the closing date by 18 months, (ii) formally changing the implementing agency from BRR to MinDA, and (iii) enhancing procurement capacity and financial management. There was little disruption to the project implementation progress as a result of the handover from BRR because MinDA had already been the de facto implementing agency even when BRR was still nominally leading project implementation. Implementation performance. Beginning in 2009, the project started to deliver as designed. Full block grants were disbursed in 2009, 2010 and 2011 (except in Nias, for political reasons). The main reason for this turn-around was improved support from the NMC and PIU, a strengthening of the MinDA management team, and that the capacity development provided by MinDA and the Bank Task Team to project stakeholders started to bear fruit. The effectiveness of the program depended on intensive facilitation, technical assistance and supervision. The SPADA projects had in 2010-11 a team of 20 technical specialists in Jakarta. At the district level in Aceh and Nias, SPADA had 115 professional staff members. During the second half of 2011, after the completion of sub-projects, much effort went into project documentation and closing. This included the formal hand-over of SPADA sub-projects from MinDA to respective local governments, so that O&M funds could be allocated. Project documentation and inventory were completed by December 2011, and sub-projects were formally handed over by MinDA to the respective district governments for all 18 participating districts by December 2011 for formal registration. “SPADA

7 The undisbursed portion in 2007 and 2008 was mainly due to the low number of sub-projects that were implemented in Nias, and in 2009 because of the political tensions after a split of Nias district into four new districts.

10

Maintenance Teams” have been established in all districts, and guidelines and training provided by MinDA. Although outside of the direct control of MinDA, most districts have committed themselves to provide budgets for operations and maintenance. Owing to the delays in initial mobilization, however, and despite the improvements post-MTR and extension of the closing date, USD 4.2 million (approximately 20 percent of the total grant amount) remained undisbursed and was returned to MDF in 2011 for reprogramming because: (i) block grants fell almost 20 percent short of the estimates; (ii) implementation support and technical assistance during 2007-08 was funded by the nationwide SPADA project; and, (iii) none of the monitoring and evaluation studies were implemented due to procurement problems. Trust fund support. As noted, implementation of the core project was enhanced by a number of Bank-administered TFs, as per the following table:

Source/TF No. Grant (US$ million)

Implementing agency

Components supported Activities / comments

MDF (Multi Donor Fund for Aceh and Nias) 093074 0.60 Asia Foundation Component B “Economic Governance in

Aceh”, promote local business environment

090807 2.53 Research Triangle Institute

Component B “Local Governance Support Project”, promoting LG governance, transparency and participation

DFID (Bank-administered umbrella TF070672) 090789 1.50 Asia Foundation Component B Improve local business

environment 090666 0.50 Bappenas Component D M&E capacity building for

Bappenas and KPDT 090725 1.30 Asia Foundation Component B Women's civic participation

and justice issues 091149 1.61 Int'l Organization

for Migration Component B Physical and mental health

services 091675 0.80 Relawan

Perempuan Untuk Kemanusiaan

Component B Increase women’s access to justice

094014 0.46 Forum LSM Aceh Component B Enhance the capacity of PNPM Rural and SPADA facilitators to deal with conflict emerging from projects

094912 0.17 Search for Common Ground Indonesia

Component C Enhance platform for youth and media on reconciliation and conflict

11

090761 0.80 MinDA Component C Grant cancelled, and the mediation and community legal empowerment activity implemented through Justice for the Poor Program.

Implementation and achievement of the PDO was very effectively supported and complemented by these trust funds (TFs) as follows. More details can be found in Annex 2a.

1. “Economic Governance in Aceh” (EGA), a separate grant from the MDF (TF-093074), was implemented by the Asia Foundation. The objective was to support Aceh’s post-earthquake and tsunami reconstruction and rehabilitation program by improving the local business climate, streamlining business regulations and strengthening businesses’ capacity to advocate for change. Overall, the EGA program was implemented successfully and achieved its objectives. Through the establishment of the “one stop service shop” (OSS) for business licenses, local businesses in the five participating districts benefited from better access, simplified procedures and requirements, and less time and money spent when obtaining business licenses. Most of the OSS are still in operation in 2012.

2. The nationwide “Local Governance Support Program” (LGSP) was launched by USAID in 2005 and implemented by the Research Triangle Institute (RTI). It was subsequently supported by a grant from the MDF (TF-090807) with the objective of strengthening district capacity and anchoring bottom-up planning in government institutions and civil society organizations, with a particular focus on the local ability to prepare social and infrastructure projects and deliver quality public service development programs. Through the LGSP, capacity was built in ten districts to: (i) use participatory planning processes for budgeting development activities; (ii) develop a performance-based budget; and, (iii) improve public financial management. As a result, policies and regulations for community participation in planning, decision-making and oversight were enacted in five districts, and hundreds of legislators, LG staff and civil society members were trained in various aspects of improving local governance. LGSP was an innovative partnership of donors and local partners.

3. The DFID, through TF070672, provided an US$ 9.89 million grant for parallel financing of the project, out of which US$7.16 was Recipient Executed. The objectives were: (a) strengthening the capacity-building components of SPADA Aceh-Nias to ensure the sustainability and impact of investments; (b) helping to ensure that different vulnerable groups were included; (c) addressing specific tsunami and conflict recovery needs in the dynamic war-to-peace transition context in Aceh; and, (d) providing frequent and intensive project implementation support by Bank staff. Overall, implementation of most grant activities, particularly those funding the supporting activities (access to justice, PSD strengthening, support for vulnerable groups), was satisfactory, and was rated as

12

such by DFID. Targets were achieved or exceeded, and the project generated ten relevant analytical studies and evaluations. The Bank provided adequate and timely support to local governments and other implementing partners in the conduct of Bank-executed activities, and fulfilled tasks and responsibilities related to overall program secretariat, supervision and M&E.

While the range of complementary trust funds had undeniable benefits and positive impacts, as illustrated above, arguably, owing to the considerable number of trust funds that supported the PDO and funded extra initiatives, it was difficult to coordinate and integrate all these activities, especially at the national level. Opportunities were missed to take full advantage of the various trust funds and achieve greater synergy and benefits by linking the EGA, LGSP and DFID project closer to the main SPADA project, and more broadly incorporating the achievements in all districts. For future projects adopting such a model, the PMU should take on this coordination role.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Design. Through the DFID TF, the PMU (Bappenas) was assisted to set up an M&E Framework and Manual for project implementation. This framework was designed to be conducted through a series of exercises, including community participatory monitoring, regular field supervision and monthly progress reporting through MIS, independent monitoring by civil society groups, qualitative and quantitative household surveys, technical/sectoral studies (EIRR, facility and user surveys), private sector development assessment tools, conflict incidence mapping, vulnerable population impact assessments, training tracer impact assessments, and financial audits.

Implementation. Although the concept for the M&E design in the PAD was ambitiously comprehensive, it appears that neither GOI agencies nor the Bank paid sufficient attention to the overall execution of the framework. Most seriously, no base-line study was commissioned – the rapid roll out of the project in post-disaster Aceh was deemed a priority. The MIS for the entire SPADA program, including the SPADA Aceh-Nias, had serious problems, and “availability and accuracy of data” was rated at 5% during 2007/2008 by Bank staff. By mid-2008 Bank supervision still reported that there was no MIS standard on reporting, and procurement delays and poor quality technical assistance were blamed for the MIS problems. Reasons include inadequate resources at the field level for data collection, unreliable MIS hardware and software (including lack of internet connection), and insufficient resources at MinDA for procurement and oversight. To somewhat compensate for the lack of analytical studies, the Task Team commissioned in 2010 an independent review of the development planning aspects of the SPADA program, including in two districts in Aceh.8 Towards the end of the project, the PIU, in collaboration with the Bank, decided to collect data on levels of school and health center utilization and to determine the economic rate of return of sub-projects. A full independent project impact study was finally carried out by MinDA in October-November 2012 (see Annex 5), which allowed KPI data to be collected. The PMU at

8 PATTIRO –World Bank/PNPM Support Facility: Lesson Learned Study on SPADA Planning and Service Delivery. 2011.

13

Bappenas carried out an evaluation of the project using the DFID trust fund resources and lesson learned workshops with stakeholders were held in 2011 in Jakarta and Medan. The DFID trust funds also generated ten relevant analytical studies and evaluations on various aspects of post-disaster governance and conflict mitigation. Together, these attempts from 2010 onwards allowed sufficient data to be collected to track indicators in the project results framework. As a result, it is possible to show the achievement of the PDO using the KPIs. However, this was achieved not through the M&E component designed into the project per se but through actions supported by associated trust funds.

Utilization. MinDA’s monitoring deficiencies adversely affected the availability of progress reports from MinDA to the Bank. Annual “clinics” to improve the MIS were organized by the Bank between 2007 and 2010, and there were also special weekly clinics held in 2010 and 2011. The MIS reached only 30% of data availability and accuracy in 2009, but had improved to more than 85% by project closing in 2011. The utilization of the MIS was limited until the later stages of the project when data availability and accuracy had improved. During project hand-over in 2011, training was provided to local governments in the utilization of the MIS, and the full data set was handed over to respective district governments (Bappeda).

2.4 Safeguard and Fiduciary Compliance Environmental Safeguards. Project appraisal had concluded that the small-scale construction or rehabilitation sub-projects financed by SPADA would have minor environmental impacts only. This assumption was based on the experience and review of earlier CDD projects, particularly the KDP series of projects. Similar to these projects, environmental impacts were to be mitigated and managed through a checklist approach, supplemented by a negative list of known, potentially damaging, types of sub-projects. Implementation of these mitigation measures was to be the responsibility of technical facilitators at the district level, and relevant training was to be provided. However, there are limited records of any monitoring that may have been done, and there are also no reports of any environmental problems or complaints arising during implementation. Environmental supervision and monitoring began rather late for the SPADA projects, and there was only one training event for environmental safeguards, held by the Bank in 2011. Social Safeguards. The PAD had proposed to use guidelines for land acquisition – the major potential social issue in CDD projects – as had been used, and refined over the years by the KDP projects. These guidelines were incorporated into the project operating manual: two methods were allowed and well socialized, i.e. voluntary land donation, or donation with compensation, and consultation was required, and also a record of donations and associated agreements. However, no records are available of any systematic monitoring of land acquisition. There were some complaints through the complaints handling system related to land acquisition but they are reported as resolved. Appraisal had concluded that there were no indigenous people (IP) in Aceh and the entire population of Nias island was considered to be indigenous, thus there were no IP issues to be addressed in the project.

14

Procurement. Procurement performance was initially very poor with only a few packages finalized according to the procurement schedule, and many delayed by up to a year and frequently with more than two amendments. The packages for procurement and FM support were contracted more than two years behind schedule, and subsequently had to be procured again because the selected firm was debarred. And five packages (including some monitoring/evaluation activities) were dropped altogether because, owing to delay, they were either not relevant to project activities anymore or the MTR had recommended their cancellation. The causes for the poor procurement performance were: (i) the complexity of the procedures; (ii) the lack of experience and technical skills of MinDA staff who were new to World Bank processes; and (iii) insufficient capacity (although the Bank provided procurement training in 2007 and 2008, and MinDA in 2010). The procurement delays and cancellations of mainly technical assistance packages had adverse impacts on the progress and quality of project implementation. Mitigation actions by the Bank only started to take effect in late 2008, when disbursement figures began to normalize. By 2010, procurement performance had improved sufficiently to give the project a rating of “Moderately Satisfactory”. Financial Management, including Audits and Complaints Handling. Financial management (FM) had been identified as high risk and mitigation measures had been proposed in the PAD, but the performance of FM throughout much of the project implementation was poor. By November 2007, MinDA had not submitted any financial monitoring reports since the grant became effective. The Bank urged that an individual FM specialist consultant be appointed to assist on all FM aspects of the project. FM training at the community level was either inadequate or not done. Late procurement and the low quality of technical assistance contributed to the poor performance of FM, and poor FM was one of the causes of the moderately unsatisfactory rating of the project in 2008. In early 2009, the Bank again recommended to address FM problems with additional training and technical assistance. Funds were allocated for independent project audits, but not used because MinDA preferred to use GoI auditors. Audit findings related mainly to the interpretation of tax liability by local governments. The complaints handling system – modeled on the system used by the KDP projects – was the core of the anti-corruption plan and did demonstrate effectiveness in reducing fraud and applying sanctions, such as dismissal of staff and / or legal action, especially if misused funds are not returned. Overall, the rate of follow-up and resolution for complaints was high and reached almost 100% by grant closing. It is unfortunate that this system – used in a number of CDD projects over the years – is solely project-based and not established within government structures9. By mid-2011, with additional attention from the Bank task team and responsiveness by MinDA (including appointment of key additional TA),

9 In this context it is important to note that in 2011 the Bank, through the PSF office, has initiated discussions with government with aim of working with the national Corruption Eradication Commission (KPK) to further reduce fraud and misuse of funds in all CDD projects in the Bank’s Indonesia portfolio. The National Anti-Corruption Commission (KPK) and the National Steering Team for PNPM (Pokja Pengendali PNPM) is conducting a media campaign to build awareness and socialization, including positive lessons from SPADA.

15

financial performance improved sufficiently to give the project a FM rating of “Moderately Satisfactory”.

2.5 Post-completion Operation/Next Phase In collaboration with local governments project documentation and inventory of physical assets were completed by December 2011, and infrastructure projects were formally handed over by the Minister of Disadvantaged Areas to the respective local governments in early 2012. All districts committed themselves to provide O&M budgets in 2012-13 after project closure. Training in infrastructure maintenance was provided to local government units and community groups, and a manual on O&M distributed to all project stakeholders. Based on what in the end was a well operating MIS system. MinDA has also committed itself to incorporate SPADA’s MIS system in their regular tracking of disadvantaged areas. A free-standing project supported through the national budget by MinDA (“Bedah Desa”) is using similar approaches to SPADA by providing block grants to communities to meet their basic development needs. Administrative systems are also based on SPADA. Finally, reforms achieved through LGSP and EGA that led to new policies and regulations (for participatory planning, performance-based budgeting, and improved public financial management) in ten districts are likely to be sustained, and local businesses in the target districts are expected to continue to benefit better access, simplified procedures and the “one stop shop” when obtaining business licenses.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation The project’s objectives were highly relevant to Indonesia’s priorities in the critical period after the tsunami and earthquakes in Aceh and Nias in 2004, especially since they focused not only on the recovery of disaster-affected areas, but also aimed to strengthen governance, promote growth and improve service delivery through the local district governments. This was highly relevant in both regions because they had been neglected developmentally, Aceh because of the armed conflict, and Nias due to its remoteness. The MTR in February 2009 reconfirmed the relevance of the project’s objectives and design while attempting to improve implementation performance. In addition to the broadly applicable objectives and in view of the post-conflict situation in Aceh, the project also sought to increase the use of dispute resolution services – highly relevant in Aceh where a fragile peace had been attained in 2005. The CDD approach to investing in community assets and improving participatory planning was also highly relevant, and provided much-needed community infrastructure and support for rural livelihoods, while building trust in government processes. SPADA showed that communities, given resources and support, can meet a range of their own development needs even within sectors such as basic health care and education. However, this needs also to be matched with supply-side responses, such as done by the project at its best: linking up with local government sectoral agencies and getting the commitment of local legislative councils to improve budget allocations.

16

The project design and implementation was also highly relevant since they focused on several key areas of the Bank’s 2009 – 2012 Country Partnership Strategy (CPS) for Indonesia, namely (i) community development, (ii) sub-national government institutions and systems, and (iii) infrastructure. In fact, “the centerpiece of Bank engagement has been a long-term relationship providing knowledge and financing services to the PNPM program. Institution building is a fundamental part of Bank support for PNPM, with a wide range of activities to strengthen accountability and enhance capacity at the community level.”10 As SPADA was part of the PNPM Mandiri projectsy, it was fully relevant to the CPS.

3.2 Achievement of Project Development Objectives The project has largely achieved its PDO as per the following:

a. Local-level governance was strengthened through the project’s contribution to the rebuilding and strengthening of district governments that had been disrupted by the tsunami and decades of armed conflict. 100 percent of districts have school committees and use the Musrenbang participatory planning process. The participatory planning process allowed community groups to identify their real needs and enabled local governments to accommodate these priorities. Specifically the local planning boards (Bappeda) and the three technical agencies responsible for project activities (Public Works, Education and Health) were strengthened by the project. All participating districts report use of participatory planning processes and the concept of participatory “technical review teams”.

b. Growth was improved as measured by an increase of 90 percent in business registration in the target districts. Through the establishment of one stop service shops (OSS) for business licenses, local businesses benefited from better access, simplified procedures and requirements, and less time and money spent when obtaining business licenses. The one stop shops are still in operation in 2012. The establishment of “Forum Daerah UKM” (district forum for small and medium scale enterprises) in the target districts enhanced the capacity of the SMEs to organize themselves, have a better policy dialogue with local authorities, and advocate for local economic governance reform. Several local regulations were revised to reduce the costs of doing business for SMEs.

c. Access to basic public services was improved. 91 percent of respondents reported satisfaction with education services provided by the project; 89 percent with health services; and 98 percent classified infrastructure sub-projects as “good” or “satisfactory”. Almost 10,000 teachers and 4,500 health workers received training, and primary school enrollment rates increased drastically, although the project team acknowledges that, for this last indicator, SPADA was one contributor amongst others.

10 Verbatim quote from the Indonesia CPS Progress Report 2009 – 2012, dated February 8, 2011.

17

d. Post-disaster and post-conflict recovery, improved justice and community cohesion was promoted through training of community legal advisors in five districts (the target was three) and more than 2,300 persons were provided direct legal assistance through the project. An estimated 230,000 vulnerable women, internally displaced persons (IDP) and ex-combatants were provided financial support and technical assistance to rebuild their lives.

SPADA was one of many development projects active in Aceh in the aftermath of the 2004 disaster, and therefore cannot claim sole responsibility for some achievements relating to more ‘upstream’ outcome areas, where broader exogenous factors were also at play. Nonetheless, the project benefitted 29% of the total population in the participating districts (the target was 30%), where better access to water, markets, and education, and improvement in the health of young children and mothers is reported.

3.3 Efficiency In December 2011 and June 2012, the SPADA PIU conducted an assessment of a small selected sample 11 of typical infrastructure sub-projects to determine the actual EIRR. Results indicate that average EIRRs for SPADA Aceh-Nias projects are approximately 31.44%, with higher values for roads, bridges and clean water projects, and lower values for irrigation/drainage and public toilet projects. These EIRR levels are roughly commensurate - when adjusted to similar time horizons - with EIRR levels calculated for similar subprojects completed by the PNPM Rural program in seven provinces in 2004 and 2012. Similar patterns were also found in these previous PNPM-Rural studies, where EIRR levels were higher for road/bridge and clean water projects, and slightly lower for irrigation/drainage projects. Although based on a relatively small sample, the SPADA Aceh-Nias results suggest substantive net economic benefits, well above the targeted values. Greater efficiency was also achieved through the use of village labor and construction management in villages compared to similar, contractor-built infrastructure. Although no specific study was made in SPADA, evaluations of KPD/PNPM had shown that, on average, community-implemented infrastructure projects cost 30 to 50% less than those executed by contractors.

3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory The project was highly relevant in all regards for the post-disaster and post-conflict situation in Aceh and Nias, and was also fully relevant to the Indonesia Country Partnership Strategy, as noted above. Results demonstrate that the physical investments achieved a high degree of efficiency, and that SPADA Aceh-Nias met all three PDO indicator targets and 18 out of 21 intermediate outcome indicators. Given the broad range of assistance provided to Aceh after the tsunami and peace process, including sectoral initiatives, SPADA alone may not have been the sole contributor to some of the PDO indicators. Therefore, while the figures indicated are accurate, they are not derived

11 15 sub-projects – including road/bridge, water supply and irrigation – were selected; they represent 3.5% of the total number of sub-projects and about 5.2% of the total amount spent on infrastructure. The sample included sub-projects from six districts, five in Aceh and one in Nias.

18

from methodologies for which attribution can be statistically verified or judged to be as a result of the SPADA intervention alone. Given this, the project outcome rating is rated as MS.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Although not a formal objective, nor measured directly in its narrowest income dimension, the project is likely to have significant poverty impacts as it was targeted to poor, disadvantaged and post-disaster areas, all with high rates of poverty incidence. SPADA Aceh-Nias improved local infrastructure and services, the quality of basic health and education, and also promoted small business growth and thereby employment creation. Women’s participation in planning and decision meetings averaged 30%. The BRR activities to assist conflict victims (including ex-combatants) that were supported by a SPADA-linked trust fund, contributed towards improving community cohesion. SPADA was also found to increase the variation of community proposals, adding other community concerns to the ‘general CDD project pattern’ of village roads and bridges.12 (b) Institutional Change/Strengthening In addition to the achievements listed under Section 3.2, institutional change and strengthening happened at the district level in: (i) those ten districts that benefited from the LGSP and improved participatory planning, assets-based budgeting and financial management; and, (ii) the five districts that participated in the EGA program for simplifying the cumbersome registration procedures for small businesses, such as the one-stop-shop that are still operating. At grant closing, MinDA projected that the new policies will likely be sustained and that the skills and capacities acquired will assure longer term benefits from these reforms. This was also confirmed by joint missions to Aceh in December 2011 and August 2012. (c) Other Unintended Outcomes and Impacts (positive or negative) Few, if any, unintended outcomes were identified, but feedback in stakeholder sessions would suggest that a project like SPADA raises expectations of a next phase with another round of grants to satisfy unmet needs of poor and disadvantaged communities. The PATTIRO study also found that SPADA and PNPM programs with their assured funding are having a ‘substitution effect’ with regard to community participation in the regular planning processes at the local level; this is likely due to the minimum level of accommodation of community proposals in the district budget. This is one of the reasons why in PNPM and future CDD projects, the new GoI policy of “One Village, One Plan” will be adhered to.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops In November 2011 and again in February 2012, workshops attended by project staff, government officials and community representatives were held in Medan and Banda Aceh respectively, to draw lessons from project implementation. Feedback included:

12 Lessons Learned from SPADA Planning and Service Delivery. PATTIRO – PSF / World Bank. October 2011

19

- Confirmation of the need to continue funding CDD projects, since villages do not

have enough funds to cover their development needs; - Activities funded through the block grants and planning grants were highly relevant,

especially rural infrastructure and capacity building of teachers and health workers; - Provincial and district planning boards (Bappeda) were effective in their coordination

role with other technical agencies; - Late issuance of central government budget (DIPA) caused poor quality planning and

implementation of subprojects in the short time that was available; - Different schedules of SPADA and regular planning processes led to a lack of

synergy between project activities and regular government programs; - There was very limited community involvement in monitoring project activities, but

good involvement in planning and implementation; - Low quality of consultants/facilitators resulted in inadequate supervision - the latter

was due to insufficient budget allocation for consultants in a very competitive employment environment.

A more detailed report on these workshops can be found in Annex 6.

4. Assessment of Risk to Development Outcome Rating: Moderate The public infrastructure funded under the project is likely to be sustained. Sub-projects were designed and built in a simple and cost effective manner, and are reported as easy and affordable to operate and maintain. Sub-projects were selected by the involved communities, increasing stakeholder commitment to future operations. There are however uncertainties – despite a number of projects and initiatives producing economic growth – whether Aceh’s economy will continue to flourish. Lastly, even though the latest provincial election in April 2012 was peaceful, there is a risk that increased levels of violence and political instability could affect governance and growth outcomes, even at the local level.

5. Assessment of Bank and Recipient Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory - The Bank responded quickly to the post-disaster and post-conflict, and adapted the

design of a relevant emerging project - The Bank was successful in mobilizing a large number of grants through multi- and

bilateral trust funds that supported the PDOs - Due to the emergency situation, haste in preparation and multiple stakeholder

demands, the project, as designed, was too ambitious and attempted to cover too

20

many diverse activities and locations, including some difficult pilots such as improving access to justice, and support for conflict resolution

- There were too many procurement packages (the splitting of large procurements into more manageable packages was done as a mitigation effort in light of the low capacity of MinDA; however this also led to a work overload)

- There were too many performance indicators some of which were difficult to measure, those where output targets were set too low, or where no baseline was available or properly planned. Furthermore, there were differences between some of the intermediary indicators listed in the PAD, the GA and the M&E framework developed by the project

- Implementation was expected to rely on two new and inexperienced government agencies (BRR and MinDA), with the risks of this not well addressed

The Bank took on a project under very difficult circumstances and although the high risks had been identified, not all of them could be effectively mitigated, with the expedited timeline leading to some shortcomings in implementation and institutional readiness and inconsistencies in the project documentation.. (b) Quality of Supervision Rating: Satisfactory - Supervision was frequent and intensive and attempted to address the considerable

number of implementation problems, funded through Bank Executed Trust Funds. Findings and recommendations were discussed on a regular basis with GOI’s project team

- Despite the slow and difficult launch of the project, Bank staff persisted in providing constructive implementation support under at times difficult conditions in the post-disaster areas. Assignment of special Task Team ‘representatives’ in the Bank Office in Banda Aceh helped the project to improve its performance

- Advice and reporting by Bank staff (e.g. through back-to-office reports, aide memoires and management letters) was good throughout implementation

- The availability of a number of trust funds was well used for doing much more intense supervision and implementation support. Collaboration with other units (e.g. PREM, OSU, PSF’s Field Team) in the missions, supported the task team to comprehensively analyze findings and develop recommendations

- Bank staff worked assiduously on the problem areas, where tensions existed between GoI and Bank guidelines, in FM, procurement (especially for mobilizing consultants in the field) and poor capacity, but only towards the end of the project, with new leadership in MinDA, some success was achieved in overcoming the impediments to efficient procurement, good FM and a well-functioning MIS

- Regular coordination meetings with GOI to monitor and discuss progress of the agreed follow-up actions allowed the Task Team and GOI to appropriately identify further actions required

21

- Bank staff may have missed an opportunity at the MTR to decisively improve implementation and, in particular, to galvanize attention to the weak execution of the M&E framework

- There was no reporting – and likely no specific monitoring – of the implementation of the safeguards mitigation measures set out in the PAD (Note: There were no indications that any serious safeguard problems arose during implementation).

Despite the initial problems and procurement delays, the Bank’s supervision efforts and implementation support helped to slowly improve performance, especially after the MTR. It significantly improved the way the task team and GOI’s project team worked together in addressing systematic project issues (e.g. financial management, procurement, monitoring and reporting, and project sustainability) and in managing project risks. Based on these considerations, the rating is Satisfactory. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The rating for overall performance is the aggregate of (a) and (b) above.

5.2 Recipient Performance (a) Government Performance Rating: Moderately Satisfactory - GoI was quick to agree with the MDF and the Bank on adapting the SPADA project

to the urgent needs in Aceh and Nias - The appointment of the BRR assured efficient channeling of funds to the local

governments during the early part of project implementation - Bappenas/PMU provided strong guidance and quality enhancement support to

MinDA - GoI provided budget resources for financing the block grants and administrative

support - Release of the central government funds (DIPA) was consistently late with adverse

effects on project implementation at the sub-district levels - A sizable portion of the grant, i.e. about 20%, was returned to the MDF and re-

programmed because the project was unable to fully utilize them even though an 18-month extension was agreed

- Even though having significant “teething problems”, MinDA and the various levels of management consultants gained experience and capacity, and the project disbursed fully in 2009, 2010 and 2011.

Although there were delays early during implementation, GoI efforts and performance were sufficient to overcome the numerous initial difficulties. Based on this consideration, the rating is Moderately Satisfactory. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory

22

- MinDA, initially as the co-implementing agency (until BRR’s mandate expired in

April 2009), succeeded to implement the project, and a total of 1,770 infrastructure sub-projects and training activities were completed in 19 districts, some of which are very remote

- Key outcomes in education and health system improvements were achieved, and some 781 essential infrastructure sub-projects were built

- The capacity of the provincial planning board (Bappeda) in Aceh was improved substantially in terms of overall planning and coordination of district level agencies

- Women and vulnerable groups were reported to have participated in the planning and implementation of the project at the community level

- The complaints handling system was generally effective in resolving fraud cases - After the MTR, MinDA managed to improve implementation performance

sufficiently to meet the Bank’s pre-conditions for an extension of the closing date - Due to their lack of experience and capacity MinDA staff were not able to procure

consultants services in a timely manner; many packages were delayed - 30% by more than a year

- Procurement, FM and the MIS improved during the latter two years of the project - Only limited M&E studies were carried out during project implementation, and no

base-line study.

Similar to the rating of the Recipient performance, the implementing agency improved their performance throughout the implementation period, and especially following the MTR, so that a rating of Moderately Satisfactory is considered appropriate. (c) Justification of Rating for Overall Recipient Performance Rating: Moderately Satisfactory The rating for overall Recipient performance is the aggregate of (a) and (b) above. After a slow start with significant “teething problems”, project performance improved during project implementation to warrant a “moderately satisfactory” rating over the full period of the project.

6. Lessons Learned Adapting a project for post-disaster operation. Following a major natural disaster, such as the 2004 tsunami in Aceh, it is possible to adapt an approach and implementation arrangements of a previously designed project, facilitating quick project start-up to provide urgently needed assistance. It can also attract major sources of multi- and bilateral grant funding to support the project objectives. However, it is crucial to carefully draw lessons from the existing project before emulating: challenges arose with SPADA Aceh-Nias since the nationwide SPADA was too new to fully provide lessons learned, and the end result was an overly complex project with too many components and indicators. The new Bank operational policy that has been drafted recently for work in fragile states and post-conflict settings will be useful for future operations.

23

Working with a new ministry as implementing agency. Project design and appraisal had identified that institutional weakness of MinDA as a major risk because it was newly established, lacked experience, without line agencies at the local level, and dependent on local agencies whose post-disaster capacity was limited. A newly created Ministry, appointed as the implementing agency of a newly designed and complex project in a post-disaster area for which they actually have neither a mandate nor the expertise, will take considerable time to get organized and up to speed with project implementation. When dealing with a new ministry or implementing agency, additional training, special consultants and associated implementation support must be provided in the project and by the Bank’s task team. With the intense support provided by the Bank – only possible through additional trust funds - MinDA eventually managed to provide quality project delivery. Trust fund support. Implementation of a complex project dealing with low-capacity stakeholders cannot rely solely on Bank budget for effective implementation support. SPADA Aceh-Nias would likely have been a failed project without the additional support afforded by trust funds. These grants allowed frequent and comprehensive implementation support involving specialist resources. However, when using a large number of trust funds, coordination and integration of trust-funded activities is difficult to achieve, especially at the national level. Also, government ownership in SPADA turned out to be low since the trust-funded activities were implemented by third parties without clear links and reporting channels to government agencies. The project missed opportunities to take full advantage of the various trust funds and achieve greater synergy and benefits by linking the EGA, LGSP and DFID project closer to the main SPADA core activities, and more broadly incorporating the achievements in all districts.

7. Comments on Issues Raised by Recipient/Implementing Agencies/Donors (a) Recipient/Implementing agencies Both MinDA (PIU) and Bappenas (PMU) have in their written comments to the draft ICR provided broad agreement with the ICR. They have reported that the main issues during implementation were: (i) Aide Memoire follow-up was not always done and updated; (ii) several activities were planned by the SPADA task team using local budget financing but were not properly communicated right from the start; (iii) these activities were not allocated in the DIPA (on-budget) which made implementation difficult; (iii) the procurement committee lacked support for procurement processes as advice was based on Bank guidelines and did not take GoI guidelines into consideration; (iv) delays in project effectiveness reduced initial performance; and (v) advice was provided by the task team without reference to the state administration process - this created difficulties with implementation. These comments reflect the tension that existed throughout the project between the Bank’s fiduciary requirements and the reliance among local governments on country systems. In future projects, it is recommended that clarifications and capacity building to local governments is done early in the implementation stage.

(b) Cofinanciers/Donors

24

Comments were received from the Multi Donor Fund for Aceh and Nias (see Annex 8) that have been incorporated into the present version. (c) Other partners and stakeholders N/A

25

Annex 1. Project Costs and Financing

(a) Project Cost for the MinDA project by Component (TF057955) (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

1. District Block Grants 12.76 10.71 83.97 2. Planning Grants 0.41 0.31 76.54 3. Implementation Support 8.90 6.18 69.07 4. Monitoring and Evaluation 0.41 0.00 0.00

Total Baseline Cost 22.48 17.20 89.10 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00

Total Project Costs 22.48 17.20 89.10 Project Preparation Costs 0.00 0.00 .00 0.00 0.00 .00

Total Financing Required 22.48 17.20 89.10

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Government 14.60 15.87 108.6 Trust Funds N/A 0.00 0.00 0.0 Special Financing (MDF) Grant 25.60 19.72 77.0 Sub-Totals 40.20 35.59 88.53 Special Financing (DFID) Grant 9.68 8.41 86.88 Totals incl. DFID TF 49.88 44.00 88.21 Note: The financing included GOI reported contributions and might not be identical to SAP/CC generated figures.

26

Annex 2A. Outputs by Component All data on project outputs have been drawn from MinDA’s draft completion report as recorded in the project MIS. Component 1 – District Grants Types and details of outputs The main project outputs were delivered under this component. They span three sectors, i.e. education, health and infrastructure, and include physical, as well as non-physical outputs, as detailed further below in Table 2.1:

Table 2.1: Types of Sub-projects and activities SECTOR

Aceh

% Nias %

Type of activities

EDUCATION Quality improvements 59 42 Teacher training Provision of access 37 58 Rehab schools, equipment, furniture Management 4 - Training for school-based management HEALTH Quality improvements 73 Training of health staff (incl. midwives) Extension of coverage 27 Rehab clinics, equipment, transport INFRASTRUCTURE Roads 49 Bridges 11 Drainage works 25 Irrigation systems 3 Clean water systems 12 The detailed outputs of the various sub-components are presented in the following table 2.2:

Table 2.2: Details types of sub-projects

TYPE OF SUB-PROJECT / ACTIVITY

Number of Activities / Persons

EDUCATION Rehabilitation of schools 125 Purchase of books, tools, furniture and equipment 185 Scholarships 10 School-based management training 75 Other training 125 SUB-TOTAL EDUCATION ACTIVITIES 520

27

HEALTH Rehabilitation of clinics 29 Training of health staff (nurses, assistants, midwives) 301 Nutrition program, health information, books 139 SUB-TOTAL HEALTH ACTIVITIES 469 In sum, almost 1,000 activities were implemented that focused on improving the overall quality of education and health services in response to community demand in Aceh and Nias. In total, 9,551 teachers and 4,891 health cadres were trained by the project. INFRASTRUCTURE Transport (roads (97 km) and bridges) 431 Sanitation 88 Drainage works 108 Irrigation systems 23 Clean water supply systems 46 Other (electricity, moorings, function room, etc) 85 SUB-TOTAL INFRASTRUCTURE ACTIVITIES 781 A total of 1770 sub-projects were completed in Aceh and Nias by SPADA. By opening up and improving physical access, especially to more remote areas, they also contributed to the promotion of private investment and job creation. Quality of outputs During the project, the completed infrastructure sub-projects and training activities were assessed for quality, and 98% of all outputs were rated as “good” or “satisfactory”. The ratings of sub-project quality are based on inspections by a local certification team that involved the communities concerned. In terms of outcomes for Component A, 1,770 activities were implemented that focused on improving the overall quality of education and health services in response to community demand in Aceh and Nias. Component 2 – Planning Grants The outputs under this component were mainly planning activities to support the utilization of block grants. Total planning grant utilized in the main SPADA project (TF-057955) was US$ 310,006 (77% of grant allocated) since according to the Recipient’s PCR the project used national budget funds during the early years of project implementation. Total recipient’s contribution for this allocation was US$ 473,571 (60% of total planning grant). The planning activities supported by this grant included district technical review meetings, decision meeting forums, and capacity building to support the planning activities. Component 3 – Implementation Support This component financed consultants at the national, provincial and district levels. In Aceh, due to large numbers of participating districts, the number of provincial consultants was adjusted to support the responsibilities of PIU- MinDA. The district level consultants worked as facilitators and liaison with KDP to collect the still unfunded KDP sub-

28

districts needs and proposals. Below is the number of consultants that supported SPADA Aceh Nias. Note that some consultant services for activities such as private sector development, mediation and community legal empowerment and conflict monitoring, were cancelled due to late procurement process.

Table 2.3: Mobilization of Consultants and Facilitators

Type of consultants Planned Actual National Management Consultants (NMC) 14 20 Provincial Management Consultants (PMC) 10 8 District Management Consultants (DMC) 57 57 Procurement and FM Consultants 42 50 Private Sector Development 8 0 TOTALS 131 135

Component 4 – Monitoring, Evaluation, and Studies This component was not implemented. The Recipient utilized local budget resources to support the monitoring and evaluation studies. With support from DFID’s grant, the PMU-Bappenas prepared the monitoring and evaluation framework and manual. In late 2009, they also conducted an analysis of a small sample of health and education sub-projects to assessment stakeholders’ (see Annex 2b on summary of linked trust funds for detailed information). Prior to project closing in December 2012, MinDA committed to conduct a rapid assessment of EIRR, health and education indicators in six districts of Aceh and one district in Nias in order to complement the post-completion output study that was scheduled for October-November 2012; it was financed by the nationwide SPADA project.

29

ANNEX 2B: Summary of Linked Trust Funds and Supporting Operations No Trust Fund #

(Donor) Disbursed in US$ equiv.

Name of Funded Activity

Effective Date Closing date

I. TF-093074 (MDF)

600,000 Aceh Economic Governance (EGA)

04/02/2009 31/12/2010

The objective of this project, implemented by the Asia Foundation, was to support Indonesia’s post-earthquake and tsunami reconstruction and rehabilitation program by improving the local business climate in selected districts in Aceh by streamlining business regulations and strengthening businesses’ capacity to advocate for change. The project comprised four components: (i) assessment of economic governance and regulatory environment, or the “Economic Governance Index (EGI) Component”, implemented in all 23 local governments in Aceh, (ii) streamlined business licensing and registration, or the “One Stop Shop (OSS) Component”, piloted in five districts in Aceh, (iii) improved district regulatory review process, or the “Regulatory Impact Assessment (RIA) Component”, and (iv) improved advocacy through small and medium enterprise (SME) networks, or the “SME Component”. Outcomes: Overall, the EGA program was implemented successfully and achieved its objectives. Through the establishment of the OSS, local businesses in the five participating districts benefited from better access, simplified procedures and requirements, and less time and money spent when obtaining business licenses. The introduction of the RIA method improved the capacity of the LGs (both the executive and the legislative) to prepare and review local regulations. Based on the RIA results, several local regulations were revised to reduce the costs of doing business, particularly for SMEs. And the establishment of “Forum Daerah UKM” (district forum for SMEs) in five districts enhanced the capacity of the SMEs to organize themselves, have a better policy dialogue with the LGs, and advocate for local economic governance reform. Note: EGA was co-funded by a US$1.5 million DFID grant (TF-090789) linked to the nationwide SPADA, with the same objectives and outcomes; US$1,380,542 were disbursed when that TF closed on 31/12/2009. II. TF-090807

(MDF) 1,949,782 Additional Financing

for Support for the Poor

26/09/2007 31/12/2009

This project extended the support that USAID had been providing under the nationwide Local Government Support Program (LGSP) to five post-disaster districts in Aceh since September 2005. The MDF grant supported an expansion of the scope to eleven districts and sought to integrate the LGSP activities into the SPADA good governance program. The objectives of the project, implemented by the Research Triangle Institute (RTI), were to assist districts to: (i) strengthen capacity and anchor bottom-up planning in government institutions and civil society organizations, with a particular focus on local ability to prepare social and infrastructure projects and deliver quality public service development programs, (ii) improve budgeting, accounting and assets management, (iii) strengthen local councils, focusing on budgeting, oversight, legislation and outreach to constituents, (iv) strengthen civil society through citizen engagement in planning and budgeting, drafting of local regulations, and public service advocacy and monitoring, and (v) improve public services by restoring basic services, and by improving access, transparency and accountability. Outcomes: USAID funding for LGSP had launched the good governance program, and LGs, DPRDs and citizen groups worked together to experiment and build trust. Subsequently, MDF/SPADA funds permitted expansion to the east coast of Aceh, leading to learning across Aceh and ultimately underwriting many of the innovative practices emerging in the final 18 months of the program. As a result, a total of ten districts

30

(i) used participatory planning processes for budgeting development activities, (ii) developed a performance-based budget, and (iii) improved public financial management. A few of the LGs enacted policies and regulations for community participation in planning, decision-making and oversight, and hundreds of legislators, LG staff and civil society members were trained in various aspects of improving local governance. LGSP-Aceh USAID-MDF/SPADA was an innovative partnership of donors and local partners, with LGSP being a good channel for collaboration. III. TF-070672

(DFID) 9,983,647

Strengthening Post Conflict Reintegration Aspects of the Support for Poor and Disadvantaged Areas Project13

15/02/2007 30/04/2010

The purpose of this project, which listed the Bank as the “Recipient” and was partly implemented by the Bank, was to complement and co-finance the SPADA project. The objectives of this project were: (a) strengthening the capacity-building components of SPADA Aceh-Nias to ensure the sustainability and impact of investments; (b) helping to ensure that different vulnerable groups – who were often marginalized in the development processes – were included, and (c) addressing specific tsunami and conflict recovery needs in the dynamic war-to-peace transition context in Aceh. There were 12 ‘child’ trust funds under this grant and eight of them directly supported the PDO of SPADA Aceh Nias. These child trust funds are described further below in table III.1 to III.8. Outcomes: Overall, implementation of most grant activities, particularly those supporting the "satellite” activities (access to justice, PSD strengthening, support for vulnerable groups), was satisfactory, and was rated as such by the donor agency (DFID). Targets were achieved, except for two government-executed activities: Mediation and Community Legal Empowerment (MCLE) and Support to Internally Displaced People (IDP). Implementation of these activities failed due to broader procurement issues with the SPADA program. The project also generated ten analytical studies and evaluations. The Bank provided adequate and timely support to local government and other implementing partners in the conduct of Recipient-executed activities, and fulfilled tasks and responsibilities related to overall program secretariat, supervision and M&E (targets for Bank-executed activities were met). A follow-on project, Consolidating Peaceful Development in Aceh (CPDA) funded by the Netherlands and AusAID has continued some of the initiatives started by this project, such as addressing post-conflict vulnerabilities, improving access to justice, and capacity-building assistance to local governments and civil society. Note: The summary of outcomes above draws from a more detailed assessment of the implementation of this TF that can be found in the separate Implementation Completion Memorandum (ICM) for TF-070672.

This latter DFID trust fund comprised eight Recipient-executed TFs and four Bank-executed TFs that directly supported the PDO of this project. Details on these TFs are presented below. Following are the Recipient-Executed Trust Funds: III.1. Trust Fund #


Name of Funded Activity Effective Date

Closing date

TF090789 1,380,542.49 SPADA for Aceh - Economic 26/09/2007 31/12/2009

13 Different from the other two TFs described above, this TF funded by DFID, was not formally linked to the SPADA Aceh-Nias and therefore is not reflected in the data sheet. However, the project was clearly a closely related and directly supporting parallel operation.

31

Governance in Aceh (EGA) The objective of this project, implemented by the Asia Foundation, was to support Indonesia’s post-earthquake and tsunami reconstruction and rehabilitation program by improving the local business climate in selected districts in Aceh by streamlining business regulations and strengthening businesses’ capacity to advocate for change. The project comprised four components: (i) assessment of economic governance and regulatory environment, or the “Economic Governance Index (EGI) Component”, implemented in all 23 local governments in Aceh, (ii) streamlined business licensing and registration, or the “One Stop Shop (OSS) Component”, piloted in five districts in Aceh, (iii) improved district regulatory review process, or the “Regulatory Impact Assessment (RIA) Component”, and (iv) improved advocacy through small and medium enterprise (SME) networks, or the “SME Component”. Outcomes: The project was linked with a similar project funded by MDF (see above on TF-093074): some of the objectives were achieved through DFID support and the project was implemented successfully and achieved its objectives. Four project components were implemented using DFIF support. The first assessment of economic governance and regulatory environments covering 23 districts in Aceh was carried out in 2008. The outcomes of this component were achieved through MDF support. The second component – One Stop Shop for Business Licensing – used five baseline assessments of district level licensing; it then delivered training for 212 staff of One-Stop Service centers. The materials provided focused on how to use OSS to provide better services and make them work effectively. The objective of this component was also achieved with MDF support. The third component improved district regulatory processes and was delivered by introducing Regulatory Impact Assessment (RIA) methods as tools to review current local regulation that had been identified as obstacles to a good business enabling environment at the project locations. 10 RIA statements where produced from five project locations as an input for revising the existing local regulations. At the end of the project, ten draft local regulations related to nuisance and construction permits and traditional market regulations were produced, and regulatory impact assessment statement recommendations were included in the local legislation program for 2010 and 2011. Two districts – Aceh Tenggara and Aceh Barat Daya – issued district head decrees to institutionalize RIA as part of their regular regulation-making process. The last component for strengthening SME associations was fully funded by MDF.

III.2. Trust Fund # (Donor)

Disbursed in US$ equiv.


Closing date

TF090666 326,184.81 SPADA for Aceh Trust Fund - Monitoring and Evaluation Capacity Building Project

04/09/2007 31/12/2009

The objective of this project, implemented by Bappenas, was to support the Government of Indonesia in preparing a detailed action plan to strengthen monitoring and evaluation capacity of the Ministry of National Development Planning (Bappenas) and the Ministry of Development for Disadvantage Areas in implementing the Support for Poor and Disadvantage Areas Project (SPADA Project). Outcome: A Monitoring and Evaluation framework and manual were produced, introduced, and used; also a functioning MIS system was established. Capacity has been increased through formal training, workshops and on-the job training. The overall objective of strengthening monitoring and evaluation capacity of Bappenas and MinDA has been partially achieved. The outputs include: (a) a manual on SPADA monitoring and evaluation, including testing the manual in 7 SPADA provinces and 10 districts, and (b) a study on SPADA acceptance and effectiveness levels through EIRR analysis and clients’ satisfaction surveys.

III.3. Trust Fund # (Donor)



Closing date

TF090725 1,219,110.27 SPADA for Empowering Women in Aceh (EWA)

25/09/2007 31/12/2009

32

The objective of this project, implemented by The Asia Foundation, was to contribute to gender equality in Aceh by increasing women's civic participation and by strengthening gender justice in selected villages (Empowering Women in Aceh, or EWA). Outcome: 32 Syariah court judges trained to evaluate obstacles to women’s access to justice in their jurisdiction, two rounds of gender training was conducted with 32 Syariah court judges, who subsequently conducted outreach programs to women's groups in 4 districts. A reference book and documents to help evaluate and increase awareness of gender sensitivity in religious and civil court decisions was published and disseminated (i.e., 3,500 booklets on women's legal issues, 3,000 stickers, and 1,500 calendars). The outcome is likely to be sustainable across most areas. Program beneficiaries were more likely to engage government to access services and actively participate in Musrenbang planning meeting, local leader elections, school committee and PNPM processes. Increases in knowledge and attitude amongst program beneficiaries indicate they will continue to be more pro-active in engaging government and other stakeholders. The work with the Shariah Court has been entrenched into the Court's training and performance assessment mechanisms. Additional resources are being considered to the CPCRS in order to strengthen its sustainability.

III.4 Trust Fund # (Donor)



Closing date

TF 091149 1,491,963.82 Grant for Direct Health and Psychosocial Assistance Program

19/11/2008 31/10/2009

The objective of this project, implemented by International Organization for Migration, was to provide direct medical and psycho-social assistance focused on mental health. Outcome: The program provided general medical services to 5,992 patients, and mental health care to 1,530 patients (including 980 women). The program provided additional livelihood support to 200 patients (142 women). The final impact evaluation of the program conducted by Harvard Medical School shows that the mental health outreach program was highly effective in treating the primary mental health symptoms suffered by patients, with 83 % of them reporting improvements. The study shows that the treatment of mental health problems had dramatic effects on the economic functioning of patients and their families, enabling patients to return to work and reducing the time and money families must invest in caring for a family member with a mental health problem. 67% of patients reported being more able to work, and the average number of hours worked per week per patient went from 29 prior to treatment to 40 hours or more after treatment. The pilot study of providing combined mental health care and livelihood aid to individuals shows greater improvements in psychiatric symptoms for patients receiving livelihood support compared to those receiving mental health care alone, and even greater benefits in their ability to work and improvement in the economic levels of their family. The study thus indicates a significant leveraging effect of combining mental health and livelihood interventions.




Closing date

TF091675 730,079.61 SPADA for Aceh - Women Legal Empowerment (WLE)

23/06/2008 31/10/2009

The objective of this project, implemented by Relawan Perempuan untuk Kemanusiaan, was to increase women's access to justice, e.g.: (1) to increase women's legal knowledge; (2) to strengthen the capacity of local legal and other related institutions in the formal and informal sectors to address women's legal issues; and (3) to increase women's policy advocacy and policy reform. Outcomes: Paralegals in 5 districts were trained and they are providing assistance to women's groups (avg. size 35) in 23 sub-districts and 67 villages (total of more than 2,300 women). In addition, formal

33

legal support was provided in 44 cases, some with multiple beneficiaries, and 85 formal legal dialogues were conducted with community members throughout program locations. The overall objectives of the program were met, despite the shortened timeframe. At the community level sustainability is likely as trained paralegals were from the villages themselves and as such will remain resources on these issues within the village. At an institutional level, the program developed the capacity of the implementing NGOs to administer grants and implement programs. As a result two of the four NGOs involved in implementing the program have subsequently been able to access resources from other donors.




Closing date

TF094014 370,865.19 Conflict Resolution and Mediation

30/03/2009 31/10/2009

The objective of this program, implemented by the Forum LSM Aceh, was to enhance the capacity of PNPM Rural and SPADA facilitators in Aceh to deal with conflict emerging from the community development and participatory planning project. Outcome: 408 PNPM (including SPADA) facilitators, and an additional 405 village leaders (i.e. a total of 813), including village heads and youth representatives, were trained in conflict mediation, leadership, and civic education in October 2009. This output used about half of the total DFID funds. Core training was planned to be followed by a round of advanced training before the end of the grant. Delays in implementation, mostly due to procedural delays on the government side, led to the core training being held only in October 2009, towards the end of the project Advanced training could therefore not happen before the closing of the grant. III.7 Trust Fund #



Closing date

TF094912 168,254.44 Aceh Youth Radio for Peacebuilding II

13/08/2009 31/10/2009

The objective of this project, implemented by Search for Common Ground Indonesia (SFCGI), was to (i) establish an enhanced platform for youth in Aceh to explore issues affecting reconciliation and conflict transformation in a constructive and solutions-oriented way, (ii) enhance the capacity and sustainability of media in Aceh to play more active role in reconciliation and peace-building processes, and (iii) increase collaboration between youth, their communities and other relevant stakeholders. Outcomes: Overall, the project achieved its objectives but it fell somewhat short of its intended objective to stimulate dialogue between youth and other Acehnese stakeholders, due to low stakeholder turnout for the Youth View Report presentation. Although the program was able to mobilize and stimulate informal youth mechanisms for engagement, the youth of the participants was also a liability in the sense that cultural norms constrained them from engaging with older generations effectively. If youth are going to engage in the democratization and peace building process, more efforts need to be made to prepare them for that role. It would be necessary to assist the nascent “Aceh Youth Forum” and youth network with development and capacity building to give it an opportunity to implement the crucial points of the Aceh Youth Forum Declaration to ensure its sustainability, However, SFCGI was unable to mobilize donor support to finance follow-up activities.




Closing date

TF090761 0 SPADA MCLE Aceh 11/10/2007 31/12/2009 The objective of this project, implemented by Ministry of Development for Disadvantaged Areas was to (i)

34

strengthen the capacity of institutions and individuals at community level in the Province of Aceh to resolve conflict in an open, independent and just manner; (ii) assist and facilitate the resolution of disputes involving community-wide interests; and (iii) increase public trust in the formal legal system by enhancing community legal awareness and access to the formal system. Outcomes: No outcomes have been achieved as the entire grant was canceled. The procurement for consultant services for implementing this activity was delayed, and following the MTR, this activity was dropped because it did not fit the project time line. Alternative resources were identified to implement the activity in accordance with the original design after this grant’s closing date. The additional funds were provided by the Justice for the Poor Program.

The Bank-executed DFID trust funds comprised of the following:

III.9 Bank-executed Trust Fund



Closing date

TF0900030 398,564.81 SPADA-ENHANCED SUPERVISION

06/03/2007 31/12/2009

This trust fund was utilized to support the SPADA Task Team. III.10 Bank-executed Trust

Fund Disbursed

in US$ equiv. Name of Funded Activity Effective

Date Closing date

TF0900031 298,816.88 SPADA-POST CONFLICT DEVELOPMENT IN MID COUNTRIES

06/03/2007 31/12/2009

This trust fund was utilized to support the SPADA Task Team, and part was allocated for the Conflict Team. III.11 Bank-executed Trust

Fund Disbursed


Date Closing date

TF0900032 553,988.36 SPADA ACCESS TO JUSTICE

06/03/2007 31/12/2009

The trust fund was utilized to support the Justice for the Poor Team. III.12 Bank-executed Trust

Fund Disbursed


Date Closing date

TF0900039 1,869,906.70 SPADA-ASSISTANCE TO VULNERABLE GROUPS

10/04/2007 31/12/2009

This trust fund was utilized to support the Conflict Team.

35

Annex 3. Economic and Financial Analysis A sample of about 30 SPADA Aceh-Nias infrastructure subprojects was used for EIRR calculations in 2012. These subprojects represent about three percent of all subprojects. They were chosen to roughly represent the overall sub-project mix in SPADA Aceh-Nias and were analyzed according to an EIRR methodology developed by the KDP/PNPM projects. Results indicate that the average EIRR for SPADA Aceh-Nias sub-projects is 31.44%, with higher values for road & bridge and water supply sub-projects, and lower values for irrigation and drainage and public toilet sub-projects.

Table : Average result for Indicator of Cost and Benefit Analyses14

Type of Infrastructure

No Sub Project

Average EIRR

Average NPV in Rp

Average BCR

Clean water 4 40.25 % 249,682,579 1.93

Road and Bridge*) 11 35.64 % 166,741,912 1.82 Irrigation 4 31.08 % 161,269,438 1,86 Public toilets 2 29.78 % 126,253,518 1,88 Drainage 2 20.46 % 120,378,887 1,61 Total Sub Project 23 31.44% 164,865,267 1.82

These EIRRs are roughly equivalent to EIRRs calculated for similar subprojects completed by PNPM Rural in East and Central Java, South and West Sulawesi, West Kalimantan, North Sumatera, and NTT provinces in 2004 and 2012 (see Torrens 2005 and World Bank 2012). The patterns are also similar to these previous PNPM-Rural studies, where EIRR levels were higher for road & bridge and water supply projects, and slightly lower for irrigation and drainage projects. This and previous EIRR studies have been relatively expensive given the methodology's preference for technical specialists (engineers and economists) to both gather data from the field and perform the EIRR analysis and calculations. Additionally, the usefulness of the EIRR methodology itself to reliably capture the actual gains from these types of community-planned and community-built small-scale infrastructure projects is increasingly being called into question. Given these concerns, but equally importantly because the SPADA Aceh-Nias results from the smaller sample conform broadly to previous analyses, the Task Team and MinDA decided not to launch a full-scale, SPADA-wide EIRR.

14 With five outlier projects with much higher benefits excluded.

36

Annex 4. Grant Preparation and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending/Grant Preparation Scott Guggenheim Lead Social Development

Specialist/Consultant EASID TTL

Sri Kuntari Social Development Specialist EASID Co-TTL Linda Ayu Citra Consultant EASID Complaint handling Purri Andriaty Consultant EASID Private sector

development (PSD) Stefano Gnes Consultant EASID PSD Lauren Serota Consultant EASID Local governments Muhammad Ruslan Consultant EASID Operations H.Gultom Consultant EASID FM Roesfita Roesli Operations Officer in HD EASHD Education Rizal Rifai Procurement Specialist EAPPR Procurement Christina I Donna Financial Management Specialist EAPFM FM Edhi Pambudhi Operations Officer in HD EASHD Health

Supervision/ICR

Natasha Hayward Senior Social Development Specialist EASID TTL

Hans Antlov Senior Social Development Specialist EASID Co-TTL

Heru Darmawan Consultant EASID Operations Muhammad Fithriadi E T Consultant EASID Operations Heinrich M. Gultom Operations Analyst EASID Operations Teuku Budi Hermawan Consultant EASIS Operations Bisma Husen Senior Procurement Specialist EAPPR Procurement Hanggar Irawan E T Consultant EASID FM Sri Kuntari Social Development Specialist EASID Original TTL

Victor Bottini Senior Social Development Specialist EASID TTL

Linda Ayu Citra Ningsih Consultant EACIF Anti-corruption Endah Nurdiana Consultant EASIS M&E Vicki Diane Peterson E T Consultant EASFP PSD Enggar Prasetyaningsih Procurement Analyst EAPPR Procurement Yogana Prasta Operations Adviser EACIF Operations Restu Pratiwi Consultant EASIS Purwanto Consultant EACIF Finance Rizal H. Rivai Senior Procurement Specialist EAPPR Procurement Virza S. Sasmitawidjaja Consultant EASIS Safeguards Purri Andriaty Subana Consultant EASID Operations

37

Unggul Suprayitno Sr Financial Management Specialist EAPFM FM Aang Sutrisna Consultant EASIS M&E Chatarina Ayu Widiarti Program Assistant EACIF Operation I Gusti Ngurah Wijaya K. E T Consultant EAPFM FM Marcellinus Jerry Winata Communications Assistant EACIF Communications Yoseph Lucky Consultant EASID Operations

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (MDF-TF055080)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending (FY07)

Total: N/A 108.14 Supervision/ICR

FY07 N/A 46.48 FY08 4.2 46.28 FY09 60.7 121.24 FY10 27.5 73.84 FY11 24.7 61.80 FY12 16.9 78.39

Total: 134.1 428.05

38

Annex 5. Beneficiary Survey Results 1. Study Objective This study collects beneficiary satisfaction information in Aceh and Nias provinces in order to summarize the performance of the SPADA initiative against its achievement indicators and targets. The study focuses on satisfaction with the provision of health and education services provided through, or financed in part by, SPADA. 2. Study Design The beneficiary survey is quantitative in nature: by sampling a roughly representative set of confirmed project beneficiaries, it will provide the information necessary to estimate average beneficiary satisfaction. This estimate is limited, however, by both SPADA operational details and study design. Firstly, baseline, pre-project satisfaction information does not exist. Secondly, SPADA typically finances ongoing services and programs (in health and education) delivered by local governments and service providers and in so doing expands the coverage of these previously existing services. For both reasons, the study as designed cannot measure the impact of the SPADA program against a valid counterfactual. In order to ensure that satisfaction as measured can be attributed to particular SPADA projects (rather than just general or ongoing activities in a locality), the survey instruments and sampling strategy will generate enough identifying information to link confirmed beneficiaries with particular projects and locations. 3. District Coverage Data collection (carried out by Ceriajasa Engineering Consultants) was completed in the week 9 to 17 October, 2012 in six districts in Aceh – Aceh Besar, Pidie, Aceh Barat, Aceh Tamiang, Aceh Selatan, and Aceh Tenggara –and 1 district –Nias Selatan – on Nias island. These districts were selected purposively by PIU for their confirmed delivery of education and health services supported by SPADA block grants – see below. 4. Samples Many SPADA-supported projects are quite small in geographic or population coverage and are specific to needs and capacities of each locality, so comparability between SPADA-supported projects is not always straightforward. In order to achieve a measure of performance that is as general as possible – a measure that could reasonably represent performance across all SPADA-supported initiatives – and to be able to compare between geographic regions, the study sampled only projects that are widely implemented in nearly all Nias and Aceh kabupaten. SPADA MIS data indicates only two such projects/services –Desa Siaga (training for health provider and cadre) and interactive learning (training for teachers) – are so general, so these two project types were sampled in all study regions.

In order to reduce the influence of experience with goods, services, and initiatives provided without SPADA support (but which nonetheless may be complements to, or substitutes for, SPADA-supported initiatives), the study design randomly selects confirmed participants in each type of project. For education-related projects, three

39

primary school teachers and one junior high school teacher (from different schools) were sampled and interviewed in each district. For each teacher sampled, 25 students who learn from that teacher were interviewed. For health-related projects, three midwives were sampled for interview. For each midwife selected, the study contacted for interview one or two village health posts (Posyandu) where the midwife is confirmed to practice. For each midwife and associated Posyandu thus contacted, the study interviewed ten patients who are confirmed recipients of health services from those providers. The total number of providers and end-users sampled is adjusted based on the expected number of participants in each program in each region. The expected number of participants in each program in each region has not previously been estimated, so the study design included a data collection specifically to estimate these participation rates. 5. Method of Analyses The analysis estimates rates of satisfaction by project type, by geographic region, and by participant type (as well as overall satisfaction across all participants contacted). As the survey instrument includes several distinct (but not independent) questions regarding satisfaction with different, each distinct question and response will be weighted in order to estimate average satisfaction. The formulation of these weights is part of the analytic program and the analysis will include a section on their derivation. The study provides information about individual, household, and village-level factors influencing beneficiary satisfaction to provide SPADA implementers and practitioners with advice on more effective implementation. 6. Result The results showed positive impacts, as follows: The draft report of the output study survey shows that, on average, 87.5% of patients were satisfied with health services provided by midwives and cadres trained by the project. And 91% of students were satisfied with education services provided by teachers trained by the project. The results show that the services needed by patients were prenatal, immunization, childbirth and general health checkup. The percentage for health services satisfaction was derived from patients who were very satisfied and satisfied with their midwives and cadres (see Table 5.1). The survey sample included 215 patients served by midwives and 215 patients served by cadres. Table 5.1. Patients’ satisfaction of midwives service

Midwives patients Health Cadres patients Very Satisfied 19% 14% Satisfied 68% 74% Not satisfied 13% 12% Total 100% 100% % patients satisfied 87% 88%

87.5%

40

For the project’s education component, the study focused on levels understanding shown by students to teaching methods in three majors: Indonesian, physics, and mathematics. The percentage for education services satisfaction was derived from student’s perception of the teacher’s explanation, e.g., very clear and clear. There were 847 students interviewed and the results are summarized on the Table 5.2. Table 5.2. Percentage of Students satisfied by Education Services Bahasa Indonesia Physic Mathematics Very clear 22.31% 21.25% 24.32% Clear 72.61% 70.60% 62.81% Unclear 4.84% 7.79% 12.16% Very unclear 0.24% 0.35% 0.71% Total 100.00% 100.00% 100.00%

% of students satisfied 94.92% 91.85% 87.13%

91%

41

Annex 6. Stakeholder Workshop Report and Results 1. Multi-stakeholder ICR workshop in February 2012 in Banda Aceh A stakeholder workshop was held in the World Bank Office in Banda Aceh, the capital of Aceh Province, on February 16, 2012. About 20 people participated very actively in the half-day group event; a separate discussion was held later with Bappeda, the provincial coordination team. Participants provided feedback on what worked and what did not work during the implementation of SPADA Aceh-Nias. Below is a list of workshop participants from four districts in Aceh Province who were project stakeholders:

• District Holding Commitment Officer (SATKER) • District Sub-project Implementing Unit (UPKD) for health • District Sub-project Implementing Unit (UPKD) for education • District Sub-project Implementing Unit (UPKD) for infrastructure • District Technical Review Team • District Management Team (TPK) • District Management Consultant (DMC) • School principal, a beneficiary

The participants were divided into two groups by district, and each group discussed and reflected on three main issues: planning processes, implementation, and sustainability; the group discussion was facilitated by National Management Consultant staff. Then, the outputs of the group discussions were brought to the workshop plenum to agree on the findings. Some personal insights from the stakeholder event are presented below: Reflections on the Implementation of SPADA Aceh-Nias

Former District Holding Commitment, Aceh Jaya District: “The planning process, as set out in the project timeline, was to be finalized within two months. This limited amount of time did not allow the technical review team to produce a good technical and financial proposal; moreover, public consultation was also not properly facilitated. In future, more time should be allowed for planning, and we would not need a consultat or facilitator if adequate time were given to us.” School Principal, Aceh Jaya District:

“In the past, projects fund were mainly for infrastructure improvements but there were few activities in capacity building to improve the quality of human resources, such as training for teachers. We are happy that the SPADA Aceh-Nias project did fund capacity building activities, so that teachers received six months of training; this had a positive impact on increasing the graduation rate of our students. We hope the local government could provide a contribution towards sustainability, especially through provision of classroom furniture and school equipment. The benefits of the SPADA project were acknowledged by the community, the school committee, and the students. The training that the teachers received focused on improving teachers’ understanding of teaching materials, curriculum, methods of instruction, and other supporting teaching tools. The real impact was an increase of the graduation rate

42

from 65% to 85% in Aceh Jaya, and most of the graduated students were subsequently accepted by the best schools in the neighborhood. Therefore, the project benefits are very significant, and we hope that there will be similar programs in the future. We expect to receive local government contributions in the future to sustain capacity building of our teachers”.

District Sub Project Implementing Unit (UPKD) Health, Aceh Besar: “There should be inputs, both the technical as well as health considerations, during the design stage of any health sub-projects, especially if construction is involved. For example, when designing the examination room and the bathroom, as well as during the design of the maternity ward and delivery room, they should be close to the sterilizer and the room for medical equipment. The infrastructure team should also seek advice from the health sector specialist staff to make sure that all health service standards are fully met.”

Former Technical Review Team, Aceh Besar: “The positive impact of a sub-project could be seen with the increase of the student graduation rate due to the improved quality of teachers, and adequate teaching facilities. And in the health sector, where we are facing endemic diseases, such as malaria and dengue hemorrhagic fever we hope that more patients can be treated with the provision of fogging equipment [for vector control] through the project. The critical issue will be the continued supply of chemicals, and the maintenance and operational cost of the fogging equipment.”

Summary of Workshop Findings - Project socialization was weak, and not all communities and technical agencies,

received adequate information. Public consultations were facilitated only as ceremonial events, and there was a lack of coordination among technical agencies and programs.

- Bappeda, as the lead agency at the province and district levels, were successful in ensuring vertical coordination with other technical agencies.

- The efforts of involving communities in project monitoring were limited; communities were mainly involved in sub-project planning and implementation.

- The late issuance of the central government budgets (DIPA) caused poor quality of sub-project planning and implementation. Adequate time for preparing budgets and design is required for good quality infrastructure sub-projects.

- The different implementation schedules of the SPADA program and the regular planning processes caused a lack of synergy between SPADA and government programs.

- The selection and composition of Technical Review Teams is a critical factor for generating good quality draft proposals that address the problems and needs that were identified through the participatory approach. A follow-up public consultation would be a good way to finalize the proposals for discussion in the district forum before decisions are made

43

- The issue of project sustainability was not properly socialized at the beginning of the project, thus the follow-up for sustainability was rather late. Some of the simpler sub-projects can be maintained by the communities, but the follow-up on non-physical sub-projects, such as teacher training will be a problem because of the limited budgets of local governments, and also because there are no records of past training participants.

2. Observations and views of the SPADA Provincial Coordination Team of Aceh Province provided as part of February 2012 stakeholder workshop in Banda Aceh

Achievement of PDO and implementation: The SPADA Project supported Aceh’s recovery from conflict and tsunami disaster through comprehensive problem identification at the community, sub-district and district levels. The provincial government also provided support for poverty alleviation by allocating local budget resources to supplement grants at the village level. In the future, better and wider coverage by socialization should include more provincial stakeholders since it is key to avoid any misunderstandings on the project concept: all actors need to know their roles so that they could support project implementation more effectively. Performance of institutions, quality of maintenance and sustainability: There was some discussion about political interests and involvement of local governments and communities in the district forums during planning and implementation. “Political interests” would help to ensure the availability of local government contributions for project administration and monitoring, but it could decrease the opportunities for communities to participate in project implementation, such as during the decision-making forum for funding and the selection of sub-projects by the community. Implementation was often by private contractors, and as a result, there were little community involvement and contributions that would have enhanced sustainability. In the future, the enforcement of accountability, transparency and good governance in implementation should be strengthened.

Based on monitoring by government, all sub-projects are well established and the commitment of local governments is contributing to successful project implementation. The provincial government will continue to work hard to ensure that all sub-projects are functioning and will become local assets. There is hope for continued support for all districts, including also newly established districts. There were bi-weekly meetings with project stakeholders to discuss problems and find solutions. There was also monitoring through field visits.

There were issues with the quality of the technical assistance, i.e. the consultants and facilitators for the project, and overall there was not enough technical assistance. There were extended periods when important positions, such as the district FM consultants, were vacant [mainly due to procurement delays]. This problem contributed to the poor quality of financial reports, an issue identified by the government audit findings.

44

Regarding the project management structure, the project should have been managed and executed by the province for more effective implementation, such as the implementation of the BOS program (national program for operational support to schools).

45

Annex 7. Summary of Recipient’s Project Completion Report Project Achievements Planning Process: The bottom-up participatory planning that was introduced by the SPADA program has increased the confidence of communities in development planning and implementation; it has given them new hope because their proposals went through the SPADA planning process, and they were involved all the way through, deciding the funding priorities and attending various meetings. And they could see the outputs and benefits of the project in the same year. A positive trend is emerging and communities are becoming more open in expressing their opinions and aspirations – a critical factor for successful participatory planning activities involving communities. Participation of marginalized groups, i.e. the poor and women, met the standard requirements because the participatory, bottom-up planning process used by SPADA facilitated the involvement of these groups. Beneficiaries: The project reached a total of 1,364,561 beneficiaries, or 29% of the total population in Aceh-Nias (end target 30%). A total of 57% of the beneficiaries are poor. Table 2: Total Beneficiaries.

Province Total Population

Total no. of Beneficiaries

% of Beneficiaries

Beneficiaries (poor households) Total %

Aceh 3.862.331 1.251.632 32% 741.761 59% Nias and South Nias 799.517 112.929 14% 42.169 37%

TOTAL 4.661.848 1.364.561 29% 783.930 57%

46

Major Outputs/intermediate outcomes: The project finalized 1,770 sub-projects (infrastructure, education, and health) and have been handed over to local governments. Most of sub-projects categorized as ‘satisfactory’ and ‘acceptable’ (for education sub-projects, 76% satisfactory and 24% acceptable’; for health sub-projects, 95% satisfactory and 5% acceptable; while for infrastructure sub-projects, 83% categorized satisfactory, 16% acceptable, and 1% need improvement). All targeted districts have increased their capacity in procurement and financial management, while 13 districts (and target 13 districts) have improved their policies and mechanisms related to local regulation development to support implementation of transparency and accountability principles. The project facilitated establishment of 19 private sector forums (end target 19 forums). A total of 59% of village with active participation in school management committees (end target 40%). Specific study to assess respondent satisfaction for education and health services provided by the project is to be conducted in August-October 2012. Development objectives: results for rapid assessment exercise to 23 sub-projects shows that the average of EIRR is 23% (end target >20%), with average net economic benefit for each sub-project is IDR 164,865,267, indicating the activities provide positive economic benefits for the community. Percentage of children aged 7-12 not attending primary school in Aceh is 4.82% (end target >5%), and in Nias is 6.36 (end target >11%), while percentage of children aged 13-15 not attending junior secondary school in Aceh is 23.55% (end target >13%), and in Nias 50.09% (end target >29%). Challenges Adoption of planning approach by local governments: very few local authorities have adopted the planning process promoted by SPADA. Some factors contributed, including: (1) time frame for SPADA planning process is not coherent with the time frame for regular planning process; (2) unclear policies and mechanisms for many of government agencies at local level to adopt the SPADA planning process. Sustainability: maintenance and continuity of project activities continue to be the issues in many districts. Major challenges identified, including: (1) unclear local regulation and mechanism to continue the activities when the project ended; (2) insufficient local budget to support the maintenance; (3) inadequate commitment from the local authorities to adopt the approaches promoted by the project; and (4) lack of capacity of local level institutions and human resources to continue project activities. Project Management:

1. Insufficient ‘socialization’ of project objectives and processes to local stakeholders during the initial stage of project (lack of coordination with the locals, insufficient fund allocated).

47

2. Inadequate project work plan (unsatisfactory baseline data, lack of resource plan and financial plan, insufficient preparation of project results framework, lack of coordination with other similar programs)

3. Insufficient technical assistance to local stakeholders (lack of training activities for consultants, inconsistence policies/mechanisms for project implementation –particularly in fund disbursement, inadequate financial plan for consultants’ supervision and operational costs).

4. Ineffective monitoring and evaluation (inaccurate data in management information system, inadequate plans for studies/evaluation).

5. Insufficient coordination with local authorities (multi-interpretation of project manual, high turn-over of consultants at local level).

6. Inadequate communication (insufficient communication strategy/plans, inadequate financial plan)

7. Delay of budget approval.

48

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

This Annex contains comments on SPADA Aceh-Nias prepared by the Secretariat of the Multi Donor Fund for Aceh and Nias (MDF) and do not necessarily represent the views of the MDF’s Steering Committee or donors.

The MDF supports the Government of Indonesia (GoI) in its rehabilitation and reconstruction of Aceh and Nias following the tsunami and earthquake disasters in Aceh and Nias in 2004/2005. The MDF, supported by 15 donors, provided grants for the implementation of 23 post-disaster reconstruction projects in Aceh and Nias. Following the GoI’s rehabilitation and reconstruction efforts, the MDF program prioritized six outcome areas – recovery of communities, rehabilitation and reconstruction of large infrastructure, strengthening governance and capacity building, sustaining the environment, and recovery of livelihoods – in a three-phase approach. The MDF financed SPADA project contributes to the strengthening governance and capacity outcome area of the MDF.

This project leveraged the existing Bank-financed Support for Poor and Disadvantaged Areas (SPADA) Project that covers the country’s poorest and disadvantaged areas at the district level. The original SPADA project concept was reviewed in May 2002, approved by the Bank in June 2005, and became effective in December 2005 with seven poorest districts in Aceh already slated to be part of the program. It was therefore logical and cost effective to scale up the existing program to cover additional districts in Aceh and Nias as a response to the GoI’s recovery efforts in Aceh and Nias. At the government’s request, the MDF agreed to provide grant funds to support the scaling up covering Aceh and Nias.

The MDF supported SPADA in Aceh and Nias benefited from quick project start-up to deliver urgently needed assistance to disaster-affected area. The project continued the model adopted by the national SPADA program with an aim to strengthen district government capacity to incorporate bottom-up (village, sub-district and inter-sub-district) planning and needs analysis into its planning and budgeting process (Musrenbang) and to provide block grants to improve public services and the recovery of basic infrastructure post-disaster. The concept review took place in March 2006 and became effective in May 2007. The project took some time to start up and was extended for an additional 18 months to close in December 2011.

Despite significant early delays, at project closure, a total of 1,738 planned sub-projects from all sectors – education, health, and infrastructure – were completed, achieving 99% of the total proposed sub-projects. 100% of the SPADA districts now participate in the GoI’s participatory planning and budgeting process (Musrenbang) through SPADA’s capacity development program. More than 9,551 teachers received school management training; 125 NGO staff and formal justice officials (judges, police, and prosecutors) were trained to provide judicial services and legal education; through the SPADA supported Women’s and Legal Empowerment program, more than 2,300 benefited from legal aid assistance; and through the BRA-KDP project, 233,115 vulnerable women, IDPs, and ex-combatants were provided with technical assistance and financial support to

49

rebuild their lives. Through disbursement of block grants, SPADA supported the construction and rehabilitation of 97km of roads, 23 units of irrigation systems, 125 schools, and 29 village health clinics. In sum, the SPADA project in Aceh and Nias has contributed to the government’s efforts to develop rural poor and disadvantaged areas to create economic development opportunities and to provide improved services for its citizens, particularly in health, education, and community infrastructure, contributing to an important outcome area in the MDF portfolio.

Despite the embedded difficulties, the project was agile in working outside the box by leveraging an existing program, the USAID funded LGSP, to provide needed capacity building activities to local governments as well as partnering with TAF to improve investment climate in the affected region.

Due to budget delays and other problems in the earlier years of implementation, some block grant funds could not be distributed resulting in savings of US$4.2 million. These unspent residual funds were returned to the MDF pool six months before project closing date through a restructuring of the project and cancellation of funds. The funds thus released were subsequently reprogrammed to other ongoing projects. This flexible approach by the project team allowed the MDF to optimize the use of the funds for the reconstruction of Aceh and Nias, within the tight timeline for the overall MDF program to address unmet needs. This approach is particularly significant for projects that are funded from programs with a finite implementation period and funds that are usable for very specific purposes, so that time and funds are most efficiently utilized across a wider portfolio.

The ICR identifies areas that contributed to the implementation delays, ranging from consistent lateness in the annual DIPA issuance and protracted procurement processes on the government side, to the revolving TTL from the Bank side, resulting in less than satisfactory performance during the first half of the program. The ICR further cites that the project recognized at the outset that the lack of experience and capacity of the newly established implementing agency, MinDA, would be a significant risk factor to mitigate. However, while the project could have partnered with a more experienced ministry for the implementation of SPADA for Aceh and Nias to ensure, for example, more efficient procurement execution, the challenges of delays in the annual DIPA issuance is entrenched in the Indonesia planning process. As one of the lessons learned, in post disaster settings where speed of implementation is important, risk factors in the implementation arrangements, such as working with a newly-established ministry, should be carefully considered and risk mitigation measures fully defined. Furthermore, projects operating in post-disaster situations should take full advantage of the Bank’s streamlined procurement procedures.

The project built in a comprehensive monitoring and evaluation program at the onset, but unfortunately very little of it was implemented due to, as stated in the ICR, lack of qualified personnel, procurement delays, and DIPA delays. Had the M&E system been implemented as designed, with quantitative and qualitative management information system (MIS) augmented by surveys and case studies to track results due to project

50

interventions. The deficiency of the MIS adversely affected the availability of data for the project to report on. These data would have greatly enhanced the project to report on results. Impact of the numerous capacity building activities, an important MDF outcome area, were never well captured in the project’s reporting.

51

Annex 9. List of Supporting Documents

• Project Appraisal Document

• Grant Agreements

• Aide Memoires and Management Letters

• Back-to-Office Reports and Implementation Status Reports

• Project Annual and Quarterly Reports

• The World Bank. Implementation Completion Memorandum (ICM) : DFID Trust Fund for Strengthening Post Conflict Reintegration Aspects of the Support of the Poor and Disadvantaged Areas Project for Aceh.September 2011

• Research Triangle Institute (RTI). Local Governance Support Program Final Report – Aceh USAID - SPADA. 2009

• The Asia Foundation (TAF). Economic Governance in Aceh Support for Poor and Disadvantaged Area – Revised MDF Grant Completion Report. September 2011

• Pusat Telaah dan Informasi Regional (PATTIRO) - PSF. Consolidated Report: Study Lesson Learned on SPADA Planning and Service Delivery. 2011

• SMERU Research Institute. Research Report: The State of Local Governance and Public Services in the Decentralized Indonesia in 2006: Findings from the Governance and Decentralization Survey 2 (GDS 2).February 2008

• Anthony Torrens – World Bank. Economic Impact Analysis of Kecamatan Development Program Infrastructure Projects. January 2005

• World Bank. Small scale replication of economic impact analysis of PNPM infrastructure projects. April 2012

Banda AcehBanda Aceh

MedanMedan

PekanbaruPekanbaru

PadangPadang

M A L AY S I AM A L AY S I A

THAILANDTHAILAND

4

3

2

5

1010

6

8

9

1414

1313

1

1212

71616

1818

1111

1515

1717

1919

3

5

4

2

1

Banda Aceh

Medan

Pekanbaru

Padang

M A L AY S I A

THAILAND

4

3

2

5

10

6

8

9

14

13

1

12

716

18

11

15

17

19

3

5

4

2

1

Nias

Siberut

Simeulue

Me n

t aw

ai I s .

Andaman Sea

I N D I A N

O C E A N

Gulf of

Thailand

1 ACEH PROVINCE01. SIMEULUE02. ACEH SELATAN03. ACEH TENGGARA04. ACEH TIMUR05. ACEH BARAT06. ACEH BESAR07. BIREUEN08. ACEH UTARA09. ACEH BARAT DAYA10. GAYO LUES11. ACEH TAMIANG12. NAGAN RAYA13. ACEH JAYA14. ACEH TENGAH15. BENER MERIAH16. PIDIE17. ACEH SINGKIL2 SUMATERA UTARA (NORTH) PROVINCE18. NIAS19. NIAS SELATAN

NANGGROE ACEH DARUSSALAMSUMATERA UTARARIAUSUMATERA BARATJAMBI

12345

PROVINCES

IBRD 39270

APRIL 2012

INDONESIA

S U P P O R T F O R P O O R A N DD I S A D VA N TA G E D A R E A S ( S PA D A )

I N A C E H A N D N I A S

This map was produced by the Map Design Unit of The World Bank.

The boundaries, colors, denominations and any other information

shown on this map do not imply, on the part of The World Bank

Group, any judgment on the legal status of any territory, or any

endorsement or acceptance of such boundaries.

NATIONAL CAPITAL

PROVINCE CAPITALS

INTERNATIONAL BOUNDARIES

PROVINCE BOUNDARIES

KABUPATEN BOUNDARIES