peru - second phase of health reform program ... · web view2016/06/29 · (vi) training of 380...

Document of The World Bank

Report No: ICR00003691

IMPLEMENTATION COMPLETION AND RESULTS REPORT(IBRD-76430)

ON A

LOAN

IN THE AMOUNT OF US$15.0 MILLION

TO THE

REPUBLIC OF PERU

FOR A

SECOND PHASE OF THE HEALTH REFORM PROGRAM

June 30, 2016

Health, Nutrition and PopulationBolivia, Ecuador, Peru and Venezuela Country Management UnitLatin America and the Caribbean Region

CURRENCY EQUIVALENTS(Exchange Rate Effective December 31, 2015)

Currency Unit = Peruvian Nuevo SolPEN 300.35 = US$ 1.00US$ 1.00 = PEN 3.33

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

APL Adaptable Program LoanAUS Aseguramiento Universal en Salud –

Universal Health InsuranceCPS Country Partnership StrategyCRED Control de Crecimiento y DesarrolloCRVS Civil Registration and Vital StatisticsDIA Derecho a la Identidad y

Aseguramiento DIRESA Dirección Regional de SaludEA Environmental AssessmentEDA Extreme DiarrheaENDES Encuesta Demográfica y de Salud

FamiliarEPPES Estrategia de Promoción de Práctica

y Entornos Saludables FS Feasibility StudyGOP Government of PeruIADB Inter-American Development BankICR Implementation Completion and

Results ReportIMR Infant Mortality RateINT Department of Institutional Integrity IOI Intermediate Outcome IndicatorIPP Indigenous People PlanISR Implementation Status and Results

Report

KPI Key Performance IndicatorM MillionM&E Monitory and EvaluationMA Management AgreementMEF Ministerio de Economía y Finanzas

– Ministry of Economy and FinanceMINSA Ministerio de Salud de Perú –

Ministry of Health of PeruMMR Maternal Mortality RatePAD Project Appraisal DocumentPARSALUD Programa de Apoyo a la

Reforma del Sector SaludPDO Project Development ObjectivePIU Project Implementation UnitPRONIS Programa Nacional de Inversión de

Salud QUALYQuality Adjusted Life YearsRF Results FrameworkSIAF Sistema Integrado de Administración

FinancieraSIS Seguro Integral de Salud –

Comprehensive Health InsuranceSNIP Sistema Nacional de Inversión

PublicaSUNASA Superintendencia Nacional de Salud

Senior Global Practice Director: Timothy G. EvansPractice Manager: Daniel Dulitzky

Project Team Leader: Andre MediciICR Team Leader/Author: Federica Secci

ii

PERUSecond Phase of the Health Reform Program (P095563)

TABLE OF CONTENTS

Data SheetB. Key Dates...................................................................................................................ivC. Ratings Summary.......................................................................................................ivD. Sector and Theme Codes............................................................................................vE. Bank Staff....................................................................................................................vF. Results Framework Analysis.......................................................................................vG. Ratings of Project Performance in ISRs.....................................................................xH. Restructuring...............................................................................................................xI. Disbursement Profile..................................................................................................xi

1. Project Context, Development Objectives and Design................................................12. Key Factors Affecting Implementation and Outcomes...............................................53. Assessment of Outcomes...........................................................................................114. Assessment of Risk to Development Outcome.........................................................215. Assessment of Bank and Borrower Performance......................................................226. Lessons Learned........................................................................................................267. Comments on Issues Raised by Borrower/Implementing Agencies/Partners...........27Annex 1. Project Costs and Financing...........................................................................28Annex 2. Outputs by Component..................................................................................29Annex 3. Economic and Financial Analysis..................................................................38Annex 4. Bank Lending and Implementation Support/Supervision Processes.............44Annex 5. Beneficiary Survey Results............................................................................46Annex 6. Stakeholder Workshop Report and Results...................................................47Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR......................48Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders........................69Annex 9. List of Supporting Documents.......................................................................70Annex 10: Analysis of PDO Achievement....................................................................71Annex 11: Loan Amount Allocation.............................................................................75Annex 12: Organization of PARSALUD II Project Implementation Unit (PIU)..........76MAP...............................................................................................................................77

iii

Data Sheet

A. Basic Information

Country: Peru Project Name:PE- (APL2) Health Reform Program

Project ID: P095563 L/C/TF Number(s): IBRD-76430ICR Date: 06/30/2016 ICR Type: Core ICR

Lending Instrument: APL Borrower:GOVERNMENT OF PERU

Original Total Commitment:

USD 15.00M Disbursed Amount: USD 11.98M

Revised Amount: USD 15.00MEnvironmental Category: BImplementing Agencies: PARSALUD Cofinanciers and Other External Partners: Inter-American Development Bank (IADB)

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 11/02/2005 Effectiveness: 12/14/2009 12/15/2009

Appraisal: 12/11/2008 Restructuring(s):06/20/201108/25/2014

Approval: 02/17/2009 Mid-term Review: 10/30/2012 02/26/2013 Closing: 01/31/2015 12/31/2015

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately

Unsatisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Moderately Satisfactory

iv

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators QAG Assessments (if any) Rating

Potential Problem Project at any time (Yes/No):

NoQuality at Entry (QEA):

None

Problem Project at any time (Yes/No):

NoQuality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes Original Actual

Sector Code (as % of total Bank financing) Compulsory health finance 4 4 Health 62 62 Public administration- Health 34 34

Theme Code (as % of total Bank financing) Child health 30 30 Health system performance 30 30 Indigenous peoples 10 10 Population and reproductive health 30 30

E. Bank Staff Positions At ICR At Approval

Vice President: Jorge Familiar Pamela Cox Country Director: Alberto Rodriguez Carlos Felipe Jaramillo Practice Manager: Daniel Dulitzky Keith E. Hansen Project Team Leader: Andre C. Medici Fernando Lavadenz ICR Team Leader: Federica Secci ICR Primary Author: Federica Secci

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document)Framed within the long-term objective of the Health Reform Program (APL in all its phases) to reduce maternal and infant mortality rates in Peru’s nine poorest regions of the country (Amazonas, Huánuco, Huancavelica, Ayacucho, Apurimac, Cusco, Cajamarca,

v

Ucayali, and Puno); the specific APL 2 Project Development Objectives are to: (i) improve family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three; (ii) strengthen health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three; and (iii) support MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment (same as in the Loan Agreement).

Revised Project Development Objectives (as approved by original approving authority)The PDOs, the Key Performance Indicators (KPIs) and the outcome targets were not revised. However, some targets were revised during the 2014 project restructuring for the Intermediate Outcome Indicators (IOIs) (see below).

(a) PDO Indicator(s)Achievement of targets was assessed as follows: Target Surpassed: the value of the indicator at the end of the project (December 2015, based

on 2014 data) is higher than the target Target Achieved: the value of the indicator at the end of the project (December 2015, based

on 2014 data) is equal to the target, or it is equal to or greater than 85% of the target Target Partially achieved: the value of the indicator at the end of the project (December

2015, based on 2014 data) is equal to or greater than 65% and lower than 85% of the target Target Not Achieved: the value of the indicator at the end of the project (December 2015,

based on 2014 data) is equal to or lower than 64% of the target Not evaluated: no data is available to assess the achievement of the target and/or the value of

the indicator.

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Overall program indicator: Reduce infant mortality rateValue: 42 25 17

Date: 2005 January 2015 (2013 data)

December 2015(2014 data)

Comments: Target SurpassedOverall program indicator: Reduce chronic malnutrition of children under the age of 5Value: 38.2 30.2 23.7%



Comments: Target SurpassedKPI #1: Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2013) – proxy indicator for Maternal MortalityValue: 44% 78% 74.2%



vi

Comments: Target Achieved (89%). KPI #2: Reduce the prevalence of anemia among children under age in the nine regions from 69.5% (2005) to 60% (2013)Value: 69.5% 60% 57.3%



Comments: Target Surpassed KPI #3: Increase from 64% to 80% the share of children in the nine selected regions who are exclusively breastfed until 6 months of ageValue: 64% 80% 87%



Comments: Target SurpassedKPI #4: Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2013)Value: 41.5% 35% 36.4%



Comments: Target Partially Achieved (78%). KPI #5: Reduce the hospital lethality rate among neonates in the nine selected Regions from 9.5% (2005) to 5% (2013)Value: 9.5% 5% 5%



Comments: Target Achieved (100%). KPI #6: Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2013)Value: 20% 45% 69.1%



Comments: Target Surpassed

vii

(b) Intermediate Outcome Indicator(s)The rating scale is the same as the one above used for the PDO Indicators.

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

IOI #1: Percentage of SIS affiliated children who received growth and development controls (CRED) according to their ageValue 34% 66% 56.8%

Date 2005 January 2015 (2013 data)


Comments: Target Partially Achieved (71%). IOI #2: Percentage of health facilities with improvement in infrastructure (minor construction, rehabilitation and/or new equipment)Value 0 104 Dropped 104

Date 2005 January 2015 (2013 data) 2014 December 2015

(2014 data)Comments: Target Achieved (100%) (dropped at the 2014 restructuring but reintroduced for the ICR, based on government’s data). IOI #3: Percentage of SIS affiliated rural pregnant women with laboratory tests on hemoglobin, urine and syphilisValue 37% 80% 53.5% 69%

Date2005 January 2015

(2013 data)

December 2015

(2014 data)


Comments: Surpassed against revised target; 74% achievement against original target.IOI #4: Percentage of pregnant women under SIS that receive iron and folic acid supplementsValue 37% 80% 60% 55%


(2013 data)

December 2015

(2014 data)


Comments: Revised Target Partially Achieved (78%); 42% achievement against original target.IOI #5: Percentage of women satisfied with the services in selected facilities by confidence indexValue N/A 25% Dropped 74.4%


(2014 data)Comments: Surpassed (dropped at 2014 restructuring, but reintroduced for the ICR). Based on ISR data, the target was 25% and the value after the survey was conducted was 74.4%. IOI #6: Percentage of cesareans in SIS affiliated pregnant rural womenValue 3% 5% 10% 9.5%


(2013 data)

December 2015

(2014 data)


Comments: Revised Target Achieved (93%); surpassed against original target. IOI #7: Percentage of references among SIS-affiliated women (during pregnancy and puerperium) and neonatesValue N/A 5% Dropped N/A


(2014 data)

viii

Comments: Not evaluated. Indicator not monitored throughout the project and dropped at restructuring (2014)IOI #8: Percentage of SIS affiliated households that make out-of-pocket expenditures in medicinesValue 67.3% 25% 55% 56%


(2013 data)

December 2015

(2014 data)


Comments: Revised Target Achieved (92%); 27% achievement against original target.IOI #9: Number of accredited health establishment by type of resolutionValue 80 169 Dropped N/A


(2014 data)Comments: Not evaluated. Indicator not monitored throughout the project and dropped at restructuring (2014)IOI #10: Number of Management Agreements in placeValue N/A N/A Dropped N/A


(2014 data)Comments: Not evaluated. Indicator not monitored throughout the project and dropped at restructuring (2014)IOI #11: Number of health personnel and community health workers trained within the behavior change campaign (EPPES)Value 0 758 1178



Comments: Surpassed (added for the ICR - based on Government data)IOI #12: Number of health facilities improved Value 0 73 69



Comments: Achieved (95%) (added for the ICR - based on Government data). This indicator measured the number of facilities that were either built, re-built or expanded by the project (i.e. major constructions).IOI #13: Norms and regulations to improve efficiency and equity of the health delivery system preparedValue 0 27 32



Comments: Surpassed (added for the ICR - based on Government data)IOI #14: Clinical pathways and corresponding financing systems designedValue 0 11 16



Comments: Surpassed (added for the ICR - based on Government data)IOI #15: Periodic evaluations of the performance of the health networks preparedValue 0 17 19



ix

Comments: Surpassed (added for the ICR - based on Government data)

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements(USD millions)

1 06/19/2009 Satisfactory Satisfactory 0.00 2 12/11/2009 Satisfactory Satisfactory 0.00 3 06/28/2010 Satisfactory Satisfactory 0.00 4 02/23/2011 Satisfactory Satisfactory 0.86 5 08/05/2011 Satisfactory Satisfactory 1.20 6 01/23/2012 Satisfactory Satisfactory 3.48 7 09/22/2012 Satisfactory Satisfactory 4.05 8 05/10/2013 Satisfactory Moderately Satisfactory 5.65 9 12/21/2013 Satisfactory Moderately Satisfactory 7.40

10 07/12/2014 Satisfactory Moderately Satisfactory 7.96 11 12/19/2014 Satisfactory Moderately Satisfactory 9.27 12 06/17/2015 Satisfactory Moderately Satisfactory 9.73 13 12/30/2015 Moderately Satisfactory Moderately Satisfactory 11.98

H. Restructuring

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes MadeDO IP

06/20/2011 No S S 1.20

Reallocation of funds to increase funds allocated to Component 2 and reduce those for Components 1 and 3.

08/25/2014 No S MS 8.58

Based on the MTR, the restructuring (a) revised the results framework; (b) extended the Closing Date of the Project to Dec 31, 2015; (c) increased the threshold for firm contracts to US$300,000; and (d) changed the disbursement estimates.

x

I. Disbursement Profile

xi

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

1. Peru was a growing economy, but with persistent inequalities, and undergoing a demographic transition. At appraisal, it had registered strong economic growth, with a 7.6% GDP growth in 2006, 9.0% in 2007 and 9.3% in 2008. This progress contributed to a substantial reduction of poverty. The national poverty rate dropped from 48.6% in 2004 to 39.3% in 2007, while extreme poverty fell from 17.1% to 13.7%. However, inequalities and disparities across regions remained a challenge. Extreme poverty was 3.5% in urban areas and 32.9% in rural areas in 2007. From being heavily rural in 1950 with 33% of the population living in urban areas, by 2007 this proportion had increased to 76%. In terms of age structure, of a population of more than 27 million in 2007, 33% were less than 15 years old and 4.8% over 65.

2. By appraisal, Peru had advanced on some health-related MDG outcome indicators; yet, improvements were not uniform ― across all socio-economic groups, regions, and between rural and urban settings, revealing persistent inequalities. Despite its overall decrease, in 2006 the infant mortality rate (IMR) varied from 5 per 1,000 live births in the richest quintile to 45 in the poorest; Lima had a low IMR of 20, but Cusco has the highest at 84. While mortality in the post-neonatal period decreased, the relative share of perinatal mortality as a cause of infant deaths increased. This was due to conditions related to both demand and supply side factors (e.g. low institutional delivery rate, lack of immediate attention for newborns) and strongly linked with maternal malnutrition – more than a quarter of pregnant women, age 15 to 49, suffered from anemia (ENDES 2000-1). Despite progress on nutrition outcomes, one-quarter of Peruvian children under five suffered from chronic malnutrition, while 69% of children under two suffered from anemia. Located at high altitude, the regions of Huancavelica, Huánuco, and Ayacucho, among the poorest of Peru, were the ones with the highest stunting levels (more than 40 percent). At 164 deaths per 100,000 live births, Peru’s maternal mortality ratio (MMR) was almost double the Latin American average. In 2006, institutional delivery in urban areas was 92% and only 44% in rural areas. Finally, financial obstacles still represented a significant barrier to access. In the poorest quintile, 34% of individuals reported they had no access to health care for lack of money, while in the richest quintile only 6% did (ENAHO 2006).

3. Evolving health system. The Ministry of Health (Ministerio de Salud, MINSA) had taken some steps to strengthen accountability within a fragmented health care system in an increasingly decentralized environment. Management Agreements (MAs) were adopted to set goals for the Regions/municipalities’ health networks and results-based budgeting was increasingly used. The Comprehensive Health Insurance (Seguro Integral de Salud, SIS), created in 2001 and covering over 16% of the population, reimbursed MINSA public providers based on agreed upon health plans and covered predominantly vulnerable population living in poverty or extreme poverty, although not all in need.

1

4. Rationale for Bank assistance. The project evaluated in this ICR was part of a two-phase Adaptable Program Loan (APL) to support the Government of Peru (GOP)’s Health Reform Program (Programa de Apoyo a la Reforma del Sector Salud – PARSALUD). In both its phases, the overall PARSALUD program aimed at improving maternal and child health outcomes in Peru. The APL series was built on an ongoing dialogue with the Government of Peru (GOP) and on prior analytical work (e.g. RECURSO) and lending operations in the health sector (e.g. Basic Health and Nutrition Project – P008048).

5. The first phase (APL 1) in support of PARSALUD I was the Mother and Child Insurance and Decentralization of Health Services Project – P062932, which started in July 2001 and closed in June 2006. The planned investment under PARSALUD I amounted to US$239 M, jointly funded by the International Bank for Reconstruction and Development (IBRD, i.e. the Bank) (US$ 87 M), the Inter-American Development Bank (IADB) (US$ 87 M), the GOP (US$ 64.3 M), and the OPEC Fund (US$ 8 M); the total actual investment was US$232 M ― GOP (US$ 176.80 M), Bank (US$ 27 M), and IADB (US$ 28 M). The objective of the first phase of the program was to improve maternal and child health and to help reduce morbidity and deaths of the poor from communicable diseases and environmental conditions. The specific objective of the APL 1 was to increase access of the poor to better quality health programs and services. PARSALUD I was successful in reducing perinatal mortality and IMR and increasing skilled birth attendance by strengthening the demand and improving the quality of the supply of health programs and services. APL 1 contributed to the success of the overall program and its ICR (ICR000073) rated efficacy as substantial and the overall project as satisfactory ― confirmed by the IEG evaluation. Finally, a set of nine triggers was agreed on to demonstrate readiness for transition from phase I to phase II. All triggers were met at the end of phase I, with the exception of one, due to changes in regulatory framework, independent from PARSALUD1.

6. The project evaluated by this ICR (P095563) supported the second phase of the GOP’s PARSALUD and intended to sustain the achievements of the first phase. The overall objective of the second phase of the program was to reduce maternal and infant mortality and reduce chronic malnutrition of children under the age of 5. The specific objective of the APL 2 are described in section 1.2. The total planned investment under PARSALUD II amounted to US$162.40 M, funded largely by GOP (US$ 132.40 M) and co-funded by the Bank and the IADB (US$ 15 M each); the total actual investment was US$165 M ― GOP (US$ 138 M), Bank (US$ 11.98 M), and IADB (US$ 15 M). The project represented a small portion of domestic financing; yet, there was considerable demand from the GOP for the Bank’s support to policy reforms and for its fiduciary contributions to leverage an expeditious and efficient execution of policies and investments. The project also did provide additional funding for regional governments for investments and interventions not covered by budgetary allocations. Finally, the project was aligned with

1 A summary of triggers is reported in the PAD, pp 10-11. The trigger that was not achieved was the separation of financing and service provision within the Social Security Fund (ESSALUD). Changes in the regulatory framework granted more autonomy to ESSALUD and reduced the ability of MINSA to influence its institutional processes. Eventually, ESSALUD started piloting a separation of functions, but this was decided independently from the PARSALUD trigger.

2

the national and sector policies and with the Bank’s Country Partnership Strategy (CPS) for Peru for FY12-16, as well as with the Ministry of Economy and Finance (MEF)’s efforts towards results-based budgeting.

1.2 Original Project Development Objectives (PDO) and Key Indicators

7. This project was the second of the APL series to continue supporting the broader GOP program (PARSALD II). The specific program objective was to reduce maternal and infant mortality and reducing chronic malnutrition of children under the age of 5. The program indicators were: MMR2, IMR and chronic malnutrition of children under 5.

8. The stated objectives of APL2 were to continue supporting the Borrower’s effort to reduce maternal and infant mortality rates in intervened rural areas in Selected Regions in the Borrower’s territory, in particular through: (i) the improvement of family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three; (ii) the strengthening of health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three; and (iii) the supporting of MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment. The objectives were aligned between the Loan Agreement and the PAD.

The PDO-level indicators (Key Performance Indicators, KPIs #1-6) were:a) Increase the proportion of institutional deliveries in rural areas of the nine selected

Regions from 44% (2005) to 78% (2013)b) Reduce the prevalence of anemia among children under age 3 in the nine regions

from 69.5% to 60%c) Increase from 64% to 80% the share of children in the nine selected regions who

are exclusively breastfed until 6 months of aged) Reduce the prevalence of anemia among pregnant women in the nine Regions

from 41.5% (2005) to 35% (2013)e) Reduce the hospital lethality rate among neonates in the nine selected Regions

from 9.5% (2005) to 5% (2013)f) Increase in the proportion of pregnant women of the nine regions with at least 1

prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2013).

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification

9. Neither the PDO nor the PDO indicators were revised. However, as per restructuring paper dated August 20, 2014, the Results Framework (RF) was revised to reflect the new proposed end date of the project, which was moved from January 31, 2015 to December

2 Due to difficulties in monitoring MMR, institutional delivery was used as a proxy for maternal mortality by PARSALUD.

3

31, 2015. The targets of some Intermediate Outcome Indicators (IOIs) were revised and some IOIs were dropped (IOIs # 2, 5, 7, 9, 10). No additional indicators were added.

1.4 Main Beneficiaries,

10. The main beneficiaries were meant to be families (preponderantly rural) with pregnant women and children under age of three in the nine poorest regions in the country: Amazonas, Huánuco, Huancavelica, Ayacucho, Apurimac, Cusco, Puno, Cajamarca and Ucayali. The last two regions were also prioritized due to slow advances on IMR and MMR. Particular attention was to be targeted on indigenous populations as part of vulnerable and poor groups. Other beneficiaries included health professionals who benefited from training and improved infrastructure; local health managers, who benefited from training, tools, and technical assistance; and MINSA who benefited from technical assistance for enhancing government capacities, regulatory framework for local services, and M&E functions.

1.5 Original Components

11. The project consisted of four components:

Component 1 (total estimated costs US$6.00 million). Improving health practices at the household level for women (during pregnancy, delivery and breastfeeding) and children under the age of three in rural areas of selected Regions (demand-side interventions), by: a) design, implementation and monitoring of a behavioral change communication and education program to promote healthy practices at the household level, including increased demand for health services (Estrategia de Promocion de Practica y Entornos Saludables, EPPES); and b) promotion of SIS enrollment rights and identity rights of the targeted population (Derecho a la Identidad y Aseguramiento, DIA).

Component 2 (total estimated costs US$142.30 million). Increasing the capacity to provide better maternal and child health services for the poor (supply-side interventions); through: a) the improvement of the quality of services in health facilities of the nine regions; and b) the provision of support for the integrated health delivery model and the development of support systems to raise the efficiency and effectiveness of health networks.

Component 3 (total estimated costs US$5.20 million). Strengthening government capacities to offer more equitable and efficient health system in a decentralized environment (governance and financing) by: (a) supporting a regulatory framework and increasing quality in the provision of health services, (b) expanding the health insurance system (SIS) enrollment; (c) strengthening data monitoring and accountability in the system; and (d) supporting the decentralization of health services.

Component 4 (total estimated costs US$8.9 million) Project Coordination and Monitoring and Evaluation (M&E), through the provision of technical assistance, financing of incremental operating costs, and external and concurrent audits.

4

1.6 Revised Components

12. During the 2011 restructuring, the reallocation of Loan proceeds among disbursement categories was revised, as shown in Annex 11. The contribution to Components 1 and 3 was halved (from 50% of program costs to 25%), while the contribution to Component 2 was increased (from 6% of program costs to 8%). No changes were made to Component 4.

1.7 Other significant changes

13. The project underwent two level 2 restructurings. The first, in June 2011, changed funding allocations among components and supported more timely disbursement. It was motivated by the fact that, at the onset of the project, the GOP used domestic resources to advance expenditures for technical assistance activities originally planned to be financed with the loan funds, given that the budget allocation for the loan had not yet been approved by the borrower.

14. The second restructuring, in August 2014: a) Extended the Closing Date to December 31, 2015, to complete all Project

activities and to ensure full disbursement of loan proceeds, focusing on component 2 and 3;

b) Revised the RF to increase clarity, improve the accuracy of indicator definitions and data, and revised project targets in line with available evidence and feasibility of achieving targets. The indicator target dates were also adjusted to the new Closing Date;

c) Changed disbursement estimates to reflect the new action plan and respective procurement plan, and

d) Increased procurement threshold for prior review for consulting firms from US$100,000 to US$300,000.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

15. Project Design. The project design was based on the lessons of APL 1 outlined in its ICR (ICR000073). In particular, it was emphasized that: (1) the lack of a clear implementation strategy that would ensure results, among other factors, caused difficulties in the first phase of the program, which were overcome by using evidence-based research to focus on results; (2) political volatility was detrimental on project implementation and impact and, while ministerial influence was necessary; it was also important to implement project activities in coordination with the areas of MINSA responsible for specific project components; (3) there was a need to build institutional and managerial capacity at regional level before transfer funds directly to these local management units; and (4) intercultural strategies were vital when beneficiaries belonged to diverse indigenous groups and community participation was key to ensure sustainability. In addition, the second phase of the program, including its activities and

5

the Results Framework (RF), was informed by a comprehensive feasibility study finalized in 2008 (with a baseline for the indicators taken in 2005).

16. Due to a strong interest of the GOP at the time of appraisal on infrastructure investment, the project was largely focus on upgrading health facilities in the nine regions, where lack of infrastructure was identified as an important barrier to service access. Yet, these infrastructure investments were used as entry point to promote broader sectoral reforms and the use of evidence-based policy. These “soft interventions”, which were at the core of PARSALUD I, were less prominent in PARSALUD II; however, they are fundamental in ensuring sustainability of progress.

17. Project Preparation. Preparation of APL 2 started promptly, even before the closing of APL 1. As was the case for the first phase, APL 2 was also co-financed by the World Bank and the IADB, and therefore, project preparation was conducted in close collaboration with the IADB team3. However, a three years gap stands between the end of phase I and the effectiveness of the project supporting phase II. Changes in political priorities and leadership due to frequent changes in Government were the main causes of this delay. When a new Minister of Health was appointed in October 2008, an opportunity window opened up and project preparation regained traction. The project was negotiated in December 2008 and approved by the Board of Directors on February 17, 2009. The signing of the Loan Agreement only happened 9 months later, on November 16, 2009 and the project was declared effective on December 15, 2009.

18. Quality Enhancement Review (QER) and Decision Review Meeting (DM). The project underwent a QER in May 2006; the DM was held in December 2008. During both meetings, the element of the project that was mostly appreciated by the reviewers was the cultural adaptation of all the planned activities. Some of the issues and recommendations raised during these meetings not only revealed to be crucial to determine quality at entry, but were eventually found to be critical during implementation. These included:a) Establishing clearer links with the previous APL and, more broadly, better explaining

how the new operation would fit within the Peruvian institutional environment, which had changed from the end of the first phase of the program and, even more, from its original conceptualization;

b) Strengthening the M&E system by: (i) reducing the number of indicators, but establishing a clear results chain from activities to outputs and outcomes; (ii) reducing the number of data sources from which the indicators would be derived; and (iii) ensuring that the counterpart had the capacity to monitor the RF;

c) Assessing more realistically the risks deriving from the institutional environment which are outside the scope of the project, especially with regard to the links with SIS and the dependence upon approvals from the National Investment System (Sistema Nacional de Inversión Pública, SNIP). During implementation, it became clear that

3 The IADB project and the Bank project in support of PARSALUD II were aligned in terms of development objective, timeline and resources committed (US$15 million each). The results frameworks of the two projects slightly differed on some of the KPIs and IOIs. While the Mid-Term Review was conducted jointly and communications between the two teams was maintained throughout the life of the projects, supervision missions were largely carried out separately by the two co-founders.

6

SIS had gain much more independence than anticipated and coordination with PARSALUD gradually reduced. SNIP, which was created in Peru in the early 2000s, was rather rigid in its conceptualization of investment projects, with a non-participatory decision-making process and long approval time of programs, thereby compromising the possibility to efficiently make any changes to the projects during implementation. Furthermore, its clear preference for financing infrastructure investment left very little margin to incorporate other investment approaches, such as those adopted by APL 1, despite their proven effectiveness.

2.2 Implementation

19. Implementation of the project did not suffer from any major complications. The project had a slow start. While activities related to Components 1 and 3 started immediately, delays in disbursement and execution related mainly with the infrastructure investments under Component 2. Once construction works began and medical equipment started being purchased in 2011, project disbursement picked up towards the end of 2012.

20. As a result of the nature of the activities, project implementation was characterized by a high volume of transactions. Given the commitment to co-finance all civil works of the PARSALUD program with 6% funding coming from loan resources (i.e. 3% from the Bank and 3% from the IADB) and the remaining 94% from domestic resources, all procurement processes followed the Bank’s procedures. This was highly desirable from the perspective of MINSA, given the stricter Bank procurement guidelines, and it reflected a general trend in Peru at that time ― when the GOP was seeking external resources to fund rather small portions of broader national investment programs to benefit from streamlined procurement processes and technical assistance. Yet, this posed stress on the task team for the supervision of all transactions related to the program, efforts that were, therefore, disproportionate with respect to the resources committed with the loan.

21. While the program was successful in achieving its broader goals of reducing maternal and infant mortality and chronic malnutrition in children under 5, progress on the indicators in the RF was mixed during the life of the project. Some of the targets were achieved even before the MTR in February 2013, while others had a more fluctuating trajectory. To some extent, this was related to the delays in construction works, which shortened the time horizon available to see the impact of the infrastructure investments on the selected health outcomes and outputs. In other cases, the trend of some indicators reflected those at the national level, such as in the case of the prevalence of anemia among pregnant women. The delays in progress on some indicators motivated the downgrade of the Overall Project Implementation (IP) rating from satisfactory to moderately satisfactory in 2013, rating which was then kept in consideration of the disbursement delays ― eventually, the project disbursed 80% of the planned amount.

22. The Mid-Term Review (MTR) in February 2013 identified some of the challenges and correcting measures, including the needs to better coordinate with other relevant units within the MINSA and regional governments, strengthen M&E, increase loan disbursement (by then only 35%), modify the RF to reflect the actual starting and end date of the project, and support the reform process within MINSA by increasing capacity

7

of and coordination with the regional and levels. Some of these recommendations were immediately taken on board, including for example strengthening the M&E function within PARSALUD. Others were only partially addressed during the life of the project.Overall, the following implementation strengths were identified:

(i) Project Implementation Unit (PIU)’s capacity and commitment. The PARSALUD PIU was reconstituted in 2009, after a period of two years from its closure after the program’s first phase. The capacity of the PIU for project coordination was assessed as satisfactory and the unit was generally appropriately staffed (see Annex 12). The role of project coordinator was stable (two main coordinators and two acting for a very brief period of time). The project coordinators and many of the key personnel were very committed and remained within the team throughout project implementation, ensuring continuity and supporting improved capacity of the PIU. The PIU was found to be very proactive in a number of areas. For example, the PIU accepted all communications and documents to be sent electronically to the IADB and the Bank, which is not common practice in Peru. The PIU developed and made available checklists to constructors to ensure environmental safeguards were abided by and monitored compliance independently. Finally, the PIU worked closely with each Direccion Regional de Salud (DIRESA) to support local and regional-level interventions (such as the EPPES) and to strengthen capacity at the regional and local level based on the needs of each region.

(ii) Cultural adaptation of interventions. The systematic strengthening of health rights and empowerment of the population in rural areas about social participation in health through the EPPES and DIA campaigns were very well received by the local communities. These initiatives, together with the prior consultations held with local communities before the start of all civil works under PARSALUD, helped regional administrations build capacity for intercultural strategies, including communication in local languages (e.g. Quechua and Aymara). As a result of those culturally-sensitive interventions, coverage from SIS increased in the lowest quintiles in the Project areas.

(iii) Contribution to the Identity and Insurance Rights movement. PARSALUD was very active in catalyzing efforts to support the Derecho a la Identidad y Aseguramiento (DIA) for the health sector. This was an intersectoral initiative, in collaboration with the RENIEC and civil society, and with a strong regional and local commitment, to which PARSALUD contributed by supporting a campaign to promote the issuance of the Live Birth Certificate to children under 3 years of age and the National Identity Document to pregnant women and mothers.

The following implementation weaknesses were identified:

(i) Political changes within MINSA and regional governments. Four Ministers of Health changed during project implementation. Despite posing some concerns about stability of political commitment for the project and its key staff, eventually political changes at the national level did not substantially impact on project implementation. On the other hand, frequent political changes in the regional governments meant that PARSALUD PIU had to constantly re-engage with new administrators and staff, requiring duplication of efforts for capacity building at the local level.

8

(ii) Delays in civil works and consultancies. As of December 2012, after 2 years of implementation, about 27% of project's civil works were delivered, with heterogeneous patterns among the regions. Logistic difficulties for timely civil works completion were associated with difficult climate and access to sites, as well as with a few cases of collusion, properly addressed by the counterpart. Delays in hiring consultants were due to, among others, frequent changes in regional administration, scarcity of professionals adequately qualified for some tasks, delays in approval processes from the regions, and political attention diverted to sudden public health emergencies (e.g. pneumonia in Puno and dengue in Ucayali). Despite all delays, almost all planned civil works were completed before the project closing date, with a few being delivered in 2016.

(iii) Underestimation of the impact of other programs or initiatives on progress on the PARSALUD indicators. In some cases, project indicators showed irregular progress. This was partially due to other government programs providing incentives contrasting with those provided by PARSALUD (e.g. distribution of formula milk which at times affected exclusive breastfeeding of infants), and partially with changes in procedures for the registration with SIS (e.g. requirement of national identification document to register with SIS, which negatively affected the number of affiliates).

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

23. Design. The project’s Results Framework (RF) was derived from the PARSALUD program RF, which was based on a feasibility study (FS) finalized in 20084 and approved as the program proposal by the SNIP.

24. The indicators in the RF reflected all parts of the PDO. The PDO-level indicators (Key Performance Indicators, KPIs) focused on improving practices at the household level (PDO 1) and on strengthening the health service network (PDO 2). The Intermediate Outcome Indicators (IOIs) focused on PDO 2 and 3 (supporting MINSA’s government functions). The first three components of the project were also aligned with the three parts of the PDO. Given the funds allocation, KPIs and IOIs focused largely on PDO 2 and Component 2 (strengthening of health services networks – supply side). Component 3 (governance and financing) is reflected by a small number of IOIs, most of which were dropped in 2014.

25. Baseline data as of 2005 was available for all but three indicators, based on the FS. Targets were set against that baseline by imposing improvements greater than the expected improvements based on historical trends. However, delays in project preparation and effectiveness made the baseline and the targets outdated, but neither of these were revised. If unable to change the baseline and/or the targets at the time of appraisal, due to time constraints and lengthy government processes, within the context of the first restructuring in 2011, when data from 2009 was becoming available, these should have been used to update the baseline and the targets.

4 Segunda fase del Programa de Apoyo a la Reforma del Sector Salud - PARSALUD II. ESTUDIO DE FACTIBILIDAD. Nov 2008

9

26. Implementation. The project indicators were monitored using government’s systems and surveys. This had the advantage of not creating a parallel system. However, the PIU had to request or download data from different institutions responsible for data collection before being able to analyze it. There was a time lag of one year between the data collection and the data availability, so that data for a given year became available only in the following calendar year.

27. Due to unavailability of data at the time of the ISRs, two KPIs only started being monitored in December 2014 (ISR 11, with data of December 2013). Given that they had not been monitored, five IOIs were dropped during the restructuring in 2014. However, data was available for two of those, which were reintroduced for the purpose of the ICR. The other three indicators had not been clearly defined and monitoring was therefore problematic. Unfortunately, all three of those IOIs measured progress against the same part of the PDO (3). Since the PARSALUD PIU monitored more indicators than those monitored by the Bank, additional intermediate indicators might have introduced to replace the problematic IOIs to better assess improvements on PDO 3 ― for example during the first restructuring in 2011 or immediately after the MTR in 2013.

28. Within the PIU the M&E Unit originally included two specialists, one focused on Monitoring and the other on Evaluation. Following suggestions from the MTR, the M&E function was strengthened and the original unit was split into two to focus and strengthen each area ― supervision of program performance, and management of scientific evidence for enhanced effectiveness of the overall program (see Annex 12).

29. Utilization. Once data was processed, the M&E team analyzed data for each indicator and informed the technical team of the trends. If needed, the local coordinators within the DIRESA were contacted to understand the reasons for the variation in the indicators, especially with regard to birth, death and maternal anemia. However, given the difficulties in monitoring the full results chain and the delay in obtaining information on the indicators, data was not used by the local level to inform decision-making or revise practices and procedures in real time. It was used by the PIU to promote studies that supported increased knowledge and evidence-base policy, informing the design of technical guidelines and regulations that improved MINSA’s regulatory capacity.

2.4 Safeguard and Fiduciary Compliance

30. Safeguards. Given that the project (environmental category B) triggered the Environmental Assessment (OP/BP/GP 4.01) and the Indigenous People Safeguard Policies (OP 4.10), environmental and social safeguards were monitored. The Environmental Assessment was conducted in 2005; the Indigenous People Plan (IPP) was prepared in 2006. In 2013, specific missions assessed the compliance with environmental and social safeguards and proposed corrective recommendations where needed. More targeted safeguards supervision missions were conducted in 2013 to review implementation of the action plans in line with the EA and the IPP. The implementation of the activities under an intercultural approach related to Component 1 (including the EPPES, the DIA, and the prior consultations to IP for infrastructure construction) were rated as satisfactory. Similarly, the implementation of environmental safeguards was

10

rated satisfactory, given the proactivity of the PIU in promoting the use of checklists for solid waste and water management during infrastructure construction under Component 2.

31. Financial Management. The counterpart’s financial management performance was considered generally satisfactory. The PIU was appropriately staffed from the start and, despite delays in systematically adopting the official system for managing transactions used in all implementation units in the public sector (Sistema Integrado de Administración Financiera, SIAF), financial reports were timely and were found to be of satisfactory quality. Audit reports were provided on time and there were no qualified opinions. Until 2013, disbursement was very slow (39% in August 2013). Problems were related to a number of factors, including previous delays in civil works as a consequence of inadequate planning and due to the remote project locations, which did not offer incentives for enterprises to participate in the bidding process; and the contractors’ noncompliance with contractual clauses. The GOP established a condition of Pari-Passu for all civil works (6% of external resources), which limited the scope for accelerating disbursement of the loan. Finally, deferred payments for civil works at the end of the project (Ocongate Health Center and Health Center Chuquibambilla) and the cancelation of the scheduled execution of large amounts of consulting services (i.e. Diplomado APS PROFAM and Sistema Nacional de Sangre Segura) expected to be contracted before the project closing and executed during the project grace period, negatively impacted disbursement. As a result, total disbursement reached 80% at project closing.

32. Procurement. Given that the entire Government program followed the Bank’s procurement guidelines, the Bank reviewed and approved a large volume of transactions. The initial delays due to lack of planning by the counterpart and long processing time for No Objections by the Bank were addressed and resolved after the first two years of project implementation. Procurement delays remained associated mainly with the scarcity of qualified contractors and remoteness of the work sites.

2.5 Post-completion Operation/Next Phase

33. Building on the PARSALUD know-how, a follow on GOP investment program, the Programa Nacional de Inversion de Salud (PRONIS) was approved in 2015 and is now operational. In line with the decentralization, PRONIS allows regions to set their own priorities and request funding from the central level. The possibility of a follow-on Bank-funded operation to support this new investment strategy has been discussed. MINSA and PARSALUD presented concrete proposals to MEF and the Bank (including a logic framework for the new operation). Discussions are still ongoing until the political situation stabilizes after the national elections which are taking place between April and June 2016.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

The Overall Relevance Rating is: Substantial.

11

34. Relevance of Objectives: Rating High. The project was fully aligned with the CPS for FY12-16 for Peru (February 1, 2012; p.21) and with the priorities of MINSA for an investment program with a strong focus on reducing maternal mortality, infant mortality and undernutrition, and an interest in improving equity in access to services. Today, reduction of maternal mortality and especially of infant mortality remain important, while non-communicable diseases are increasingly capturing attention and funding due to increased burden. Malnutrition and anemia still remain very high on the Government agenda given that progress has been fluctuating, not only in the project areas, but more generally across the country. In addition, poverty and inequalities, especially in the project areas, are still far from being resolved. The World Bank is currently undertaking the first Systematic Country Diagnostics for Peru (the review meeting will be held before the end of FY16) and will start working on the Country Partnership Framework in the first quarter of FY17. Addressing inequalities, support to effective decentralization, and improved nutrition, all of which were at the core of PARSALUD, still remain highly relevant.

35. Relevance of Design: Rating Substantial. Given that this project supported phase II of the program and all triggers were met at the end of phase I, the APL design was maintained. MINSA gained experience in managing APL fiduciary rules and believed these would help reinforce accountability, expressing preference for this lending instrument.

36. The project design appropriately aimed to address both demand and supply side factors to improve maternal and child health outcomes in the nine regions, as well as to increase the capacity of the government at the national and local level. It had a strong intercultural footprint, which was very appropriate given the areas of focus under the program. It was also designed in parallel and to build synergies with another Bank-financed project aimed at improving nutrition outcomes by using conditional-cash transfers, the Juntos Results for Nutrition Project (P117310), approved in 2011.

37. As mentioned above, the RF was aligned with the project components and reflected the different parts of the PDO. At approval, the design of the RF still reflected baseline data of 2005, which could have been updated with more recent data, either at that stage or at the first restructuring in 2011. Data for some IOIs was not available and five indicators were dropped at the 2014 restructuring. More effort could have gone to incorporate in the project’s RF some IOIs included in the program’s RF to better assess progress with Component 1 and 3; to compensate for these weaknesses, several IOIs were included at the time of the ICR.

3.2 Achievement of Project Development Objectives

The Efficacy Rating is: Substantial. The overall efficacy rating is the result of the assessment of achievement of the three parts of the PDO, all of which are rated substantial.

Assessment of achievement of the program’s objectives

12

38. The project objective was to contribute to the overall GOP’s effort to reduce maternal and child mortality and improve chronic malnutrition in children ― and, as explained below, the project did contribute to this. A comprehensive impact evaluation would be needed to demonstrate the attribution of the achievement of the program objectives and of the PDO-level indicators the project. However, the case for project contribution to the achievement of the PDO, is supported by progress on the intermediate outcome indicators that are more directly attributable to the project.

39. In terms of the overall results of the program, between 2009 and 2014, Peru was able to reduce IMR from 42 to 17 against a target of 25, and chronic malnutrition of children under 5 years of age from 38.2% to 23.7%, against a target of 30.2% (last PARSALUD progress report). MMR per se was not monitored by the program due to uncertainty about the reliability of data; however, the indicator on institutional deliveries was used as a proxy. As seen below, this indicator moved from 62.2% to 74.2% against a target of 78%, between 2009 and 2014. The total number of maternal deaths was also reduced from 186 to 159 in the project areas, against a national average of 481 and 411 between 2009 and 2014. Hence, overall, PARSALUD was successful in reducing MMR and IMR and chronic malnutrition in children under 5 years of age. More detailed program results are found in Annex 3.

Assessment of achievement of the project’s objectives

40. In terms of the three specific project objectives, the ICR assesses the original indicators as well as additional indicators added at the time of the ICR to introduce additional evidence in support of the results chain to the PDO . All indicators were related to the three parts of the PDO statement, as follows:(i) Improvement of family care practices for women (during pregnancy, delivery and

breast-feeding), and children under the age of three:KPIs# 1, 2, 3, 4, 6; IOIs # 1, 11

(ii) Strengthening of health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three:KPIs# 1, 2, 4, 5, 6; IOIs # 2, 3, 4, 5, 6 and 12 (added)

(iii) Supporting MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environmentIOIs # 7, 8, 9, 10; and 13, 14, 15 (added).

41. IOIs #2 and 5 (PDO 2), 7, 9 and 10 (PDO 3) were dropped at the 2014 restructuring. At the time of the ICR data was only available to support IOIs #2 and 5, which were therefore reintroduced. Additional qualitative data was collected to support the analysis of PDO 3.

42. When assessed against the original baseline, all the indicators have surpassed, achieved or partially achieved the targets at the end of the project, which supports achievement of all parts of the PDO (Table 1). Also, notably, the final percentage of achievement does not reflect that roughly half of all indicators surpassed their targets. Therefore, the actual level of achievement is far greater than the table suggests.

13

Table 1: Achievement of PDO (targets against original 2005 baseline)

Program

levelPDO level Intermediate PDO 1* PDO 2* PDO 3

Target surpassed 2 3 6 4 4 3Target achieved or substantially achieved (>=85% met) 2 4 1 5 1Target partially achieved (65%-84% met) 1 2 2 2 0Target not achieved (<65% met) 0 0 0 0 0Unknown 0 3 0 0 3Total 2 6 15 7 11 7% surpassed and achieved 100% 83% 83% 71% 82% 100%

* Note: The table double-counts some indicators to assess PDO 1 and 2.

43. Given the fact that the baseline for this project which was approved in 2009 actually used 2005 data (for reasons explained above), the ICR team reviewed achievement of project indicators against the 2009 baseline (see Annex 10). Notably, many of the indicators already registered improvements in 2009 compared with the baseline values of 2005. This overall trend continued during project implementation, with all but one indicator (prevalence of anemia in pregnant women) following the positive trajectory. This supports the positive contribution of the project to the achievement of the PDO. In addition, many of the indicators for the project regions showed performance at least in line with the national average (see Annex 3), a very positive result considering the geographical and socio-economic conditions of the project regions.

44. The assessment below is based on the official 2005 baseline data, and when appropriate, also uses revised targets for IOIs.

PDO 1: improvement of family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three. Rating: Substantial.

KPI #1: Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2014) – Achieved

14

KPI #2: Reduce the prevalence of anemia among children under the age of 3 in the nine regions from 69.5% (2005) to 60% (2014) – Surpassed KPI #3: Increase from 64% to 80% the share of children in the nine selected regions who are exclusively breastfed until 6 months of age – Surpassed KPI #4: Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2014) – Partially Achieved KPI #6: Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2014) – Surpassed IOI #1: Percentage of SIS affiliated children who received growth and development controls (CRED) according to their age – Partially AchievedIOI #11: Number of health personnel and community health workers trained within the behavior change campaign (EPPES) – Surpassed

45. The project contributed to the increase of breastfeeding practices, one of the most important indicators for Component 1. Exclusive breastfeeding has been low in Peru, although in the project areas it was higher than the national average in 2009 (82.2% vs. 68.5%). The project greatly contributed to the uptake of breastfeeding practices in the nine regions, since the share of exclusive breastfed children in project areas rose to 87%, while the national average remained flat at 68.4%. This was in line with other GOP programs such as Juntos, although other programs aimed at providing food and formula supplements to poor families; therefore, achievements under the project seem to be remarkable.

46. Progress on anemia in the nine regions has been slow, reflecting national trends . Anemia has traditionally been a problem in Peru, often associated with lack of knowledge. Anemia in children under 3 has been stable between 2009 and 2014 in the project areas, while it has slightly declined as a national average (50.4 and 46.8). Anemia in pregnant women declined between 2005 and 2014 but it did not follow a steady trajectory. The prevalence of anemia, higher in the nine regions compared to national average, was relatively stable at the beginning of the project, at around 30%. However, after dropping to 24.3% in 2013, well below the national average of 28%, it started rising again to levels higher than the national average (36.4% vs 32.5%). In the project regions, the reasons for this trend seems to be related more to demand-side factors than supply-side factors. The availability of iron supplements at facility level has increased over time and the proportion of women who receives those supplements has also increased. According to PARSALUD staff, women admit that they are not taking iron supplements due to their bad taste, color and smell. New supplements have recently been purchased to overcome this issue; their effectiveness is yet to be assessed once the data for 2015 becomes available. The slow progress on the prevalence of anemia in women in Peru requires additional efforts to ensure stable and sustainable improvements.

47. The achievement of PDO 1 is supported by successful progress on a number of IOIs. All facilities that needed to be provided with audiovisual equipment to support the behavior change campaigns were in fact provided one (1,423). The project trained more a total of 1,178 health personnel and community health workers within the communication

15

and behavior change campaign (400 more than planned). According to SIS data, the biggest increase in the number of services provided to SIS-enrolled beneficiaries between 2011 and 2015 was for primary care services, which demonstrates greater accessibility of basic services for the poor, with a sharp increase in prevention as opposed to curative services. Finally, the campaign promoting identity and insurance rights (Derecho a la Identidad y Aseguramiento, DIA) was conducted twice in every region for 3 months each time, for a total of 6 months of campaing in every region ― 3 months less than originally planned.

48. PDO 2: strengthening of health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three. Rating: Substantial.

KPI #1: Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2014) –AchievedKPI #2: Reduce the prevalence of anemia among children under age in the nine regions from 69.5% (2005) to 60% (2014) – SurpassedKPI #4: Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2014) – Partially AchievedKPI #5: Reduce the hospital lethality rate among neonates in the nine selected Regions from 9.5% (2005) to 5% (2014) – AchievedKPI #6: Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2014) – Surpassed IOI #2: Percentage of health facilities with improvement in infrastructure (minor construction and/or equipment) -- AchievedIOI #3: Percentage of SIS affiliated rural pregnant women with laboratory tests on hemoglobin, urine and syphilis – Surpassed IOI #4: Percentage of pregnant women under SIS that receive iron and folic acid supplements – Partially AchievedIOI #5: Percentage of women satisfied with the services in selected facilities by confidence index – Surpassed IOI #6: Percentage of cesareans in SIS affiliated pregnant rural women – Achieved IOI #12: Number of health facilities improved – Achieved

49. Health infrastructure improved in the nine regions. The project supported the construction and/or renovation of 695 out of the planned 73 hospitals and basic health centers that offered obstetric and neonatal services; the remaining 4 are being completed in 2016, financed by GOP. It provided all of the 104 pre-identified health centers with medical equipment for maternal and child care and installed IT systems in 55 centers to support the implementation of the e-Health plans (planned 54). 5 More precisely, one health center underwent two types of renovations and received two sets of equipment; hence although technically the number of health centers renovated and equipped was 68 and 103 respectively, the number of renovation works done and sets of equipment delivered was 69 and 104 respectively.

16

50. The project delivered capacity building programs for different groups of health professionals using a culturally-sensitive approach. The project contributed to the training of 956 health workers (nearly 300 more than planned) on maternal and child care and specifically of 282 doctors and midwives on the vertical delivery (Parto Vertical) in 7 health facilities located in 4 regions within PARSALUD. Together with improved infrastructure and training, cultural adaptation of health facilities for the provision of vertical deliveries and support to Maternal Waiting Homes (Casas Maternas) were key in improving rates of institutional deliveries among indigenous populations. This includes the creation, dissemination, and systematization of knowledge around vertical delivery practices among health care providers and the institutionalization of this method in both regions: in the Amazon and Cusco, 54% and 33% of all deliveries were vertical, respectively, in 2012. The Casas Maternas, communal space managed, built, and maintained by communities and local governments, helped address the physical and cultural barriers faced by indigenous women residing far from health centers. There are 475 operational Casas Maternas throughout the country, with Cusco, Puno, Huancavelica and Apurimac the most important areas of reference.

51. The project strengthened the networks of services and improved quality of care and access to emergency services. In order to ensure a more effective network of services, all 9 regions elaborated a plan for the improvement of the referral and counter-referral system. The project also trained key health personnel in management of human and financial resources (1,336 actual vs. 1,143 planned).

52. On quality of care, the project funded a specific training and implementation program to reduce maternal pre- and post-partum bleeding through the introduction of a supervised medical protocol based on the use of specific evidence-based interventions and medicines, which has been shown to reduce post-partum hemorrhage. According to a survey done in 2012, the reported satisfaction among women who used the services was 75%.

53. The rates of C-sections in women affiliated with SIS increased in line with the intention of ensuring that women in need were actually able to access this service. While the national average has increased to 12%, in the PARSALUD region it seems to have been stable since 2012. This seems to suggest that while excessive use of those services might be starting in Peru, as it is already common practice across Latin America, in the project area those services are provided only when required.

54. PDO 3: supporting MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment. Rating: Substantial.

KPI #8: Percentage of SIS affiliated households that make out-of-pocket expenditures in medicines – Partially AchievedKPI #13: Norms and regulations to improve efficiency and equity of the health delivery system prepared – Surpassed

17

KPI #14: Clinical pathways and corresponding financing systems designed – Surpassed KPI #15: Periodic evaluations of the performance of the health networks – Surpassed

55. The achievement of the IOIs suggests that progress towards PDO 3 was significant. PARSALUD II developed 32 proposals for norms and legal acts to improve efficiency and equity of the health system for approval by MINSA, when they had only planned for 27. The design of the clinical pathways and corresponding financing system was also finalized by the project (16 actual vs 11 planned). The capacity of analyzing performance of the health system also improved through the completion of regular performance reports (19 produced vs. 17 planned) – although the majority of those were released in 2015, rather than being released constantly throughout the project and periodically updated.

56. PARSALUD II supported MINSA in reformulating its regulatory framework to ensure better quality of the health delivery system, in particular on purchasing, prescribing, and monitoring the quality of pharmaceuticals. Peru’s consolidation of a purchasing system for essential medicines lowered their costs for the country. In addition, a comprehensive training system in essential drug prescriptions was implemented in the project’s health facilities, and support was provided for improving the regulatory framework for quality control using SUNASA (Superintendencia Nacional de Salud).

57. The project strengthened the regional management capabilities on MCH through specialized training in neonatal and obstetric competencies for health professionals and managerial capacities for the Health Regional Directorates (DIRESA). Budget executions for functions related to maternal and child care and especially nutrition improved in the nine regions following capacity building interventions between 2012 and 2014 (e.g. Cusco moved from 88.7% in 2012 to 97% budget execution in 2014 on maternal and child health, while Amazonas moved from 77.5% in 2012 to 91.6% in 2014 on nutrition)

58. Additional qualitative information supports significant progress on PDO 3. This information was collected during the ICR mission and triangulated with findings by PARSALUD evaluations derived from focus group discussions and individual interviews conducted with informants at MINSA, PARSALUD team, regional administrators, health professionals, and beneficiaries. The technical assistance provided by PARSALUD supported a cultural change within the regional administrations and health professionals towards an approach that recognizes the human right of pregnant women and children to be treated fairly and with dignity, which has been appreciated by the beneficiaries. The project assisted the decentralization process by providing targeted assistance to the regions that needed more specific support with the regulation function. Topics of focus varied depending on the needs of each region, ranging from human resources to supply chain management. Progress were made on enhancing transparency, with a portal and website created for PARSALUD. In addition, central planning for civil works was strengthened as delays in starting and concluding civil works at the beginning of the project reduced over time. Overall, this evidence suggests that PARSALUD promoted significant progress in improving the government capacity on regulation, quality, equity, and efficiency.

18

3.3 Efficiency

The Efficiency Rating is: Modest

59. Rather than carrying out a separate economic analysis, the PAD referred to the one done in the FS. The ICR could not update the analysis done in the FS; instead, a more qualitative assessment of project efficiency and cost-effectiveness was undertaken (see Annex 3).

60. Cost-effectiveness was overall substantial, especially considering that the Bank loan leveraged a much more significant amount of domestic resources; on balance, in consideration of the delays and the disbursement level, efficiency is considered modest.Allocative efficiency

61. Maternal, newborn and child health (MNCH) interventions are recognized by the literature as highly cost-effective investments. Particularly effective interventions in MNCH packages would be management of labor and delivery, care of preterm births, and treatment of serious infectious diseases and acute malnutrition 6 ― all key focus of PARSALUD. Such MNCH interventions, targeted to those most in need, can prevent maternal and infant deaths and reduce the healthy years of life lost due to disability, thereby benefiting the health sector and society as a whole 7,8,9.

62. Lack of infrastructure and professional attention, under registration of beneficiaries on the public health insurance (SIS) and poor managerial and logistic capacity on local health units were identified as barriers to health service availability for women and children in rural areas. Therefore, closing the gap in health care access required targeting resources on the identified barriers and on vulnerable rural populations. The return on infrastructure investments have clear long-term benefits, which facilitate improvements in the provision of quality services. In addition to infrastructure investments, the project funded some important and highly cost-effective demand-side interventions under Component 1. These evidence-based interventions supported the growing health service demand, improved the quality of care and were at the core of PARSALUD I; a stronger focus on similar interventions would have further increased the cost-effectiveness of PARSALUD II.

Efficiency of project preparation and implementation

6 Black R, Levin C, Walker N, Chou D, Liu L, and others. 2016. "Reproductive, Maternal, Newborn, and Child Health: Key Messages from Disease Control Priorities, 3rd Edition." The Lancet. Published online 9 April 2016 7 World Health Organization the case for Asia and the Pacific, Investing in Maternal Newborn and Child Health, Geneva, Switzerland. 8 Bill and Melinda Gates Foundation, Maternal, Newborn and Child Health Strategy Overview, www.gatesfoundation.org. 9 Black R, Laxminarayan R, Temmerman M, and Walker N. editors. 2016. “Reproductive, Maternal, Newborn, and Child Health. Disease Control Priorities, third edition, volume 2”. Washington, DC: World Bank. doi:10.1596/978-1-4648-0348-2. License: Creative Commons Attribution CC BY 3.0 IGO

19

http://www.gatesfoundation.org/

63. Project preparation was characterized by delays leading to a three-year gap between the end of APL 1 and APL 2 effectiveness. Delays were largely due to the political economy in the country. Once the project gained political traction, it was rapidly appraised and negotiated.

64. The project was implemented within the time period originally planned. The project was extended by 11 months, which balanced the 10-month gap between Bank approval and effectiveness. However, despite the extension, the project disbursed 80% of the planned amount, due to delays in procurement and civil works10.

65. Given the nature of the project, the evaluation of Project efficiency should consider the Bank’s overall program financial contribution. In this sense, the project was extremely cost-effective, since by supporting less than 10% of the total program’s costs, it leveraged nearly US$ 140 million of domestic resources, in addition to the US$ 15 million provided by the IADB. Despite the initial civil works delays and the 80% final loan disbursement ratio, the program’s actual costs were fundamentally in line with the amount of resources identified at appraisal.

66. The implementation efficiency of the overall Government program was leveraged by the project funds and the use of Bank fiduciary safeguards that provided cover for the weaker Government systems and processes. The Bank project team reviewed procurement bidding documents and contracts for civil works, consultancy services and provision of training for the overall program. This effort increased the leverage of technical and fiduciary implementation support beyond the loan funds and enhanced the technical skills of the PARSALUD technical and fiduciary team.

Fiscal impact and sustainability

67. The fiscal impact of the project was marginal, limiting sustainability concerns . Project implementation did not have a major impact on the MINSA budget, as it weighed an average of 0.08 percent throughout the period analyzed. Once the facilities were built, improved or better equipped through the program’s funds, the responsibility for the management of those was transferred to the regional government. By signing the Convenios (contracts between PARSALUD and each of the nine regions), regional governments committed to allocate budget for maintenance of the upgraded facilities as a condition for starting civil works in pre-identified facilities in that specific region.

3.4 Justification of Overall Outcome Rating

The Overall Outcome Rating is: Moderately Satisfactory. This rating takes into account that the project is the second in a series of two APLs to support a broader GOP program that has been largely successful in achieving its goal of improving maternal and child health.

10 Until three months before the project closing date, the PARSALUD team assured that the project would disburse 100% of the loan. However, this did not materialize due to legal problems with regard to a big consultancy contract and delays in civil works, which are being funded by domestic resources.

20

Table 6: Summary of Outcome RatingsProject Outcome Ratings

Relevance SubstantialEfficacy SubstantialEfficiency ModestOverall Outcome Rating Moderately Satisfactory

3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

68. The project targeted specifically nine of the poorest regions, home of indigenous people and isolated communities, and within those communities focused on the most vulnerable groups ― women and children in the early years of life. The equity dimension was important in conceptualizing the project, which contributed to social inclusion and promoted a solid basis for personal identification and for the affiliation to the SIS. The project was mindful of local identities and practices and successfully built on those to select interventions that were culturally appropriate for a specific context, such as the parto vertical. This was key in ensuring that people were more comfortable in accessing the services, with the aim of sustaining improvements in access to basic services.

(b) Institutional Change/Strengthening

69. The project showed progress in building capacity at the central level and most importantly at the regional and local levels, benefiting government representative and different categories of health professionals. For example, PARSALUD II contributed to the local and regional capabilities to conduct social dialogue with various stakeholders prior to project implementation. The prior consultations on project’s infrastructure (to build, improve and equip the health facilities for obstetric and neonatal care), was a coordinated effort involving indigenous populations, regional and local governments, which strengthened the ability of local and regional actors to seek consensus on health infrastructure building.

(c) Other Unintended Outcomes and Impacts (positive or negative)

70. The project contributed to increased inclusion of indigenous communities and to provide space for communities and especially women to be heard, increasing accountability of the system. At the end of 2013, the project organized a South-South Knowledge workshop on intercultural interventions in health aimed to showcasing PARSALUD’s best lessons learned and bringing regional experiences that informed and strengthened PARSALUD’s intercultural agenda. The main attendants to the workshop were representatives of the Pan American Health Organization (PAHO), the National Autonomous University of Mexico, the Ministry of Health of Venezuela and the Ministry of Cultures of Peru, among others.

21

71. The project had a pivotal role in fostering and guiding the identity rights movement in the health sector and in creating a platform for dialogue of different actors at the national and local levels. The systematic introduction of birth certificates as the first means of personal identification provided the legal basis for the registration to the SIS as well as other GOP programs and access to the related benefits.

72. The project improved awareness of environmental issues, helped create a conducive institutional environment to ensure environmental hazards are considered and dealt with, and built capacity at the central, regional and local level.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder WorkshopsNot available.

4. Assessment of Risk to Development Outcome

Risk to Development Outcome Rating: Moderate

73. The follow-on GOP program, Programa Nacional de Inversion de Salud (PRONIS), built on the lessons learned and know-how from PARSALUD. Financial sustainability of the program does not seem to be a concern, given that PARSALUD was already largely funded by domestic resources and funds have already been allocated for the new program investments. While the Project Coordinator and the Technical Coordinator have been transferred, many of the key PARSALUD personnel are now working on PRONIS, ensuring greater continuity of the follow-on investment program with PARSALUD. PRONIS is also supervising the completion of the outstanding PARSALUD consultancies and civil works to be delivered in 2016.

74. PRONIS focuses on 748 key health centers and facilities, many of which PARSALUD rehabilitated or improved. However, contrary to PARSALUD, PRONIS is not specifically targeted to the poorest regions and it does not support an explicit, a priori objective or topic. The regions will identify the specific outcomes of focus for the investments depending on their demographic and epidemiological characteristics and political priorities.

75. PARSALUD put in place systems to ensure sustainability at the local level, by promoting and supporting decentralization of responsibilities and functions and building capacity for better use of resources for maternal and child services, which would benefit the regions even in the context of PRONIS. However, the high turnover of staff in the regional administrations and the irregular progress of some of the indicators pose some questions on sustainability of achieved results if a systematic strategy to address those is not adopted. Nutrition and anemia in particular remain problematic at the national level. Therefore, ensuring more systematic progress on those indicators, especially among the most vulnerable population, would require alignment of national, regional and local commitment.

22

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry

Quality at Entry rating is: Moderately Unsatisfactory.

76. The project suffered from a long preparation time. The Concept Note review was held in November 2005, but appraisal and negotiations were only conducted in December 2008. However, the time gap between the end of PARSALUD I and the beginning of phase II was mainly due to changes in the political arena and consequent inconsistent commitment from the GOP to maternal and child health and nutrition through the program. When the project regained political traction, the project was appraised, negotiated and approved by the Board of Directors within 3 months (December 2008-February 2009). Once approved by the GOP, the Loan Agreement was signed in November 2009 and declared effective in December 2009.

77. The long preparation time might have been used more effectively for persuading the GOP to incorporate more of the “soft interventions” that characterized PARSALUD I. The main focus of the APL 2 was on infrastructure investment, reflecting a strong interest of the GOP. This was used as an entry point to incorporate some activities to support a reform agenda, evidence-based interventions and changes of practices at the individual and local level. These were in line with the highly effective activities promoted by PARSALUD I, although in PARSALUD II they had lost the prominence they had in phase I.

78. Finally, more effort could have been placed to address suggestions from the QER and the DM, which proved to be important during implementation. Concerns were raised during the QER and DM with regard to the need to strengthen the M&E capacity of the counterpart and the M&E design, and to better assess the project’s fit with the new institutional environment, evaluating the risks that this entailed. These concerns were only partially addressed. Additional measures to strengthen M&E capacity of the counterpart had to be taken during the MTR. The RF reflected a baseline which was outdated by the time the project started and it included a high number of indicators; yet it could have more clearly incorporated some IOIs to support the achievement of PDO 1. Institutional constraints, such as the rigidity of the SNIP, were underestimated by the project. Given that the project supported a broader program, any substantial changes would have required additional approvals by SNIP causing further details; hence, the RF was never revised during preparation, reflecting a baseline taken in 2005 and targets set against that baseline.

(b) Quality of Supervision

The quality of supervision is rated Moderately Satisfactory.

23

79. The task team conducted regular supervision missions approximately every six months to monitor the implementation of the project. Financial management supervision missions were at times held separately. In addition to formal missions, the team was in close communication with the PIU. Records of ISRs and Aide Memoire were kept, although in some instances these were not available in the Bank’s online system.

80. The ISRs reported progress on most of the indicators, although data on two PDO indicators only became available in late 2014, at which time five IOIs were dropped. The team was aware of the outdated baseline and targets and repeatedly asked the counterpart to update the RF; however, while changes were made to the RF, the counterpart did not agree to revise the baseline. The team could have taken the opportunity of the restructuring in 2011 to address this issue so as to better be able to monitor the performance of the project.

81. Given that the project triggered the Environmental Assessment (EA) (OP/BP/GP 4.01) and the Indigenous People Safeguard Policies (OP 4.10), environmental and social safeguards were monitored. An environmental safeguard mission carried out in June 2013, reported that, although the EA’s recommendations were not fully met, the seriousness of the counterpart on environmental issues and on their proactivity was remarkable. The mission produced a series of recommendations aimed to systematize the PIU’s efforts, which were ratified and implemented by the counterpart. A social safeguard supervision mission carried out on May 2013, made recommendations to improve the use of waiting homes for deliveries (casas de espera) and to increase the sustainability of the mobile health care teams (mobile brigades), both of which were adopted by the counterpart.

(c) Justification of Rating for Overall Bank Performance

82. On balance, the overall Bank performance is rated as Moderately Satisfactory.

5.2 Borrower Performance

(a) Government Performance

Government Performance: Moderately Unsatisfactory

83. The project could not count on constant political support. After a long preparation period in which government commitment to the project was variable, an incoming Minister of Health in October 2008 ensured that MINSA supported the investment program during its implementation, since it aligned well with the new political priorities. Throughout the life of the project, four ministers of health were in power. Laws and regulations were approved timely and Management Agreement with the regions were also finalized and implemented in line with the decentralization. While support from the highest levels of the Ministry was more stable, commitment from the Sub-Ministry level

24

was more variable, requiring additional efforts from the PARSALUD team and the co-founders to keep momentum and ensure continuity of implementation.

The political economy of the investment system in Peru and the relationship between MINSA and MEF/SNIP posed some difficulties for both project design and implementation. At the time of project preparation, MEF and SNIP, which had recently been instituted, were strongly focused on hard investments rather than incorporate soft elements in investment program. Unfortunately, given the SNIP requirements at the time, options to adjust the project in line with “soft” interventions were very limited for MINSA. For the same reasons, even though aware that the RFs for the project and for the program should have been improved, MINSA could not support the request to update the baseline, given that it would have required additional political approvals by SNIP. During the last supervision mission in December 2015, the team was informed that MEF had not secured funds for key PARSALUD personnel to be in place during the grace period (until April 2016); the task team urged MINSA to address this issue. In addition, while in 2013 MEF had in principle agreed on increasing the proportion of loan resources to be used under Component 2 (which was supposed to be 6% for the Pari-Passu), this commitment never materialized and MINSA was not successful in its negotiations with MEF on this aspect. As documented in the ISRs, financial planning and coordination between MINSA and MEF remained an issue throughout the project.

84. Coordination between MINSA and the project team was variable depending upon the political changes within MINSA. Tensions reached the highest levels in 2012, when the Project Coordinator left the PIU, but overall improved after the appointment of a new Minister of Health in the same year. In addition to informal and ongoing dialogue between MINSA and the PARSALUD team, representative of MINSA participated in the Steering Committee that regularly reviewed progress of the program, together with MEF, PARSALUD, and regional representatives. From 2011, improved planning and coordination between MINSA and the project team allowed for a reduction in the number of requests for No Objections and a smother processing of the same by the Bank, which resulted in less delays in implementation of Component 2.

(b) Implementing Agency or Agencies Performance

Implementing Agency Performance Rating: Moderately Satisfactory

85. PARSALUD PIU, an independent unit created within and by MINSA, was responsible for the technical coordination, planning, M&E, financing and administration, including procurement, contracting and payments, and accountability for the program; however, it had limited decision making power. The execution of the technical component remained under the relevant Directories of MINSA. The coordination capacity of the unit was assessed as satisfactory from the start, given prior experience with coordination of the first phase of the program and with the related Bank procedures.

86. The unit was adequately staffed. It included approximately 70 employees in 2015, in addition to some consultants. Its key personnel have been pretty stable throughout the life of the project. The Project Coordinator changed in 2012, after a political change in

25

MINSA. The new coordinator used to be the Technical Coordinator of PARSALUD, which ensured institutional memory and consistent support to the initiative. Some issues of coordination and dialogue between different teams within the unit could be detected.

87. The PIU complied with financial management procedures and reporting. The Finance and Administration Unit was appropriately staffed (6 employees) since the start of the project. For the first two years, integration of the information system and the improvement of the TASK POA to issue Interim Financial Reports and Statement of Expenditures was not complete. However, the PIU was still able to provide timely and reliable information for project monitoring through Excel. Transactions were well documented and financial reporting done in accordance with Bank requirements, as the financial audits confirmed.

88. The unit was proactive in flagging any suspected cases of collusion in procurement . For example, an INT case was opened in April 2011 on risks during execution of works in the Region of Huancavelica. PARSALUD II timely identified and corrected fraudulent practices from an enterprise and it responded positively to all requests made by the Bank, designing and implementing a Governance and Anti-Corruption Action Plan.

89. The unit was very active in addressing social and environmental safeguards. For example, even before the supervision mission in 2013, the unit had a dedicated and growing team that not only monitored compliance with the EA, but had also developed practical tools to be used during construction works to appropriately deal with solid waste and water management. The unit had a system to ensure these checklists were available to the construction companies and it monitored timeliness of use and compliance. The unit supported the development of a technical norm on liquid waste management, training, and the adoption and monitoring of plans for adequate solid waste and water management in facilities already constructed and those being constructed.

(c) Justification of Rating for Overall Borrower Performance

90. On balance, the overall rating for the Borrower Performance is: Moderately Satisfactory.

6. Lessons Learned

91. Some of the lessons learned and reflections that emerge from the ex-post evaluation of the project, considering the strengths and weaknesses of design and implementation, include: The importance of culturally-sensitive interventions to ensure effective access of

indigenous populations to health services (e.g. vertical delivery and casas de espera); this requires action on the demand as well as on the supply side, since both families and healthcare professionals need to understand their respective roles and perceptions;

The pivotal role of the health system in promoting civil registration and vital statistics (CRVS) through awareness campaigns and through its network of clinics and hospitals, so that, from birth, individuals can be identified ― with benefits that go

26

beyond the sector; this requires a strong coordination between the Ministry of Health and the authorities in charge of coordinating and administering CRVS programs. The health sector is well positioned to pioneer innovative ways of engaging with marginalized communities to this purpose, given the privileged entry point that health services offer;

The crucial role of M&E systems to monitor project performance and to ensure correction measures can be taken in a timely fashion and at the appropriate level; particularly, being able to recognize and address the weaknesses of M&E design early on and having a strong and meaningful RF from the start, which can be revised as needed to reflect the project’s success and implementation pace. The RF should reflect appropriate baseline and targets, identify indicators for outcomes and outputs directly attributable to the project and those to which the project only contributes, and present a clear results chain of PDO, outputs and project activities;

The importance of ensuring continuity of projects when financing a programmatic series; institutional environment and contextual factors can change substantially in the time frame of APL or DPL (Development Policy Lending) operations and keeping the momentum when transitioning from one phase of investments to the next is crucial for the relevance of the series of operations;

The difficult balance between the importance of maintaining ongoing dialogue and long-term engagement with a country, on the one hand, and the efficiency in the use of scares resources when pursuing relatively small investments in projects that require substantial levels of supervisions and high volumes of transactions, on the other. In this project, supervision efforts to oversee all program’s transactions were disproportionate compared to the size of the loan, which accounted for less than 10% of the program resources. As many countries transition from low-middle income to higher income levels, this questions becomes more and more relevant for Bank operations.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Borrower/implementing agenciesComments were received by the PIU on June 17 and 21, 2016 and were further discussed between the PIU team and the ICR Task Team Leader on June 21, 2016. Where appropriate, comments were incorporated in the main text of the ICR. All comments provided by the PIU are reported, as received, in Annex 7 after the executive summary of the Borrower’s ICR.

(b) CofinanciersNo comments were received from the IADB.

27

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

Strengthening of demand 6.00 6.43 107%Improvement of service delivery network 142.30 138.54 97%Government and Financing 5.20 3.25 62%M&E and Administration 8.90 16.86 189%

Total Baseline Cost 162.40 165.08 102%

Total Financing Required 162.40 165.08 102%

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest

Estimate(USD

millions)


Borrower 132.40 138.23 104% Inter-American Development Bank 15.00 15.00 100%

International Bank for Reconstruction and Development 15.00 11.98 80%

28

Annex 2. Outputs by Component

1. Component 1. Improving health practices at the household level for women (during pregnancy, delivery and breastfeeding) and children under the age of three in rural areas of selected Regions

a) Design, implementation and monitoring of a behavioral change communication and education program to promote healthy practices at the household level, including increased demand for health services (Estrategia de Promoción de Práctica y Entornos Saludables, EPPES);

(i) Identification of specific practices to be promoted, including antenatal care, institutional delivery, neonatal visits, exclusive breastfeeding, newborn feeding, care for sick newborns at home, feeding a sick newborn; within the affiliation to SIS and the health care rights and responsibilities.

(ii) Development and implementation of tailored EPPES strategies by region, focused on 202 districts.

(iii) Development, validation, and production of culturally sensitive printed and audio-visual materials (radio spots, soap operas, videos, etc.) in different languages, including Spanish, Quechan, Aymara, Awaji, Shipibo, and Wampi; for example, 13 modules of radio soap opera “Mi derecho a crecer” (My right to grow), and 5 short videos on healthy practices.

(iv) Behavior change campaign being featured in 45 radio channels and 18 television regional or local channels in local languages.

(v) Distribution of equipment for basic training and dissemination (PCs, data display devices, TVs, DVD) in 1,423 health centers;

(vi) Training of 380 community leaders, 412 local authorities, 720 healthcare personnel, and 720 community/civil society agents for the local implementation of the EPPES;

b) Promotion of SIS enrollment rights and identity rights of the targeted population (Derecho a la Identidad y Aseguramiento, DIA)

(i) 666,993 children under 3 years old and 1,888,531 women have been provided with the live birth certificate, national identification document and affiliation with SIS.

(ii) Design, production and dissemination of materials promoting SIS rights and identity rights for all nine Regions in Spanish, Quechuan, Aymara, Shipibo and Awaji.

29

(iii) Design and implementation of a campaign promoting SIS rights and identity rights, carried out twice (in 2011 and in 2013-2014) in the nine regions for the duration of three months each, focused on mothers, fathers, and careers.

(iv) Implementation of the Live Birth Registry and support to the Auxiliary Registry Offices for the prompt registration in six regions (Amazonas, Apurímac, Ayacucho, Cajamarca, Huánuco, and Huancavelica).

(v) Online system of birth registration in hospitals and more complex health centers in eighth regions (Amazonas, Ayacucho, Apurímac, Cajamarca, Cusco, Huancavelica, Puno and Ucayali).

(vi) Thirty six Auxiliary Registry Offices were installed in seven regions (Amazonas, Apurimac, Ayacucho, Cajamarca, Huánuco, Huancavelica, and Ucayali) and training workshops were organized for midwifes and RENIEC and SIS registrars.

(vii) Creation and strengthening of spaces for intercultural dialogue (Grupos Impulsores del Derecho a la Identidad y Aseguramiento) in the nine regions, with representatives of the regional Directorates of Social Development; RENIEC; JUNTOS Program; local government representatives for the sectors: Education, Health, and Women and Vulnerable Groups; and other NGOs and civil society organizations.

(viii) Seven regions approved regional directives for the issuance of the Live Birth Certificate (Amazonas, Apurímac, Ayacucho, Cusco, Huánuco, Huancavelica, and Ucayali).

(ix) All nine regions adopted regulations related to the timely and free-of-charge issuance of the Live Birth Certificate.

2. Component 2. Increasing the capacity to provide better maternal and child health services for the poor

a) Improvement of the quality of services in health facilities of the nine regions;

(i) Sixty nine health facilities were improved; these included new constructions and expansion or remodeling of existing facilities in the nine regions; the remaining four prioritized facilities will be delivered in 2016.

(ii) One hundred and four facilities were provided with new medical equipment in the nine regions.

(iii) Technical assistance to ensure proper use and maintenance of the new infrastructure and equipment for the administrators and the health teams.

(vi) All construction sites used the approved tools for environmental managements.

30

(vii) Fifty Environmental Impact Studies certified by DIGESA.

(viii) Evaluation of environmental liabilities and elaboration of Environmental Management Plan for 22 projects implemented.

(ix) Capacity building to 805 staff in comprehensive solid waste managementin 104 health facilities.

(x) Design of the Technical Standard proposal "Integrated Management of fluidwaste in health facilities and medical support services”.

(xi) Technical assistance to DIRESA for simplification of procedures related to authorization of septic tanks and infiltration and approval of the Program of Adequacy and Environmental Management for the transfer, treatment and disposal of solid waste.

(xii) Implementation of an Internship program for Emergency Obstetrics and Neonatal Care for a total of 674 participants, including 382 interns in diagnosis, stabilization and referral of obstetrics and neonatal emergencies (FONB), 256 interns in basic emergency obstetrics and neonatal care (FONE), and 36 interns in intensive care (FONI).

(xiii) Technical assistance to 11 hospitals in the nine regions to be qualified to host interns.

(xiv) Training of 282 health professionals in the prioritized health facilities (FONB) on vertical delivery, with 33 tutors and 6 training sites in 5 regions (Ayacucho (2), Cajamarca, Cusco, Huancavelica, and Ucayali).

(xv) Development and distribution of 180 DVDs with videos promoting vertical delivery across health centers in Ayacucho.

(xvi) Training of 440 health professionals in intercultural communication with Quechua-speaking population in Ayacucho, Apurímac, Cusco, Huancavelica, and Puno.

(xvii) Twenty four initiatives implemented though an ad-hoc fund (Fondo Concursable) in 167 health centers in the nine regions, by providing technical assistance to 24 technical teams to strengthen both technical and managerial skills to implement 333 activities identified in the action plans – benefiting a population of more than 370,000.

b) Provision of support for the integrated health delivery model and the development of support systems to raise the efficiency and effectiveness of health networks.

(i) Proposals for improving the referral and counter-referral system, in particular for maternal and neonatal service referrals, finalized in Puno and Ucayali and under development in Amazonas, Ayacucho, Apurímac, Cajamarca, Cusco, Huánuco y Huancavelica.

31

(ii) Technical assistance provided to DIRESAs and the health networks in the nine regions to improve budget execution by better planning for maternal and child health and nutrition between 2012 and 2014.

(iii) Training provided to 1,007 pharmacists and personnel responsible for pharmaceuticals on stock management and good storage practices for pharmaceutical products, medical devices and medical devices.

(iv) Implementation of the redesign of network storage and distribution of pharmaceutical products, medical devices and medical devices in eighth regions.

(v) Software developed for the regions to monitor the availability of pharmaceuticals and medical devices in the health centers.

(vi) Implementation of the Health Care Standards in 128 health centers by conducting 793 visits to provide technical assistance.

(vii) Implementation of the e-Health Network model of teleconsultations and tele-training, using a platform installed in two centers acting as national reference points (Nacional Materno Perinatal y Hospital Nacional Docente Madre Niño San Bartolomé) and in the Regional Hospital and health centers in the regions of Amazonas, Huanuco and Ucayali and Amazon, for a total of 48 primary care centers connected. Ongoing plans to establish the e-Health model in Huancavelica, Apurímac y Ayacucho.

(viii) Started the development of the interoperable national system of Registry of Teleconsultations.

c) Inclusion of intercultural focus in service provision

(i) Consultations in 12 districts in the nine regions through Intercultural Dialogues whose purpose was to promote cultural understanding and participatory decision-making in relation to the implementation of the program’s civil works.

(ii) Technical meetings for the analysis of maternal and neonatal health indicators with representatives of DIRESAs, Directorate of Social Development, Ombudsmen, universities, professional associations, CUNA MAS, UDR - SIS.

d) Increasing knowledge of effective delivery systems

(i) Study of prescribing practices in health facilities FONB and FONE in Huánuco and Cajamarca.

(ii) Mixed-method study on the provision of health care to children under 3 years of age in health facilities in 9 poor regions, Peru. 2013.

(iii) Qualitative evaluation of neonatal mortality in the regions of Huánuco y Ucayali, Peru. Biomedica: Revista del Instituto Nacional de Salud (Colombia)

32

(iv) Cause-effect analysis of hospital neonatal mortality in two Andean-Amazonian departments of Peru. Revista Panamericana de Salud Pública (OPS).

(v) Neonatal mortality, analysis of surveillance registries and clinical histories for neonates in 2011 in Huánuco and Ucayali, Peru. Rev. Peru Med. Exp. Salud Pública. 2014.

(vi) Study of factors related to anemia in children under 3 years of age in Peru: analysis of data from ENDES 2007-2013. Biomedica: Revista del Instituto Nacional de Salud (Colombia).

(vii) Four studies on users’ perceptions on drugs and their use in medical facilities.

(viii) Determinants of the use of contraceptives in adolescent and young girls who are sexually active, Peru 2012.

(ix) Analysis of the impact of investing in health facilities on maternal health indicators in regions under PARSALUD II.

(x) Study of knowledge, perceptions, and attitudes towards C-sections and blood transfusion in rural areas of the regions under PARSALUD II.

3. Component 3. Strengthening government capacities to offer more equitable and efficient health system in a decentralized environment.

This component aimed at: a) Supporting a regulatory framework and increasing quality in the provision of health services; b) Expanding the health insurance system (SIS) enrollment; c) strengthening data monitoring and accountability in the system; d) supporting the decentralization of health services

Component 3 outputs:

(i) Preparation of 32 technical norms and regulations for: (a) the accreditation and certification system, currently proposed by the law but not regulated, (b) infrastructure maintenance systems, (c) a reference laboratory system, (d) hemotherapy (e) hospital financing, (f) pharmaceutical purchasing and logistics system, and (g) a health communication and promotion system.

Proposal for the regulatory framework for Universal Health Coverage Policy guidelines on citizen participation and oversight under the Universal

Health Insurance. In 2011 the Guidelines for Citizen Oversight prepared in 2010, were approved

Proposed technical standards on Obstetric and Neonatal functions in health facilities

33

Inspection manual and guide of good practices in manufacturing pharmaceutical products (and validation)

Project for regulating organization and functions of SUNASA Technical standards of the Service Delivery Unit, Obstetric Center Regulation of the organization and functions of SIS – Personnel Allocation Table Regulation for the registry, control and surveillance of pharmaceutical products

and medical devices SIS regulations under the Universal Health Insurance Analysis of the national health policy framework in line with the regulations of

transferred functions (decentralization) Operationalization of the model of comprehensive health care at the primary care

level Document defining physical targets for 2012 for the Strategic Programs for

Maternal and Child Health and Nutrition, through the implementation of the IPMF (adjusted with recommendations of the MEF and MINSA) at the level of the DIRESAs, UE, Networks, Micro-networks and health facilities

Bill of law on financing for the subsidized and semi-contributory insurance Proposal for the salary scale of the MINSA and Regional Government personnel

in the Medical Career track Roadmap of the decentralization process Criteria to estimate the needs for pharmaceutical products and medical devices

used for the Health Priorities Supreme decree on the Mobile System for Emergency Care Clinical guideline for emergency obstetric care according to the level of care (x2) Strategic Plan of the AUS Manual of the organization and functions for the regulatory framework on

Universal Health Insurance and its regulations New LOF of MINSA Proposal for the Users’ Committee of the SUNASA; health norm (maternal,

newborn and child care); regional norm on the care for newborn and children under 3 years of age in the region of Huancavelica

Regulation related to the law on financing for the subsidized and semi-contributory system

Whitepaper on Identity and Insurance Rights Technical standards for the Service Delivery Unit for Intensive and Intermediate

Newborn Care Norm of SISMED Proposal for the standard identifier for medical devices Proposal for local decentralization Criteria for the Definition of Health Care Networks, with emphasis on maternal-

neonatal care Update of the technical standards for the vertical delivery (2015) Proposal for the regulation of telecare – remote diagnostics (2015) Accreditation model of the IPRESS, which includes:

34

o product 2: (i) proposal of whitepaper: model of quality health care accreditation by IPRESS, and (ii) proposal of health standards: manual of quality health care accreditation by IPRESS in Peru

o product 3: (i) proposal of regulation for the accreditation by IPRESS in Peru; (ii) Proposal of regulation for the certification of the agents qualified for certifying accreditation by IPRESS in Peru

o product 4: (i) proposal of methodology and clinical guidelines and tools for the accreditation by IPRESS (including the model of supervision of accreditation by IPRESS); (ii) proposal of the financing model for accreditation by IPRESS in Peru; and (iii) proposal for the model of information management for accreditation by IPRESS in Peru

(ii) Technical assistance supporting SIS and the decentralization of responsibilities in health care

Redesign of the Technical Document on Quality in Health, in the context of the National Policy on Quality in Health, Decentralization, and Universal Health Insurance, which sets the guidelines for the design and implementation of the Quality Management System in the health facilities

Systematization of the implementation process of the Universal Health Insurance in the MINSA and in the regions of Ayacucho, Apurímac and Huancavelica

Systematization of the evaluation of the exercise of the functions transferred to the regions, based on the application of MED

Technical and financial assistance for the development of the model for the supervision of SUNASA

Systematization of the M&E implementation process of the decentralization aimed at improving performance of the health functions

Computer application that automate the analysis and reporting of information from the monitoring system of availability of medicines and supplies for the delivery and obstetrics and neonatal emergencies (available on the PARSALUD II website)

Technical assistance for the proposal for the evaluation of results and monitoring of the implementation of the universal health insurance, which includes a set of indicators for the AUS baseline and the design of the evaluation

Technical and financial assistance for the development of the model for implementing the National System of Conciliation and Arbitration in Health

Design of the model for evaluation of staff and monitoring of the supervisions to IPRESS and the Management Units (2015)

Strengthening of the management of multi-year investments in health (2015) Proposal of the Coverage of the Benefit Package of FISSAL (2015) Design of the management model of FISSAL (including tax policy, relationships

with providers, payment mechanisms and incentives) (2015) Proposal for the baseline, evaluation of results and monitoring of activities for the

implementation of the AUS, presented to CTIN and SETEC Cost-benefit analysis of pre-hospital emergency and urgent care of the Mobile

System for Emergency Care National Program

35

Study of the status of transfer (of functions) from SIS to the implementing units Elaboration of the Annual Report to the Congress of the Republic of Peru on the

progress with the implementation of the AUS Evaluation of the potential impact of measures for the protection of intellectual

property in the access to biological products Analysis, identification and proposal for the legal regulations to support the

activities of the innovative model of Telemedicine Diagnosis of the care provided to SIS patients in pharmacies, both public and

private, in the context of the implementation of Inclusive Pharmacies

(iii) Support to capacity development through the creation of ad-hoc training programs, including:

Government and Management in Health (176 tutors and 332 health professionals trained in the I edition and 346 in the II edition, including doctors, nurses, obstetricians, and pharmacists), consisting of a Master in Government and Management in Health, Diplomas in different areas (Strategic Planning and Public Investment Programs, Health Management and Administration, Management of Health Networks, Hospital Management), and certificates for specific modules

Diploma in Information management for Health Interventions (76 tutors and 266 health professionals trained, including doctors, nurses, obstetricians, nutritionists, psychologists, and biologists)

(iv) Studies supporting the implementation of the PARSALUD program and the development of technical documents and proposals for regulations:

Analysis of factors associated with out-of-pocket spending in health, among thepoor population, in a context of a progressively increasing funding for comprehensive health insuranceDiagnostics of the information system in the nine regions

Evaluation of the implementation of the accreditation process of health serviceswithin the scope of PARSALUD II and proposal for improvement

Evaluation of maternal and child care practices in areas of extreme poverty in Peru, 2012 (Published in Rev. Peru Med. Exp. Salud Pública. 2014; 31(2):243-53)

Systematic review on effectiveness of community interventions on the reduction of neonatal mortality. Revista Peruana de Medicina Experimental y Salud Pública

Systematic review on effectiveness of community interventions on the growth and development of children under 5 years of age in rural areas. Revista Cadernos de Saude Pública – Brasil

Technical document generated from the International Conference “Towards Universal Health Coverage” (Hacia la Cobertura Universal de Salud)

Literature review on Comprehensive Health Care Networks

36

Literature review on the efficacy or effectiveness of pharmaceutical policies which contribute to improve access to essential drugs for the population

Review and development of proposals for improvement to the document “Guidelines and measures for the reform of the health sector”

E-Health in Peru: systematization of the experiences from 2002 to 2013. E-Health in maternal and neonatal services in the Peruvian Amazonia: towards an

integrated model. Evaluation and redesign of the health care model for dispersed populations Systematization of the experience with the behavior change campaign Estrategia

de Promoción de Practica y Entornos Saludables (EPPES) in PARSALUD II Systematization of the experience with the strategy for the Derecho a la Identidad y

Aseguramiento (DIA) in PARSALUD II Systematization of the experience with the Fund (Fondo Concursable) for the

selection and implementation of interventions in PARSALUD II Systematization of the experience with the strategy for Prior Consultations and

Intercultural Dialogue implemented by PARSALUD II Systematization of the experience with the management of PARSALUD II Systematization of the experience with the implementation of the training

program via internships on obstetrics and neonatal emergencies and vertical delivery

Systematization of the experience with community participation and transparency: strategies for citizen oversight and accountability

Systematization of the experience with the improvement of infrastructure and equipment of the health facilities classified as FONB and FONE, developed under the investment framework within PARSALUD II

4. Component 4. Project Coordination and Monitoring and Evaluation (M&E), through the provision of technical assistance, financing of incremental operating costs, and external and concurrent audits.

(i) PARSALUD PIU operating costs.

(ii) Mid-Term Evaluation of PARSALUD program.

(iii) Final evaluation of the PARSALUD program.

(iv) Yearly external financial audits.

(v) Elaboration and implementation of risk mitigation plans related to procurement and contracting.

37

Annex 3. Economic and Financial Analysis

Introduction

1. Peru’s (APL2) Health Reform Program Project sought to reduce maternal and infant mortality rates in Peru’s nine poorest, largely rural and indigenous, regions. The targeted regions (Amazonas, Huánuco, Huancavelica, Ayacucho, Apurimac, Cusco, Cajamarca, Ucayali and Puno) are characterized by greater population dispersion, fewer health facilities, lower service demand and generally a higher incidence of infant and maternal morbidity and mortality. Therefore, the interventions targeted by the Project improved maternal and infant mortality in the regions where health improvements are more difficult to achieve.

2. Higher maternal and infant mortality rates and malnutrition in children under 5 are associated with higher poverty levels and lower access to health services.11 According to data from ENDES 2005-2007 neonatal mortality was 11 times higher among newborns of the poorest income quintiles (23 per 1000 live births) compared to those of the richest income quintiles (2 per 1,000 live births). Likewise chronic malnutrition has a very unequal geographic and income distribution.12 Maternal mortality, double the LAC average also reflects wide disparities in Peru, with Lima measuring an MMR of 52 in 2000 while the MMR for Huancavelica and Puno were 302 and 361, respectively in the same year. In these two regions only 21 and 27.8 percent of the total births were professionally attended.

3. Studies have demonstrated that public spending on rural infrastructure is one of the most powerful instruments that governments can use to promote economic growth and poverty reduction.13,14 In particular, maternal, newborn and child health (MNCH) interventions are recognized by the literature as highly cost-effective investments. Particularly effective interventions in MNCH packages include labor and delivery management, preterm birth care, and serious infectious diseases and acute malnutrition treatment 15 ― all key focus of PARSALUD. Many maternal and infant deaths can be prevented with cost-effective health interventions and services targeted at those most in need.16,17 Studies show that the direct health benefits of investing in family planning and maternal and newborn health 11In 2005 a rural sick child affiliated to SIS had a 2.9 higher likelihood to demand health services than a sick rural child with no SIS affiliation (http://www.parsalud.gob.pe/factibilidad-del-programa, Estudio de Factibilidad: Modulo III: Formulacion, pag 434). 12 In 2004 (INEI, 2006) malnutrition was almost four times higher among children living in the rural areas (39 percent) than for those living in urban areas (10 percent). The regions of Huancavelica, Huánuco and Ayacucho, among the poorest in Peru, have more than 40 percent stunting levels. 13 Fan, Shenggen, Infrastructure and Pro-poor Growth, Paper prepared for the OECD DACT POVNET Agriculture and Pro-poor Growth, Helsibki Workshop, 17-18 June 2004. 14 Many of the health complications women face during childbirth could be prevented with better access to skilled health care professionals during labor (World Health Organization).15 Black R, Levin C, Walker N, Chou D, Liu L, and others. 2016. "Reproductive, Maternal, Newborn, and Child Health: Key Messages from Disease Control Priorities, 3rd Edition." The Lancet. Published online 9 April 2016 16 World Health Organization the case for Asia and the Pacific, Investing in Maternal Newborn and Child Health, Geneva, Switzerland.

38

http://www.parsalud.gob.pe/factibilidad-del-programa

services is dramatic, reducing the healthy years of life lost due to disability and pre-mature death. Furthermore, the implementation of cost effective MNCH interventions not only has direct benefits for women and children but also for the health sector and societies as a whole.18 Targeted investments can support the response of health systems to other urgent medical needs, curb sexually transmitted diseases, while reduce unplanned births and family size, thereby improving educational and employment opportunities for women, saving public-sector spending for health, water, sanitation and social services and reducing pressure on scarce natural resources.

Project Costs

4. Lack of infrastructure and medical personnel attention are an obstacle to health service provision for women and children in rural areas. Closing the gap in health care access requires targeting resources on those least likely to be receiving care, such as the indigenous and rural population in Peru. Investment in health care infrastructure is cost effective and supports the improvement of health outcomes in rural and poor areas. A recent study prepared by Juan Jose Diaz and Miguel Jaramillo evaluating Peru’s PARSALUD program found infrastructure investments (and training) cost effective through the prevention of blood loss.19 Eighty three percent of Peru’s Health Reform project funds (USD$138 million) went to fund Component 2 (demand side interventions), which was one of the most effective in terms of PDO achievement.20

Table 1: Component Effectiveness21

PDO 1 PDO 2 PDO 3PDO Achievement 0.70 0.79 0.94Costs for related component 5,861,770.32 138,555,014.74 3,262,626.36Indirect costs per related component 4,522,123.35 4,522,123.35 7,787,229.78Total costs for related component 10,383,893.67 143,077,138.09 11,049,856.14

Feasibility Study Benefits

5. The GOP conducted a feasibility study (FS) during project preparation evaluating two project investment options. The project was selected for its low cost-effectiveness ratio vis-à-vis alternative projects. Though the FS was not updated during the ICR, the assumptions made regarding the estimation of Project costs and effectiveness for the

17 Bill and Melinda Gates Foundation, Maternal, Newborn and Child Health Strategy Overview, www.gatesfoundation.org.18 Black R, Laxminarayan R, Temmerman M, and Walker N. editors. 2016. “Reproductive, Maternal, Newborn, and Child Health. Disease Control Priorities, third edition, volume 2”. Washington, DC: World Bank. doi:10.1596/978-1-4648-0348-2. License: Creative Commons Attribution CC BY 3.0 IGO19 The study found that infrastructure investments and training supported the prevention of blood loss (above 500 milimeters) for an average cost of US$3328 per case (and US$29,897 for a case with blood loss above 3,000 mililiters). Evaluating Interventions to reduce maternal mortality: evidence from Peru’s PARSalud program, Journal of Development Effectiveness, Volume 1, Issue 4, 2009).20 See annex 2 for detailed information on indicator achievement.21 This chart does not include the almost USD$17 million spent on monitoring, evaluation, administration and auditing.

39

feasibility study appear adequate. The economic valuation was undertaken following accepted international standards for estimating the present value of future costs avoided in the target population. The Project’s costs were assumed to be US$162.4 million, the discount rate 11 percent in soles and the benefit of Project implementation USD$4.45 million. The actual project cost was USD$164 million, (USD$13 million World Bank financed, USD$15 million IADB financed and USD$138 million GOP financed). The benefits were estimated applying the methodology of Quality Adjusted Life Years (QUALY). Measuring benefits through QUALY’s not only measures the number of years gained due to loss of mortality but also as a result of the decrease in chronic or temporary incapacity due to chronic illnesses.22 The method considered the following effectiveness indicators: the number of avoided deaths; the number of avoided disease cases and the number of days that an individual is prevented from of being ill due to the project effects. Chart 2 includes some of the main health benefits assumed and quantified from project implementation.

6. In order to evaluate the study’s assumptions we compared (when possible) the actual change in indicators impacted by the targeted interventions. The interventions used in the FS study for applying the QUALY methodology (and for which a reduction in deaths or cases is assumed) are similar to those targeted in the Project. Generally the assumptions regarding health improvements (and years of life gained due to reduction in deaths and incapacities) were reflected in health improvements in the Project regions though the actual number of death avoided/cases is difficult to compare. The largest benefits (in economic terms) from the interventions assumed in the study stem from the leading causes of maternal and infant mortality (hemorrhages and preeclampsia/eclampsia, and delayed fetal growth, fetal malnutrition, short gestations and low birth rate, respectively), since one year of premature death is the equivalent of one year of healthy life lost (see Table 2).

Table 2: Feasibility Study Calculated Benefits

Source: FS Module IV: Evaluation

22 The method converts the loss of mortality/incapacity avoided years into years of life gained.

40

7. The FS assumed project interventions would result in a reduction in maternal death cases, from 321 cases if the project was implemented to 237 without the project (difference of 84).23 The actual decrease in maternal deaths in the Project areas was of 75 women between 2007 and 2015.24 A FS-assumed reduction in anemia cases was reflected in an actual reduction of anemia among children under the age of 5 in the nine project regions (PDO 2) from 69.5 percent (2005) to 57.3 percent (December 2014) and among pregnant women in the same area (PDO 4) from 41.5 percent (2005) to 36.4 percent (2014).25 The availability of iron/folic acid supplements during Project implementation in the targeted regions also suggests that the use of iron/folic acid supplements increased with Project implementation (as indicated in the FS study) thereby boosting maternal nutrition (with associated impacts on infant deaths).

8. The FS study assumed a significant reduction in infant deaths with a high associated monetary benefit due to the prevention of considerable years of healthy life lost. The main targeted interventions (which are the main causes of infant mortality) are related to the diagnosis and treatment of asphyxia, sepsis, prematurity, low birthrate, neonatal hospitalization and postpartum control. The projected number of deaths avoided according to the FS due to various interventions ranged from 24,200 without the Project to 28,531 if the Project was implemented (difference of 4331). Though the Project did not measure infant mortality, one of the Project PDO indicators (PDO indicator 5) measured the hospital lethality rate among neonates in the nine selected Regions, which fell from 9.5 percent (2005) to 5 percent (2014). The under-5 mortality rate (per 1,000 live births) in Peru fell from 22.1 in 2009 to 16.9 in 2015, also displaying a downward trend.26 In addition to the interventions targeted, the project also lowered chronic malnutrition in children under 5 from 36.6 percent of the Project population to 23.7 percent. This was supported by an increase in exclusive breastfeeding in children under 6 months from 79.7 percent to 87 percent and in hand washing (for mothers) from 36.3 percent to 44.1 percent (project data). These interventions support the prevention of diarrhea, pneumonia, and respiratory diseases, all common causes of child illnesses.

Comparison among regions

9. An experimental approach comparing the regions with and without Project intervention reveal substantial effectiveness due to favorable results in Project implemented regions. The fall in malnutrition and maternal mortality were greater in the Project regions than in the non-project regions. This is particularly significant when one takes into account the lag in health service demand in the project regions, partly due to the associated service access difficulties. A study prepared by the GOP revealed that (pre-project) only 57

23 The study focuses on: complicated abortion, normal birth, hemorrhaging, eclampsia, sepsis and obstructed cesarean.24 National Epidemiological Network25 At the time of project preparation more than ¼ of women between 15 and 49 suffered from anemia largely because of inadequate nutrition.. 26 Estimates developed by the UN Inter-agency Group for Child Mortality Estimation (UNICEF, WHO, World Bank, UN DESA Population Division) at www.childmortality.org.

41

percent of the poorest quintile had physical access to the obstetric network less than 2 hours from their residence.27.

10. Malnutrition fell 12.8 percentage points in the project areas, compared to 6.9 percentage points in the non-project regions. The fall in the project areas narrowed the gap between the project and national averages from 12.8 percent in 2009 to 9.1 percent in 2014. Decreases in the rural project areas (13.5 percent) were significantly higher than in urban areas (7.9 percentage points). Furthermore, as hoped, the fall in malnutrition was higher for the lower income quintile (14.5 for quintile 1 and 16.7 for quintile 2 compared to no change in the superior quintile)

11. Malnutrition and infant mortality outcomes were driven by a number of improvements in interventions supported by the project. The proportion of exclusively breastfeed children in the project areas increased 7.3 percentage points while remaining practically the same in the non-project areas. However, in the project areas the increase was driven by the urban and higher quintile population (since approximately 90 percent of the lower quintile already exclusively breastfeed). Changes measuring the prevalence of anemia and EDA (extreme diarrhea) in children under 3 for the years 2009-2014 were similar for project and non-project areas. Both project and non-project areas experienced an increase in hand washing, 7.8 percentage points and 11.4 percentage points respectively, supporting a reduction in diarrhea and other illnesses. In the Project areas the increase was higher in the rural and lower income quintile populations. The proportion of children under 3 with health child appointment (control de crecimiento y desarollo – CRED) increased significantly in both the project areas and nationally.

12. Maternal mortality in the project areas fell by 38 percent between 2007 and 2015, compared with a 28 percent drop in the non-project areas suggesting that the project had an important impact. This is particularly true when comparing the more difficult terrain and poorer access to services, in project versus non-project areas. The proportion of institutional births in Peru, a proxy for maternal mortality, increased 13.3 in the project areas, 16.3 in the non-project areas and 14.3 nationally between 2009 and

27 (http://www.parsalud.gob.pe/factibilidad-del-programa, Estudio de Factibilidad: Resumen Ejecutivo, p. XIII).

42

http://www.parsalud.gob.pe/factibilidad-del-programa

2014. The increase was most marked in rural project areas (13.3 percent increase) and in the lower income quintile (16.2 percent for quintile 1 and 14.5 percent for 2). The proportion of rural pregnant women that have an appointment within the first trimester increased for both groups. Though the neonatal mortality did not reach the target for 2014 of 3.8 it decreased slightly from 5.6 in 2009 to 5.02 in 2014.

Fiscal Impact and Sustainability

13. The fiscal impact of the project was marginal limiting any sustainability concerns. As identified during project preparation and as revealed in Table 3 project implementation did not have a major impact on the MINSA budget, as it weighed an average of 0.08 percent throughout the period analyzed.

Table 3: Project Financial and Sustainability Analysis

Source: World Bank DataBank, at 2010 constant price

43

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title UnitLendingFernando Lavadenz Senior Health Specialist GHN04Amparo Elena Gordillo-Tobar Sr Economist (Health) GHN04Nelson Gutierrez Sr Social Protection Specialist GSP04Alessandra Marini Senior Economist GSP03Jose Pablo Gomez-Meza Senior Economist (Health) LCSHHLivia M. Benavides Country Operations Adviser LCC6CPatricia Mc Kenzie Practice Manager GGOPRRocio Schmunis Operations Officer GHN05Mariana Montiel Senior Counsel LEGLEFabiola Altimari Senior Counsel LEGLEKeisgner De Jesus Alfaro Senior Procurement Specialist GGODRPatricia de la Fuente Hoyes Senior Financial Management Specialist GGO22Tomas Socias Senior Procurement Specialist LCSPTXiomara Morel Lead Financial Management Specialist GGO22Lourdes Linares Senior Financial Management Specialist GGO22Nelly Ikeda Financial Management Specialist GGO22Robert Leonard O'Leary Senior Finance Officer WFAFOMonique Francine Mrazek Senior Investment Officer CMGCSAlonso Zarzar Casis Sr Social Scientist GSURRIsabel Tomadin Social Sector Specialist GSURRPablo Lavado Junior Professional Associate LCSHSCarmen Rosa Osorio Junior Professional Associate LCSHEClaudia Sanchez Junior Professional Associate LCSHEPatricia Bernedo Senior Program Assistant GSP04Luisa Yesquen Program Assistant LCC6CErika Bazan Lavanda Program Assistant LCSHDNatalia Moncada Senior Executive Assistance GSUSDJulia Nanucci Language Program Assistant LCSHDZulma Ortiz Consultant LCSHHSilvana Vargas Consultant LCSHSPablo Augusto Lavado Consultant GMFDRSupervision/ICRFernando Lavadenz Senior Health Specialist GHN04Andre Medici Senior Economist (Health) GHN04

44

Nelson Gutierrez Sr Social Protection Specialist GSP04Omar S. Arias Diaz Lead Economist GSPDRAmparo Elena Gordillo -Tobar Sr Economist (Health) GHN04Alvaro Larrea Lead Procurement Specialist GGO04Rocio La Vera Procurement Specialist GGO04Patricia de la Fuente Hoyes Senior Financial Management Specialist GGO22Nelly Ikeda Financial Management Specialist GGO22Monica Tambucho Senior Finance Officer WFALNMaria Virginia Hormazabal Finance Officer WFALNRenata Pantoja Financial Analyst WFALNMariana Montiel Senior Counsel LEGLERocio Schmunis Operations Officer GHN05Federica Secci Health Specialist GHNGEClaudia Sanchez Lanning Junior Professional Associate GSPDRCarmen Cornejo Junior Professional Associate LCSHDGabriela Moreno Zevallos Program Assistant GHN04Sara Burga Program Assistant LCC6CCristian Pereira Stambuk Consultant GEDDRFernanda Bahia Consultant LCSHHIsabella Bablumian Consultant GHNDR

(b) Staff Time and Cost

Stage of Project CycleStaff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY06 36.18 184.20 FY07 24.35 85.85 FY08 37.94 140.46 FY09 43.55 144.57

Total: 142.02 555.08Supervision/ICR

FY09 12.94 41.70FY10 32.17 93.30FY11 41.00 137.90FY12 47.00 196.22FY13 51.78 215.81FY14 60.35 254.61FY15 28.56 152.98FY16 10.94 78.14

Total: 284.74 1170.66

45

Annex 5. Beneficiary Survey Results

Not applicable

46

Annex 6. Stakeholder Workshop Report and Results

Not applicable

47

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

1. The evaluation of the PARSALUD II program was conducted in 2015 by the Peruvian University of Cayetano Heredia and the National Institute of Mexican Public Health. An English summary of the main findings of the evaluation is reported below and the original Executive Summary of the evaluation report follows (in Spanish)

2. PARSALUD II was established as a program to support the modernization process and reform of the health system, while seeking to improve the health and lower the mortality and morbidity of the maternal and infant population in the 9 poorest regions of Peru. Project design included various components targeting health service demand and supply, health sector regulations and national and regional management aspects.

3. The objectives of the evaluations were the following: Determine program design relevance in terms of PDO achievement. Determine target achievement in relation to program activities/components and

achievement of objectives and targets in relation to the baseline and the intermediate evaluation

Identify the Project’s main limitations, successes and lessons learned in order to provide recommendations.

4. Results and Conclusions: PARSALUD II targets were aligned with Peru’s national health strategy. Interventions prioritized rural and areas with greater poverty levels where

maternal and infant mortality rates were greater than in the rest of the country. Program design was relevant, appropriate, and had a logical structure. Indicators were partially in line with the Program’s strategic actions and

components. Project expenditure was 100% efficiency in 6 of the 14 analyzed activities. Some of the most notable program successes include: (a) a reduction in the

prevalence of malnutrition in children under 3 (from 37 to 21%), (b) the institutionalization of the care of women’s health during pregnancy, birth and during the postpartum period, and (c) the promotion of good health practices related to children.

When comparing Project implementation and non-Project implementation regions, the evaluation found that the project regions presented better health conditions regarding indicators measuring prenatal care attention and quality of attention and institutional births. However, women in these regions also presented greater complications during the postpartum period.

In terms of the health of infants 3 years and younger the greatest advances were related to nutritional indicators. The reduction in chronic malnutrition was almost 12 percentage points in both groups between 2009 and 2014.

Though the results reveal changes that favor the improvement of health in population groups exposed to PARSALUD2 it is difficult to directly attribute the positive impacts to the Project.

48

“ESTUDIO DE EVALUACIÓN FINAL DE LA SEGUNDA FASE DEL PROGRAMA DE APOYO A LA REFORMA DEL SECTOR SALUD -

PARSALUD II” (Resumen Ejecutivo)

INTRODUCCIÓN

5. PARSALUD II se crea como programa para apoyar el proceso de modernización y reforma del sistema de salud, buscando mejorar el estado de salud de la población materno – infantil, mediante el incremento del uso de servicios de salud materno infantiles y la reducción de la morbilidad de los niños y niñas menores de 3 años de familias de la zona rural de 09 regiones más pobres del Perú (Amazonas, Apurímac, Ayacucho, Cajamarca, Cusco, Huancavelica, Huánuco, Puno y Ucayali).

6. PARSALUD II combina varios componentes que se relacionan y actúan tanto en la oferta como en la demanda de servicios de salud, así como a nivel normativo y de gestión a nivel nacional y regional. Para cumplir con los objetivos, el Programa se ha organizado sobre la base de tres componentes que han sido considerados como fundamentales: Fortalecimiento de la demanda, fortalecimiento de la oferta y gobierno, y financiamiento y adicionalmente un componente transversal de gestión que incluye administración, evaluación, monitoreo y auditoría.

7. La Universidad Peruana Cayetano Heredia con el Instituto Nacional de Salud Pública de México realizaron la evaluación final de PARSALUD II en el 2015, desde una perspectiva integral de todo el ciclo de la acción: Planificación, implementación resultados e impactos, utilizando la teoría del cambio.

8. Los objetivos específicos de la evaluación son los siguientes:

1. Determinar la pertinencia, relevancia y suficiencia del diseño del Programa en relación al logro de los objetivos.

2. Determinar el grado de cumplimiento de las metas previstas con relación a las actividades y componentes del Programa.

3. Evaluar el nivel de cumplimiento de los objetivos y metas a nivel de resultado y evaluar su tendencia contrastándolos con los de la línea de base y evaluación intermedia.

4. Identificar las principales limitaciones y los factores de éxito y lecciones aprendidas del Programa y formular recomendaciones para la mejora del cumplimiento de sus objetivos.

METODOLOGÍA

9. En la evaluación del diseño para la evaluación de la pertinencia se revisaron documentos proporcionados por PARSALUD II; estos son: Resumen Ejecutivo, Módulo

49

I: Aspectos generales, Módulo II Identificación: Definición del problema, Módulo II Identificación: Diagnóstico del contexto, definición del problema, Módulo III Formulación: Análisis de la demanda- Análisis de la oferta-Descripción de la alternativa. Se revisaron adicionalmente las ENDES 2000, 2004, 2005, 2006, informes de la Organización Mundial de la Salud, Organización Panamericana de la Salud, Informes del Instituto Nacional de Estadística e Informática de Perú. Así como lineamientos de política nacional, normas y documentos técnicos en el campo de la salud materna infantil. Con base en la revisión de documentos, se elaboraron informes por categorías de análisis vinculadas a la pertinencia.

10. Para la evaluación de coherencia del diseño de PARSALUD II, se utilizó el análisis de teoría causal y teoría de cambio del Programa, para lo cual se retomó el árbol de problemas28 y el árbol de objetivos propuestos por PARSALUD II.29 Se organizaron diagramas de causalidad con la finalidad de dar un ordenamiento lógico a la teoría causal.

11. Para identificar la relevancia de las intervenciones de PARSALUD II para atender el problema de salud materno infantil en zonas rurales se realizó una búsqueda intencionada de propuestas para abordar la problemática en la literatura utilizada para construir la teoría causal y la teoría de cambio del Programa que fuera publicada entre 2000 a 2006. Se elaboró una matriz con el resumen de cada uno de los artículos a partir de la cual se elaboró el informe.

12. Para evaluar la suficiencia en el diseño de PARSALUD II, se evaluó la lógica vertical y la lógica horizontal del marco lógico del Programa. En la lógica vertical del Programa se realizó una evaluación de la suficiencia de las actividades (cantidad, oportunidad y concentración en función a la población beneficiaria) para generar un determinado producto y en qué medida los productos producidos por las actividades permiten alcanzar el componente del Programa. Este tipo de evaluación se realizó para cada uno de los componentes considerados en el Programa, igualmente se realizó una evaluación de la suficiencia de los componentes para producir el propósito del Programa y de éste como contribuye al fin. El análisis de la lógica vertical respondió a las siguientes preguntas: ¿Las metas a nivel de acciones son adecuadas para alcanzar las actividades

principales? ¿Las actividades especificadas para cada componente son las necesarias para

producir el componente? ¿Los componentes son necesarios para lograr el Propósito del Programa? ¿El Propósito del Programa contribuye al fin?

13. Con respecto a la evaluación de desempeño y eficiencia, se realizó un análisis comparativo de la programación y la ejecución de los productos relacionados a los componentes del Programa en tres dimensiones: cantidades producidas, tiempos de producción y monto gastado; buscando identificar los factores relacionados a la eficiencia. Las fuentes de información utilizadas fueron las siguientes: los estudios de

28 PARSALUD II. Módulo II: Identificación, definición del problema y sus causas. pág 344. 29 PARSALUD II. Módulo II: Identificación, objetivos del proyecto pág. 377

50

factibilidad del Programa (nacional y regionales), Planes Operativos Anuales (POA), Matrices del Progress Monitoring Report para el BID, Reportes del SIAF (MEF) y SIMAF (PARSALUD) II, Informes de Evaluación, Convenios con Regiones, Actas de Sesiones del Comité Directivo de PARSALUD II, entre otros. La estrategia de análisis tomó en cuenta los lineamientos del Ministerio de Economía y Finanzas para la evaluación Ex - post de proyectos de inversión (general y en salud) y constó de tres etapas. La primera fue dirigida a la recopilación y procesamiento de información, que incluyó trabajo de campo en Perú, así como la definición de los productos por componente a trabajar. La segunda etapa constó de un análisis cuantitativo de los datos de producción, tiempo de ejecución y gasto; generándose medidas de eficiencia a partir de razones entre lo planeado y lo realmente obtenido. Finalmente, la tercera etapa estuvo encaminada a identificar los factores relacionados a la eficiencia del Programa, a partir de revisión documental y compilación de información brindada en reuniones con el grupo de tarea. Para sistematizar esta información se consideraron cuatro categorías: factores políticos, económicos, sociales y tecnológicos (PEST).

14. En relación al análisis del logro de los resultados, se ha aplicado el análisis siguiendo el enfoque del marco lógico. Hasta donde ha sido posible, se han vuelto a estimar los indicadores a partir de información primaria. En la interpretación de los resultados se consideró no solamente la diferencia aritmética entre el valor proyectado y el valor observado, sino el significado de la diferencia considerando dos criterios: la naturaleza dinámica interna de los procesos que requiere una adecuación periódica de las metas y el ambiente externo cambiante que implica también una adecuación periódica de las metas. Se recurrió en lo posible a fuentes de información y, en su defecto, a fuentes secundarias.

15. PARSALUD II, midió en ¿qué medida las intervenciones de PARSALUD II han contribuido en el mejoramiento de los indicadores intermedios de salud materna e infantil en las regiones establecidas como prioritarias por el propio Programa? e identificó si el efecto sobre los indicadores intermedios de salud depende del grado de exposición que tiene la población de interés al Programa, entendiendo como exposición al número de intervenciones realizadas en un distrito o provincia en un tiempo determinado.

16. El análisis de efectos se basó en un diseño observacional pre-post con grupo de comparación. Se llevó a cabo un análisis intención al tratamiento de diferencias en diferencias tomando como línea base la información disponible en 2009 y como información post-intervención la disponible en 2014. A partir de la utilización de pruebas estadísticas de diferencias de medias se decidió que el grupo control esté conformado por los distritos no expuestos al programa de las nueve regiones donde interviene PARSALUD. Los indicadores de resultados planteados en la presente evaluación se encuentran relacionados a los indicadores de propósito y componente del PARSALUD II.

17. Con el fin de explorar diferencias de efecto según intensidad del tratamiento, se conformaron los siguientes grupos de comparación: P1: Hogares residentes de distritos con una acción PARSALUD II entre 2009 y 2014, P2: Hogares residentes de distritos con

51

más de una acción PARSALUD II entre 2009 y 2014 y C: Hogares residentes de distritos en las regiones PARSALUD II pero sin acciones del Programa entre 2009 y 2014.

18. En el análisis se utilizó un modelo de regresión multivariada, ajustándose por la edad de la madre, si tuvo algún aborto, nivel de escolaridad, condición laboral, edad del jefe del hogar, estado conyugal, edad de la mujer en el primer nacimiento, lengua indígena, total de niños que alguna vez nacieron, nivel de riqueza30, conformación del hogar, urbano/rural, y si tiene la cultura de lavado de manos. En el caso de los indicadores de salud de niños menores de tres años, adicionalmente se ajustó por las siguientes variables: si al momento del nacimiento hubo contacto piel a piel con la madre, lugar del parto, y tamaño del niño al nacer. En el caso del análisis de los indicadores de gasto en salud, en el análisis de se controló por: sexo, edad y educación del jefe del hogar, algunas características demográficas del hogar como la presencia de niños pequeños, adultos mayores o mujeres en edad fértil, también se controló por el nivel de pobreza del hogar y otras condiciones sociales y de saneamiento en la vivienda. Con excepción de la variable que identifica a los hogares con gasto catastrófico en salud, las variables de gasto fueron transformadas en logaritmos.

19. La aproximación cualitativa buscó caracterizar la implementación de PARSALUD II, describir los resultados de este Programa, las lecciones aprendidas, los principales retos y los factores de éxito de éste, desde la perspectiva de los actores del MINSA, PARSALUD II, las agencias financiadoras y la población beneficiaria. La metodología de esta aproximación consistió en entrevistas a decisores de política a nivel nacional, regional y local, a operadores de EESS y a gestantes, puérperas y madres de niños menores de 3 años de las regiones Amazonas, Apurímac y Cajamarca, las cuales fueron seleccionadas en razón de que cuentan con un gran número de estrategias implementadas y en base a que presentan diversos niveles de eficiencia (alto, medio y bajo). Asimismo, se realizó un grupo focal con los nueve coordinadores zonales de PARSALUD II. La información recolectada y grabada fue transcrita y analizada.

20. Entre las limitaciones identificadas resalta el corto tiempo programado para realizar la presente evaluación, asimismo, el periodo de corte establecido (2009 hasta agosto 2015), no hizo posible una evaluación que incluya las actividades desarrollas hasta el cierre del Programa en diciembre del 2015.

21. Finalmente, el que el Programa no contemplara en el documento de factibilidad el diseño de una evaluación de efecto dificulta establecer categóricamente en qué medida los resultados identificados en la mejora de la salud materno infantil se deben al Programa.

RESULTADOS

22. En cuanto la pertinencia del diseño, El PARSALUD II propuso como problema central para la intervención del Programa el “Bajo acceso a servicios de salud materna, alta 30 El nivel de riqueza es un indicador disponible para su uso en las bases de datos de la ENDES. De acuerdo a su metodología, se construye a partir de los activos de los hogares, siguiendo la metodología de Shea Rutstein y Kiersten Johnson de Macro Internacional Inc. y Deon Filmer y Lant Pritchett del Banco Mundial (The DHS Wealth Index: Approaches for Rural and Urban Areas. DHS Working Papers.USAID. 2008)

52

morbilidad infantil y deficiente ingesta de micronutrientes de los niños y las niñas menores de 3 años de familias de zona rural de 09 regiones. Como se puede observar existen tres problemas; el bajo acceso es un problema relacionado con la atención de servicios de salud, la morbilidad infantil relacionada con el estado de salud y la deficiente ingesta de micronutrientes es la ausencia de una solución que debe integrarse como parte de la atención integral del niño. En consecuencia, los otros dos problemas requieren de un análisis de causa efecto y de un marco lógico específico que permita definir con claridad el propósito del sector salud en el caso de las IRA y EDA en niños menores de tres años ya que la mayoría de factores explicativos para este problema están fuera del sector salud. Consideramos que el problema más importante a resolver en la población rural en 2006 se expresa en el “Bajo acceso a servicios de salud maternos y en niños y las niñas menores de 3 años de familias de zona rural de 9 regiones.

23. Según los documentos revisados en torno a las metas a alcanzar en salud materno infantil, se aprecia que PARSALUD II se alinea perfectamente a las metas estratégicas nacionales propuestas por el gobierno peruano. Por otro lado también se priorizó la intervención de ámbitos rurales, dispersos y con la mayor pobreza. Se seleccionó las regiones para su intervención en base a indicadores con mayores desventajas sociales y sanitarias. Es evidente que intervinieron en regiones con mayor pobreza y ruralidad en las cuales la razón de mortalidad materna e infantil fue superior al resto de regiones del país. Para la formulación del PARSALUD II convocaron a diversas instituciones, lo que hace inferir que su formulación se realizó desde una perspectiva intersectorial. Para la implementación del Programa se consideraron instancias vinculadas al MINSA, las regiones y sus organismos públicos descentralizados.

24. En relación a la evaluación de la coherencia, existen actividades que se contemplaron en el diseño del Programa, pero que no se incluyeron en los planes. Así como actividades que son parte de los planes, pero que no se contemplaron en el diseño de la alternativa seleccionada. Las actividades de diseño se contemplaron acertadamente en los tres primeros años del Programa; no obstante algunas actividades de evaluación se programaron antes de haber terminado las acciones de ejecución.

25. El análisis de la teoría causal fue elaborado utilizándose la evidencia científica antes que el MEF contará con los programas basados en la lógica de los presupuestos por resultados. Por lo que el 31% de ellas fueron publicadas antes de 1996 y 54% antes del 2000. En cuanto a la documentación de evidencia para la causalidad del problema central relacionado con el bajo acceso de servicios de salud de la población rural sólo utilizaron un documento del año 1991, lo que podría calificarse como una revisión insuficiente para el contexto actual. La mayoría de documentos revisados dan cuenta de las IRA, EDA y desnutrición.

26. En cuanto la evaluación de la relevancia, en el diseño se incluyó aspectos claves para mejorar el acceso de la población rural a los servicios de salud materno infantiles reportados en la literatura. Abordó aspectos como: fortalecimiento de las competencias del personal de salud para la atención integral de la gestante, del niño menor de 3 años así

53

como para la atención de la emergencia. Incluyó acciones para dotación de equipos, materiales e insumos a los establecimientos de salud así como la construcción de establecimientos de salud para garantizar mayor accesibilidad de la población a un paquete de servicios de salud. Contempló el fortalecimiento de las acciones que realiza el SIS en cuanto al aseguramiento en salud, otras instancias del MINSA, SUSALUD (Ex SUNASA), y finalmente desarrolló estrategias para la adecuación cultural de los servicios de salud

27. Con respecto a la evaluación de suficiencia, se observó que a nivel de la lógica vertical del programa los componentes se alinearon perfectamente con el propósito y éste con el fin. No obstante, las acciones estratégicas (propósitos) y actividades (acciones) requirieron revisarse para alinearse totalmente, al igual que la unidad de medida y la cantidad a producir de las metas de las acciones estratégicas. En cuanto a los componentes los resultados a alcanzar fueron los adecuados.

28. Los indicadores se alinearon parcialmente a la medición de las acciones estratégicas y componentes del Programa.

29. En términos del desempeño del Programa en cuanto al cumplimiento de actividades y logro de productos, y el gasto ejercido por producto, se observó que el Programa fue eficiente en niveles de 100% o más en 6 de los 14 productos analizados. El componente con mayor eficiencia fue el Componente III de Gobierno, destacando la generación de propuestas de normas y disposiciones legales, y el diseño de sistemas de seguimiento. Otro componente con producción eficiente fue el Componente I de Demanda, donde destaca la producción de personal capacitado en el Programa de Comunicación y Educación para la Salud. Se debe mencionar también que la producción de capacitación de personal en gestión de recursos humanos (PREG, Parto Vertical, Emergencias obstétricas y neonatales, Quechua) y en atención materno neonatal, del Componente II de Oferta, presentaron procesos eficientes. No obstante, entre los productos generados con menos eficiencia se encuentran: los relacionados a infraestructura, debido al retraso en el desarrollo de las obras y el equipamiento en Telesalud, en este último caso por el punto de corte de la evaluación.

30. Entre los hallazgos relacionados al cumplimiento de objetivos y metas a nivel de los resultados destaca el logro en cuanto a la reducción de la prevalencia de la desnutrición en menores de 3 años (37 a 21%); destacan también los logros en la institucionalización del cuidado de la salud de la madre durante el embarazo, el parto y el puerperio, así como en la promoción de las prácticas del cuidado de la salud del niño; pero se advierte aún una diferencia rural urbana importante.

31. Con relación a la evaluación del efecto, cuando se realiza una comparación simple de los indicadores intermedios de salud materna e infantil en regiones atendidas por PARSALUD II y regiones sin atención, entre 2009 y 2014, se encuentra que el grupo de comparación presentó mejores condiciones de salud en los indicadores de atención prenatal, atención prenatal ajustada por calidad y parto institucional. No obstante, se

54

observó que las mujeres del grupo de comparación presentaron mayor porcentaje de complicaciones durante el puerperio. Al analizar los indicadores con la información de 2014, se observó que en algunos de los indicadores las brechas entre los grupos de exposición y de comparación se redujeron, sin embargo en la mayoría, las diferencias entre los grupos se mantiene. Los indicadores de salud relacionados con la atención prenatal, lugar del parto y situación de salud durante el puerperio mejoraron entre 2009 y 2014 en toda la región PARSALUD II, tanto en el grupo de expuestos como en los no expuestos a las acciones del Programa.

32. Con relación a la salud de los niños de 3 años y menores, los mayores avances se dieron en los indicadores nutricionales, sobre todo en el indicador de desnutrición crónica y anemia. La reducción de la prevalencia de la desnutrición crónica, entre 2009 y 2014, fue de casi 12 puntos porcentuales en ambos grupos. Con relación a las variables de gasto en salud de los hogares, se encontraron cambios en los niveles observados en 2009 y 2014.

33. Si bien los resultados anteriores muestran algunos cambios que parecen favorecer las condiciones de salud en el grupo de hogares expuestos a PARSALUD II, no es posible argumentar si los cambios observados responden a las acciones implementadas por el Programa o a otros factores. Por ello se realiza un análisis para la estimación de efectos, controlando por otros factores que inciden en los resultados de salud analizados, de tal manera que podamos aproximarnos al verdadero efecto del PARSALUD II sobre los indicadores de interés. Los resultados de este ejercicio muestran que, con excepción del indicador de parto institucional donde se encontró un ligero efecto del programa, en las demás variables analizadas no se halló evidencia de que el Programa tuviera un efecto significativo en mejorar la atención prenatal, en reducir las complicaciones durante el puerperio, en mejorar algunas de las condiciones nutricionales de los niños menores de 3 años, o en reducir el gasto que realizan los hogares en salud.

34. Finalmente, hubo consenso en señalar que el PARSALUD II cumplió con su objetivo de apoyar la reforma del sistema de salud de Perú, favoreciendo el aseguramiento público, el desarrollo de infraestructura, así como la capacitación del personal y el equipamiento de establecimientos de salud. Se destacó su diseño organizacional y administrativo como una de sus fortalezas más importantes. Para las Regiones el PARSALUD II es considerado que, más allá de su acción financiadora, es un valioso apoyo técnico estratégico. Se reconocieron problemas de retraso de obras y la necesidad de mejorar la coordinación entre el Programa y el MINSA. La transparencia y el diseño organizacional han sido factores de éxito. En la relación intercultural si bien gestantes y puérperas valoran el “aliento”, la “importancia” y el “encariñamiento” en la atención a sus procesos de parto y puerperio, aún es débil el reconocimiento de su derecho a comprender todos los procedimientos (resultados de análisis, revisión de su gestación, asistencia a casas maternas, cesárea y lactancia) a los que están sujetas para tranquilizarlas y no infundirles temor.

35. La posibilidad del rol municipal en la Promoción de la salud es más bien una perspectiva a desarrollar. Lo avanzado con gobiernos municipales ha demostrado ser valioso,

55

requiere persistir en restablecer vínculos que aseguren la aún débil articulación con este actor público.CONCLUSIONES

1. Respecto a la pertinencia del Programa, la formulación tiene un alto valor que recoge la problemática y plantea la solución de manera adecuada

2. Respecto a la coherencia responde parcialmente a una teoría causal 3. El Programa es relevante porque incluyó la formulación las acciones de intervención

relacionadas en la literatura científica consideradas relevantes para problemas similares.

4. Respecto a la suficiencia, la lógica vertical del programa se alinean perfectamente los componentes, con el propósito y este con el fin y que las acciones estratégicas (actividades) y actividades( acciones) requieren revisarse para alinearse adecuadamente necesita mejorar el sentido de la lógica horizontal más que la lógica vertical

5. PARSALUD II se planteó generar 14 productos relacionados a actividades dentro de sus componentes, para así lograr los objetivos del Programa. En su logro fue eficiente para los componentes de Gobierno y Demanda; y parcialmente eficiente para el componente de Oferta, principalmente por el retraso en el inicio de obras así como problemas con las empresas contratistas. A pesar que el desarrollo del Programa se dio en un periodo con constantes cambios políticos y ante la carencia de postores competentes para llevar a cabo las acciones encomendadas; el PARSALUD II logró coordinación con los gobiernos regionales para llevar a cabo su labor, desarrolló sistemas de monitoreo e información que facilitaron la detección de problemas de ejecución, así como procesos administrativos para agilizar la gestión financiera.

6. Los resultados de los indicadores en términos de la diferencia entre lo proyectado y lo observado, en números absolutos y resultados, muestran que el Programa ha alcanzado los logros que se había propuesto.

7. El Programa coadyuvó a objetivos de la reforma del sistema de salud en la atención del embarazo, parto y la salud infantil, a la vez que favoreció el desarrollo de innovaciones gerenciales en la gestión gubernamental, lo que hace necesario conservar lo aprendido en futuras intervenciones del MINSA.

8. En las regiones hay un reconocimiento del importante apoyo de la asistencia técnica para producir ordenamientos organizacionales y capacitar al personal de salud que debe acompañar la inversión en infraestructura.

9. Se reconoce un avance significativo en todas las regiones acerca del reconocimiento del derecho de las mujeres gestantes, puérperas y madres a ser tratadas con igualdad, quedan aún desafíos para el MINSA para avanzar en una relación cívica de reconocimiento a sus derechos y que no experimenten que se el servicio “les está haciendo un favor” como señalaba una decisora del nivel nacional.

RECOMENDACIONES

56

1. Las propuestas de implementación de programas de salud deben responder a necesidades de salud de la población a quien se pretende beneficiar, las mismas que deben estar respaldadas en fuentes de información locales y los lineamientos de política de salud nacional. PARSALUD II en este sentido puede constituirse en un referente nacional para formular intervenciones de salud en poblaciones rurales.

2. El análisis de teoría causal para la elaboración del marco lógico de PARSALUD II, utilizó artículos científicos, no obstante algunos de ellos con una temporalidad de publicación de más de diez años previos a la implementación del programa, dado el retraso en su inicio. Por lo cual se sugiere que en estos procesos las evidencias utilizadas deben ser el más actuales y procedente de países con el mismo nivel de ingresos.

3. La coherencia interna de los niveles de causalidad en la formulación de programas de salud deben revisarse con la finalidad de proponer estrategias y acciones altamente vinculadas con los componentes del programa y así alcanzar el propósito del mismo.

4. Los indicadores deben seleccionarse en base a su especificidad para medir los componentes del Programa y debe contemplarse la mejor fuente de información, ya sea esta primaria o secundaria para su medición desde el inicio del Programa. Por lo cual como parte del diseño los programas deben contemplar presupuesto para la evaluación integral del mismo.

5. Los indicadores de las actividades deben ser definidos como una cantidad a ejecutar y los indicadores de componentes y propósito como resultados a alcanzar.

6. Rescatar y replicar formas de gestión, como el Monitoreo de la Gestión Financiera, que permite un mejor desempeño del Programa y una identificación temprana de problemas de ejecución.

7. Establecer mecanismos que limiten la vulnerabilidad de la operación del Programa ante factores externos como la inestabilidad política y de personal. Es importante desarrollar una masa crítica y de técnicos que aseguren la continuidad a las acciones del Programa.

8. En términos del diseño para la evaluación del impacto de futuros programas, se recomienda planear un diseño de evaluación de impacto con asignación aleatoria de los grupos de exposición o intervención y de control, que permita estimar el efecto del programa en la población que realmente recibe los beneficios de éste, y que permita, además, controlar por aspectos como la calidad de la atención.

9. Retomar la experiencia gerencial desarrollada en materia de buenas prácticas y gestión por resultados.

10. Fortalecer el vínculo con los Municipios en planes concertados para la vigilancia y disminuir la desnutrición infantil.

11. Reforzar la relación social de cuidado y encuentro cultural con gestantes y madres que solicitan explicaciones para comprender cómo los procedimientos que susciten confianza y no temor.

12.

57

COMENTARIOS SOBRE EL REPORTE DE IMPLEMENTACION, FINALIZACION Y RESULTADOS (BIRF-76430)

SEGUNDA FASE DEL PROGRAMA DE APOYO A LA REFORMA DEL SECTOR SALUD

AVANCE DE INFORMACION 1

17 junio del 2016

UNIDAD EJECUTORA 123

PROGRAMA NACIONAL DE INVERSION EN SALUD - PRONIS

Equipo de Gestión PRONIS

1. Rocío Espino Goycochea, Coordinadora General (e) del PRONIS.

2. Francisco Solís Coronado, Coordinador (e) de la Unidad de Planeamiento, Calidad y

Desarrollo.

3. Ana Cano Bobadilla, Coordinadora de la Unidad de Pre Inversión.

4. Juan Manuel Pizarro Garcés, Coordinador (e) de la Unidad de Estudios Definitivos.

5. Carlos López Chamorro, Coordinador (e) de la Unidad de Obras.

6. Bárbara Lem Conde, Coordinadora de la Unidad de Administración y Finanzas.

7. Elizabeth Martínez Galván, Coordinadora de la Unidad Asesoría Legal.

8. Paola Tamayo Medina, Coordinadora de la Unidad de Relaciones Institucionales y de

Comunicación.

58

UNIDAD EJECUTORA 123 Av. Javier Prado Oeste 2108 - San Isidro Lima – Perúwww.pronis.gob.peTeléfono: 611-8181

Lima - Perú

59

2.2 Implementation (page 7)

19. Implementation of the project did not suffer from any major complications. The project had a slow start. While activities related to Components 1 and 3 started immediately, delays in disbursement and execution related mainly with the infrastructure investments under Component 2. Once construction works began and medical equipment started being purchased in 2011, project disbursement picked up towards the end of 2012.Comentario:

Respecto al contrato de obras debemos indicar que el primer contrato se suscribió el 12 de mayo de 2010 correspondiente a la ejecución de obras de la región Apurímac; y que respecto a los contratos de equipamiento debemos indicar que el primer contrato se suscribió en diciembre del 2010.

Asimismo, se registra pagos en obras con recursos de endeudamiento externo de aproximadamente 5 millones de nuevos soles y es importante destacar que el MEF no autorizo el reembolso de US$ 731,22.7 de gastos efectuados con Recursos Ordinarios del Tesoro Público y que inicialmente estaban programados a financiarse con Recursos de Endeudamiento Externo.

21. While the program was successful in achieving its broader goals of reducing maternal and infant mortality and chronic malnutrition in children under 5, progress on the indicators in the RF was mixed during the life of the project. Some of the targets were achieved even before the MTR in February 2013, while others had a more fluctuating trajectory. To some extent, this was related to the delays in construction works, which shortened the time horizon available to see the impact of the infrastructure investments on the selected health outcomes and outputs. In other cases, the trend of some indicators reflected those at the national level, such as in the case of the prevalence of anemia among pregnant women. The delays in progress on some indicators motivated the downgrade of the Overall Project Implementation (IP) rating from satisfactory to moderately satisfactory in 2013, rating which was then kept in consideration of the disbursement delays ― eventually, the project disbursed 80% of the planned amount.

Comentario:En el PARSALUD II, los indicadores no fueron evaluados relacionando el retraso en las obras de construcción ni con los retrasos en los desembolsos. Para relacionarlos tendría que hacerse la evaluación de impacto.

22. The Mid-Term Review (MTR) in February 2013 identified some of the challenges and correcting measures (page 8)(ii) Delays in civil works and consultancies. As of December 2012, after 2 years of implementation, about 27% of project's civil works were delivered, with heterogeneous patterns among the regions. Logistic difficulties for timely civil works completion were associated with difficult climate and access to sites, as well as with a few cases of collusion, properly addressed by the counterpart. Delays in hiring consultants were due

60

to, among others, frequent changes in regional administration, scarcity of professionals adequately qualified for some tasks, delays in approval processes from the regions, and political attention diverted to sudden public health emergencies (e.g. pneumonia in Puno and dengue in Ucayali). Despite all delays, almost all planned civil works were completed before the project closing date, with a few being delivered in 2016.

Comentario:El término dificultades logísticas, es general y puede inducir a pensar que estos fueron problemas atribuibles a la entidad, se sugiere modificar e indicar directamente que las obras civiles presentaron retrasos por diversos factores entre ellos: factores climaticos, rutas de acceso así como también por problemas de colusión.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization - Implementation. (page 10)27. Due to unavailability of data, two of the six KPIs only started being measured in December 2014 (ISR 11, with data of December 2013). Given that they had not been monitored, five of the intermediate indicators were dropped during the restructuring in 2014. However, data was available for two of those, which were reintroduced for the purpose of the ICR. The other three indicators had not been clearly defined and monitoring was therefore problematic. Unfortunately, all three of those IOIs measured progress against the same part of the PDO (3). Since the PARSALUD PIU monitored more indicators than those monitored by the Bank, additional intermediate indicators might have introduced to replace the problematic IOIs to better assess improvements on PDO 3 ― for example during the first restructuring in 2011 or immediately after the MTR in 2013.

Comentarios:Todos los indicadores que solicitó el banco, se presentaban lo avances en los informes de progreso semestrales. Cuáles son los dos indicadores de los seis que de acuerdo a lo que se señala se midieron a partir de diciembre 2014.

Según lo afirmado en el reporte son 5 indicadores eliminados y la entidad tiene identificado 6; los que se retiraron por lo descrito en la columna de Comentarios en la siguiente tabla, de acuerdo a lo coordinado con el responsable del BM y el equipo de AEGE del PARSALUD II.

Indicadores de Resultados de Nivel ODP Protocolo Fuente de

Información Comentarios

Componente 2: % de establecimientos de salud con mejora en infraestructura (construcción menor y nuevo equipamiento)

NO HAY SISTEMA DE INFORMACIÓN QUE SOPORTE ESTE DATO

% de mujeres (gestantes, parturientas y lactantes) que reportan satisfacción


61

Indicadores de Resultados de Nivel ODP Protocolo Fuente de

Información Comentarios

con la atención recibida% de establecimientos de salud con mejora en infraestructura (construcción menor y nuevo equipamiento)


Componente 3:

% de referencias correspondientes a mujeres (gestante, parturienta, puérpera) o neonatos afiliados al SIS


Número de establecimientos de salud acreditados por tipo de resolución


Número de acuerdos de gestión


28. Within the PIU and, specifically, within the Technical Coordination Unit, the M&E Unit originally included two specialists, one focused on Monitoring and the other on Evaluation. Following suggestions from the MTR, the M&E function was strengthened and the original unit was split into two to focus and strengthen each area ― supervision of program performance, and management of scientific evidence for enhanced effectiveness of the overall program (see Annex 12).

Comentario:La Unidad de Monitoreo y Evaluación no dependía de la Unidad de Coordinación Técnica. Además el organigrama según el Manual es el siguiente, debiéndose modificar el presentado en el reporte.

62

2.4 Safeguard and Fiduciary Compliance (page 11)

31. Financial Management. The counterpart’s financial management performance was considered generally satisfactory. The PIU was appropriately staffed from the start and, despite delays in systematically adopting the official system for managing transactions used in all implementation units in the public sector (Sistema Integrado de Administración Financiera, SIAF), financial reports were timely and were found to be of satisfactory quality. Audit reports were provided on time and there were no qualified opinions. Until 2013, disbursement was very slow (39% in August 2013). Problems were related to a number of factors, including previous delays in civil works as a consequence of inadequate planning and due to the remote project locations, which did not offer incentives for enterprises to participate in the bidding process; and the contractors’ noncompliance with contractual clauses. The GOP established a condition of Pari-Passu for all civil works (6% of external resources), which limited the scope for accelerating disbursement of the loan. Finally, deferred payments for civil works at the end of the project (Ocongate Health Center and Health Center Chuquibambilla) and the cancelation of the scheduled execution of large amounts of consulting services expected to be contracted before the project closing and executed during the project grace period, negatively impacted disbursement. As a result, total disbursement reached 80% at project closing.

Comentario:Respecto a los retrasos iniciales debido a la falta de planificación, el informe debería precisar que un elemento importante que explica el retraso fue la demora en la firma del contrato de préstamo.Además mencionar, que no se concretó el financiamiento de una consultoría programada las (Diplomado APS PROFAM) y la recisión del contrato de otra consultoría sobre el Sistema Nacional de Sangre Segura; junto a las obras ya mencionadas en este párrafo.En relación al porcentaje de desembolsos a finales del 2013, tenemos registrado 49.33% con respecto a los fondos provenientes del Banco Mundial. Se sugiere indicar la fuente para verificar la data.

32. Procurement (page 11). Given that the entire Government program followed the Bank’s procurement guidelines, the Bank reviewed and approved a large volume of transactions. The initial delays due to lack of planning by the counterpart and long processing time for No Objections by the Bank were addressed and resolved after the first two years of project implementation. Procurement delays remained associated mainly with the scarcity of qualified contractors and remoteness of the work sites.

Comentario:Respecto a los retrasos iniciales debido a la falta de planificación, el informe debería precisar que el retraso se debió a la demora en la firma de los contratos de préstamo (16 de noviembre del 2009); lo que no permitió solicitar los créditos presupuestales

63

oportunamente ni para el 2009 ni el 2010 debido a los plazos que fijan las normas de presupuesto.Con respecto a la lejanía de los centros de trabajo, se debería de modificar la expresión, a que las intervenciones se realizaban en lugares de difícil acceso y que el problema de la planificación se debió a esto y a la calidad de los contratistas y la baja participación de los mismos. 3.2 Achievement of Project Development Objectives

42. When assessed against the original baseline, all the indicators have surpassed, achieved or partially achieved the targets at the end of the project, which supports achievement of all parts of the PDO (Table 1). Also, notably, the final percentage of achievement does not reflect that roughly half of all indicators surpassed their targets. Therefore, the actual level of achievement is far greater than the table suggests.

Table 1: Achievement of PDO (targets against original 2005 baseline)

Program

levelPDO level Intermediate PDO 1* PDO 2* PDO 3

Target surpassed 2 3 6 4 4 3Target achieved or substantially achieved (>=85% met) 2 4 1 5 1Target partially achieved (65%-84% met) 1 2 2 2 0Target not achieved (<65% met) 0 0 0 0 0Unknown 0 3 0 0 3Total 2 6 15 7 11 7% surpassed and achieved 100% 83% 83% 71% 82% 100%

Comentario:Se requiere la tabla con los cálculos realizados para la elaboración de la Tabla 1 del Informe.

49. Health infrastructure improved in the nine regions (page 16). The project supported the construction and/or renovation of 69 out of the planned 73 hospitals and basic health centers that offered obstetric and neonatal services; the remaining 4 are being completed in 2016, financed by GOP. It provided all of the 104 pre-identified health centers with medical equipment for maternal and child care and installed IT systems in 55 (planned 54).

Comentario:

64

Hasta el año 2015 se culminó las obras en 68 establecimientos de salud (en un establecimiento se realizaron 2 obras). Para este año 2016 se deben culminar 5 Establecimientos de Salud adicionales. Asimismo, el equipamiento médico y mobiliario se distribuyó en 103 establecimientos de salud. Finalmente se sugiere que con respecto a las TI señaladas, se debe especificar que esto se refiere al equipamiento para la implementación de la estrategia de Telesalud.50. The project delivered capacity building programs for different groups of health professionals using a culturally-sensitive approach. The project contributed to the training of 956 health workers (nearly 300 more than planned) on maternal and child care and specifically of 282 doctors and midwives on the vertical delivery (Parto Vertical) in 7 health facilities located in 4 regions within PARSALUD. Together with improved infrastructure and training, cultural adaptation of health facilities for the provision of vertical deliveries and support to Maternal Waiting Homes (Casas Maternas) were key in improving rates of institutional deliveries among indigenous populations. This includes the creation, dissemination, and systematization of knowledge around vertical delivery practices among health care providers and the institutionalization of this method in both regions: in the Amazon and Cusco, 54% and 33% of all deliveries were vertical, respectively, in 2012. The Casas Maternas, communal space managed, built, and maintained by communities and local governments, helped address the physical and cultural barriers faced by indigenous women residing far from health centers. There are 475 operational Casas Maternas throughout the country, with Cusco, Puno, Huancavelica and Apurimac the most important areas of reference.

Comentario:Cuál es la Fuente de las 475 casas maternas?

3.3 Efficiency Efficiency of project preparation and implementation (page 19)

64. The project was implemented within the time period originally planned. The project was extended by 11 months, which balanced the 10-month gap between Bank approval and effectiveness. However, despite the extension, the project disbursed 80% of the planned amount, due to delays in procurement and civil works. Comentario:El proyecto desembolsó el 80% del préstamo programado por el Banco, ello debido a que no se concretó el financiamiento de una consultoría programada (Diplomado APS PROFAM) ya la rescisión del contrato de otra consultoría(Sistema Nacional de Sangre Segura), por causal atribuible al contratista, por un total de aproximadamente US$1,900,000.Adicionalmente, se debió a la postergación de los pagos por la ejecución de obras que estaban financiándose con los recursos del préstamo, debido a que su ejecución excedía el plazo de vigencia del Programa (31 de diciembre de 2015), como es el caso del

65

Centro de Salud Ocongate y Centro de Salud Chuquibambilla por un importe aproximado de US$ 925,945.59Adicionalmente, el retraso de las contrataciones y ejecución se debió a que inicialmente fue necesario resolver contratos de obras, por situaciones atribuibles a los contratistas. Efficiency of project preparation and implementation (page 20) 66. The implementation efficiency of the overall Government program was leveraged by the project funds and the use of Bank fiduciary safeguards that provided cover for the weaker Government systems and processes. The Bank project team reviewed all procurement bidding documents and contracts for civil works, consultancy services and provision of training. This effort increased the leverage of technical and fiduciary implementation support beyond the loan funds and enhanced the technical skills of the PARSALUD technical and fiduciary team. Comentario:No todos los documentos de licitación y contratos para obras, fueron revisados por los bancos. En tal sentido, se debería precisar que se efectuaron las revisiones de losdocumentos de licitación y contratos para las obras, según el tipo de revisión, en este caso, la revisión ex ante, otorgándoselas No Objeciones correspondientes. En los casos de procesos con revisión ex post, las revisiones fueron aleatorias en las misiones fiduciarias del propio banco, además de las revisiones de la Auditoría Externa.

4. Assessment of Risk to Development Outcome (page 22)

74. PRONIS focuses on about 800 key health centers and facilities, many of which PARSALUD rehabilitated or improved. However, contrary to PARSALUD, PRONIS is not specifically targeted to the poorest regions and it does not support an explicit, a priori objective or topic. The regions will identify the specific outcomes of focus for the investments depending on their demographic and epidemiological characteristics and political priorities.

Comentario:

El número de Establecimientos de Salud Estratégicos que el MINSA ha definido son 748.

5.2 Borrower Performance

(b) Implementing Agency or Agencies Performance (page 25)

89.The unit was proactive in flagging any suspected cases of collusion in procurement. For example, an INT case was opened in April 2011 on risks during execution of works in the Region of Huancavelica. PARSALUD II timely identified and corrected fraudulent practices from an enterprise and it responded positively to all requests made by the Bank, designing and implementing a Governance and Anti-Corruption Action Plan.

66

Comentario:

Cabe señalar que la primera denuncia de práctica fraudulenta se efectuó ante el Banco Mundial el 12 de noviembre de 2010, en el marco del contrato suscrito para la ejecución de obras de la región Huancavelica entre el PARSALUD y el Consorcio Huancavelica. Por lo tanto, se deberá precisar que el caso de dicha denuncia se efectúo en Noviembre de 2010.

ANNEX 1. Projects Costs and Financing (page 28)

(a) Project Cost by Component (in USD Million equivalent)

ComponentsAppraisal

Estimate (USD millions)

Actual/Latest Estimate (USD

millions)


Strengthening of demand 6.00 6.43 107%Improvement of service delivery network 142.30 138.54 97%Government and Financing 5.20 3.25 62%M&E and Administration 8.90 16.86 189%

Total Baseline Cost 162.40 165.08 102%

Total Financing Required 162.40 165.08 102%

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest

Estimate(USD

millions)


Borrower 132.40 138.09 104% Inter-American Development Bank 15.00 15.00 100%

International Bank for Reconstruction and Development 15.00 11.98 80%

Comentarios:

Sobre el cuadro a)

67

El 03 de octubre del 2013 el PARSALUD envía al Ministerio de Economía y Finanzas el documento donde se señala la Modificatoria en la Estructura de Financiamiento de los contratos de préstamo (OFICIO N° 0740-2013-PARSALUD/CG) en donde se adjunta el OFICIO N° 0275-2013-PARSALUD/CG que anexa la propuesta de modificación.El Ministerio de Economía y Finanzas con Oficio 1457-2013-EF/52.04 del 26 de diciembre del 2013 envía la Nueva Estructura de Financiamiento de los contratos de préstamo aprobada de acuerdo al OFICIO N° 0740-2013-PARSALUD/CG.Con documento Re. Loan N° 7643 – PE – Health Reform Program (APL 12) – PARSALUD Project Restructuring del 26 de agosto del 2014, el Banco Mundial da opinión favorable a la propuesta de estructura de financiamiento de acuerdo al Oficio 1457-2013-EF/52.04, la misma que hace referencia a la modificatoria del 2013.Por lo que se solicita considerar la información del siguiente cuadro – En la Columna Modificado.

Distribución por componente, Montos originales (Viabilidad) VS última reasignación aprobada (Año: 2014)

Components

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of

Appraisal

Strengthening of demand 5.98 5.9 99%Improvement of service delivery network 142.25 141.8 100%

Government and Financing 5.19 4.83 93%M&E and Administration 8.96 17.17 192%

Total Baseline Cost 162.38 169.7 105%Total Financing Required 162.38 169.7 105%

Al respecto, se adjunta los documentos en PDF del Sustento de Modificación de la Estructura de Financiamiento Aprobada.Sobre el cuadro b)Igualmente se observa una diferencia en la data del cuadro enviado, la información que tiene la entidad es la siguiente:

Source of FundsType of Cofinan

cing

Appraisal Estimate

(USD millions)

Actual/Latest

Estimate(USD

millions)


Borrower 132.40 138.23 104% Inter-American Development Bank 15.00 15.00 100% International Bank for Reconstruction and Development 15.00 11.98 80%

68

ANEXO 7: Estudio de Evaluación Final de la Segunda Fase del Programa de Apoyo a la Reforma del Sector Salud – PARSALUD II (Resumen Ejecutivo)

Punto 29 RESULTADOS (página 54)En términos del desempeño del Programa en cuanto al cumplimiento de actividades y logro de productos, y el gasto ejercido por producto, se observó que el Programa fue eficiente en niveles de 100% o más en 6 de los 14 productos analizados. El componente con mayor eficiencia fue el Componente III de Gobierno, destacando la generación de propuestas de normas y disposiciones legales, y el diseño de sistemas de seguimiento. Otro componente con producción eficiente fue el Componente I de Demanda, donde destaca la producción de personal capacitado en el Programa de Comunicación y Educación para la Salud. Se debe mencionar también que la producción de capacitación de personal en gestión de recursos humanos (PREG, Parto Vertical, Emergencias obstétricas y neonatales, quechua) y en atención materno neonatal, del Componente II de Oferta, presentaron procesos eficientes. No obstante, entre los productos generados con menos eficiencia se encuentran: los relacionados a infraestructura, debido al retraso en el desarrollo de las obras y el equipamiento en Telesalud, en este último caso por el punto de corte de la evaluación.Comentario:

1. Las actividades de Telesalud solo fueron financiadas con recursos del BID y Recursos Ordinarios. No hubo recursos ni actividades asignadas al presupuesto del Banco Mundial.

2. Con referencia a lo colocado en el informe: “entre los productos generados con menos eficiencia se encuentran: los relacionados a infraestructura, debido al retraso en el desarrollo de las obras y el equipamiento en Telesalud, en este último caso por el punto de corte de la evaluación”

a. No hubo ningún retraso ni incumplimiento de las actividades ni productos programados en la línea de Telesalud del PARSALUD II.

b. En relación a la última entrega de equipos adquiridos para las DIRESA’s, en diciembre del 2015 se realizaron: (i) Las entregas a las Direcciones Regionales de Salud de Amazonas, Huánuco y Ucayali; (ii) Se entregaron las pecosas y actas con la conformidad de pago final a la UAF del Programa.

c. El giro al proveedor se efectuó en este ejercicio presupuestal 2016.

69

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

70

Annex 9. List of Supporting Documents

Project Appraisal Document, January 22, 2009 Minutes of Concept Note Review Meeting, November 2, 2005 Minutes of Quality Enhancement Review Meeting, May 4, 2006 Minutes of Decision Review Meeting and Matrix of Comments and Team

Reponses, December 8, 2008 Loan Agreement between Republic of Peru and International Bank for

Reconstruction and Development (7643-PE), signed November 16, 2009 Minutes of Negotiations, December 15, 2008 Implementation Completion Report of Health Reform Program (Programa de

Apoyo a la Reforma del Sector Salud - PARSALUD I; First Phase: Mother and Child Insurance and Decentralization of Health Services), dated March 30, 2006 (ICR000073)

Feasibility Study of PARSALUD II (Segunda fase del Programa de Apoyo a la Reforma del Sector Salud - PARSALUD II. Estudio de Factibilidad), November 2008

Restructuring Papers:a) Restructuring Paper (level 2), dated May 17, 2011, approved June 20, 2011b) Restructuring Paper (level 2), dated August 20, 2014, approved August 26,

2014 Implementation Documents:

a) Implementation Status and Results Reportsb) Aide Memoiresc) Mid-Term Review Report (Revisión de Medio Término de la Segunda Fase

del Programa de Apoyo a la Reforma del Sector Salud (PARSALUD II) en el Perú, April 2013)

d) Financial Audits Reportse) PARSALUD Progress Reportsf) Government’s Final Evaluation of PARSALUD (Estudio de Evaluación Final

de la Segunda Fase del Programa de Apoyo a la Reforma del Sector Salud ― PARSALUD II, December 2015)

PARSALUD Studies: a) Sistematización de la Estrategia de Promoción de Practica y Entornos

Saludables (EPPES) por el PARSALUD IIb) Sistematización de la Estrategia de Derecho a la Identidad y Aseguramiento-

DIA del PARSALUD IIc) Sistematización del Fondo Concursable para la Selección e implementación

de las iniciativas en el ámbito de intervención del PARSALUD IId) Sistematización de la Experiencia en Consulta Previa y Diálogos

Interculturales, Implementadas por el PARSALUD II

71

Annex 10: Analysis of PDO Achievement

# Indicator 2005 2009 2014End

target

Average annual growth

rate 2005-2009


rate 2009-2014 Ratings

End target met (100% achieved or surpassed)?

In/ before 2009?

Indicator going in desired

direction in 2009-2014?

Annual growth

rate >=0.5%

?1 Infant mortality rate 42 17 25 Surpassed Yes Yes

2

Prevalence of chronic malnutrition in children under 5 years of age 38.20% 23.70% 30.20% Surpassed Yes Yes

3

Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2014) 44.00% 58.80% 74.20% 78.00% 3.70% 2.20%

Partially achieved No Yes Yes

4

Reduce the prevalence of anemia among children under age in the nine regions from 69.5% (2005) to 60% (2014) 69.50% 58.20% 57.30% 60.00% -2.83% -0.13%

Partially achieved Yes Yes Yes No

5

Increase from 64% to 80% the share of children in the nine selected regions who are exclusively breastfed until 6 months of age 64.00% 82.20% 87.00% 80.00% 4.55% 0.69% Achieved Yes Yes Yes Yes

6

Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2014) 41.50% 31.80% 36.40% 35.00% -2.43% 0.66%

Not Achieved No No

7 Reduce the hospital lethality 9.50% 9.50% 5.00% 5.00% 0.00% -0.64% Achieved Yes Yes Yes

72


target


rate 2005-2009




In/ before 2009?



Annual growth

rate >=0.5%

?rate among neonates in the nine selected Regions from 9.5% (2005) to 5% (2014)

8

Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2014) 20.00% 63.90% 69.10% 45.00% 10.98% 0.74% Achieved Yes Yes Yes Yes

9

Percentage of SIS affiliated children who received growth and development controls (CRED) according to their age 34.00% 28.80% 56.80% 66.00% -1.30% 4.00%


10

Percentage of health facilities with improvement in infrastructure (minor construction, rehabilitation and/or new equipment) 0.00 0.00 103.00 103.00 0.00% 1471.43% Achieved Yes No Yes Yes

11

Percentage of SIS affiliated rural pregnant women with laboratory tests on hemoglobin, urine and syphilis 37.00% 42.00% 69.00% 53.50% 1.25% 3.86% Surpassed Yes No Yes Yes

12

Percentage of pregnant women under SIS that receive iron and folic acid 37.00% 49.90% 55.00% 60.00% 3.23% 0.73%


73


target


rate 2005-2009




In/ before 2009?



Annual growth

rate >=0.5%

?supplements

13

Percentage of women satisfied with the services in selected facilities by confidence index 0.00% 1.00% 74.40% 25.00% 0.25% 10.49% Surpassed Yes No

14

Percentage of cesareans in SIS affiliated pregnant rural women 3.00% 3.50% 9.50% 10.00% 0.13% 0.86%


15

Percentage of references among SIS-affiliated women (during pregnancy and puerperium) and neonates N/A N/A

Dropped Dropped Dropped Dropped

16

Percentage of SIS affiliated households that make out-of-pocket expenditures in medicines 67.30% 62.00% 56.00% 55.00% -1.33% -0.86%


17

Number of accredited health establishment by type of resolution 80.00 N/A


18Number of Management Agreements in place N/A N/A


19

Number of health personnel and community health workers trained within the behavior change campaign (EPPES) 0.00 0.00 1178.00 758.00 0.00% 16828.57% Surpassed Yes No Yes Yes

20 Number of health facilities 0.00 0.00 69.00 73.00 0.00% 985.71% Partially No Yes Yes

74


target


rate 2005-2009




In/ before 2009?



Annual growth

rate >=0.5%

?improved achieved

21

Norms and regulations to improve efficiency and equity of the health delivery system prepared 0.00 0.00 32.00 27.00 0.00% 457.14% Surpassed Yes No Yes Yes

22

Clinical pathways and corresponding financing systems designed 0.00 0.00 16.00 11.00 0.00% 228.57% Surpassed Yes No Yes Yes

23

Periodic evaluations of the performance of the health networks 0.00 0.00 19.00 17.00 0.00% 271.43% Surpassed Yes No Yes Yes* Explanation of Ratings: Surpassed: end target achieved/surpassed and indicator value at the start of the project lower than the end target; Achieved: end target met/surpassed (at any point in time), growth rate during the project going in desired direction and >=0.5%; Partially Achieved: end target not met, growth rate during the project going in desired direction and >=0.5% OR end target met before 2009, growth rate during the project going in desired direction and <0.5%; Not Achieved: end target not met and undesired trajectory of indicator

75

Annex 11: Loan Amount Allocation

Loan Amount Allocation (in millions of US$)Approved Revised Actual

(1) Goods, works, consultant’s services and Training under Part 1 of the Project

2.99 1.49 0.52


9.16 11.96 10.32


2.59 1.30 0.88

(4) Goods, consultant’s services including audit and Operating Costs

0.25 0.25 0.25

76

Annex 12: Organization of PARSALUD II Project Implementation Unit (PIU)

Source: http://www.parsalud.gob.pe/organizacion

General Coordinator: Dr Walter Vigo ValdezTechnical Coordinator: Dr Rosa Ines Bejar CaceresFinancial Administration Coordinator: Mr Fernando Masumura Tanaka

77

http://www.parsalud.gob.pe/organizacion

MAP

78