acknowledgment - pdf.usaid.govpdf.usaid.gov/pdf_docs/pa00k6gk.pdf · introduction and background 11...
TRANSCRIPT
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 2
ACKNOWLEDGMENT
This report was made for review by the United States Agency for International
Development and prepared by Lucas Pinxten, Siti Nurul Qomariyah, and Irma Anintya
Tasya for the USAID Indonesia Mission under the contract number AID-497-0-14-00097.
The following individuals have contributed to the preparation of this report:
John Rogosch, Masee Bateman, Maria Pinto, Edhie S. Rahmat, Mildred Pantouw, Ratna
Suwandono of the United States Agency for International Development (USAID), Robin
Nandy, Bill Hawley, Ferdinand Laihad, Karina Widowati, Budhi Setiawan, Olivi Silalahi,
Hellen Parrera, Sudhir Kanal, Jana Fitria Kartika Sari, Ratih Woelandaroe, Nurlely Bethesda
Sinaga of the United Nations Children’s Fund (UNICEF), Asik Surya, Erna Muliati, Lukas
Hermawan, Riskiyana Sukandhi Putra of the Indonesia Ministry of Health (MoH) Meike
Pontoh (PHO Maluku) Aloysius Giyai (PHO Papua) Victor Eka Nugrahaputra (PHO West
Papua) Bitzael S. Temmar (Maluku Tenggara Barat(MTB)/ West South-East Maluku district).
Special thanks for the warm welcome and input from the Head of DHO and Staff, District
Hospital Staff, Community Health Center Staff and clients, and the community leaders at
MTB District, Jayapura District and Sorong District.
Contact persons: Lucas Pinxten, MD, MPH, Global Health Management, Maastricht, The
Netherlands. ([email protected]) and Siti Nurul Qomariyah, MD, PhD, Center for
Family Welfare, Faculty of Public Health, University of Indonesia.([email protected])
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 3
Content page
A. Acronyms and Abbreviations 4 B. Executive Summary 6 1. Introduction and background 11
1.1. Introduction 11 1.2. Background 15 1.3. Scope of Work 18
2. Methodology 20 2.1. Overarching framework 20 2.2. Time line design, selection of sites and informants 21 2.3. Data collection and instruments 23 2.4. Data process and interpretation 24
3. Findings 25
3.1 Progress on objectives against combined MIP / ACHIEVE Indicators 25 3.1.1. Objective I: Improve service quality 27 3.1.2. Objective II: Improve health management 29 3.1.3. Objective III: Use of evidence for implementation 30 3.1.4. Project-specific objectives 30 3.1.5. Outcome Indicators 31
3.2. UNICEF inputs and results to strengthen MoH Malaria and MCH program 33
3.3. Program focus and coherence 36 3.4. Coordination and harmonization with other (non-) USAID
funded activities 37 3.5. Integration and synergy MIP and ACHIEVE 38 3.6. Health system challenges and adaption to MoH policy changes 39 3.7. Monitoring and evaluation 40 3.8. Sustainability and scalability 41
4. Recommendations 43
4.1. Recommendation for the remaining years 43 4.2. Observations and recommendations for the future programming 46 4.3. Measuring programs contributions towards USAID’s higher-level
health indicators 47 4.4. Lessons learned 49
List of Annexes: Deliverables, Timeline, Itinerary and list of informants, Overview structural interview detailed questionnaires, National Performance on MCH/ICM indicators, Task division, Matrix overview of interviews, focus group discussions with Clients/Patients List of MCH indicators, Recommendation and Priority Matrix, List of documents/literature, and statement regarding conflict of interests.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 4
A. ACRONYMS and ABREVIATIONS ACT Artemisinin-based Combination Therapy ANC Antenatal Care ABER Annual Blood Examination Rate AKBID Akademi Kebidanan: Midwifery Academy APBD Anggaran Pendapatan dan Belanja Daerah (District Revenue and
Expenditure Budget) API Annual Parasites Incidence ART Antiretroviral Therapy ATM AIDS, Tuberculosis and Malaria Center BAPPEDA Badan Perencanan Pembangunan Daerah (Local Development
Planning Board) BEONC Basic Emergency Obstetric and Neonatal Care BPS Badan Pusat Statistik (National Statics Office) BOK Biaya Operational Kesehatan, or Operational Health Funds Bapelkes Provincial Health Training Center Bupati Head of district Cadre Community volunteer CCM Country Coordinating Mechanism CDC Centers for Disease Control and Prevention CHC Community Health Center (Puskesmas) CEONC Comprehensive Emergency Obstetric and Neonatal Care DHO District Health Office DHS Demographic and Health Survey EMAS Expanding Maternal and Neonatal Survival, a USAID program EPI Expanded Program on Immunization GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria GOI Government of Indonesia NGO Non-governmental Organization IBI Ikatan Bidan Indonesia (Indonesian Midwives Association) IDAI Ikatan Dokter Anak Indonesia (Indonesian Pediatricians
Association) IMP Integrated Microplanning IPT Intermitted Preventive Treatment Jamkesmas Jaminan Kesehatan Masyarakat (Community Health Insurance) Jampersal Jaminan Persalinan (Delivery Insurance) JKN Jaminan Kesehatan Nasional (Universal Health Coverage) KIE Knowledge, Information and Education LLIN Long-lasting insecticide treated bed net MCH Maternal and Child Health MDG Millennium Development Goal MIP Malaria in Pregnancy MNCH Maternal, Neonatal and Child Health MCH/IMC Maternal and Child Health / Integrated Malaria Control MMR Maternal Mortality Rate NMR Neonatal Mortality Rate MOH Ministry of Health
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 5
MTB Maluku Tenggara Barat District (district in eastern Indonesia) MWH Maternity Waiting Home NMCP National Malaria Control Program NTB Nusa Tenggara Barat, (West Nusa Tenggara, a province in
Eastern Indonesia NTT Nusa Tenggara Timur (East Nusa Tenggara), a province in
Eastern Indonesia PERDA Peraturan Daerah (District or province regulation) PHO Province Health Office PKK Pembinaan Kesejahteraan Keluarga (Family Welfare Movement) PLA Participatory Learning and Action PMTCT Prevention of Mother-to-Child Transmission Polindes Pondok Bersalin Desa (Village delivery post) Poltekkes Politeknik Kesehatan (Polytechnic health academy) Posyandu Pos Pelayanan Terpadu (Integrated Health Service Post) Puskemas Pusat Kesehatan Masyarakat (Community Health Center) Pustu Puskemas Pembantu (Sub-CHC) RDT Rapid Diagnostic Test RSCM Rumah Sakit Dr. Cipto Mangunkusumo RSUD Rumah Sakit Umum Daerah / District Hospital SBA Skilled Birth Attendant SB Sekolah Bidan: midwifery school SOP Standard Operating Procedure TB Tuberculosis TBA Traditional Birth Attendant TWG Technical Working Group UNDIP Diponegoro University UNHAS Hasanuddin University UNICEF United Nations Children’s Fund US United States USAID United States Agency for International Development WHO World Health Organization
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 6
B. Executive Summary
Introduction
Indonesia has a population of 238 million: the fourth largest population in the world
and is the largest economy in South East Asia. Despite impressive macro economic growth
figures, significant health challenges remain: over 40% of the population or around 105
million are living on less than 2 US$ a day and the progress on maternal health (MDG 5)
has slowed down: maternal mortality rate (MMR) (number of deaths among pregnant
women / within 42 days after delivery per 100,000) declined from 390 / 100,000 in 1991 to
228 / 100,000 in 2007 and was in 2012 pegged at 359 / 100,000. Infant and child mortality
(MDG 4) is currently 32 respectively 40 per 1,000 live births. Especially the reduction of
neonatal mortality rate (NMR) has stalled: still at the 2007 level: 19 deaths per 1,000
births.1 This performance is, compared to other Asian socio-economic less developed
countries, rather poor and Indonesia will probably not meet these MDG targets next
years.2 East Indonesia is a high Malaria transmission area because it has ideal mosquito
breeding grounds: dense tropic forests, swamps, marshes and shallow water basins. This
part of Indonesia also has a myriad of remote islands in Maluku, an extremely difficult
highland terrain in Papua, and is home to the 4 provinces with the highest percentage of
relative poverty3,4. Malaria is important in endemic areas, like Eastern Indonesia and can be
responsible for maternal deaths both during acute malarial episodes and afterward because
of the effects of the disease on maternal anemia5.
In East Indonesia the 3 leading causes of maternal mortality are: 1. Haemorrhage:
34% (including: post- and antenatal haemorrhage, placenta previa, and premature
separation of placenta). This is higher compared to other Indonesian areas. 2. Eclampsia:
26% (including: edema, proteinuria, and hypertensive disorder). This is lower compared to
other Indonesian areas. 3. Infection and puerperium problems, including Malaria (30%)
(One of the highest compared to other Indonesian areas)6. Neonatal mortality also in
Indonesia is typically attributed to one of three major causes: infection, asphyxia, or
1 Indonesia - Demographic and Health Survey 20122013; Statistic Indonesia, National Population and Family Planning Board, Ministry
of Health, Measure DHS, ICF International: Accessed from: http://dhsprogram.com/publications/publication-FR275-DHS-Final-Reports.cfm.
2 Issue Briefs: Maternal and Child Health [database on the Internet]. UNICEF Indonesia,. 2012 [cited August 7, 2014]. Accessed from: http://www.unicef.org/indonesia/A5-_E_Issue_Brief_Maternal_REV.pdf.
3 World Bank. Indonesia Economic Quarterly - Slower Growth; High Risks2013: Accessed from: http://www.worldbank.org/content/dam/Worldbank/document/EAP/Indonesia/IEQ-Dec13-ENGLISH.pdf.
4 North Maluku, Maluku, West Nusa Tenggara, East Nusta Tenggara, West Papua and Papua - Poverty - Percentage of Poor People, Rural [database on the Internet]. knoema.com. [cited September 8, 2014]. Accessed from: http://knoema.com/atlas/embed/Indonesia/North-Maluku/topics/Living-Conditions/Poverty/percent-of-Poor-People-Rural?compareTo=ID-MA,ID-PB.
5 Ibid. 6 Joint Committee on Reducing Maternal and Neonatal Mortality in Indonesia. Reducing Maternal and Neonatal Mortality in
Indonesia : Saving Lives, Saving the Future. 2013.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 7
prematurity. Indonesia relies on national IDHS data. The IDHS National NMR: 20/1000,
which is about 50% of the Infant Mortality rate (which was according to IDHS 2012
45/1000 live births). According to IDHS 2012 NMR per province: Papua: 27, West Papua
35, NTT 26, Maluku 24, North Maluku 27/1000 live births.1 Most if not all of these
maternal and neonatal deaths are preventable. About 50% of maternal deaths are managed
by a health professional before death7 while in Malaria endemic situations up to 25% of the
maternal deaths can be attributed to Malaria.8
Socio-economic, educational, cultural factors and the poor access to services, due
to the extreme geographical settings, are problems beyond short time interventions. In
addition delays in recognition of a maternal/neonatal emergency, decision making and
transport/referral, and the quality of services: the classic "three delays and lack of quality of
services" can be targeted as we can learn from other countries who managed to reduce
both maternal and neonatal mortality to the MDG targets in a rather short period of time
span (5-10) years. The MCH/MIP program approach of using Malaria prevention, diagnoses
and treatment as a vehicle to strengthen the complex system- prevention- and care
interventions is an innovative approach that will lead to professional synergy9 and is
planned to reduce malaria in pregnancy and maternal and neonatal death3.
The current USAID-funded and GoI/UNICEF implemented Maternal Child Health
and Integrated Malaria Control in Eastern Indonesia project (MCH/MIP) covers past years
activities from two separate projects: Malaria in Pregnancy (MIP) and the ACHIEVE: the
former MNCH project. This integrated project started on October 1, 2010, will expire on
September 29, 2015, and is funded through a five-year grant, which is as MCH Umbrella
Grant awarded to UNICEF/NY Health Section for health and immunization response
support. The program is in sync with both USAID and UNICEF’s overall goal to support
GoI in attaining the MDG 4, 5 and 6, and is designed to implement activities to improve
MCH through utilizing evidence-based activities and practices, to improve the continuum
of care as it links to communities, primary level facilities, and hospitals in Eastern
Indonesia10.
The purpose of the mid-term evaluation was to assess UNICEF’s progress towards
their stated objectives and indicators in this project, and to make specific
recommendations to improve the project’s performance and future programming.
7 Ronsmans C, Scott S, Qomariyah SN, Achadi E, Braunholtz D, Marshall T, et al. Professional assistance during birth and maternal
mortality in two Indonesian districts. Bull World Health Organ. 2009 Jun;87(6):416-23. 8 Steketee RW, Nahlen BL, Parise ME, Menendez C. The burden of malaria in pregnancy in malaria-endemic areas. Am J Trop Med
Hyg. 2001 Jan-Feb;64(1-2 Suppl):28-35. 9 WHO Bulletin. Editorial. 2005 April;83(4). 10 Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia revised 3-08-2013. USAID project document
regarding incremental funding to immunization in collaboration with CDC. Jakarta2013.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 8
Methodology
The evaluators used a mixed quantitative-qualitative design. For the quantitative analysis
the evaluation team compared numerical aggregated provincial data from the year 2013 on
MCH/MIP program progress on targets/indicators, against 2010 Baseline MCH/MIP
program targets/indicators data. The team performed an overall progress on program
targets/indicators and an analysis of progress on MCH/MIP program targets/indicators per
province. For the qualitative analysis the evaluation team performed a desk study of
available MCH/MIP program documentation and conducted structural interviews and focus
group discussions with informants. For this purpose a structural overall interview guideline
was developed of which subsequently 5 specific interview guidelines were derived for
interviewing stakeholders form MoH national, UNICEF Country Office, DHO/PHO,
UNICEF Provincial Offices and for focus group discussion with clients. A result matrix was
developed containing detailed questions on: progress towards the project’s objectives,
target, focus and scalability of the project to support GoI investments in MCH/MIP,
appropriateness, accuracy and specificity of indicators, donor coordination and
harmonization and the integration of the two formerly-separate MIP and ACHIEVE
projects. The informants were: national and local level government staff working at both
policy and technical levels, USAID, UNICEF and CDC professional staff, a representative
from Global Fund for Aids Tuberculosis and Malaria (GFATM), representatives from two
other USAID funded programs related to mother and neonatal health care: Kinerja and
EMAS, UNICEF MCH/MIP program staff and clients/patients in 3 of the 5 Eastern
Indonesian provinces: Maluku, Papua, West Papua. Due to time limits the evaluation team,
together with USAID/UNCEF Country offices, selected the above-mentioned provinces.
The quantitative analysis covered all 5 provinces of Eastern Indonesia: Maluku, North
Maluku, Papua, West Papua and Nusa Tenggara Timor (NTT). The qualitative analysis
covered the above-mentioned 3 provinces. The evaluation lasted from August 18-28,2014. Findings
Comparing the MCH/MIP program baseline targets/indicators from 2010 against the 2013
performance indicators, it was concluded that over all provinces and over all indicators
46% reached 100% or beyond the targets/indicators and 78% of the targets/indicators were
reached at 50% or beyond. Most targets seemed to be set realistic and program progress
was good. Some specific selected findings are listed below: 1. Improvement of district health system (objective I of the MCH/MIP program) was
measured by the number of Health Centers reporting no stock outs of essential items for
the integrated Malaria program supplies: artemisinin-based combination therapy (ACT),
rapid diagnostic test (RDT) and long-lasting insecticide treated bed nets (LLIN) or maternal
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 9
health program: magnesium sulphate, oxytocin, intravenous antibiotics (MCH/MIP
indicators 5-10). It proved that no data at all were collected for MCH indicators (8, 9, 10),
while for the Malaria indicators, NTT province did not report and Maluku province
reached 86% on the stock of LLIN distribution (indicator 7) while all other provinces
(Maluku Utara, Papua, West Papua and NTT reached 100% or more on indicators covering
the stocks of LLIN, ACT and RDT (ind.5, 6, 7).
2. To measure service quality the proportion of pregnant women attending first ANC
contact who receive screening and appropriate treatment (indicator 12 & 13) were
recorded: Maluku Utara reached on both the Malaria and MCH target 84%. West Papua
reached only 30% of its Malaria target while Papua did not report at all on the MCH
indicator (13). Maluku and NTT Province reached respectively 52% and 72% on both the
Malaria and MCH indicators. Papua reached 58% on Malaria Indicator (12) and West Papua
57% on the MCH Indicator.
3. In order to measure increase of coverage of the integrated MCH/MIP program through
advocacy and leveraging of local Government/GFATM funds (Malaria Objective 3) the
number of districts with budget allocation for integrated MCH /MIP program that is at least
equal to the total contribution of the GFATM and UNICEF, was recorded (Indicator 18).
Analysis of the routine reported data revealed that only Maluku Utara and Papua reached
100% or more on the targets of this indicator. Maluku and West Papua showed no
progress and NTT did not report on this indicator.
4. East Indonesia combines a very poor referral system, due to extreme geophysical setting
with a low professional density: of the 1700 OB&GY’s, 14 work in Papua and only 9 in
Maluku. Also neonatology and anesthesiology are underdeveloped in this part of Indonesia.
5. In order to ensure that all women and newborns receive comprehensive and quality care
during pregnancy, delivery and postnatal periods the proportion of delivery assisted by
skilled birth attendance was recorded. The results showed that Maluku Utara and Maluku
reached respectively 75% and 53% of their targets, while Papua, West Papua and NTT
reached over 100% of their targets. Recommendations
1. To counter the regular Malaria and MCH stock-outs it is recommended that UNICEF
advocates for a buffer stock at DHO-PHO level for Oxytocin, Magnesium sulphate, RDT
and ACT and ILLNs, and increases its efforts to strengthening the referral system and the
quality of BEONC sites whereby action is required from UNICEF and the District and
Provincial Health offices (DHO/PHO). 2. The successes of the ILLN, Malaria screening and treatment, in terms of increased first
ANC visit and increased immunization figures, probably does not contribute much to
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 10
increase the proportion of deliveries assisted by skilled midwives. Beside the development
of buffer stocks it is recommended to use the synergy obtained by the integrated program
to inform expecting mothers of the benefits of the JKN health insurance scheme, and to
promote skilled delivery at an official BEONC site whereby action is required from
UNICEF and DHO/PHO level.
3. The integrated MCH/MIP program has proven to be successful in leverage of other funds
like APBD (Local Government) funds in Maluku and North Maluku for Malaria screening,
RDT and ILLNs for pregnant women and children and GFATM funds for scaling-up the
integrated MCH/MIP approach. However there is no indicator to measure this success. For
future and current program planning and to strengthen monitoring and evaluation, it would
be advisable adding strong output indicators for leverage, such as ensuring sustainable
funding from the local government to support the program whereby action is required
from UNICEF and USAID.
4. With the new budget allocation based on Village Law, villages in the remote area are
encouraged to secure some funds to support transportation to refer emergency patients.
Further formalizing the referral system at all levels is needed to ensure the roles of each
level to stimulate their ownership through actions by UNICEF and DHO/PHO.
5. Given the fact that over 30% of the neonatal mortality happens at delivery and about 50%
of maternal deaths are managed by a health professionals before death it is highly
recommended that UNICEF develops a stronger focus on strengthening the basic
integrated maternal/neonatal health services including safe deliver practices and neonatal
resuscitation at Health Center/BEONC level instead of trying to cover the whole spectrum
of Maternal and Neonatal health: Action required at UNICEF Country office level.
Selected lessons learned
1. Midwives at BEONC centers had a patient caseload below 100 deliveries per year. Below
this international recognized standard, especially emergency delivery skills are eroding. As a
result of the low caseload of midwives, a trained attendant is not necessarily a skilled
attendant and there is no guarantee for a safe delivery / appropriate neonatal care.
2. When the under-funded MoH system is faced with budget cuts it is the supervision budget
at all levels that seems to be sacrificed first. Lack of quality monitoring / supervision /
feedback mechanism results, not only in low motivation of field staff and reduced service
quality, but also in poor data collection which hampers evidence-based policy making and
planning: these budget cuts come at a very high cost.
3. Integrating the MIP guidelines and introduced to the curriculum of medical/midwifery
schools is an excellent way to ensure the continuation transfer of learning / knowledge
related to the integration and production of high quality of health service providers.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 11
1. Introduction and background
1.1. Introduction
Indonesia has a population of 238 million; is the fourth largest population in the world and
is the largest economy in South East Asia. Extreme poverty fell from 23.4% in 1999 to
11.4% in 2013, while over 40% of the population or around 105 million are living on less
than 2 US$ a day.11 The Eastern Indonesian provinces rank among the poorest in Indonesia.
Papua is the poorest of all Indonesian provinces ranking at 33, followed by West Papua
(32), Maluku (31), East Nusa Tenggara (30) and West Nusa Tenggara (29); North Maluku is
the exception in Eastern Indonesia ranking at 14 but still with about 10% of the population
living below the poverty line.12
From 1990 to 2007 Indonesian reduction of Maternal Mortality (MDG 5) and of U-
five mortality (MDG 4) figures showed a steady progress with having the largest reductions
happening before the financial crisis and the decentralization. The hasty decentralization in
2001 has increased the difficulty of establishing coordinated national health programs and
accounting for health funds and as a result may have contributed to the slowdown of U-five
mortality and especially maternal mortality (MMR) reduction. Despite last years and
current strong economic growth, significant health challenges remain: the progress on
maternal health (MDG 5) has slowed down maternal mortality rate (MMR) (number of
deaths among pregnant women / within 42 days after delivery per 100,000) declined from
390 / 100,000 in 1991 to 228 / 100,000 in 2007 and was in 2012 pegged at 359 / 100,000.
Infant and child mortality (MDG 4) is currently 32 respectively 40per 1,000 live births.
Especially the reduction of neonatal mortality rate (NMR) has stalled: still at the 2007 level:
19 deaths per 1,000 births. 13 This performance is, compared to other Asian countries,
rather poor and as a result Indonesia will probably need a few more years to meet these
MGD targets. 14
11 World Bank. Indonesia Economic Quarterly - Slower Growth; High Risks2013: Accessed from:
http://www.worldbank.org/content/dam/Worldbank/document/EAP/Indonesia/IEQ-Dec13-ENGLISH.pdf. 12 North Maluku, Maluku, West Nusa Tenggara, East Nusta Tenggara, West Papua and Papua - Poverty - Percentage of Poor People,
Rural [database on the Internet]. knoema.com. [cited September 8, 2014]. Accessed from: http://knoema.com/atlas/embed/Indonesia/North-Maluku/topics/Living-Conditions/Poverty/percent-of-Poor-People-Rural?compareTo=ID-MA,ID-PB.
13 Indonesia - Demographic and Health Survey 20122013; Statistic Indonesia, National Population and Family Planning Board, Ministry of Health, Measure DHS, ICF International: Accessed from: http://dhsprogram.com/publications/publication-FR275-DHS-Final-Reports.cfm.
14 Joint Committee on Reducing Maternal and Neonatal Mortality in Indonesia. Reducing Maternal and Neonatal Mortality in Indonesia : Saving Lives, Saving the Future. 2013.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 12
In Indonesia, availability and quality of basic emergency obstetric and neonatal care
(BEONC) is below standard: according to the 2011 Health Facility Review, by WHO and
World Bank, about 60% of districts have less than the government’s recommended 4
Health Centers providing BEONC. Shortages of drugs and supplies at Health Center level
are common also for key commodities to address maternal emergencies.
Both health professionals and families have to deal with the harsh reality of poor
BEONC services leading to maternal and neonatal death. According to the above-
mentioned survey almost 25% of the Health Center in-patients, including patients in
BEONC Health Centers, had no transport for referral. Service quality and rates of
maternal and neonatal death are the poorest in the provinces of Eastern Indonesia and
among the poorest women and children.
East Indonesia has a myriad of remote islands in Maluku, an extremely difficult
highland terrain in Papua, and has a huge variety in quality of healthcare due to differing
levels of commitment, availability and level of skills of health professionals in each district.
Maternal and child mortality was, compared to the national average, always higher in
Eastern Indonesia because of lower demand, a weaker health system, the geophysical
challenges and the higher burden of infectious diseases, including Malaria. In East Indonesia
the 3 leading causes of maternal mortality are: 1. Haemorrhage: 34% (including: post- and
antenatal haemorrhage, placenta previa, and premature separation of placenta). This is
higher compared to other Indonesian areas. 2. Eclampsia: 26% (including: edema,
proteinuria, and hypertensive disorder). This is lower compared to other Indonesian areas.
3. Puerperium problems, mostly infections including Malaria (30%) (one of the highest
compared to other Indonesian area's)15. Malaria is important in endemic areas, like Eastern
Indonesia and can be responsible for maternal deaths both during acute malarial episodes
and afterwards because of the effects of the disease on maternal anemia.16 Most if not all of
these maternal and neonatal deaths are preventable. About 50% of maternal deaths are
managed by a health professional before death17 while in a Malaria endemic situation up to
25% of the maternal deaths can be attributed to Malaria.18,19 The medical factors that cause
15 Ibid. 16 Ibid. 17 Ronsmans C, Scott S, Qomariyah SN, Achadi E, Braunholtz D, Marshall T, et al. Professional assistance during birth and maternal
mortality in two Indonesian districts. Bull World Health Organ. 2009 Jun;87(6):416-23. 18 Steketee RW, Nahlen BL, Parise ME, Menendez C. The burden of malaria in pregnancy in malaria-endemic areas. Am J Trop Med
Hyg. 2001 Jan-Feb;64(1-2 Suppl):28-35. 19 Schantz-Dunn J, Nour NM. Malaria and pregnancy: a global health perspective. Rev Obstet Gynecol. 2009 Summer;2(3):186-92.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 13
maternal, fetal (stillbirth), and neonatal deaths are often the same and the causes of
increased maternal, fetal, and neonatal mortality include poor nutrition, living in poverty,
poor access to care, and poor quality of care. High risk of maternal death is strongly
related to high risk of both stillbirth and neonatal death and interventions aimed at
reducing maternal mortality will frequently reduce stillbirths and many neonatal deaths.20
Socio-economic, educational and cultural factors and the poor access to services
due to the geographical setting, are problems beyond short time interventions. However,
delays in recognition of a maternal/neonatal emergency, decision making and
transport/referral: the classic "three delays" and the quality of services, can be targeted as
we can learn from other countries who managed to reduce both maternal and neonatal
mortality to the MDG targets in a rather short period of time span (5-10) years.21,22
East Indonesia has vast tropical forests, swamps, coastal marshes and shallow lakes
and as a result this is a high Malaria transmission area (>1 case per 1000 population) and
annual parasites incidence (API) between 10% (Lesser Sunda Island) and 15% (Papua).23,24
Pregnancy weakens the pregnant women’s immunity, increasing the risk of infection, severe
anemia and death. Malaria in pregnancy results in hemolytic anemia and is often the cause
of maternal death. Also acute renal insufficiency, spontaneous abortion and stillbirth are
related to Malaria in pregnancy while Malaria infection of the placenta leads to low birth
weight.25 Globally Malaria case fatality rate in pregnant women varies from 13% in a stable
epidemiological setting to 71% in a Malaria outbreak situation.26 Malaria contributes at
about 8% to the child's mortality27 while fetal loss or neonatal death rate in a Malaria
outbreak situation can be up to 67%.28 In Africa where Malaria is endemic (Malaria
Plasmodium only) it may directly contribute to about 25% of all maternal deaths.29 Sub-
Saharan African (SSA) interventions illustrate that 90% of the maternal Malaria burden can
20 Joint Committee on Reducing Maternal and Neonatal Mortality in Indonesia. 21 Freedman LP, Graham WJ, Brazier E, Smith JM, Ensor T, Fauveau V, et al. Practical lessons from global safe motherhood initiatives:
time for a new focus on implementation. Lancet. 2007 Oct 13;370(9595):1383-91. 22 Lazanoet. Progress towards Millennium Development Goals 4 and 5 on maternal and child mortality: an updated systematic analysis.
The Lancet. 2010;378:113-6. 23 WHO. International Malaria profiles: Indonesia Accessed from: http://www.who.int/Malaria/publications/country-
profiles/profile_idn_en.pdf. 24 Elyazar IR, Gething PW, Patil AP, Rogayah H, Kusriastuti R, Wismarini DM, et al. Plasmodium falciparum malaria endemicity in
Indonesia in 2010. PLoS One. 2011;6(6):e21315. 25 Manson's Tropical Diseases. 23 ed. Jeremy Farrar et all, editor. London: Elsevier Saunders; 2014. 26 Wickramasuriya. Malaria and Ankylostomiasis in the Pregnant Woman. Oxford.: Oxford University Press; 1937. 27 Hammerich A, Campbell OM, Chandramohan D. Unstable malaria transmission and maternal mortality--experiences from Rwanda.
Trop Med Int Health. 2002 Jul;7(7):573-6. 28 Steketee RW, Nahlen BL, Parise ME, Menendez C. The burden of malaria in pregnancy in malaria-endemic areas. Am J Trop Med
Hyg. 2001 Jan-Feb;64(1-2 Suppl):28-35. 29 Schantz-Dunn J, Nour NM. Malaria and pregnancy: a global health perspective. Rev Obstet Gynecol. 2009 Summer;2(3):186-
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 14
be reduced by a combined approach: preventive chemoprophylaxis/ intermittent
preventive treatment (IPT) with Sulphadoxine-pyrimethamine (SP), and the use of long
lasting insecticide treated bed nets (LLINs).30 However the traditional IPT utilized in SSA is
not applicable to Indonesia, because of the SP resistance and the fact that Indonesia has
four species of human malaria parasites: P. Vivax is the predominant species except in
Papua where P. Falciparum slightly predominates. P. Malariae and P. Ovaleare mostly found
in the eastern part of Indonesia, Nusa Tenggara Timur and Papua. The current national
treatment policy is Artemisin Combination Therapy (ACT) for pregnant women except for
1st trimester infections.
According to the 2012 IDHS in Maluku 87%, in North Maluku 90%, in West Papua
87% and in Papua 58%, received at least one check-up from public doctors, nurses, or
trained midwives.31 With this coverage level strengthening ANC service delivery through
increased coverage of interventions like LLINs and IPT with ACT, will be an incentive for
the use of other services such as delivery with the assistance of a skilled birth attendant.
According to B. Hawley, a CDC specialist on Malaria in Indonesia "Combining the vertical
Malaria program stimulates like a wedge the horizontal MCH program."32 Integration of
MCH/MIP also has a positive synergetic effect on the health professionals involved: they
are stimulated to cooperate and to work as a team.33 According to the WHO: using
Malaria in pregnancy as an entry point to comprehensive maternal and child health services,
seems to be a very sound approach because it will not only reduce the burden of Malaria
during pregnancy but will also improve MCH outcome.34
The Government of Indonesia introduced in 2014 a national health insurance
scheme, Jaminan Kesehatan Nasional (JKN), aiming to achieve universal health coverage by
the end of 2019. In 2015-16, substantial financial resources will be channelled to over
68,000 villages through arrangements under the new Village Law (800 million to 1.4 billion
Rupiah per village). Coming years the village will have authority over a much larger budget,
which will lead to increased accountability and inclusive decision-making. Especially point 3
of article 74 of the Village Law: improving public services for village citizens in order to
30 WHO. A Strategic Framework for Malaria Prevention and Control in Africa: Accessed from:
http://www.who.int/malaria/publications/atoz/afr_mal_04_01/en/. 31 Indonesia - Demographic and Health Survey 2012. 32 Personal comment during discussion with UNICEF country office staff, August 8 2014. Jakarta. 33 WHO Bulletin. Editorial. 2005 April;83(4). 34 WHO Global Malaria Programme. Malaria: Global Fund proposal development (Round 11) - WHO Policy Brief2011: Accessed
from: http://www.who.int/Malaria/publications/atoz/Malaria_gf_proposal_dev_who_policy_brief_201106.pdf.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 15
accelerate the realization of public welfare, offers opportunity to advocate for local funding
of village level, strengthening of LOCAL maternal and neonatal preventive and care
services.35
1.2. Background
In October 2010, USAID decided through a five-year grant and under the UNICEF
Umbrella Grant, (Agreement GHA-G-00-07-00007), to continue support for control of
Malaria in Pregnancy (MIP) in 5 provinces in eastern Indonesia. The intent of the program is
to reduce Malaria-related maternal and child mortality in provinces facing the highest
burden of Malaria in Indonesia and home to 15 million people living on about 700.000 km2
and a population density of 0.05 persons per km2.. Also in October 2010, the new maternal
and child health (MCH) UNICEF project was funded to accelerate progress in reducing
maternal mortality in 4 of the provinces supported by the MIP program. In 2012 the two
separate projects: Malaria in Pregnancy (MIP) and the MCH (ACHIEVE) were combined
into one project titled: Maternal and Child Health and Integrated Malaria Control in
Eastern Indonesia (MCH/MIP).This program expires in September 29, 2015.
The MCH/MIP program is in sync with both USAID and UNICEF’s overall goal in
supporting GoI in attaining the MDG 4,5 and 6.This integrated project covers past years
activities of two separate projects and is measured against the same indicators. The
integrated project is designed to enhance efficiency and alignment in the joint effort to
strengthen GoI systems for greater coverage and quality of integrated MCH/MIP services in
Eastern Indonesia.
The medical events related to maternal, fetal (stillbirth) and neonatal deaths are
mostly strongly related but the factors behind the causes include poor nutrition, living in
poverty, poor access to care, and poor quality of care.36
As stated above Malaria is endemic in Eastern Indonesia with particularly high rates
in the provinces of Papua, West Papua, Maluku, North Maluku and East Nusa Tenggara.
These provinces have 8% of Indonesia’s population but over 70% of its Malaria cases.37 In
these specific target provinces, people still lack basic preventive measures, receive poor
35 Republic of Indonesia, Village Law Number 6/2014, accessed from http://menpan.go.id/jdih/perundang-undangan/undang-undang. 36 Joint Committee on Reducing Maternal and Neonatal Mortality in Indonesia. 37 Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia revised 3-08-2013. USAID project document
regarding incremental funding to immunization in collaboration with CDC. Jakarta2013.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 16
diagnosis and inappropriate treatment and are highly mobile and Malaria is an important
factor in maternal and newborn health. Survey data estimate about 20,000 deaths annually,
although officially reported, deaths due to Malaria are relatively small.38 The USAID-funded
MCH/MIP project uses the single disease approach of Malaria, through leverage of Global
Fund, to strengthen the efforts to improve MCH and health systems in the remote areas of
Eastern Indonesia, supports and facilitates the MoH Malaria and maternal health sub-
directorate’s implementation of integrated antenatal care. Using Malaria control as a wedge
to force improvement to health systems in the most remote parts of the archipelago is an
effort to maximize indirect beneficial effects of the primary Malaria prevention tool the
LLIN, to increase demand for both antenatal health care services and a kind of "quick fix"
for rapidly decreasing the MMR in the target provinces.39,40
The Overall goal of the MCH/MIP program is "to accelerate Indonesia’s progress
towards attaining MDG5 in Eastern Indonesia, using Malaria control as a wedge to force
improvement to health systems in the most remote parts of the archipelago". This goal is
consistent with discussions with the Ministry of Health (MoH) on goals for the next several
years, and is in line with both the US Government’s Global Health Initiative and UNICEF’s
Country Program Action Plan, which aims for development of integrated Malaria control in
all highly Malaria-endemic areas of the entire country of Indonesia and reductions in
maternal and neonatal mortality in all communities. In order to counter major access and
quality of service, monitoring, planning and evaluation, human resources challenges and the
lack of community participation and accountability of service providers, the MCH/MIP
project supports MoH at sub-national level to implement district/provincial-wide Malaria in
pregnancy program by Malaria screening, IPT and distribution of LLINs to pregnant women
visiting the firsts ANC, by improving reporting on maternal and neonatal mortality, the
referral system and the quality of Basic and Comprehensive Emergency Obstetric and
Neonatal Care and referral.41
The specific objectives of the MCH/MIP program are:
Objective I: Improve service quality.
38 UNICEF Indonesia. Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP and ACHIEVE) - Seventh
Progress Report (November 2013 - April 2014) May 2014. 39 Ibid. 40 USAID. Report on a joint evaluation of UNICEF's USAID-funded program to control malaria in pregnancy in Eastern Indonesia.
Jakarta May 13, 2009. 41 UNICEF Indonesia. Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP and ACHIEVE) - Sixth
Progress Report (May 2013 - October 2013) November 2013.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 17
This covers the following original ACHIEVE and MIP objectives:
• Objective 1.(MIP): Malaria in Pregnancy (MIP) services provided by midwives and
nurses improved.
• Objective 2.(ACHIEVE): Ensure that all women and new-borns receive
comprehensive and quality care during pregnancy, delivery and postnatal periods.
Objective II: Improve health management.
This covers the following original ACHIEVE and MIP objectives
• Objective 2.(ACHIEVE): Improve district health system management for maternal
and neonatal health.
• Objective 3.(MIP): Quality monitoring and evaluation, and supervision at provincial,
district, and health center level are implemented.
Objective III: Use of evidence for implementation.
This covers the following original ACHIEVE and MIP objectives:
• Objective 4.(ACHIEVE): Enhance management and coordination for policy advocacy
and sustainable and effective program outcomes through sharing of experiences and
good practices.
• Objective 4.(MIP): Support operational research in Indonesia related to the control
of Malaria in pregnancy.
Project-specific objectives
There are two project specific objectives which are not re-formulated into new objectives
of the project. They include objective 3 of MIP and objective 1 of ACHIEVE.
• Objective 3. (MIP): Integrated Malaria program coverage is increased within and
among districts through advocacy and leveraging of local government/GFATM funds.
• Objective 1. (ACHIEVE):Improve the district based referral system to ensure
accessibility of quality emergency care for women and newborns. There has been
no indicator set for this specific objective. This reflects that improving the referral
system has not been a focus for UNICEF in this project period.
For the remaining years it is expected that the USAID-funded and UNICEF implemented
Maternal and Child Health and Integrated Malaria Control (MCH/MIP) in Eastern Indonesia
project will continue the following activities.42
42 Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia revised 3-08-2013. USAID project document
regarding incremental funding to immunization in collaboration with CDC.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 18
1. To support and sustain the control of Malaria in pregnant women and young children
by strengthening management, coordination, advocacy, sustainable and effective program
outcomes through sharing of experiences and best practices and replication/scaling up of
integrated Malaria program coverage among districts through advocacy and leveraging of
local Government/GFATM funds and government schemes.
2. To provide technical assistance to improve district-based referral systems to ensure
accessibility of quality emergency care for women and newborns and to improve
documentation and advocacy for needful policy and program formulation based on the
evidence of current and upcoming initiatives.
3. To ensure that all women and newborns receive comprehensive and quality care
including interventions to improve nutrition status of children and women during
pregnancy, delivery and postnatal periods.
For the detailed list of expected project activities: see ANNEX I: Scope of Work, Purpose
and Objectives of the Evaluation.
1.3. Scope of Work
The purpose of this mid-term performance evaluation is to assess UNICEF’s progress
towards their stated objectives and indicators in this project, and to make specific
recommendations to improve the project’s performance and to guide future programming.
Given the fact that the integrated MCH/MIP program is at a mid-term, there is a
need to assess the project's progress; whether the program is well targeted and focused;
in-sync and in line with current GoI strategies and actions towards the MDG's; increased
its effectivity and finally to assess whether the program indicators still cover program
activities, input, process and outcome.
USAID's and UNICEF's overall goal is to support the GoI's efforts regarding the
MDG’s 4,5 and 6. The results of the mid-term evaluation will provide these three key
partners with information on the current "state of the project", which allows midcourse
corrections and changes when needed and might guide future programming. The specific
objectives of this mid-term evaluation are: 1) to assess the performance of the project
against indicators and targets, 2) to make recommendations for the remaining years, 3) and
to make observations and recommendations for current efforts or future programming
with specific focus on the program indicators. Detailed expected project activities,
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 19
statement of work, purpose and objectives of the mid-term evaluation are listed in:
ANNEX I.
These three specific objectives are translated into the following overarching
evaluation questions:
1) What progress has been made towards the project’s objectives of improving the
quality of Maternal and Child Health and Malaria in Pregnancy programs, and
replicating good quality programs broadly? How have UNICEF’s inputs and efforts
contributed to the Government of Indonesia’s investments in Maternal and Child
Health in Eastern Indonesia, such as the Island Cluster Approach?
2) Is the project appropriately targeted and focused in the breadth of what it does,
and how it supports the GoI’s efforts to improve Maternal and Child Health in
Eastern Indonesia?
3) How scalable is the impact of UNICEF’s interventions, and how replicable is their
approach?
4) Are the project indicators accurately and sufficiently capturing the full scope of the
project’s impact? Are the indicators sufficiently specified and appropriately designed
to measure the project’s impact? If not, how can they be improved?
5) How successfully have the management and implementation of the two formerly-
separate MIP and ACHIEVE projects been integrated into a single coherent effort,
with a unified set of indicators and performance measures? How effectively do the
two components achieve synergies and/or leverage each other for greatest impact?
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 20
2. Methodology
2.1. Overarching framework
The CDC Working Group on Evaluation has developed a comprehensive and step-by-step
evaluation framework to improve and account for public health and curative actions as
pictured in Fig 1 below.43
Figure 1. CDC Evaluation framework for public health and curative actions.
Evaluation in general involves procedures that are useful, feasible, ethical, and accurate: this
CDC Framework for Program Evaluation in Public Health is a practical, non-prescriptive
tool, which summarizes and organizes the steps and standards of an effective program
evaluation. We use the above-mentioned CDC framework as a guideline for this mid-term
evaluation process and to assess the MCH/MIP progress towards the program objectives.
The Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP
and ACHIEVE) has three objectives, which are related to both MIP and ACHIEVE original
43 Framework for Program Evaluation in Public Health. CDC; 1999 [cited 2014 August 2]; Accessed from:
http://www.cdc.gov/EVAL/framework/.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 21
objectives. In addition, there are two project-specifics objectives: all summarized in Table
1. To asses whether these objectives are met, UNICEF in consultation with the
partners (i.e. the government of Indonesia and USAID) set 22 performance indicators. The
evaluation team involved UNICEF and stakeholders from province and district levels in the
data collection process. This approach may introduce bias to the data collection, as
beneficiaries might not want to inform the real situation in the presence of UNICEF and
other health offices staff. However, by informing the stakeholders on the objectives and
the bottom-up approach of the evaluation, which includes getting their recommendation
and creating commitment for the future programming, we assume that the bias is limited.
New objectives MIP objectives ACHIEVE Objective
Objective 1: Improve Service Quality
Objective 1: Malaria in Pregnancy (MIP) services provided by midwives and nurses are improved.
Objective 2: Ensure that all women and newborns receive comprehensive and quality care during pregnancy, delivery and postnatal periods.
Objective 2: Improve Health Management
Objective 2: Quality monitoring and evaluation, and supervision at provincial, district, and health center level are implemented.
Objective 3: Improve district health system management for maternal and neonatal health.
Objective 3: Use of evidence for implementation.
Objective 4: Support operational research in Indonesia related to the control of malaria in pregnancy.
Objective 4: Enhance management and coordination for policy advocacy and sustainable and effective program outcomes through sharing of experiences and good practices.
Project-specific objectives Objective 3: Integrated malaria program coverage is increased within and among districts through advocacy and leveraging of local government/GFATM funds.
Objective 1: Improve the district based referral system to ensure accessibility of quality emergency care for women and newborns.
Table. 1.Objectives of the Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP and ACHIEVE).
2.2. Time line, design, selection of sites and informants
The assignment started with the formation of the team and introduction of the team at
USAID offices on August 8, and the final report was due on September 16, 2014. The
overview of the Time Line and Deliverables is attached as ANNEX II, the Itinerary is
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 22
attached as ANNEX III and the list of informants as ANNEX IV. Discussions were
conducted with UNICEF staff, as well as the PHO/DHO staff after field visits to clarify
findings and to extract further information related to the issues discussed. The discussions
were also used to share lessons learned right after the data collection in each site. For
efficiency reasons the team sometimes split up some evaluation tasks. The team approach
characterized this evaluation while team members had a specific task division as
documented in ANNEX V.
The evaluation team combined quantitative and qualitative approach including data
set comparison, documents review, interviews and focused group discussion as explained
in the following chapters: data collection/instruments (2.3) and data process/interpretation
(2.4). In sequence, the approaches were conducted as follow:
A. Desk-base study: review of project reports/documents data sets and other
relevant documentation/literature before the start of the mid-term evaluation
and during the first week in order to be introduced to the MCH/MIP program.
B. Interviews in Jakarta with MoH officials, UNICEF Country Office staff and
representatives of other (non) USAID-funded programs in order to understand
the context depth of the program and to assess progress, scalability and
sustainability of the MCH/MIP program and the coordination/collaboration with
other (non) USAID-funded programs.
Site visits to three project areas during week 2 and 3 of the evaluation.
A) The evaluation started with a desk-study of relevant program documents. The team
reviewed documents provided by USAID/Indonesia, and UNICEF/Indonesian Country
Office, including the project SOW, routine UNICEF reports and other relevant documents
related to specific Indonesian and global MCH and Malaria programs. The main objective
of this document review was to compare program progress against the program indicators
and targets, which is the first evaluation question. The documents review was also used to
develop the data collection instruments for the site visits and interviews in Jakarta.
B) Discussions at UNICEF country office were held to get the overall picture of the
program implementation, including the challenges faced, efforts taken to overcome
challenges, and the program progress against the program indicators/targets. These
discussions were also instrumental for the site selection. Interviews were also conducted
with related MOH officers at national level to explore UNICEF inputs to the government
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 23
of Indonesia (GoI) and to assess whether these inputs are in sync and in line with the GoI
policies. In addition, the discussion was also used to gather information on challenges for
program implementation as well as the program sustainability.
The final site selection was conducted in consultation with UNICEF country office
as well as with USAID staff. This selection was not at random, but it was designed
purposively to get the specific information needed for the evaluation process. For example
Maluku was selected to get specific information on the cluster islands approach, which was
developed and implemented in Maluku. Papua and West-Papua were also selected for
specific reasons, such as to understand the challenges faced by the program due to
geographical barriers. The visit to Sorong was planned by the evaluation team to get a
good picture of the program implementation in an area with high a prevalence of HIV/AIDS.
The selection of sites to be visited took also in consideration the approach being used by
UNICEF in providing the specific technical assistance like the use of the "positive deviance
approach": model sites developed in some focused districts/health centers to be further
adapted and implemented in other districts/health centers.
Once the sites were selected, the selection of informants to be interviewed by the
evaluation team was in consultation with UNICEF field staff. Considering the limited time
available, again, this also was a purposively selection approach. This may also introduce bias
to the data collection process. However the data triangulation, by which the evaluation
team gathers and combines quantitative and qualitative data from available data sources and
from informants at all levels assures that the evaluation team gets the best possible picture.
The team interviewed 97 informants (as listed in ANNEX IV) including national and
local level government staff, working at both policy and technical levels and USAID staff,
UNICEF project staff, other (non) USAID program staff working in MCH and Malaria in
Eastern Indonesia. The team also had discussions with 21 Health Center clients (in 3
focused group discussion of 5-8 clients). Overall, the Evaluation team met about 200
people both through the interviews and FGDs, and at the island of Nyafar Nifmas, (Maluku)
where the team had a focus group discussion with 8 villagers and conducted an extended
village gathering with 85 villagers.
2.3 Data collection and instruments
Based on the five core evaluation questions, as presented in the background chapter (1.3)
an overall structural questionnaire (ANNEX VI) and specific interview guidelines, for MoH,
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 24
UNICEF Country Office, DHO/PHO, UNICEF Provincial Offices and clients were
developed (ANNEX VII). Informants for each evaluation question were defined based on
their relevancy to the questions; while methods of data collection were defined based on
the possible number of informants willing to be consulted for the question.
The first evaluation question (the program progress against program indicators), for
example, was answered by comparing the MCH/MIP program progress against Baseline
2010 and "National 2013 Performance" (meaning a compilation of routine quantitative
performance data on the integrated MCH/MIP program indicators in all 5 provinces in
Eastern Indonesia) and combined with results from interviews of UNICEF staff at country
and provincial office level as well as discussion with beneficiaries in supported districts
(qualitative data). The other evaluation questions will be answered based on the results
from interviews with stakeholders using the specific interview guidelines (qualitative data)
and combined with / compared against the results of the Baseline 2010-Results 2013
comparison.
2.4. Data process and interpretation
For the quantitative evaluation: measuring the MCH/MIP program progress on the
objectives against established program indicators, the evaluation team selected the best
available routine data set. In order to measure this progress the evaluation team opted for
maximal reliability by comparing the most recent available performance data (National
Performance 2013) against the 2010 baseline indicators: covering a timespan of over 3
years. As a result the evaluation team first developed an overall overview "Progress on
Program indicators 2010 baseline - 2013 results" spread sheet (see ANNEX X), to facilitate
the analyses of total progress over all indicators per province and the progress per specific
indicator per province.
For the qualitative analysis: all interviews and focused group discussion, when
possible and after consent, were audio-recorded. In addition, a matrix (by informant by
question) was developed to put brief notes based on the finding from the interviews and
discussions (see ANNEX VIII). Completion of the matrix was as much as possible
conducted on a daily basis to avoid missing information. The findings presented in this
report are based on the quantitative analysis and qualitative analyses of both the matrix and
audio-records.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 25
3. Findings
In order to fulfill the first specific objective of this evaluation: to assess the performance of
the MCH/MIP project and the related overarching core questions (1.3): we start with
paragraphs on progress against objectives and indicators followed by the paragraph on the
UNICEF inputs and efforts. The next paragraphs describe the program focus and
coherence, coordination and harmonization, integration and synergy of MIP and ACHIEVE,
followed by the advocacy and leverage of GFATM and other funds, health system
challenges and adaptation to MoH policy changes, monitoring and evaluation and ends with
a description of sustainability and scalability.
3.1. Progress on objectives against combined ACHIEVE/MIP Indicators
For the quantitative assessment of the program, the evaluation team was supplied with 3
data sets: the first set included the combined ACHIEVE/MIP (MCH/MIP) program 2010
baseline/ target indicators 2011-2015, per province and two sets "National Performance
2012 and 2013": covering MCH/MIP program target indicators, results and percentages
reached in the 5 Eastern provinces as compiled by UNICEF. However, comparing the most
recent available performance data (national performance 2013) against 2010 baseline
indicators posed a challenge because of the missing values in both data sets. In order to
increase the strength (validity and reliability) of the comparison between the 2010 baseline
and the 2013 performance indicators, missing values of the 2010 baseline indicators were
imputed using agreed 2011/2012 program target indicators (as stipulated in ANNEX X). As
for the "National Performance 2012" this data set has, compared to the 2013 data, besides
missing values, also data reporting formats that do not allow comparison with the 2010
baseline.
Table 2 offers a summary of progress on overall indicators in 4 quartiles;
respectively >/= 100%, 75-100%, 50-75% and </=50% of target reached and 1 category
"undetermined". From this Table it can be concluded that for all provinces and over all
indicators, 46% reached at 100% or beyond the targets/indicators (Maluku Utara 10,
Maluku 8, Papua 12, West Papua 13, and NTT 8 of the 22 indicators) and 78% of the
targets/indicators were reached at 50% or beyond (Maluku Utara 19, Maluku15, Papua 16,
West Papua 14 and NTT 13 of the 22 indicators) over the 3-years timespan. All provinces
have (3-8) missing values. Compared to the other 3 provinces, the provinces Maluku and
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 26
NTT are under-performing with respectively 7 and 9 of the 22 of the target indicators
reaching 100% or beyond. Papua and West Papua seemed to be the high(-er) performing
provinces, with respectively 12 and 13 of the 22 indicators having reached 100% or beyond,
however especially the results of Papua need to be treated carefully because in 2013 only
14 of the 29 districts reported their results to provincial level.
Progress summary: Indicators reached
Maluku Utara
Maluku Papua West Papua NTT
>/= 100% of target reached
10 7 12 13 9
75-100% 3 1 3 1 2 50-75% 6 7 1 0 3 </=50% 0 3 1 4 1 Undetermined 3 4 5 4 7
Table 2: Summary of progress (2010-2013) on overall indicators in 4 quartiles and 1 category undetermined across 5 provinces in Eastern Indonesia.
The USAID-funded and GoI/UNICEF implemented Maternal and Child Health integrated
Malaria Control Program (MCH/MIP) in Eastern Indonesia is measured against the 22 core
indicators listed in ANNEX X. These indicators are originally stemming from the MIP
(Malaria in Pregnancy) and the ACHIEVE / Mother and Child Health programs which were
merged together with the program objectives into the current integrated MCH/MIP
program in October 2011.
Comparing progress against objectives and indicators (2010-2012) we refer to the
22 core indicators (refer to ANNEX X Progress on program indicators 2010 baseline -
2013 results). Overall one can observe that in all provinces selected parts of the service
quality of MIP and MCH (indicators. 1-4) and for indicator 11, Maluku and Maluku Utara
seem to perform very well. However in Papua, West Papua and NTT the number of
districts with effective malaria slides crosschecking systems were very low. The malaria
part of the program was strong in logistics (reduced stock outs: indicator 5-7, except for
LLINs: 86% reported in Maluku and 0% reported in NTT) compared to MCH logistics
(indicator 8-10), which were not reported at all.
Except the result of first ANC (MCH) visits (indicator 15) in Maluku Utara: over
100% of the targets reached and the first ANC (MIP) visit in West Papua (indicators 14 and
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 27
16) over 100% of the targets reached, the overall percentage of targets reached on ANC
related indicators (indicators 12-17) seems to be low in the other provinces. On the
advocacy indicator 18 there is a remarkable split: Maluku Utara and Maluku are reaching
far beyond the 100% targets while the other three provinces are underperforming with
percentages ranging from 25%-48% reached. The mother and neonatal care (indicators 19-
22) seems to be weak in Maluku Utara and Maluku and strong in the provinces Papua,
West Papua and NTT. In the following chapter a more detailed description about the
performance against the project objectives and indicators is presented.
3.1.1. Objective I: Improve service quality
This covers two original objectives, one from ACHIEVE (objective 2) and one from MIP
(objective 1). Table 3 below describes the performance against the objectives and
indicators.
Objectives Indicators Findings Objective 1
(ACHIEVE): Ensure that all women and newborns receive comprehensive and quality care during pregnancy, delivery and postnatal periods.
Indicator1: Number of midwifery academies with pre-service training module for midwives developed and implemented:
West Papua and NTT reached their targets while of the three other provinces some exceeded beyond their targets as illustrated below in Figure 2.
Indicator3: Number of districts with standard localized model HC for internship for new staff:
Maluku Utara and NTT reached 200% of their target while the other provinces reached 100%.
Indicators12&13: Proportion of pregnant women attending first ANC contact who receive screening and appropriate treatment (indicator 12: MIP, indicator 13: MCH):
All provinces except Maluku Utara reached below 75% of their targets on both MIP and MCH. Maluku Utara reached 84% on both MIP and MCH indicators.
Indicators14&15: Proportion of pregnant women who attend ANC at least once during their pregnancy:
Maluku reached 39% of its target on indicator 14 (MIP). All other provinces reached well above 50% of their targets on both indicators. Maluku Utara reached over 100% on the MCH indicator (15) and Papua reached over 100% of target of respectively indicator 15 (MCH) and Indicator 14 (MIP).
Indicator19: Proportion of pregnant women attending four
Maluku Utara and Maluku reached respectively 63% and 50% of their
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 28
antenatal visits during their pregnancy:
targets while the other provinces reached 100% or beyond.
Indicator 20: Proportion of delivery assisted by skilled birth attendances:
Maluku Utara and Maluku reached respectively 75% and 53% of their targets, while Papua, West Papua and NTT reached respectively 184,102, and 107% of their targets.
Indicator 21&22: Proportion of newborns (Indicator 21) and mothers (Indicator 22) who received a check up by skilled health providers within 2 days of delivery (KF1 & KN1):
Maluku Utara and Maluku scored low on both indicators: Indicator 21 being 80% and 53% and for Indicator 22 being 74% and 62%. Papua, West Papua reached their targets on both indicators for 100% or more, while NTT reached 100% on target 21 and 92% on target 22.
Objective 2 (MIP): Malaria in Pregnancy (MIP) services provided by midwives and nurses improved.
Indicator12&13: Proportion of pregnant women attending first ANC contact who receive screening and appropriate treatment:
Maluku Utara reached on Indicator 12 (MIP) and 13 (MCH) 84% of its targets. West Papua reached on Indicator 12 (MIP) only 30% of its target while Papua did not report at all on indicator 13 (MCH). Provinces Maluku and NTT reached respectively 52% and 72% on both indicators. Papua reached 58% on indicator 12 and West Papua 57% on indicator 13.
Indicator16&17: Proportion of pregnant women attending first ANC contact who receive an insecticide treated bed net:
Except Maluku Utara reached 69% of its target of indicator 17, the rest of the data are poor or under-reported due to the confusing definition of Indicator 17. On indicator 16 (MIP) Papua and West-Papua reached respectively 99% and 116% while Maluku Utara and Maluku reached respectively 69% and 35% of their targets on Indicator 16.
Table 3 Performance against Objective 1: Improve service quality
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 29
Figure 2. Number of midwifery academies with pre-service training module for
bidans developed and implemented
3.1.2. Objective II: Improve health management
This new objective covers two original objectives:
1. (ACHIEVE): Improved district health system management for maternal and neonatal
health.
2. (MIP): Quality monitoring and evaluation, and supervision at provincial, district, and
health center level are implemented.
The performance of UNICEF activities against these two objectives is described below.
Objectives Indicators Findings Objective 1
(ACHIEVE): Improve district health system management for maternal and neonatal health
Indicators 5-10: Number of Health Centers reporting no stock outs of essential items for the integrated Malaria program (ACTs, RDTs, LLINs) or maternal health program (magnesium sulphate, oxytocin, intravenous antibiotics) (MIP and MCH):
Due to lack of data available for Indicators 8,9,10, progress on these Indicators is undetermined. On Indicators 5, 6, 7: except the fact that NTT did not report and Maluku reached 86% on the MIP indicator 7 (LLINs), all other provinces reached 100% or more on indicators 5, 6, 7.
Objective 2 (MIP): Quality monitoring and evaluation, and supervision at provincial, district,
Indicator 2: Number of health centers in focus districts conducting quarterly midwife peer review meeting:
All provinces reached 100% or more on the targets of this indicator.
Indicator 11: Number of Maluku Utara and Maluku both
1
2
1
3
2 2
3 3 3 3
2
3 3 3
2
0
1
1
2
2
3
3
4
Maluku Utara
Maluku Papua West Papua NTT
Baseline 2010
Result 2012
Result 2013
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 30
and health center level are implemented.
districts with effective malaria slides cross-checking systems:
reached over 100% of the targets of this Indicator. Papua, West Papua and NTT reached respectively 25%, 44% and 48% of the targets of this Indicator.
Table 4. Performance against Objective II: Improve Health Management
3.1.3. Objective III: Use of evidence for implementation.
This covers two original objectives, both are Objective 4 of the ACHIEVE and MIP
program.
1. ACHIEVE: Enhance management and coordination for policy advocacy and sustainable
and effective program outcomes through sharing of experiences and good practices. This
objective is measured by Indicator 18, which is: ‘Number of districts with budget allocation
for integrated MCH program that is at least equal to the total contribution of the GFATM
and UNICEF.’ Only Maluku Utara and Papua reached 100% or more on the targets of this
Indicator. Maluku and West Papua showed no progress and NTT did not report on this
Indicator.
2. MIP: Support operational research in Indonesia related to the control of Malaria in
pregnancy. UNICEF is an important partner of the CDC/LSTHM/Eijkman Malaria in MCH
treatment trial. However there are no program indicators formulated to measure this
objective.
3.1.4. Project-specific objectives
There are two project specific objectives which are not re-formulated into new objectives
of the project. They include objective 3 of of the MIP and objective 1 of ACHIEVE program.
Objective 3 (MIP):Integrated malaria program coverage is increased within and among
districts through advocacy and leveraging of local government/GFATM funds.
Three indicators were set for these specific objectives:
• Indicator 3: Number of districts with standard localized model HC for internship
for new staff.
• Indicator 4:Number of health centers in focus district using mother card
integrated with malaria for program monitoring:
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 31
• Indicator 18: Number of districts with budget allocation for integrated MCH
program that is at least equal to the total contribution of the GFATM and
UNICEF:
For Indicators 3 and 4, all provinces reached 100% or more on the targets. While for
Indicator 18 only Maluku Utara and Papua reached 100% or more on the targets. Maluku
and West Papua showed no progress and NTT did not report on this Indicator.
Figure 3. Number of districts with budget allocation for integrated MCH program
Objective 1 (ACHIEVE): Improve the district based referral system to ensure accessibility
of quality emergency care for women and newborns. There has been no Indicator set for
this specific objective. This reflects that improving the referral system has not been a focus
by UNICEF in this project period.
3.1.5. Outcome Indicators
Two outcome Indicators were set by UNICEF and their partners to assess the project
performance: number of neonatal deaths and maternal deaths (ANNEX X). The GoI
routine health information was the source used for the number of maternal and neonatal
deaths.
Maternal deaths: comparing number of reported maternal deaths in the year of
2012 and 2013 showed a decrease in Maluku and North Maluku, but an increase in Papua
and West Papua. There were no data available on the performance in NTT in 2013.
Neonatal deaths: comparing data from 2012 and 2013, revealed a very high increase
of neonatal mortality in North Maluku and West Papua: 72% and 62% respectively.
However the neonatal mortality decreased in Maluku: 62% and in Papua 23%. Similar to
1 1 2 2
1
6
-‐
4
-‐ -‐ -‐ 1 2 3 4 5 6 7
Maluku Utara
Maluku Papua West Papua
NTT
Baseline 2010
Result 2013
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 32
data for maternal deaths, there is no information available on neonatal deaths for 2013 in
NTT. Given the fact that these figures are based on incomplete routine reporting data
there is a possibility of over- and underestimation of the reported cases. As a result the
data are too limited to derive a trend in maternal and neonatal mortality in Eastern
Indonesia at this moment.
Given the complexity of the MCH/MIP program is facing in all three provinces
visited: Papua, West Papua and West Maluku, the progress towards the objectives as
measured by the Indicators is sufficient but stock-outs are still reported as can be learned
from ANNEX X and the summary above. Especially the stock of LLIN seems to be a
problem in Maluku province. Interview with the CDC Technical Advisor and the Director
of the Sub-directorate Malaria confirmed that progress of the Malaria part of the program
is hampered by problems related to procurement and distribution of LLIN at national level.
As for MCH drugs and supplies, (Oxcitocin, Oxygen and Magnesium Sulphate) there
seemed to be a logistical (late ordering) problem at District-Health Center level: there are
no data available for Indicator 8-10. Staff at DHO/PHO level report that this is a temporary
problem: "due to the electronic catalogue". Malaria progress is reported at provincial level
but not covered by an appropriate outcome Indicator.
Moreover it is remarkable that the first objective: "to improve service quality" is
covered by 10 indicators and objective II: "to improve health management" is covered by
11 indicators while objective III: "the use of evidence for implementation" is covered by
only 2 indicators and the last objective: "to support operational research” has no Indicator
at all. As a result of the above it is obvious that documentation of progress on Indicators
needs to be addressed: Indicators are not balanced over the objectives. Objective I and II:
"improving service quality and health management" have sufficient Indicators while
objective III "Use of evidence for implementation" is not covered by appropriate Indicators.
The type and strength of the Indicators will be discussed in detail in the chapter
below but in general the current targets and Indicators are mostly all at input and process
level. The two outcome Indicators covering maternal and neonatal mortality lack resulting
data to generate a solid conclusion: as a result, progress on these two outcome Indicators
is undetermined.
The above-presented quantitative findings were confirmed during discussions with
UNICEF staff at country office, regional office, and MoH partners and during field
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 33
observations. Summarizing the qualitative and quantitative information one could state: the
progress towards objectives is gradual but slow, while most targets seem to be set realistic
and progress towards the targets/indicators in 2013 was, compared to the 2010 baseline,
as illustrated above good.
In general the integrated MCH/MIP program made a fair progress but given the
weak health system, the extreme working conditions and logistical challenges in Papua,
West Papua and Maluku, one could summarize the progress as follows "Despite all
challenges and barriers the USAID-funded and GoI/UNICEF implemented Maternal and Child
Health Integrated Malaria Control program in Eastern Indonesia showed, compared to the 2010
baseline, a promising progress in the year 2013 but major challenges are still ahead".
3.2. UNICEF Inputs, and results to strengthen MoH Malaria and MCH program
The inputs provided by UNICEF to the government of Indonesia in general can be
summarized as ‘strengthening the capacity’ of the stakeholders from national to community
level to implement the National Malaria Control Program and the National Maternity and
Child Health program. According to some stakeholders there is no such thing like "The
UNICEF Program". According to Dr E. Mulati from the Directorate of Child Health MOH,
Jakarta, UNICEF acts as ‘catalisator’ to ensure that the Integrated MCH/MIP program is
implemented appropriately.
The support to improve access and quality of Maternal and Child Health and
Malaria in pregnancy programs is provided through technical assistance strengthening the
health system and improving the capacity of the health providers often through the
following 3 specific UNICEF approaches:
A. Maternity Waiting Homes or Rumah Tunggu (MWH) are a well tested and equity-based
strategy implemented by UNICEF for the first time in Afghanistan about 20 years ago to
increase access to maternal and newborn services among women from remote areas. It is
a low-cost solution to skilled birth attendance, aiming to bridge the geographic gap in
obstetric care in inaccessible areas and reduce inequities. To document the maternity
waiting home approach, MoH and UNICEF have reviewed this approach44. As a result of
this review, MoH decided on 15 August 2013 to start a well monitored try-out of the
44 Evi Martha and team. A Review on Maternity Waiting Homes Effectiveness in 5 Provinces in Indonesia. Depok2013.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 34
MWH approach in Banten Province before giving the green light for development of
MWHs in 5 provinces: Jambi, Gorontalo, Maluku, West-Maluku, and NTT. According to
some informants there seems to be no funding available for the evaluation of this try-out
and as a result the official status of the MWH approach is currently not clear.45,46,47,48
B. Cluster Island Approach (CIA) was originally developed as a national economic planning
process by BAPENAS (National Development Planning Board). The CIA was modelled as a
health-planning tool for Maluku Tengah Barat (MTB) district to address access to health
services, which are the major bottleneck in the Maluku’s and Papua.
C. Integrated Micro Planning (IMP) is a technical capacity and advocacy approach initially
developed by UNICEF in Aceh Province for Health Center level "bottom-up" planning.
Through the IMP process, which includes a simple computerized data analysis process,
Health Center staff is guided to find priority problems and planning activities for each
village based on the problems found. IMP contributes to formulation of a district strategic
plan based upon minimum service standards and is used to re-socialize the MCH, malaria
and immunizations programs to professionals, stakeholders and policy makers. In Papua
Province IMP has continued to serve as a vehicle to improve BOK (local funding
mechanism) utilization rate and to contribute towards activities to reach underserved
populations.49
The last two Technical Assistance tools proved to be useful in strengthening the
local health system through facilitation of local planning of MCH/MIP programs/activities,
including logistics/procurement, reporting and advocacy to the local government to ensure
their support to the MCH and MIP programs.
To strengthen the health system, UNICEF facilitates ao:
1) Health centers conducting quarterly midwife peer reviews development.
2) Development of a standard localized model Health Center for internship for new staff.
45 GoI MoH. Pertemuan Kajian Efektivitas Rumah Tunggu Kelahiran: Accessed from: http://www.gizikia.depkes.go.id/pertemuan-kajian-
efektivitas-rumah-tunggu-kelahiran/?print=pdf. 46 Innovative Approaches to Maternal and Newborn Health Compendium of Case Studies, 2013: Accessed from:
http://www.unicef.org/health/files/Innovative_Approaches_MNH_CaseStudies-2013.pdf. 47 Evi Martha and team. A Review on Maternity Waiting Homes Effectiveness in 5 Provinces in Indonesia. Depok2013. 48 UNICEF. Annual Report 2012 for Indonesia, EAPRO, : Accessed from:
http://www.unicef.org/about/annualreport/files/Indonesia_COAR_2012.pdf 49 UNICEF Indonesia. Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP and ACHIEVE) - Seventh
Progress Report (November 2013 - April 2014).
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 35
3) Implementation of mother card integrated with malaria for program monitoring,
respectively Indicators 2, 3, and 4 all reaching 100% or beyond as presented above.
The USAID/UNICEF logistical support proved to be especially strong in the MIP part of the
integrated program as illustrated by the Indicators 5.6.7 that reached 100% and more in all
provinces (except Province Maluku on Indicator 7). MCH logistics are still a major
challenge because none of the provinces reported on the related Indicators.
In order to ensure integrated MCH/MIP capacity at pre-service level, UNICEF has
successfully developed and advocated implementation of integrated MCH/MIP training
curriculum in medical schools and midwifery academics as is measured by Indicator 1,
which reached beyond 100% of the target Indicators in all provinces.
Despite current logistical / procurement LLINs problems and in lesser degree of
supply of Rapid Diagnostic Malaria Tests (RDT) reported by several informants including
UNICEF and DHO staff, the LLIN coverage with support from GFATM is (except in
Province Maluku) in all other provinces realizing 100% or more their targets. However,
scaling-up beyond GF supported districts in highly endemic Eastern Indonesia (most of
Papua, the Maluku’s, and East and West Nusa Tenggara) is hampered by financial
constraints. Quality LLINs have to be imported until now. The purchase of LLINs with GF
support cost is less than $5 per net if free of tax. If purchased with MoH funds, the cost
more than doubles due to tax50.
The Integrated MCH Malaria program has been implemented beyond the UNICEF
focus districts. In Indonesia vaccination is a very important part of the ANC program.
Integration of the MCH and Malaria opens the door for further integration of the EPI
program into the MCH/MIP program. Provision of bed nets seems to have improved the
overall attendance of at least the first ANC visit in South Halmahera, district measles
vaccination increased 10-15% against baseline data and in Jayapura district a 10% increase in
routine ANC visit was observed in the year after the initiation of the integrated program
UNICEF according to a UNICEF report.51
50 WHO Jakarta. Review of the National Malaria Control Programme of the Republic of Indonesia. 2011; (September ): Accessed
from: http://apps.who.int/iris/bitstream/10665/94367/1/2012-‐08_SEARO.ReviewMalariaControl%20Prog.Indonesia%20201.pdf.
51 UNICEF. Sharpening the equity focus: Selected Innovations and lessons learned from UNICEF-‐assisted programmes 2009-‐20102011.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 36
3.3. Program focus and coherence
The Malaria in pregnancy program is well focused with clear targets (screening of Malaria
and LLINs for all women during their first ANC visit). This is also the case for the
integration of Malaria control program with immunization for infants (providing bed nets
for all infants after their last immunization visit for measles).
The Evaluation team noted during field visits that LLIN procurement/logistical and
regular stock-outs of essential MCH supplies and drugs were the major constraints to the
implementation of the MCH/MIP and immunization program. According to Dr. Asik Surya,
Sub directorate Malaria, the implementation of the USAID/UNICEF supported integrated
MCH/MIP program in Eastern Indonesia is in line with the National Malaria Control
program policy.
To facilitate cooperation and collaboration between all MoH levels, UNICEF played
an important role (shuttle diplomacy) in bringing together the MOH departments at
central, provincial and district level. USAID/UNICEF, together with WHO, along with key
officers within the MOH (especially Dr. AsikSurya, Head of the Sub-directorate Malaria, Dr.
Lukas Hermawan, Head of the Sub-directorate Pregnant Women and Dr. Erna Mulati,
Head of the Sub-directorate Child Health) who championed the benefits of integration,
played critical roles in facilitating collaboration across the involved departments (Malaria,
maternal health, immunization) to identify mutual goals, to draft a national strategy and to
ensure joint inputs with regards to developing operational guidelines, training materials,
staff orientation, logistics systems and new reporting formats for their respective
programmes. The success of the integrated program, which managed to lever over 169
million US$ from the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) and is
now rolled-out beyond the focus of Eastern Indonesian, is a good illustration that the
program efforts and results are well appreciated.
Considering the specific geographical setting in Maluku, UNICEF helps the local
government in formulating the Cluster Islands Approach: a tool "par excellence" that helps
to develop a focus and coherent health planning. UNICEF assists the local DHO with the
Cluster Island Approach to build the capacity of the health providers in selected Health
Centers to become as ‘pusatgugus’ or ‘center of cluster’ and to strengthen some district
hospitals to become the referral CEONC hospital. Problems with human resources and
logistics will remain but the approach suits the condition in Maluku well according to DHO
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 37
staff in Ambon. Health providers need enough number of cases to maintain their skills.
From the field visit it is found that in some Health Centers the number of estimated
deliveries per year is very small compared to the number of midwives working in the area.
In the BEONC Aimas Health Center the average caseload per midwife was 7 deliveries
maximum per month: too low to maintain essential emergency skills. According to Dr.
Sudhir Kanal, the Papua-based UNICEF Child Survival and Development Specialist, the
Cluster Island Approach could solve the problem of low caseload of midwives in Eastern
Indonesia by determining the number and place of BEONC locations based on midwife
caseload and need instead of blindly following the MoH instruction to develop the 4
standard BEONC sites also in low population density districts. Improving access should be
in line with maintaining quality of care.
3.4 Coordination and harmonization with other (non-) USAID funded activities
USAID is a major donor for maternal and neonatal health. AusAID through their Australia
Indonesia Maternal and Newborn Health and Nutrition Program (AIMPNH) in 10 districts
in NTT province, is the other main bilateral donor in the field of maternal and neonatal
health.52 The USAID-supported JHPIEGO-contracted Expanding Maternal and Neonatal
Survival (EMAS) program focuses on CEONC level in East Java and the USAID-funded
Millennium Challenge Corporation (MCC) program aims to reduce stunting by integrating
maternal and child health, nutrition, water and sanitation through the GoI. Beside UNICEF,
WHO and UNFPA are other UN agencies providing maternal and neonatal technical
assistance to the GoI.
At national level, coordination and harmonization with other (non)-USAID funded activities
is realized through regular donor meetings coordinated by the MoH and high-level inter-
donor coordination at donor’s initiative.
At provincial level, the coordination meetings between development partners are through
the existing provincial government and the Bappeda donor coordination secretariat. At this
level program representatives of the integrated MCH/MIP program meet with AIMPNH
representatives and also with the USAID-funded and RTI contracted Kinerja program
professionals. Kinerja builds capacity at the local (or sub-national) level to improve
accountability and measurable enhancements of health, education and business 52 DFAT Concept Note Accessed from: http://aid.dfat.gov.au/business/Documents/mnch-‐conceptnote-‐indonesia.pdf
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 38
development programs, managed and delivered by local service providers. Kinerja's focus
in Papua is exclusively on health service delivery: strengthening leadership and management
capacities for health service delivery. However according to several informants the current
collaboration between Kinerja and UNICEF is sub-optimal: especially alignment of their
specific activities, given the comparable mandates, needed to optimize USAID return of
investment in Papua province.
In Maluku there is no other NGO working in the MCH/MIP field, GFATM is the
only international donor active for health programs; UNICEF coordinates along with PHO
closely every time needed. Professional communication lines between the GFATM
representatives Thomas Matulessy, a retired PHO officer, and UNICEF office proved very
short during the discussion the evaluation team had in Ambon on August 18 with the PHO
team, UNICEF and the GFATM representative. The GFATM representative and PHO staff
meet on a regular base to coordinated, synchronized activities and planning and support
each other in implementation and evaluation of the integrated MCH/MIP program
according to Dr. Olivi Silalahi, the UNICEF Malaria and EPI officer stationed in Ambon. As
for non-health related NGOs and donors, usually the coordination is in the hands of
Bappeda Province but since the last few years there are no other NGOs and donors active
in Maluku.
In West Papua, a development partner forum has been formed to align NGO's
working in this booming province. UNICEF and other organizations are waiting for the
governor's signature to approve this partner forum. As mentioned by Dr. Victor Nugraha
putra from the West Papua PHO, this partner forum is urgently needed to improve the
dialogue and to cope with the challenges of MoH-NGO coordination (current frictions
between MoH and Clinton Foundation illustrate the need for this platform).
In Papua, UNICEF was the driving force behind routine quarterly meetings in the
provincial health forum, which was usually held by BAPPEDA. The head of the PHO
confirmed that they just formed the partnership forum on June 1, 2014 and that NGO's
and donors are looking forward.
3.5. Integration and synergy MIP and ACHIEVE
The integration of the two formerly separated programs is successful given the fact that
comparing Baseline 2010 with results of 2013, over half of the Indicators reach 75% or
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 39
more of their target Indicators. Integration and synergy are especially visible at first line
level of the health system. Midwives at Health Centers and villages have adopted the
integrated Malaria approach. The integration of the two programs has stimulated
teamwork and task sharing at Health Center level. Guidelines have been developed, staffs
are trained and the curriculum is inserted in the medical/midwifery curriculum. Screening,
treatment for Malaria and the distribution of LLIN are now part and parcel of the ANC
package.
At national level, senior informants mentioned that there still is room for
improvement of the coordination between DG in MoH but that new roles of two DGs
(CDC and MCH) gradually developed while working together. In the last years the
integrated program is running, a "specialization" has developed: The CDC/Malaria
department mainly deals with the policy and the logistics for the integrated program, while
the MCH department proved to be effective in operations and monitoring, especially in
ensuring the quality of the services provided by health providers. However, review of the
existing recording and reporting systems, more specifically the development of integrated
Indicators is needed to ensure that the results of the integrated program are captured in
the system in the future.
3.6 Health system challenges and adaptation to MoH policy changes
UNICEF is currently facing one main challenge in supporting the government of Indonesia
through the MCH/MIP program: in the new Fund Channelling regulation, UNICEF cannot
directly provide funding to the provincial and district government. There are two new
government regulations related to fund channeling mechanisms, both were launched in
2012. The first regulation is called:Peraturan Pemerintah (PP) nomor 2 tahun 2012 tentang
Hibah Daerah (the Government Regulation on Funding/grant to local governments) and the
second is:Peraturan Menteri Keuangan (PMK) nomor 188 tahun 2012 tentang Hibah dari
Pemerintah Pusat kepada Pemerintah Daerah (the Ministry of Finance regulation on funding
from the central government to the local government). PP number 2 2012 implies that in
providing funding to local governments, development partners should channel the funding
through the central government, i.e the Ministry of Finance which acts as the countries
general finance (Bendahara Umum Negara) body.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 40
At national level, the challenges include the limited number of staff that has to deal
with the administration for the funding provided by UNICEF in each province/district. To
overcome this problem, UNICEF, together with the local government, channels the funding
through third parties, such as professional organizations, local NGOs etc. However, this
road has challenges of its own: sometimes it is difficult to find the third party, which has
the technical and administrative capacity to manage the support. These problems have
delayed the implementation of some activities in most areas.
3.7 Monitoring and evaluation
Monitoring, evaluation and feedback mechanism are key to the quality of the health service.
The field visit to Papua, West Papua and Maluku revealed that monitoring and evaluation
remains a problem for all staff at health center level. The facilitative supervision, the main
supervision method, introduced many years ago by the MoH, has not been implemented
because of lack of funding for the field visit according to a DHO staff member in Maluku.
Also in Papua and West Papua there was no budget for supervision visits, because the
allocated budget was the very first item to be slashed when DHOs were confronted with
the national budget reductions last year. UNICEF staff and representatives from other
NGO’s were reluctantly taking the role of supervisors according to Vince Bakker, Head of
the Malawe Health Center in Sorong. The evaluation team also found that there has been
no post-training evaluation after UNICEF facilitated a team of health providers trained on
BEONC in West Papua. Only West Maluku and Maluku have an effective malaria slide
cross checking in place (Indicator 11). The feedback mechanism for Malaria quality control
and evaluation is lacking in the other 3 provinces as illustrated by the microscopist at
Mayamuk Health Center, West Papua: he never got feedback from the cross checker at
district level on the results of the slides he submitted for cross checking.
The problem of lack of monitoring and evaluation (and the resulting detrimental
effect on the quality of services) is admitted by Dr. Lukas Hermawan, Head of the Sub-
directorate Pregnant Women: modules, SOP, guidelines are developed and distributed,
health providers are trained, however there has been no effort to monitor the compliance
of the health providers in using the modules, where ideally monitoring, evaluation and
feedback should be standard practice in all levels of the Indonesian health system.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 41
3.8. Sustainability and scalability
UNICEFaims at realizing sustainability and scalability of the MCH/IMC project by:
1. Successfully promoting the inserting MIP in the curricula at medicine/midwifery schools.
This is one of the efforts to ensure that new health providers will have the skills and
knowledge needed to provide integrated Malaria and MCH services (Indicator 1: over
100% of the targets reached in all provinces).
2. UNICEF also assists the provincial and district health offices to advocate the provincial
and district government to ensure their support to the integrated MCH/MIP program:
however only 2 of the 5 provinces reached their targets on Indicator 18.
3. The Cluster Island Approach has been implemented by the province government (they
put it in the Regional Long Term Development Plan (RPJMD) and also by the central
government to be used in other provinces with a similar geographical condition like Maluku.
At district level, in Maluku, the Maluku Tenggara Barat district government has formalized
the Custer Island Approach as a Peraturan Daerah (District regulation) to ensure
continuation and budget allocation for the approach in the future.
4. The MWH approach, successful in other countries, and currently under evaluation by
MoH is well established in Maluku and being scaled-up in West-Paua. The team has visited
three homes, 2 in Maluku and 1 in West Papua, which were all in use, in a good shape,
staffed with Kader, supervised by Health Center staff and maintained using district health
funds. However as a result of the pending Indonesian evaluation of MWH, there are still no
guidelines to facilitate the further scaling-up of this approach.
In order to ensure the sustainability of the integrated MCH/IMC program for a
longer period, further assistance to the provincial and district health office is still needed,
as the commitment of the district government to support the program in some districts is
still sub-optimal. As an example: the support provided by UNICEF to one hospital in Papua
to become a referral hospital for neonates, through facilitation of a training team from
RSCM (the top referral hospital in Jakarta) has not been supported optimally by the local
government to ensure the availability of the logistics needed.
Results of interviews with many stakeholders in Maluku and Papua stress the fact
that there is progress but that reaching sustainability in Eastern Indonesia takes more time.
One informant said that they are still in a premature phase: "If there is no support from
UNICEF some of the programs may not be sustainable". Another informant mentioned
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 42
that to sustain the program, assistance from UNICEF is still needed especially for the non-
focus districts. The integrated MCH/MIP program proved to be scalable through GFATM
and the routine funding especially for Malaria control program is already available in 2 of
the 3 provinces (Indicator 18) through the local government budget (APBD).
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 43
4. Recommendations
This chapter is divided into 3 paragraphs: paragraph 4.1.gives recommendations for the
remaining years; paragraph 4.2.shares observations and recommendations for future
programming and finally paragraph 4.3. The recommendations and action required by
selected stakeholders is summarized in ANNEX XI. Further chapters discuss program’s
contributions towards USAID’s higher-level health indicators. To align with the UNICEF
objectives, recommendations are ordered based on their related objective whether it is to
improve service quality, improve health management, or to use evidence for evaluation.
Every recommendation is concluded with a required action by one or more of the
stakeholders.
4.1. Recommendations for the remaining years
4.1.1. To improve service quality:
• Health Center staff implementing the integrated MCH/MIP program, including skilled
attendants, will meet their limitations when there is lack of essential drugs and supplies
and when accessibility of emergency services like caesarian section is not available. It is
recommended that UNICEF advocates for a buffer stock at DHO-PHO level for
Oxytocin, Magnesium sulphate, RDT and ACT and LLINs, and increases its efforts to
strengthening the referral system and the quality of BEONC sites: Action required by
UNICEF and DHO/PHO level.
• In Maluku, UNICEF has successfully piloted the MWH. The MWH offers a home close
to an established BEONC/CEONC site for pregnant women in the last month of
pregnancy, who cannot travel the distance between their home and the
BEONC/CEONC site in case they need emergency maternal services. A first qualitative
study commissioned by MoH and UNICEF illustrated the MWH approach. MoH has
decided on August 15, 2013 to start and evaluate a try-out of MWH in Banten Province
before giving green light to scale-up this approach. The progress of this try-out needs to
be followed up: Action required by UNICEF at National level.
• Midwife density and the output of AKBIDs in both Maluku and Papua seem to reach
saturation: it is advisable to reduce the intake of student at the midwifery school,
otherwise they will become midwives who attend deliveries at home and are ill
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 44
prepared for emergency obstetric and neonatal services, while their services are
counterproductive and will not contribute to the reduction of MMR and NMR. It is
proposed to advocate in Papua and Maluku for a reduced intake of new midwives
through a "numerus-clausus" system, to re-certify midwives having a caseload lower
than 100 deliveries per year and to promote "4 hands delivery": Action required by
National MOH level together with professional Organization (IBI).
• The successes of the LLIN, Malaria screening and treatment, in terms of increased first
ANC visit and increased immunization figures, probably does not contribute much to
increase the proportion of deliveries assisted by skilled midwives. It is recommended to
use the synergy obtained by the integrated program to inform expecting mothers of the
benefits of the JKN health insurance scheme, and to promote skilled delivery at an
official BEONC site: Action required by UNICEF at DHO level.
• Maintaining the gains and momentum of the Integrated MCH/MIP approach: urgent
action is needed: refresher and new training of health professionals to counter staff
rotations and to improved facilitative supervision by MoH staff to maintain these skills
and ensure quality of services: Action required by UNICEF at DHO level.
4.1.2. To improve health management:
• In "the ideal health setting" information gathered through a routine data collection
system is translated into knowledge, which feeds rational national- and sub-national
program planning and policymaking, improves health management and fuels advocacy for
program expansion. In order to monitor, evaluate, document and demonstrate
effectiveness of the GoI-implemented and USAID/UNICEF supported MCH/MIP
program in Eastern Indonesia, improving the current routine data collection system and
development of specific indicators to measure the results of the integrated approach of
MCH/MIP program, is needed. Action required by UNICEF/DHO/PHO level.
• The integrated MCH/MIP program is well developed, aligned with the government
program and well implemented. UNICEF has proven to be very successful in leverage of
other funds like APBD funds in Maluku and North Maluku for Malaria screening, RDT
and LLINs for pregnant women and children and GFATM funds for scaling-up the
integrated MCH/MIP approach. However, there is no Indicator to measure this success.
For future and current program planning and to strengthen monitoring and evaluation, it
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 45
would be advisable adding strong output Indicators for leverage, such as ensuring
sustainable funding from the local government to support the program.
Action required by UNICEF and USAID.
4.1.3. To use evidence for evaluation
• In order to better bridge evidence and practice it is recommended that UNICEF
strengthen its evidence-based health programming and documentation by increasing
operations research (plus budget), systematic documentation and sharing of qualitative
and quantitative results on the contextual conditions and the effect of intervention
programs. Documentation can be realized through sharing lessons learned at (inter-)
national professional level (conferences, workshops, blogging and peer-reviewed
publications), development and dissemination of policy briefs and publications beyond
the grey literature level of internal donor reports. Action required at UNICEF Country
Office level.
• Performance Indicators of the MCH/MIP program are mostly at input and process level
and use the data from the routine system or from specific records of UNICEF activities.
The MCH/MIP program lacks program outcome Indicators. It is recommended to make
program performance more tangible and focused by adding outcome Indicators, based
on agreed National Indicators for both Malaria and MCH: Action required at
UNICEF/MOH/USAID level.
• Both routine data and surveys (e.g. Indonesia Demographic and Health Surveys) show
that progress has been made in increasing the coverage of several key reproductive,
maternal, newborn and child health interventions3. However, there has been limited
progress in improving maternal and child outcomes because of a major gap between
coverage and the quality of care provided in health facilities. Therefore, it is important
to make quality of service into an integral component of scaling-up interventions to
improve health outcomes of mothers, newborns and children. Considering this, the
WHO organized consultation on improving measurement of the quality of maternal,
newborn and childcare in health facilities. The consultation proposed nineteen global
core Indicators for assessing the quality of health care provided to mothers, newborns
and children in health facilities (see ANNEX XII). It is expected that UNICEF supports
GoI in selecting the appropriate Indicators as well as finding the mechanism to measure
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 46
them. As seen in the table attached, the Indicators can be captured through routine
data collection, self-assessment or surveys. The existing facilitative supervision should be
revitalized, as it is one of the main mechanisms to ensure quality of services and data
collected to measure the progress against agreed-on Indicators: Action required by
UNICEF and MOH at National/Provincial and district level.
• Despite regular stock outs, of ACT, RDT and LLINs in Eastern Indonesia, harboring
about 80% of the Malaria incidence, the Indonesian Malaria control program is
successful: according to the WHO World Malaria 2013, the number of presumed and
confirmed cases is rather stable from 2010-2012: around 2 million a year (2010:
1 963 807 cases; in 2011: 2 384 260 cases and in 2013: 2 051 425 cases) while the
number of confirmed cases goes down (2010: 465 764 cases, 2011: 422 447 cases and in
2013: 417 819 cases): a 4.4% reduction of confirmed cases over 3 years.53 In order to
keep the momentum and prepare for Malaria elimination (in line with the WHO/GOI
Elimination target in 2030) the integrated MCH/MIP program should adopt a new
Indicator ‘proportion of villages reducing their endemicity level’. In addition, an output
Indicator, e.g. number/percent of pregnant women with presumed/confirmed and
treated malaria is advisable. Action required by UNICEF, USAID and MoH.
• Donor collaboration, cooperation and harmonization are an important efficiency
requirement in development cooperation. In Papua province the USAID-funded and RTI
contracted Kinerja project focuses exclusively on health service delivery: strengthening
leadership and management capacities for health service delivery. Kinerja and UNICEF
have overlapping mandates and focus. According to several informants the current
collaboration between Kinerja and UNICEF is sub-optimal: especially alignment of their
specific activities is needed: Action required by USAID/UNICEF.
4.2. Observations and recommendations for future programming
• UNICEF facilitates BEONC service level as observed by the evaluation team in Adaut,
Maluku and Aimas Health Center in Sorong. At Yowari Hospital in Sentani, Papua
province, UNICEF supports the neonatal health services. In Maluku, UNICEF, as part of
the cluster island approach, supports the CEONC (PONED-Plus) facility at the Magretti
53 World Malaria Report -‐ Indonesia2013: Accessed from: http://www.who.int/malaria/publications/country-‐
profiles/profile_idn_en.pdf?ua=1.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 47
hospital in Saumlaki. Given the fact that over 30% of the neonatal mortality happens at
delivery and 50% of the maternal mortality happens at service delivery level, it is highly
recommended that UNICEF has a stronger focus on strengthening the basic integrated
maternal/neonatal health services including safe delivery practices and neonatal
resuscitation at Health Center / BEONC level instead of trying to cover the whole
spectrum of Maternal and Neonatal health: Action required at UNICEF Country office
level.
• Appropriate and professional decision making to reduce maternal and neonatal
mortality starts with collection of high-quality data on the relevant rates and causes of
mortality. Due to the decentralization, routine data collection and reporting in the
Indonesian health system has eroded. National and regional planning often is based on
the limited information of periodic household surveys to obtain official MMR and NMR
figures. The evaluation team was provided with national performance data, however the
spreadsheets had a lot of missing values (blank cells). The 2012 National Performance
data sheet used different reporting formats (proportions instead of numbers) compared
to the 2013 National Performance data sheet while for 2013 only 14 of the 29 districts
in Papua submitted their reports to provincial level. Lack of appropriate program data
hampers MONEV and ultimately program planning. A combination of Integrated
Planning and Facilitative Supervision could be effective to improve quality of care,
MONEV as well as to guide program planning in all Papuan and Maluku provinces:
Action required at UNICEF/National/PHO/DHO level.
4.3 Measuring programs contributions towards USAID’s higher-level health indicators
The goal of the current Indonesian Global Health Initiative (USAID) country strategy 2010-
2014 for Indonesia is “Improved Health Impact through Collaboration” which will be
achieved with concentrated efforts in three interrelated focus areas:
1. Catalyze action to accelerate Indonesia’s progress towards achievement of Millennium
Development Goals (MDG) 4, 5 and 6. Achieving the MDGs is a very high priority for
the GoI. There is recognition that aggressive action will be required to meet MDG 5 in
particular.
2. Enhancing the use of quality research and evidence in policy and programming,
including introduction and adoption of new technologies and capacity building.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 48
3. Partnering with the GoI to address regional and global infectious disease threats and
strengthen Indonesia’s engagement and leadership in regional and global health issues
and fora.
According to the evaluation team, all UNICEF interventions are aiming at supporting
GoI/MoH priorities: reaching the MDGs 4,5 & 6, while their activities are in line with the
GHI principles: Encourage Country Ownership and Invest in Country Led Plans and
Increase Impact through Strategic Coordination and Integration. The UNICEF activities
support Focus area 1 and 2 and the following IRs:
Focus Area 1: Catalyze action to accelerate Indonesia’s progress toward achievement of MDGs
4,5&6:MDG 4&5 more specifically the following intermediate results (IR)
IR 1.1: “Improved quality and effectiveness of government and private health systems”,
IR 1.2: "District capacity, leadership and health governance improved in a decentralized,
district-led system",
IR 1.3: "High impact health interventions effectively implemented at scale in Indonesia".
The evaluation team has observed UNICEF's' actions to improved management of
complications, quality of clinical care, referrals, integrated Maternal/Child Health, Malaria in
pregnancy and in Malaria prevalence and maternal/neonatal mortality. Although attribution
of inputs, investments to outcome is rather difficult to establish in 3 of the 4 MCH/MIP
focus districts reporting (since 2 years), the number of maternal deaths and neonatal
deaths as a decreasing trend in maternal and/or neonatal mortality, is visible. The PMTCT
clinic in Sorong is contributing to the treatment of HIV infected mothers and to the
prevention of HIV among children (IR1.1.). All interviewed GoI officials confirmed
UNICEF's commitment to stimulate GoI ownership of every intervention (IR1.2.) while
especially the Malaria part of the program, despite stock-outs, showed good results and is
scaled up at Provincial level (IR1.3.).
Focus Area 2: Enhancing the Use of Quality Research and Evidence in Policy and Programming
more specifically:
IR 2.1: "Improved availability of good quality data for programming and policies to improve
public health",
IR 2.2: "New technologies and innovations introduced to impact public health outcomes".
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 49
Referring to the USAID new Country Development Cooperation Strategy/CDCS (2014-
2018), UNICEF interventions in Eastern Indonesia support all development objectives (DO
1 to DO 4) and to the following IRs:
1. IR 1.4: Sustainable Development in Targeted Districts in Eastern Indonesia
Enhanced. This is especially forSub-‐Intermediate Result 1.4.2: Basic services
enhanced.
2. IR 2.1: Services to reduce preventable deaths particularly among women and
children improved.This IR targets the reduction of preventable deaths of women
during labor and delivery and of newborns and children under five. In order to
achieve this result: (1) the quality of health services must be improved; (2) barriers
to access must be lowered.
3. IR 3.1: Control of Infectious Diseases of regional and Global Importance improved.
4. IR 4.2: Evidence-based decision making enhanced.
4.4 Lessons learned
Despite all the challenges faced in Eastern Indonesia, a lot of progress has been made with
strong commitment from the local provincial/district government. In most provinces, over
half of the Indicators reach 75% or more of their targets Indicators. Health professionals
across all levels established to have midwives adopted the Maternity Waiting Home
approach. Below are the lessons learned we derived from the evaluation.
1. East Indonesia combines a very poor referral system, an extreme geophysical setting
and high transportation costs with a low professional density: of the 1700 OB&GY’s,
14 work in Papua and only 9 in Maluku. Also neonatology is limited while
anaesthesiology underdeveloped in this part of Indonesia. With the new budget
allocation based on Village Law; villages in the remote areas are encouraged to secure
some funds to support transportation to refer emergency patients. Further formalizing
the referral system at all levels will ensure the roles of each level to stimulate their
ownership.
2. The service offered by Comprehensive Emergency Obstetric and Neonatal Care
(CEONC) centers: blood bank and 24 hours caesarean section is sub-optimal due to
lack of skilled staff. Neonatal Care is often very poor while most Basic Emergency
Obstetric and Neonatal Care (BEONC) centers are under-performing due to
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 50
recurrent stock-outs of essential drugs and supplies and under-utilized by expecting
mothers. This situation results in a low case-load of the midwives: a trained skilled
attendant needs to maintain her skills: minimum is 100-200 per year: 8-16 babies per
month. As a result a trained attendance is not necessarily a skilled attendant and is no
guarantee for a safe delivery / appropriate neonatal care.
3. Lack of quality monitoring/supervision/feedback mechanism results in poor data
collection: of the 2013 Papua figures, only 14 of the 29 districts submitted their data to
provincial level. None of the districts in Papua and Maluku had a budget for
professional monitoring and evaluation and most staff at health center level complained
about the lack of supervision. Only when monitoring and supervision are a priority
monitoring, evaluation, evidence-based planning and ultimately quality of service can
improve.
4. Central fund channelling, a good effort to increase accountability in a decentralized
setting, is hampered by the complexities at national level to translate this regulation
into administrative practice, while the collaboration through third parties at
provincial/district level such as professional organizations/universities/NGOs is
challenged by the lack of administrative and programmatic capacity. This situation can
only improve through a joined national and regional intervention.
5. Integrating the MIP guidelines and introduced to curriculum of medical/midwifery
schools is an excellent way to ensure the continuation transfer of learning/knowledge
related to the integration and production of high quality of health service providers.
On the job training for health care providers in BEONC has shown good results,
however the establishment of a BEONC monitoring system is also crucial to maintain
high standard quality of health care services while documenting best practices in
facilitating the setting-up and improvement of well-functioning BEONC facilities can
contribute, as lessons learned for other areas.
6. Investing in good data management, recording and reporting, will contribute to
updating the progress of the program in a timely manner, to avoid logistical issues by
reporting stock outs on-time. The SMS-based reporting that was developed in Sorong,
adapted from the DHIS, might be a clever way to overcome the geographical and
communication issues, and this approach should be simplified, scaled up and replicated
to other areas with a similar setting.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 51
7. The integrated MCH/MIP program resulted in "specialization": The CDC / Malaria
department is specialized to deal with the policy and the logistics for the integrated
program, while the MCH department proved to be effective in operations and
monitoring, especially in ensuring the quality of the services provided by health
providers.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 52
ANNEX I Scope of Work of the Evaluation
Major Tasks and Duties
The Evaluation team leader will work with the other members of the evaluation team to
carry out the SOW listed below. The team leader will be specifically responsible for
ensuring evaluation questions are answered, the report is complete, and deliverables are
met on time.
Background of the Program to be Evaluated
In support of its 2009-2014 Country Strategy, USAID/Indonesia is funding the 5-year
Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia project
(2010-2015). Under the 2014-2019 CDCS strategy, USAID/Indonesia will continue to focus
on improving basic services including health in Eastern Indonesia, and improving Maternal
and Child Health for vulnerable populations across Indonesia.
In October 2010, USAID/Indonesia committed to provide funds through field support to
UNICEF/Indonesia under UNICEF Umbrella Grant, Agreement GHA-G-00-07-00007, to
continue support for control of Malaria in Pregnancy (MIP) in five provinces in eastern
Indonesia. The intent of the program is to reduce malaria-related maternal and child
mortality in provinces facing the highest burden of malaria in Indonesia and home to 12
million people. At the same time in 2010, a new maternal and child health (MCH) project
was funded with UNICEF under the name ACHIEVE, to accelerate progress in reducing
maternal mortality in four of the provinces supported by the MIP program. In 2012 the
two separate projects (Malaria in Pregnancy and ACHIEVE) were combined into one
project entitled Maternal and Child Health and Integrated Malaria Control in Eastern
Indonesia. The integrated MCH and Integrated Malaria Control project retains all major
aspects of the two formerly separate projects and is measured by a similar set of indicators.
The integration was designed to enhance efficiency and alignment in the joint effort to
strengthen government systems for greater coverage and quality of MCH services in
Eastern Indonesia.
USAID/Indonesia expects UNICEF, through the Maternal and Child Health and Integrated
Malaria Control in Eastern Indonesia project, to conduct the following activities:
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 53
4. Sustain effective malaria control to pregnant women and young children in rural
Indonesia through synergistic integration of malaria control with antenatal care and
immunizations services. Support to improve routine immunization systems by
identifying key immunization system challenges and opportunities.
5. Strengthen replication of integrated malaria program coverage among districts through
advocacy and leveraging of local Government/GFATM funds and government schemes.
6. Support operational research in Indonesia related to the control of malaria in
pregnancy.
7. Enhance management and coordination for policy advocacy and sustainable and
effective program outcomes through sharing of experiences and best practices.
8. Provide technical assistance for documentation and advocacy for needful policy and
program formulation based on the evidence of current and upcoming initiatives.
9. Provide assistance to improve district based referral system to ensure accessibility of
quality emergency care for women and newborns.
10. Ensure that all women and newborns receive comprehensive and quality care during
pregnancy, delivery and postnatal periods.
Purpose and Objectives of the Evaluation
The purpose of this performance evaluation is to assess UNICEF’s progress towards their
stated objectives and indicators in this project, and to make specific recommendations to
improve the project’s performance. The specific objectives are:
1) To assess the performance of the Maternal and Child Health and Integrated Malaria
Control in Eastern Indonesia Project.
2) To make recommendations for the remaining years of the project based on the
identified strengths and weaknesses of the program.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 54
3) Make observations and recommendations to that may be used in current efforts or
future programming based on lessons learned.
Proposed Evaluation Questions
The core overarching evaluations questions are:
6) What progress has been made towards the project’s objectives of improving the
quality of Maternal and Child Health and Malaria in Pregnancy programs, and
replicating good quality programs broadly? How have UNICEF’s inputs and effort
contributed to the Government of Indonesia’s investments in Maternal and Child
Health in Eastern Indonesia, such as the Island Cluster Approach?
7) Is the project appropriately targeted and focused in the breadth of what it does,
and how it supports the Government of Indonesia’s efforts to improve Maternal
and Child Health in Eastern Indonesia?
8) How scalable is the impact of UNICEF’s interventions, and how replicable is their
approach?
9) Are the project indicators accurately and sufficiently capturing the full scope of the
project’s impact? Are the indicators sufficiently specified and appropriately designed
to measure the project’s impact? If not, how can they be improved?
10) How successfully have the management and implementation of the two formerly-
separate MIP and ACHIEVE projects been integrated into a single coherent effort,
with a unified set of indicators and performance measures? How effectively do the
two components achieve synergies and/or leverage each other for greatest impact?
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 55
Evaluation Methodology
The evaluation team is expected to develop a methodology which must include document
reviews and key informant interviews, and may also include focus groups with stakeholders
to assess program effectiveness and outcomes, and appropriate field visits. The team will
review documents provided by USAID/Indonesia, including the project SOW, mid-term
and annual reports, and other relevant documents. The team should plan to conduct
interviews and focus groups with a number of key informant stakeholders including
national and local level government staff working at both policy and technical levels, USAID
staff, UNICEF project staff, other donors and program staff working in Eastern Indonesia,
and project beneficiaries. Suggested partners to interview include, but are not limited to:
- UNICEF staff at both national and local levels
- USAID staff
- Global Fund CCM, Malaria TWG
- Indonesian Ministry of Health
- Dinas Kesehatan Provinsi, including relevant technical section staff
- Dinas Kesehatan Kabupaten, including relevant technical sections staff
- Puskesmas and health facility doctors, nurses, midwives
- Community members
- Relevant staff from BAPPENAS
- Relevant staff from BAPPEDA
- Ministry of Home Affairs
In order to answer the evaluation questions, the team should plan to conduct in-depth site
visits in two sites in Maluku, North Maluku, Papua or West Papua in order to: (a) meet
with key informants at the local level, (b) explore in detail the impact, opportunities, and
constraints of UNICEF’s program, and (c) embed the analysis within an accurate
understanding of the contexts and capacities that currently exist in Eastern Indonesia.
Specific dates and details of site visits will be finalized together with UNICEF.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 56
ANNEX II Time line and overview of deliverables No Time Line Deliverables Due Date
1 First week of
service
Approved work plan and evaluation team
design, including the following:
• Summary report of initial findings and
approach with draft schedule of field
activities (prior to field work)
• Draft and final questionnaire(s) to be used
during interviews/stakeholder meetings
(prior to field work)
• Draft Report Outline (prior to field work)
August 15,
2014
2 Second and
third week of
service
Site visits (Maluku, Papua, and when feasible
West Papua) with detailed agenda as follow:
• Day 1 à Discussion with UNICEF provincial
team
• Day 2 àDiscussion with District Health
Officer and a representative of Provincial
Office, and when possible visit and interview
relevant District Hospital staff and interview
professionals from BAPEDA and other
stakeholders.
• Day 3 à beneficiaries: community Health
Center’s staff and clients.
Daily team discussion to gather information
collected by each member of the team and
clarification to UNICEF team when needed.
3 Fourth week of
service
Draft Evaluation report and Oral debrief with
USAID, as follows:
• Detailed Evaluation Report Outline with
bulleted response to evaluation questions
September 5,
2014
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 57
and Draft PowerPoint Briefing (at the end of
the synthesis phase)
• Finalized PowerPoint De-briefing and initial
full report draft (before evaluation team
departs Indonesia)
4 Sixth week of
service
Completion and delivery of final evaluation
report based on Mission feedback:
• Final Evaluation Report following standard
reporting format and branding guidelines
(within 1 month upon completion of the field
works including Mission comments/
recommendations).
September
16, 2014
ANNEX III Itinerary and Data Collection Schedule and Informants. 08-08 to 09-16-2014
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 58
Date Time Where Agenda Informant(s) 13/8 2.00 PM
UNICEF Office Discussion with UNICEF National Office Team
Bill Hawley, Ferdinand Laihad, Karina Widowati, Robin Nandy, Peter Leth
14/8 6.30 AM MoH, Rasuna Said
Interview with MoH Ibu Erna
9.00 AM MoH, Percetakan Negara
Interview with Malaria Programme, MoH
Pak Asik Surya
18/8 8.15 AM Fly to Ambon GA 646 3.00 PM Check in Amaris Hotel, Ambon 4.00 PM PHO Maluku Interview with Head
of PHO Dr. MeikePontoh
5.00 PM PHO, Maluku FGD with PHO Staff Natje, Thomas, Ama, Daut, Hans
19/8 7.45 AM Fly to Saumlaki Wings Air IW 1514 10.00 AM
Bupati Office, Saumlaki MTB
Meet and greet with Bupati
Pak Bitzael S. Temmar
11.00 AM Trip to Alusi Kelaan
1.00 PM
Alusi Kelaan
FGD with Community Leaders
Head of sub-districts, Head of villages and 14 community leaders
FGD with Women 2 groups @6 women 3.00 PM Visit PONED facility 3.30 PM FGD with HC Staff (5
staff) Head of Health Center, Staff at MCH, Malaria, Lab Analyst
4.30 PM Back to Saumlaki 5.30 PM
RSUD PP Magretty, Saumlaki
Visit PONEK facility 6.00 PM FGD with RSUD Staff Director of Hospital, Dr.
Lex , Midwives, Clinic Manager and Anesthesia Nurse
20/8 6.00 AM Trip to Adaut 10.00 AM
Maternity Waiting Home, Adaut
FGD with Cadres 5 cadres FGD with Clients 6 clients
12.00 PM Village Hall, Adaut
FGD with PKK FGD with Community leaders
Head of sub-districts, Head of villages and community leaders
1.00 PM Adaut HC Visit PONED facility 2.00 PM Trip to Nyafar Nifmas 2.30 PM Nyafar Nifmas FGD with Community All villagers 4.30 PM Trip Back to Saumlaki 7.00 PM Harapan Indah FGD with DHO and Dr. Juliana (Head of
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 59
Hotel Bappeda Staff DHO), Bappeda Health Staf, DHO MCH and CDC Staff
21/8 8.00 AM Bupati Office
De-brief with Bupati of MTB
Bupati and Head of DHO
9.45 AM Fly to Ambon Wings Air IW 1515 1.00 PM UNICEF Office,
Ambon Discussion with UNICEF Staff
Olivi and Hellen
22/8 7.00 AM Fly to Jayapura
Lion JT 791 and JT894 via Makassar
4.30 PM UNICEF Office, Jayapura
Discussion with UNICEF Staff
Sudhir Kanal, Jana and Ratih
6.30 PM UNICEF Office, Jayapura
Discussion with SUM 2 Papua
Novia
23/8 8.00 AM Trip Depapre HC 10.00 AM
Depapre HC
Visit PONED facility
10.30 AM
Discussion with PKM Staff
Head of Health Center, Midwives, Nurses and Lab Analyst
11.30 AM
Visit Maternity Waiting Home
3 midwives in-charge of the house
12.30 PM Sentani HC
Discussion with PKM Staff
Head of PKM, Midwives and Malaria staff
1.30 PM Visit PONED facility 4.00 PM
UP2KP Office Discussion with Head of PHO Papua
Drg. Aloysius Giyai
6.30 PM Hotel
Discussion with KINERJA Papua
Achmad Tamrin
24/8 Hotel
Writing and compiling discussion
25/8 8.30 AM PHO Papua
Discussion with PHO Papua staff
Dr. Agnes, Pak Tarman and Pak Kadek
10.00 AM Poltekkes Papua
Discussion with Poltekkes Papua staff
Frans Manansang, BerlianaTampubolon, Rai, Robert, Wiwik
11.30 AM
RSUD Yowari Discussion with Director and Staff
Erni, Brian, Dr. Yerry and Dr. Frans
1.30 PM DHO Jayapura District
Discussion with DHO Jayapura District
Khairul Lie, Pak Jeffrey, Pungut, Kristanti, Eduard Sihotang, Umim, Delila
4.30 PM Hotel
Discussion with SUM 1 Papua
Gus Sutakertya
26/8 10.20 AM
Fly to Sorong GA 471
2.00 PM Hotel
Discussion with UNICEF West Papua
Sudhir Kanal, Nurlely Bethesda
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 60
Staff 6.00 PM
Hotel Discussion with PHO West Papua Staff
Dr. Victor Nugrahaputra
27/8 8.00 AM Mayamuk HC
Visit and discussion with PKM Staff
Pak Dantje (Head of Health Center), Ibu Anita, Lab Analyst
10.30 AM
Aimas HC Visit PONED facility and discussion with PKM Staff
Ibu Yatinah, Dr. Rani, Ibu Paula, Ibu Eka, Ibu Putri, Ibu Grace, Ibu Margaretha
1.00 PM DHO Sorong District
Discussion with DHO staff
Dr. Agustinus Luther, Pak Narwan, Pak Kadir, Pak Ning
6.00 PM Hotel
De-brief and clarification from PHO
Dr. Victor
28/8 8.00 AM Malawe HC
Visit demons site Ibu Sulce, Ibu Vince 10.00 AM
Interviews with PKM clients
4 Female clients
11.30 AM Hotel
Discussion with UNICEF West Papua Staff
Nurlely Bethesda
2.00 PM Fly to Jakarta
GA 699 and GA 655 via Makassar
29/8 8.00 AM Hotel
Writing and compiling discussion
1/9 10.00 AM
MoH, Rasuna Said
Discussion with MoH Pak Lukas Hermawan and Pak Risky
2/9 9.00 AM UNICEF ffice Report writing 3/9 2.00 PM USAID Office Debrief preparation
presentation
4/9 10.00 AM
Interview with COP EMAS
Mrs. Anne Hyre
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 61
ANNEX IV LIST OF INFORMANTS
No Names Institution Position National Level 1 Erna Mulati, MD, MSc
CMFM MoH Head of Child Health
Subdirectorate 2 Lukas Hermawan, MD,
MKes MoH Head of pregnant women
Subdirectorate 3 Riskiyana Sukandhi
Putra, MD, M.Kes MoH Head of delivery and post-
pregnancy Subdirectorate 4 Asik Surya, MD, MPPM MoH Deputy Director for National
Malaria Control Program 5 Edhie Rahmat USAID HSS Specialist 6 Mildred Pantouw, MPH USAID MCH Project Management
Specialist 7 Maria J. Pinto USAID MCH/Nutrition Specialist 8 Masee Bateman USAID Sr. Health Advisor 9 John Rogosch USAID Interim Director of Health
Office 10 Ratna Suwandono USAID USAID Liaison Officer 11 Dr. Robin Nandy,
MBBS, MPH UNICEF Chief, Child Survival &
Development 12 Ferdinand Laihad, MD UNICEF Malaria Health Specialist 13 Bill Hawley UNICEF Health Specialist 14 Karina Widowati, MD,
MPH UNICEF Maternal and Newborn Health
Specialist 15 Budhi Setiawan, MD,
MPH UNICEF Health Specialist
16 Peter Leth UNICEF Chief of Planning, Monitoring and Evaluation Cluster
17 Anne Hyre EMAS Country Director Provincial Level 18 Olivi Silalahi, MD UNICEF Maluku Malaria and EPI Officer 19 Hellen Parrera UNICEF Maluku Operations Assistant 20 Sudhir Kanal, MD UNICEF Papua Child Survival and Development
Specialist 21 Jana Fitria Kartika Sari,
MD, MKes.Trop UNICEF Papua Malaria and EPI Officer
22 Ratih Woelandaroe, MD, MSc
UNICEF Papua MCH Officer
23 Nurlely Bethesda Sinaga, MD, MKM
UNICEF West Papua
MCH Officer
24 Meike Pontoh, MD PHO Maluku Head of PHO 25 Hans Tanikwele PHO Maluku Immunization 26 Daud Samal PHO Maluku Malaria Program
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 62
27 Rachma Badiu PHO Maluku MCH Staf 28 Thomas Matulessy GF Maluku Program Officer 29 Anatje Noya GF Maluku PUMK Unicef 30 Drg. Aloysius Giyai PHO Papua Head of PHO Papua 31 Drg. Agnes Ang PHO Papua Key person for Health Partner
Forum of PHO Papua 32 Sutarman, SKM PHO Papua Basic Health Services Staff 33 Kadek Hermanta, SKM,
M.Si PHO Papua Program Staff also as
Operational Activities PIC 34 Victor Eka
Nugrahaputra, MD, MKes
PHO West Papua Head of Health Services Division
35 Novia Purnamasari SUM 2 Papua Regional Coordinator 36 Achmad Tamrin KINERJA Program Manager Papua 37 Ida Bagus Sutakertya SUM 1 Papua Team Leader Papua Province,
Senior Technical Officer: Prevention
38 Frans Manangsang, SKM, MKes
Poltekkes Papua Lecturer (Epidemiology, Malaria), Secretary of Midwifery major
39 Berliana Tampubolon, SKM, MKes
Poltekkes Papua Lecturer (Midwifery Nutrition Academy)
40 I Rai Ngardita, SKM, MKes
Poltekkes Papua Head of Nutrition Academy
41 Robert Poltekkes Papua Head of Nursing Academy 42 Wiwik Mulyaningsih,
SKM Poltekkes Papua Lecturer (Environmental Health
Academy) District Level 43 Drs. Bitzael. S. Temmar Saumlaki Bupati/Head of the District 44 Juliana CH Ratuanak,
MD DHO Saumlaki Head of DHO Saumlaki
45 Nel Kulalean DHO Saumlaki Head of Family Health Division 46 Candra Utukaman DHO Saumlaki Head of Disease and Disaster
Prevention 47 John Lekatompessy DHO Saumlaki PIC for Operational Activities 48 Nusye Batmomolin Bappeda Saumlaki Head of Health Division 49 Theo Resilowi, MD,
MARS RSUD PP Magretty, Saumlaki
Director
50 Lexy Ruitan, MD, MARS
RSUD PP Magretty, Saumlaki
Doctor Plus
51 Khairul Lie, SKM DHO Jayapura District
Head of DHO Jayapura district
52 Ir. Jeffrey Koloay Bappeda Kab. Jayapura
Head of Social and Cultural Division
53 Pungut Sunarto, SKM DHO Jayapura district
Head of Communicable Disease and Control Division
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 63
54 Kristanti, SKM DHO Jayapura district
Head of Prevention and Control Subdivision
55 Eduard Sihotang, Apt DHO Jayapura district
PIC for Operational Activities, Head of Program Planning
56 Usmiati, SKM DHO Jayapura district
Staff for General Affairs
57 Delila Meheu DHO Jayapura district
Head of MCH Subdivision
58 Nurhayati DHO Jayapura district
Malaria Officer of DHO Jayapura
59 Brian, MD RSUD Yowari, Jayapura
Pediatrician Assistant
60 Erni RSUD Yowari, Jayapura
Nurse, PIC Perinatology
61 Frans Sigala, MD, Sp.Rad
RSUD Yowari, Jayapura
Director
62 Yerry, MD RSUD Yowari, Jayapura
Director of Medical Services
63 Agustinus Luther, MD DHO Sorong (Temp) Head of DHO 64 Sunarwan Avrilia, MPH DHO Sorong Health Services Division Staff 65 Kadir DHO Sorong GF Representative for Malaria
program, 66 Ning Santjojo DHO Sorong Head of Family Health Division 67 Sulce Siwabessy DHO Sorong
Municipality Head of Family Health Division
Community Level 68 TH Lakafin, Amk Saumlaki Head of CHC Alusi Kelaan 69 Sarce Meren Saumlaki Imunization Staff CHC Alusi
Kelaan 70 M.C. Warmanuk Saumlaki Malaria Staff Alusi Kelaan 71 Hafni Nanavidin Saumlaki Education Staff Selaru 72 I. Zakarias Lessy Saumlaki Head of Subdistrict Selaru 73 Adam Sainyalit Saumlaki Secretary of Subdistrict Selaru 74 Ari Melalolim Saumlaki Religious Leader Selaru 75 Heri Lerobulan Saumlaki Community Leader Selaru 76 Bernard Amarduan Saumlaki Head of Sub-subdistrict Wera 77 David Hutabarat, S.Pd Saumlaki Syahbandar 78 Semy Rotngoran Saumlaki Community leader 79 Ika Bailayar Saumlaki Head of Sub-subdistrict Adaut 80 Ladewik Nuan Saumlaki Secretary of Sub-subdistrict
Adaut 81 Meky Baumase Saumlaki Head of Sub-subdistrict Lingat 82 Daniel, MD Jayapura Head of CHC Depapre 83 Vera Jayapura Lab Analyst CHC Depapre 84 Hilda Jayapura MCH and Family Planning Staff
CHC Depapre
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 64
85 Oktovina Jayapura Malaria and EPI Staff CHC Depapre
86 Lea Jayapura MCH and IMCI Staff CHC Depapre
87 Dian Gritnowati, MD Jayapura Head of CHC Sentani 88 Dantje Sorong district Head of CHC Mayamuk 89 Anita Sorong district Midwives Coordinator CHC
Mayamuk 90 Yatinah Sorong district Midwife CHC Aimas 91 Rani, MD Sorong district Doctor Plus, GP CHC Aimas 92 Paula Sorong district Nurse CHC Aimas 93 Eka Sorong district Midwife CHC Aimas 94 Putri Sorong district Midwife CHC Aimas 95 Grace Sorong district Midwife CHC Aimas 96 Margaretha Sorong district Midwife CHC Aimas 97 Vince Bakker, SKM,
MM Sorong Municipality Head of CHC Malawe
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 65
ANNEX V
Task division of the Evaluation Team
The three professionals worked as a team, interviews and discussion with UNICEF, USAID
staff and with stakeholders and authorities in the field were held as team discussions. For
efficiency reasons the team sometimes divided specific evaluation tasks when needed while
team members also kept their specific task division.
Lucas Pinxten, team leader is responsible for general evaluation methodology, professional
health system, MCH/IMC input, coordination, implementation, quality assurance, and
reporting and contract matters. Besides keeping track of the general evaluation process
and keeping the time frame he will specifically focus on the evaluation of the progress
against indicators at UNICEF office level, lessons learned and strategic choices of all
stakeholders and on the contributions of the Maternal and Child Health and Integrated
Malaria Control in Eastern Indonesia Program to the strengthening of the Indonesian health
system.
Siti Nurul Qomariyah, evaluation specialist, will guide the team in the approach of MoH and
stakeholders in the MCH/IMC field, she is also responsible for the specific evaluation
methodology: matching qualitative and quantitative methods to evaluation topics,
developing and executing qualitative and quantitative evaluation instruments to assess
UNICEF methods and progress against indicators at district level. Besides the quality
assuring of the methodology of this evaluation she will specifically focus on the district level
contributions of the Maternal and Child Health and Integrated Malaria Control in Eastern
Indonesia Program to the strengthening of the health system.
Irma Anintya Tasya, Public health project management and HIV (PMTCT) / MCH specialist
will be responsible for translation, administrative and logistical aspects. She will help the
team to execute a well-coordinated and well-implemented evaluation, pick up loose ends
and will specifically focus on provincial level contributions of the Maternal and Child Health
and Integrated Malaria Control in Eastern Indonesia Program to the strengthening of the
health system.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 66
ANNEX VI
Overview structural Interview
Questions Methods Informants
1 What progress has been made towards the project’s objectives and against indicators of improving the quality of Maternal and Child Health and Malaria in Pregnancy programs, and document learning agenda for effective replication of good quality programs
-‐ Presentation by UNICEF provincial team
-‐ Discussion of the progress reported against the list of MCH/IMC indicators
-‐ FGDs -‐ Observations to
PKM and below
-‐ UNICEF Provincial team
-‐ UNICEF JAKARTA
-‐ Beneficiaries in supported districts How do you document and attribute this?
2 How have UNICEF’s inputs and efforts contributed to the Government of Indonesia’s investments in Maternal and Child Health in Eastern Indonesia, such as the Island Cluster Approach?
-‐ UNICEF team presentation/case study to illustrate success by outcome indicators
-‐ FGDs -‐ In-depth interview
-‐ UNICEF Provincial team
-‐ Counterparts at district and provincial
-‐ UNICEF Jakarta
a. National: how many research projects do you support? Is there a research agenda, who sets the research priorities and who coordinates this agenda. How do these results translate into implementation?
b. How do you document results/best practises?
c. How do you share your research results?
d. How do you share good practises/lessons learned and on what level?
e. How does this positive deviance approach influence the decision makers at district provincial level?
f. Who initiated the Island Cluster approach and what is the UNICEF support?
g. How do they learn the positive deviance?
h. What are the moments for using structural approach? Routine meetings? Consensus and strategic meeting? How often do you have meetings? What are the results of the meetings?
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 67
i. Can you give an example of some of the decisions based on this kind of meetings?
3
Is the project appropriately targeted and focused in the breadth of what it does?
-‐ FGDs -‐ In-depth interview
-‐ UNICEF Provincial team
-‐ Counterparts at district and provincial
-‐ UNICEF Jakarta
-‐ MoH (Malaria, MCH)
Are the performance indicators coherent with the intended activities? a. Logical/results framework? b. Assumptions/requirements? c. Per each objective, what are the
indicators? d. How far are you with the deliverables? e. How do you deal with unexpected set-
backs? Are the project activities consistent with the National and Local MCH/IMC control priorities? How do you sustain Malaria control to pregnant women and young children? Are the project indicators accurately and sufficiently capturing the full scope of the project’s impact? Are the indicators sufficiently specified and appropriately designed to measure the project’s impact? If not, how can they be improved?
How does the MCH/IMC project supports the Government of Indonesia’s efforts to improve Maternal and Child Health in Eastern Indonesia? How does the project realize the integration of Malaria control with ANC and immunizations services? How does the project support/improve routine immunization system?
4 How is the coordination and harmonization of the implementation with other USAID and non-USAID funded activities? Other networks?
-‐ FGDs -‐ In-depth interview
-‐ UNICEF Provincial team
-‐ Counterparts at district and provincial level
-‐ UNICEF Jakarta
-‐ MoH (Malaria, MCH)
How is the training used for the official resources/professional association? How do you improve the district based referral system? How do you involve public and private health providers?
5 How scalable is the impact of UNICEF’s -‐ FGDs -‐ UNICEF
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 68
interventions, and how replicable is their approach?
Would counterparts take this approach on board using their own funds?
-‐ In-depth interview Provincial team
-‐ Counterparts at district and provincial level
-‐ UNICEF Jakarta
-‐ MoH (Malaria, MCH)
6 How successfully have the management and implementation of the two formerly-separate MIP and ACHIEVE projects been integrated into a single coherent effort, with a unified set of indicators and performance measures?
-‐ FGDs -‐ In-depth interview
-‐ UNICEF Provincial team
-‐ Counterparts at district and provincial level
-‐ UNICEF Jakarta
-‐ MoH (Malaria, MCH)
How effectively do the two components achieve synergies and/or leverage each other for greatest impact?
7 How does the project adapt to changes of GoI policies in implementing external assistance (e.g. fund channelling mechanisms, etc)?
-‐ FGDs -‐ In-depth interview
-‐ UNICEF Provincial team
-‐ Counterparts at district and provincial level
-‐ UNICEF Jakarta
-‐ MoH (Malaria, MCH)
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 69
ANNEX VII Interview guidelines, MoH, UNICEF Country Office, DHO/PHO, UNICEF Provincial Offices and clients.
Interview guideline MoH
Questions 1 How have UNICEF’s inputs and efforts contributed to the Government of
Indonesia’s investments in Maternal and Child Health in Eastern Indonesia, such as the Island Cluster Approach? a. How do they document results/best practises? b. How do they share their research results? c. How do they share good practises/lessons learned and on what level? d. How does positive deviance approach influence the decision makers at e. National and/or provincial and/or district level? f. Who initiated the Island Cluster approach and what is the UNICEF support?
2
Is the project appropriately targeted and focused in the breadth of what it does? Are the project activities consistent with the National and Local MCH/Malaria control priorities? What are the efforts to sustain Malaria control to pregnant women and young children? Are the indicators sufficiently specified and appropriately designed to measure the project’s impact? If not, how can they be improved? How does the MCH/IMC project support the Government of Indonesia’s efforts to improve Maternal and Child Health in Eastern Indonesia? How does the project realize the integration of Malaria control with ANC and immunizations services? How does the project support/improve routine immunization system?
3 How is the coordination and harmonization of the implementation with other USAID and non-USAID funded activities? Other networks? How do they improve the district based referral system? How do they involve public and private health providers? How often do you have meetings? What are the results of the meetings? Can you give an example of some of decision based on meetings?
4 How scalable is the impact of UNICEF’s interventions, and how replicable is their approach? Would you take this approach on board using your own funds?
5 How successfully have the management and implementation of the two formerly-separate MIP and ACHIEVE projects been integrated into a single coherent effort, with a unified set of indicators and performance measures? How effectively do the two components achieve synergies and/or leverage each other for greatest impact?
6 How does the project adapt to changes of GoI policies in implementing external assistance (e.g. fund channelling mechanism etc)?
Interview guideline UNICEF Country Office
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 70
Questions 1 What progress has been made towards the project’s objectives of improving the
quality of Maternal and Child Health and Malaria in Pregnancy programs, and document learning agenda for effective replication of good quality programs? How do you document and attribute this?
2 How have UNICEF’s inputs and efforts contributed to the Government of Indonesia’s investments in Maternal and Child Health in Eastern Indonesia, such as the Island Cluster Approach? g. National: how many research projects do you support? Is there a research agenda, who sets the research priorities and who coordinates this agenda? h. How do these results translate into implementation? i. How do you document results/best practises? j. How do you share your research results? k. How do you share good practises/lessons learned and on what level? l. How does this positive deviance approach influences the decision makers at
District and/or provincial level? m. Who initiated the Island Cluster approach and what is UNICEF’s support? n. How do they learn the positive deviance? o. What are the moments for using structural approach? Routine meetings? Consensus and strategic meetings? How often do you have meetings? What are the results of the meetings? p. Can you give an example of some of decision making based on this kind of
meetings?
Is the project appropriately targeted and focused in the breadth of what it does? Are the performance indicators coherent with the intended activities?
1. Logical/results framework? 2. Assumptions/requirements? 3. Per each objective, what are the indicators? 4. How far are you with the deliverables? 5. How do you deal with unexpected set-backs?
Are the project activities consistent with the National and Local MCH/Malaria control priorities? How do you sustain Malaria control to pregnant women and young children? Are the project indicators accurately and sufficiently capturing the full scope of the project’s impact? Are the indicators sufficiently specified and appropriately designed to measure the project’s impact? If not, how can they be improved? How does the MCH/IMC project support the Government of Indonesia’s efforts to improve Maternal and Child Health in Eastern Indonesia? How does the project realize the integration of Malaria control with ANC and immunizations services? How does the project support/improve the routine immunization system?
4 How is the coordination and harmonization of the implementation with other USAID and non-USAID funded activities? Other networks? How is the training used by the official resources / professional association? How do you improve the district based referral system?
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 71
How do you involve public and private health providers? 5 How scalable is the impact of UNICEF’s interventions, and how replicable is their
approach? Would counterparts take this approach on board using their own funds?
6 How successfully have the management and implementation of the two formerly-separate MIP and ACHIEVE projects been integrated into a single coherent effort, with a unified set of indicators and performance measures? How effectively do the two components achieve synergies and/or leverage each other for greatest impact?
7 How does the project adapt to changes of GoI policies in implementing external assistance (e.g. fund channelling mechanisms etc)?
Interview guideline for PHO and DHO
Questions
1 How have UNICEF’s inputs and efforts contributed to the Government of Indonesia’s investments in Maternal and Child Health in Eastern Indonesia, such as the Island Cluster Approach? 1. How do they document results/best practises? 2. How do they share their research results? 3. How do they share good practises/lessons learned and on what level? 4. How does positive deviance approach influence the decision makers at district/
provincial level? 5. Who initiated the Island Cluster approach and what is the UNICEF support?
2 Is the project appropriately targeted and focused in the breadth of what it does? Are the project activities consistent with the Local MCH/Malaria control priorities? What are the efforts to sustain Malaria control to pregnant women and young children? Are the indicators sufficiently specified and appropriately designed to measure the project’s impact? If not, how can they be improved? How does the MCH/IMC project support the Government of Indonesia’s efforts to improve Maternal and Child Health in Eastern Indonesia? How does the project realise the integration of Malaria control with ANC and immunizations services? How does the project support/improve the routine immunization system?
3 How is the coordination and harmonization of the implementation with other USAID and non-USAID funded activities? Other networks? How do they improve the district based referral system? How do they involve public and private health providers? How often do you have meetings? What are the results of the meetings? Can you give an example of some of decisions based on meetings?
4 How scalable is the impact of UNICEF’s interventions, and how replicable is their approach? Would you take this approach on board using your own funds?
5 How effectively do the two formerly -separate MIP and ACHIEVE projects achieve synergies and/or leverage each other for greatest impact?
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 72
6 How does the project adapt to changes of GoI policies in implementing external assistance (e.g. fund channelling mechanism etc)?
Interview guideline UNICEF Provincial Office
Questions
1 What progress has been made towards the project’s objectives of improving the quality of Maternal and Child Health and Malaria in Pregnancy programs, and document learning agenda for effective replication of good quality programs? How do you document and attribute this?
2 How have UNICEF’s inputs and efforts contributed to the Government of Indonesia’s investments in Maternal and Child Health in Eastern Indonesia, such as the Island Cluster Approach? 1. How do you document results/best practises? 2. How do you share your research results? 3. How do you share good practises/lessons learned and on what level? 4. How does this positive deviance approach influence the decision makers at
district provincial level? 5. Who initiated the Island Cluster approach and what is the UNICEF support? 6. How do they learn the positive deviance? 7. At what moments is the structural approach used? Routine meetings? Consensus
and strategic meetings? How often do you have meetings? What are the results of the meetings?
8. Can you give an example of some decision(s) based on this kind of meetings? 3
Is the project appropriately targeted and focused in the breadth of what it does? Are the performance indicators coherent with the intended activities? 1. Logical/results framework? 2. Assumptions/requirements? 3. Per each objective, what are the indicators? 4. How far are you with the deliverables? 5. How do you deal with unexpected set-backs? Are the project activities consistent with the National and Local MCH/Malaria control priorities? How do you sustain Malaria control to pregnant women and young children? Are the project indicators accurately and sufficiently capturing the full scope of the project’s impact? Are the indicators sufficiently specified and appropriately designed to measure the project’s impact? If not, how can they be improved? How does the MCH/IMC project supports the Government of Indonesia’s efforts to improve Maternal and Child Health in Eastern Indonesia? How does the project realise the integration of Malaria control with ANC and immunizations services? How does the project support/improve the routine immunization system?
4 How is the coordination and harmonization of the implementation with other
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 73
USAID and non-USAID funded activities? Other networks? How is the training used by the official resources / professional associations? How do you improve the district based referral system? How do you involve public and private health providers?
5 How scalable is the impact of UNICEF’s interventions, and how replicable is their approach? Would counterparts take this approach on board using their own funds?
6 How successfully have the management and implementation of the two formerly-separate MIP and ACHIEVE projects been integrated into a single coherent effort, with a unified set of indicators and performance measures? How effectively do the two components achieve synergies and/or leverage each other for greatest impact?
7 How does the project adapt to changes of GoI policies in implementing external assistance (e.g. fund channelling mechanism etc)?
Interview Guideline Clients
Questions
1 What are your experiences in using the health services in this CHC? 2 Do you find any constraints/benefits/barriers in using the health services in this
CHC? 3 What do you get when you have ANC in this CHC/midwives? Probing:
Malaria? (if they do not mention it) Immunization? Family planning? 4 Have you/other women had any difficulties in referral process, if needed
Probing: How is the service quality at each level (village midwives, district hospital)? Transportation? Cost? Administration?
5 Do you have any recommendation or input to improve the service quality? 6 Do you know about the MCH and IMP program?
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 74
Annex VIII. Summary of interviews with UNICEF and Ministry of Health staff from national to HC level
Questions National Provinces Districts Health Center/hospital 1. How have UNICEF’s inputs and effort contributed to the Government of Indonesia’s investments in Maternal and Child Health in Eastern Indonesia, such as the Island Cluster Approach?
a. How do they document results/best practices?
b. How do they share their research results?
c. How do they share good practices/lessons learned and on what level?
d. How does positive deviance approach influence the decision makers at district/ provincial level?
e. Who initiated the Island Cluster approach and what is the UNICEF support?
Erna Muliati, MD, MSc, CMFM (MoH): • UNICEF acts as catalisator – helping the MOH in implementing existing programs • UNICEF can see the picture more comprehensively • Strengthen the DHO in developing their plan • There has been discussion between the MOH and UNICEF to develop the work plan: what can be done by each party
UNICEF Jakarta: • Support to MoH to meet MDG's 4,5,6 and to reduce inequity and increasing service access and quality • Support through Nutrition, Wash and Health programs • Support to decentralization • Modeling and leverage • Try-out integrated programming • Working at district and CHC level offers a "field laboratory" • Advocacy sub national and national level • Advise on "double burden" challenges like obesity and non-communicable diseases. • Development and try-out of
MeikePontoh, MD (Head of PHO Maluku): • In 2008 UNICEF initiated the cluster islands approach, helping them in formulating what the approach is. • UNICEF also helping the province in formulating the maternity waiting home, facilitating training and increasing the capacity of the staff at DHO and HC in program planning. • Coordination between programs through ‘binawilayah’. One staff in PHO is in-charge for an island/a district (need to know everything about the health problem in specific island/district
UNICEF Maluku (Olivi Silalahi, MD Malaria/EPI Officer) • Health system support though consensus meeting at provincial level • Positive deviance approach through model CHC, • Capacity building through tailored TA and sponsoring workshops • Development and implementation Malaria curriculum for Poltekkes • Development of guidelines integrated management of MCH/ • Documentation and of results / lessons learned is through the standard UNICEF reporting
MTB (Juliana Ratuanak, MD - Head of DHO): • UNICEF supports the district in: • Data analysis for program planning • Formulating the integration of MCH/IMC program • Realizing the maternity waiting home • Training of health providers • Right now, there has been peraturan bupati (PERDA) on the cluster island approach to protect the health providers who work at the pusatgugus (at BEONC/PONED + facilities, to do cesarean section etc.) • Awarded for MDGs award 2013 for their cluster islands approach • Replication of program to North Tanimbar and Seira Island
Jayapura: • UNICEF facilitated us to train the HC on IMP, to develop SOP for the ANC, on the job training for microscopic analyst (funded by UNICEF) to ensure no clinical approach on Malaria
DHO Sorong:
Alusi HC, MTB UNICEF assists the district by providing funding for training for health providers, formulating guidelines and for data review. LexyRuitan, MD, MARS – RSUD PP Magretti MTB: • Training for health providers. However, some of the training was conducted in facilities which have ‘sophisticated’ medical instruments: when the health providers go back to the hospital; they couldn’t do anything as the hospital does not have the instruments. • Not enough staff at the hospital: only one nurse for anesthesia, the midwives are not skilllfull for neonates resuscitation. HC must be able to assess risks. • Faced by challenges, such as delays in diagnosis, family decision, high cost of transportation, communication barriers since not all areas have telephone signals.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 75
the Cluster Island Approach to strengthen MCH/IMC • PLA community Malaria approach is developed implemented and shared across Kabupaten. • Positive Deviance approach to develop model CHC to stimulate BEONC learning across CHC and from "sister hospital" • Development and implementation of innovative approaches like "waiting homes" • Operationalization of minimal service standards • Structural documentation of lessons learned is not happening
Lukas Hermawan, MD, Mkes (Head of Pregnant Women Division - MoH) • To support the GoI on the implementation of the program in eastern Indonesia, while on the same time GoI implement in Kalimantan & Sulawesi. • To tackle monitoring and fund channeling, maybe it’s better to have a Technical Working Group, since it is difficult if the fund is going through them, especially because of the reporting that will affect their own program
system (6-monthly) • Province and district have embraced positive deviance approaches like waiting homes and model HC • UNICEF supports development of BEONC HC but implementation is hampered by HR (intern doctor and fast turn over of the PPT doctor is the weakest link) • Island cluster approach is an old idea revitalized by UNICEF is well accepted and creates health services crossing Kabupaten borders • MCH needs more support it is still suboptimal • Malaria program runs well but is hampered by stock-outs (bed nets and treatment)
Victor Nugrahaputra, MD, Mkes (Head of Health Services Division - PHO West Papua) • UNICEF inputs: • Funding for some activities (due to limitation in the local budget). However there has been misunderstanding by the legislative à cost shifting • Technical assistances • Many training have been done but it seems that there is no result from the training – need to improve the capacity of the district staff on program monitoring • The performance of focus districts looks different but between focus districts also different, depend on
• UNICEF inputs: SMS gateway • UNICEF suggests us to do more supervision • Training for Aimas HC on BEONC • Workshop to develop referral manual IMP (in some focus HC) • Laboratory test for Malaria – training for all HCs • Constrains: fund channelling
Daniel MD, DepapreCHC : • UNICEF strengthens the system, assisting us in data analysis. • Monitoring was done by UNICEF (Dr. Ratih) • Dr.Ratih assisted in realizing the maternity waiting home, facilitated discussions etc. • UNICEF assists us in developing program planning through the IMP (MCH/IMC, slide standards, Malaria and laboratory etc.)
Sentani HC: • UNICEF assists us in developing SOPs, micro planning, training for PLA facilitators
Yowari Hospital (Nurse Erni, Brian MD – Pediatrician Assistant) • UNICEF assists the hospital by facilitating training for the hospital to be referral hospital on neonatal care. • The hospital belongs to the local government; we need UNICEF to help us advocating the government for the
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 76
• UNICEF has assisted based on the local needs, in the field.
Asik Surya, MD, MPPM (Deputy Director for National Malaria Program): • The program has benefit, either in strengthening services integration, community empowerment, social mobilization, advocacy and initiation of service model such as cluster islands approach
the commitment of the local government • In the future we need to plan in the beginning, what will be the role of each party; this is to improve the sense of belonging of the local government to the program. • Mentoring model fits very well to the situation in West Papua
UNICEF Papua ( Sudhir Kanal, MD, CSD Specialist) • UNICEF supports MoH provincial and district level with: integrated micro planning, guideline development and implementation, community IMC, flying health service, TA and Capacity building advocacy, and leverage of local and international funds • UNICEF bridges PHO-DHO
UNICEF Papua (Ratih Woelandaroe – MCH Officer): • UNICEF only supports the local government in implementing their programs. The organization structure at the PHO/DHO makes it difficult now to work with. There are conflicting priorities at the Dinkes. Also at national level, especially the fund channeling system. We work now through third party (professional organization, NGOs etc) – this is not easy. We need to do capacity building first for them. • Money is not a problem in Papua, but the problems are human
logistics problems. Although there has been Permenkes (MoH decree) that the hospital is a referral hospital for neonatal care, the local government does not know. • Stock out for Malaria drug, although the health providers have been trained • Need to evaluate the accuracy of the microscopic analysis in doing Malaria examination.
Dantje Head of CHC and Anita Midwives Coordinator (Mayamuk HC, West Papua): • UNICEF inputs: • Bed nets, training, IMP (using the approach, now we know that we have many problems in coverage etc: "we used it to have discussion with the local government"), SMS gateway: problem in the beginning but it has been fixed (simplified) • Have one maternity waiting home, managed by a midwife, started May 2014 • Problems in logistics: stock out oxytocin; no
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 77
resources, geographical situation and conflicting priorities
PHO Papua (Drg. AloysiusGiya – Head of PHO) • The increase in MMR and Child mortality are due to: 1) the increase in reporting, and 2) the policy makers do not know how to select priorities and the health providers work without enthusiasm • UNICEF helps in solving those problems. However we still need to discuss if the activities supported by UNICEF are all correctly targeted
PHO staff Papua (Agnes Ang, MD, Tarman, KadekHermanta, SKM, MSi): • UNICEF helps us developing module for MIP. Planned for module replication • Also facilitating training for hospital staff (by RSCM hospital) on neonatal care – for the hospital to be referral hospital for neonates. • Soon we will be conducting training for community-IMCI facilitators • Constraints: schedule, monitoring after training
MgSO4 Aimas HC Bidan Coordinator): • UNICEF inputs: • BEONC training in Makassar (10 days). We can manage hypertension, resuscitation, manual removal of placenta. We refer premature baby as we don’t have incubator. What we have is not the same with what we learned in Makassar. • Monitoring by Dr. Beth (UNICEF) • No ambulance at the HC • Stock out: oxytocin, MgSO4 • Problems: laboratory is not for 24 hours. Only one staff • We promoted the BEONC HC (the facility delivery) by disseminating information that delivery at HC is free but not for delivery at home. • We never perform vacuum delivery because we are afraid if there is emergency situation for the baby • No AMP.
2. Is the project appropriately targeted and focused in the breadth of what it does?
Erna Muliati, MD, MSc, CMFM (MoH): • Targets should differentiate between provinces
UNICEF Maluku (Olivi Silalahi, MD Malaria/EPI Officer) • Malaria program is well focused (vertical program) and reaches,
Jayapura (Dinkes) • Stock out is caused by the new e-catalogue system. It
Adaut HC Bidan Coordinator): • Estimated pregnant
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 78
• There should be more about indicators of progress (output) and sustainability (commitment) not impact indicators. • To sustain the program development, partners (including UNICEF) should have exit strategy (the step by step process) – this has not been considered.
UNICEF Jakarta: • Results framework and indicators as agreed upon with donor • Project is in-sync with MoH • Stimulates/advocates development and implementation of guidelines and to influence policy. • Setting professional standards through development of models in selected districts to share best practices
despite the stock outs, its targets (90% of the pregnant women receive screening and a bed net) • MCH (horizontal program) is far more complex and broad: some targets are met • Targets are set at the beginning of the program and we have to work with them • Some targets like introduction of Malaria curriculum in Poltekkes were easy to reach, increased assisted deliveries are difficult to reach • In general indicators cover what we do but some objectives and targets are not well covered by indicators, while we also have activities that have no indicators at all • For Malaria we need an output indicator: reduction of Annual Prevalence Index (API) per village
UNICEF Papua ( Sudhir Kanal, MD, CSD Specialist) • Integrated micro planning is sustainable and system oriented. A good start at HC level but also DHO and PHO level needs more support on MONEV • Flying health services developed to address inequity but financing has to come from APBD • MoH needs health system strengthening especially MONEV last year only 14 of the 29 district submitted reports • Support to curriculum
should not be a problem if HCs report appropriately
women per year: 91 only. (Clarified by the DHO – there is misunderstanding by the HC on their area of coverage. They should also cover the other three villages – because Adaut HC is the referral facility for the cluster island approach). With this coverage, the number of delivery is more than 300. • The HC was used for caesarean section, but the doctor who was trained has left for specialization
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 79
development. Poltekkes is well targeted but we get too many midwives (annual output Jayapura only about 6-700) • Model HC and micro planning are "soft Power" approaches to improve the health system
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 80
3. How is the coordination and harmonization of the implementation with other USAID and non-USAID funded activities? Other networks?
Erna Muliati, MD, MSc, CMFM (MoH): • Donor meeting at national level to discuss the focus sites • UNICEF Jakarta: • This is a challenge, seems to be necessary but does rarely happen • With other programs/donors even worse.
UNICEF Maluku (Olivi Silalahi, MD Malaria/EPI Officer) • Coordination and harmonization through regular donor meetings at national level • Never happens at provincial level UNICEF Papua ( Sudhir Kanal, MD, CSD Specialist) • We coordinate our work on a three-monthly base in the provincial health forum: KNCV, FJI, CHAI, • KINERJA Papua • Collaborates in one of the activities in PKM, where the UNICEF are working with health staff, Kinerja tries to gather information on quality of health services and gives the input to PKM and UNICEF
PHO Papua (Drg. Aloysius Giya) • We formed the health partnership forum on June 1, 2014
Victor Nugrahaputra, MD, Mkes (Head of Health Services Division - PHO West Papua) • Development partner forum has been formed, waiting for the governor signature. Need to increase the dialogue (time constrains)
a. How they improve district based referral system?
b. How often do you have meeting? What are the results of the meeting?
UNICEF Jakarta: • SMS referral system in West Papua • Desa Siaga is a good approach but very few people in East Indonesia have ever heard of
UNICEF Maluku (Olivi Silalahi, MD Malaria/EPI Officer) • Waiting homes is a great success • SMS referral system that is now tried out in West Papua would fit well in Maluku.
•
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 81
Can you give an example of some of decision based on meetings?
c. How do they involve public and private health providers?
it. • Private sector is under developed
• There are very few private health providers
UNICEF Papua ( Sudhir Kanal, MD, CSD Specialist) • Micro planning, SMS referral system, flying doctors system to support referral from remote areas, and support to establishment BEONC. • Monthly coordination meetings in health forum meeting and incidental meetings with professional organizations like IBI, IDAI and POGI PHO • We have often discussed HR shortages and quick rotation (5 PHO in 2 years in Jayapura!) • The proliferation of new districts (14 in 2012 and 40 in 2014) is hampering the health services: health staff are used for admin tasks • With adding new districts we have no denominator to measure progress against indicators we now have started with "real time" monitoring
4. How scalable is the impact of UNICEF’s interventions, and how replicable is their approach?
Would you take this approach on board using your own funds?
UNICEF Jakarta • Yes it is replicable, but it takes time and a lot of advocacy; Kabupaten and PHO are fast to take advise and approaches from UNICEF
Asik Surya, MD, MPPM (Deputy Director for National Malaria Program):
Meike Pontoh, MD (Head of PHO Maluku) • The cluster island approach has been taken by the province government (put in the RPJMD) and also by the central government. The government of Maluku now needs to launch regulation on the cluster island approach
MTB: • The approach has been replicated and self-funded to other areas, such as North Tanimbar, Seira Island.
Jayapura • For HIV test and RDT the funding mostly is from GF. It will be a challenge if there is
Depapre and Mayamuk: • Maternity waiting home: no funding yet, and it is the midwives who take care of the house
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 82
• The national government has allocated fund for bed net for all population in endemic areas (as much as 24 billion) – not only for MCH targets.. • Encourage local funds allocations for Malaria routine bed nets
• Before there were only 19 pusatgugus (the center for each cluster), but now in Maluku, there are 39 pusatgugus • To sustain the program, assistance is still needed especially for non-focus districts. UNICEF should not leave until the local government is ready to continue the program.
UNICEF Maluku (Olivi Silalahi, MD Malaria/EPI Officer • Waiting homes and Malaria program are already scaled up beyond focus districts with APBP money
UNICEF Papua ( Sudhir Kanal, MD, CSD Specialist) • Waiting homes, micro planning, model HC and Malaria screening and treatment in pregnancy are in the process to be scaled up through using local ABPD funds. • MCH is more complex: quality is still not good and midwives have a low caseload to maintain their skills. • PHO Papua (Dr. Aloysius) • There have been many Governor regulations and Provincial regulations related with health issues. They include regulations related to health insurance, and referral system. Proportion for health programs from APBD at districts mostly still below 10%. Only one which is 8% • PHO staff Papua
no support from the district/province government
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 83
• We still need guidance from UNICEF for the IMP approach, also to review if it can be replicated to other districts. This is also the case for the community-IMCI
Victor Nugrahaputra, MD, Mkes (Head of Health Services Division - PHO West Papua) • The commitment of the local governments for replication varies – also for funding of some activities (e.g the previous radio medical program) • MIP might be sustainable. In terms of training, but logistics might be a problem in the future.
5. How effectively do the two formerly -separate MIP and ACHIEVE projects achieve synergies and/or leverage each other for greatest impact?
UNICEF Jakarta • The lower in the system the easier it is. PD2 staffs are working very well together. That is not completely the case at Jakarta DG level. It works very well at MoH • The integration with Malaria makes the pregnant women class more popular • Vertical Malaria program works like a wedge in the horizontal MCH program • The challenge is to provide bed nets on time, the droppings are at irregular times. The quality of in-country bed nets are not as good as the GF has.
PHO Maluku • Buru, Tual, East Seram are now provide with their own bed nets and RDT • Guide book for midwives and cadres on MiP integration, distributed not only in focus district but in all districts
UNICEF Maluku (Olivi Silalahi, MD Malaria/EPI Officer) and UNICEF Papua (Sudhir Kanal, MD, CSD Specialist) • After development, implementation of the integrated guidelines and integration in Poltekkes curriculum. Midwives and nurses running the ANC have adopted the integrated Malaria approach: screening and treatment is now part of the ANC package
MTB: • The approach has been replicated and self-funded to other areas, such as North Tanimbar, Seira Island.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 84
• The integration is good, increased partnership among stakeholders and among related sectors.
6. How does the project adapt to changes of GoI policies in implementing external assistance (e.g. fund channelling mechanism etc)
UNICEF Jakarta • MOH: this has been a challenge for the collaboration work as the MOH does not have enough staff to look after the project administration • We adapt well now we are using village funds to scale up PLA
Lukas Hermawan, MD, Mkes (Head of Pregnant Women Division - MoH) • We have not yet figured out how to deal with the fund channelling mechanism, maybe establish a technical working group to monitor and to get approval for activities conducted in the field. But this needs to be negotiated and approved with the GoI
UNICEF Maluku (Olivi Silalahi, MD Malaria/EPI Officer • UNICEF is flexible in its approach and we are close to our partners. However the current funding channel is problematic
UNICEF Papua (Sudhir Kanal, MD, CSD Specialist and Ratih Woelandaroe, MD – MCH Specialist) • We stay very close to MoH and together with MOH we see that Maternal mortality is not improving fast enough (only 24% of the indigenous women in the lowest health quintile has assisted birth attendance) while U-5 mortality among indigenous families is 4 times higher compared to non indigenousness ones (110 vs.24 per 1000). • Now we are pushing for more attention for U-5: Integrated community child care does not exists, and virtual no U-5 receives Malaria treatment or sleeps under a bed net • U-five mortality rate doubled since 2012 in both West Papua and Papua • Budget sharing between UNICEF and PHO, discuss which program can be supported, things needed from the health office and from program itself.
DHO Sorong district: • The SMS-gateway for reporting has sped up the reporting process. Assessment is needed to see implementation to other program.
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 85
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 86
ANNEX IX. Summary of interviews with people at community level Questions Women Kaders - TBA Community leaders 1. What are your experiences in
using the health services in this CHC
Five women (pregnant, post partum) and five women experiencing Malaria in Alusi village, MTB: Sorong: referred by the midwives to go to Malawe HC. 2nd child. Was tested for 10T with counseling
- The services are good, despite there are some limitations in medication
All good, the services are all good In Alusi, they have started communal funds to build latrines in villages, initiated and partly funded by DHO
2. Do you find any constraints/benefits/barriers in using the health services in this CHC
Sorong: - Waiting time for lab is long - Discussed with midwives where they
want to deliver the baby, since the PKM doesn’t have overnight beds
Adaut + Increased knowledge on pregnancy
and baby care + able to help other villagers to maintain
their health - Low incentives for cadres
(10.000/month) while they need to reach all pregnant women and mothers in the whole village
- No permanent place for Posyandu, usually at villager’s front yard
Alusi: + FP has lifted their burden as head of the family by limiting the number of births - Limited medication - Hard to find clean water Adaut: + there is clean water in households (80%) + not difficult to refer
3. What do you get when you have ANC in this CHC/ midwives? Probing: • Malaria • immunization • family planning
Adaut - Pregnant/post partum: blood test, bed
net, MCH book (but many of them did not get it), free of charge but some only knew that it is free when they were already at HC, No FP method yet.
- Women with Malaria: Blood test, medication then blood test again.
Pregnant women (Malawe HC, Kota Sorong) Blood test etc (2 women does not know what for? Two women said – ‘I don’t know,’ two women replied to check if I have Malaria, HIV etc.)
- Limited bed nets, since it was only given for pregnant women or children, not for the whole population.
4. Have you/other women had any difficulties in referral process, if needed. Probing:
- how are the service quality in each level (village midwives,
Sorong: No, the referral is good, there are no difficulties
Reasons of some women who don’t want to deliver at health facilities: Their husband does not agree Cost constraint Transportation challenges (High-
Women group in Adaut: Problem - transportation, logistics at the HC - Transportation to referral center is
difficult, especially when the tide is
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 87
district hospital)? Transportation? Cost?
tide/waves during rainy season) high.
5. Do you have any recommendation or input to improve the service quality?
• Providing milk for all pregnant women • All health providers should be nice • Sometimes there was no bed net for the pregnant women, this should be solved • Facility for referral • Electricity • Drugs and instruments for the midwives
Puslinglaut’(PuskemasKelilingLaut/sea mobile CHC ) to tackle transportation issue.
6. Do you know about the MCH and IMP program?
yes yes yes
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 88
Annex X Overview Progress on Program Indicators: Baseline 2010-Results 2013
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 89
Annex XI Recommendations and Priority Matrix
Priority
Recommendations
Action by MOH UNICEF USAID
Nat PHO DHO For the remaining years
1
UNICEF advocates for a buffer stock at DHO-PHO level for Oxytocin, Magnesium sulphate, RDT and ACT and ILLNs, and increases its efforts to strengthening the referral system and the quality of BEONC sites
√
√
√
2
To use the synergy obtained by the integrated program to inform expecting mothers of the benefits of the JKN health insurance scheme, and to promote skilled delivery at an official BEONC site
√
3
To put strong output indicators for leverage, such as ensuring sustainable funding from the local government to support the program
√
√
4
To improve the current routine data collection system and development of specific indicators to measure the results of the integrated of MCH/MIP program
√
√
√
5
To reduce the intake of student at the midwifery school and the intake of new midwives through a "numerous clausus" system, to re-certify midwives having a caseload lower than 100 deliveries per year and to promote "4 hands delivery"
√
(with IBI)
6
UNICEF to strengthen its evidence-based health programming and documentation by increasing operations research (plus budget), systematic documentation and sharing
√
USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia 90
of qualitative and quantitative results on the contextual conditions and the effect of intervention programs.
7
UNICEF to support the GoI in selecting appropriate quality of care indicators and finding the mechanism to measure them
√
√
√
√
8
refresher and new training of health professionals to counter staff rotations and to improved facilitative supervision by MoH staff to maintain these skills and ensure quality of services
√
9
To add outcome indicators, based on agreed national indicators for both malaria and MCH
√
√
√
10
To add output indicator for integrated MCH/MIP i.e.: proportion of villages reducing their endemicity level’ and number/percent of pregnant women with presumed/confirmed and treated malaria
√
√
√
11
Follow up the progress of MWH try out in Banten Province
√
12 To align Kinerja and UNICEF specific activities
√
For future programming 1
UNICEF to have a stronger focus on strengthening the basic integrated maternal/neonatal health services including safe deliver practices and neonatal resuscitation at Health Center / BEONC level instead of trying to cover the whole spectrum of Maternal and Neonatal health
√
2
To strengthen the monitoring and evaluation with a combination of integrated planning and facilitative supervision to improve quality of care and to guide program planning
√
√
√
√
Annex XII
54
54Source: http://www.who.int/maternal_child_adolescent/documents/measuring-‐care-‐quality/en/
92 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
93 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
94 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
ANNEX XIII List of documents and references (in alphabetical order)
1. Achadi E, Scott S, Pambudi ES, Makowiecka K, Marshall T, Adisasmita A, et al. Midwifery provision and uptake of maternity care in Indonesia. Trop Med Int Health. 2007; 12(12):1490-7 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=18076557
2. Australia Support to Primary Health Care Strengthening and Maternal Newborn Health (PERMATA) Program Design. 2014.
3. Badan Perencanaan Pembangunan Nasional, Badan Pusat Statistik, United Nations Population Fund. Proyeksi Penduduk Indonesia (Indonesia Population Projection) 2010 - 2035. Jakarta: Badan Pusat Statistik; 2013.
4. Brabin BJ, Hakimi M, Pelletier D. An analysis of anemia and pregnancy-related maternal mortality. J Nutr. 2001; 131(2S-2):604S-14S; discussion 14S-15S Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=11160593
5. Brabin BJ, Premji Z, Verhoeff F. An analysis of anemia and child mortality. J Nutr. 2001; 131(2S-2): 636S-45S; discussion 46S-48S Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=11160595
6. Brownson RC, Fielding JE, Maylahn CM. Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009; 30:175-201 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=19296775
7. DFAT Concept Note Accessed from: http://aid.dfat.gov.au/business/Documents/mnch-conceptnote-indonesia.pdf
8. Elyazar IR, Gething PW, Patil AP, Rogayah H, Kusriastuti R, Wismarini DM, et al. Plasmodium falciparum malaria endemicity in Indonesia in 2010. PLoS One. 2011; 6(6):e21315 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=21738634
9. Evi Martha and team. A Review on Maternity Waiting Homes Effectiveness in 5 Provinces in Indonesia. Depok 2013.
10. Freedman LP, Graham WJ, Brazier E, Smith JM, Ensor T, Fauveau V, et al. Practical
95 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
lessons from global safe motherhood initiatives: time for a new focus on implementation. Lancet. 2007 Oct 13; 370(9595):1383-91 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=17933654
11. Framework for Program Evaluation in Public Health. CDC; 1999 [cited 2014 August 2]; Accessed from: http://www.cdc.gov/EVAL/framework/.
12. From Reformasi to Institutional Transformation - A Strategic Assesment of Indonesia's Prospects for Growth, Equity and Democratic Governance: Harvard Kennedy School - ASH Center for Democratic Governance and Innovation; 2011.
13. GoI MoH. Pertemuan Kajian Efektivitas Rumah Tunggu Kelahiran: Accessed from: http://www.gizikia.depkes.go.id/pertemuan-kajian-efektivitas-rumah-tunggu-kelahiran/?print=pdf.
14. Hammerich A, Campbell OM, Chandramohan D. Unstable malaria transmission and maternal mortality--experiences from Rwanda. Trop Med Int Health. 2002 Jul. ;7(7):573-6 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=12100439
15. Hammerich A, Campbell OM, Chandramohan D. Unstable malaria transmission and maternal mortality--experiences from Rwanda. Trop Med Int Health. 2002; 7(7):573-6 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=12100439
16. Heywood P, Nida P Harahap, Mimin Ratminah, Elmiati. Current Situation of Midwives in Indonesia: Evidence from 3 districts in West Java Province. BMC Research Notes. 2010; 3:287 Accessed from: http://www.biomedcentral.com/1756-0500/3/287
17. Hodge A, Firth S, Marthias T, Jimenez-Soto E. Location matters: trends in inequalities in child mortality in Indonesia. Evidence from repeated cross-sectional surveys. PLoS One. 2014. ;9(7):e103597 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=25061950
18. Indonesia - Demographic and Health Survey 20122013; Statistic Indonesia, National Population and Family Planning Board, Ministry of Health, Measure DHS, ICF International: Accessed from: http://dhsprogram.com/publications/publication-FR275-DHS-Final-Reports.cfm.
19. Indonesia GHI Country Strategy - Improved Health Impact through Collaboration 2011.
96 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
20. Indonesia - Neonatal and Child Health Profile2013: Accessed from: http://www.who.int/maternal_child_adolescent/epidemiology/profiles/neonatal_child/idn.pdf?ua=1.
21. Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia revised 3-08-2013. USAID project document regarding incremental funding to immunization in collaboration with CDC. Jakarta2013.
22. Innovative Approaches to Maternal and Newborn Health Compendium of Case Studies, 2013: Accessed from: http://www.unicef.org/health/files/Innovative_Approaches_MNH_CaseStudies-2013.pdf.
23. Joint Committee on Reducing Maternal and Neonatal Mortality in Indonesia. Reducing Maternal and Neonatal Mortality in Indonesia : Saving Lives, Saving the Future. 2013.
24. Kementerian Kesehatan RI. Buku Saku Pelayanan Kesehatan Neonatal Esensial - Pedoman Teknis Pelayanan Kesehatan Dasar. 2010.
25. Kementerian Kesehatan RI. Assesment GAVI - HSS 2010 - 2011 Direktorat Jenderal Bina Gizi dan KIA: Laporan Akhir Provinsi Papua: Kementerian Kesehatan RI, GAVI Alliance, Universitas Cenderawasih2011.
26. Lazanoet. Progress towards Millennium Development Goals 4 and 5 on maternal and child mortality: an updated systematic analysis. The Lancet. 2010; 378:113-6. Accessed from:
27. Makowiecka K, Achadi E, Izati Y, Ronsmans C. Midwifery provision in two districts in Indonesia: how well are rural areas served? Health Policy Plan. 2008 Jan. ;23(1):67-75 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=17971368
28. Manson's Tropical Diseases. 23 ed. Jeremy Farrar et all, editor. London: Elsevier Saunders; 2014.
29. Maternal mortality in 2005: estimates developed by WHO, UNICEF, UNFPA, and The World Bank 2007.
30. Miranti R, Vidyattama Y, Hansnata E, Cassells R, Duncan A. Trends in Poverty and Inequality in Decentralising Indonesia. OECD Social, Employment and Migration Working Papers. 2013. ;148 Accessed from: http://dx.doi.org/10.1787/5k43bvt2dwjk-en
31. North Maluku, Maluku, West Nusa Tenggara, East Nusta Tenggara, West Papua and Papua - Poverty - Percentage of Poor People, Rural [database on the Internet]. knoema.com. [cited September 8, 2014]. Accessed from:
97 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
http://knoema.com/atlas/embed/Indonesia/North-Maluku/topics/Living-Conditions/Poverty/percent-of-Poor-People-Rural?compareTo=ID-MA,ID-PB.
32. OUTREACH: Development of Indonesia's Outlying Area -- Anti-Malaria Squads. TEMPO. 2012 19 February:1-7.
33. Papua at a glance: Province - Demography and Health Status 2012.
34. Personal comment during discussion with UNICEF country office staff, August 8 2014. Jakarta.
35. Republic of Indonesia, Village Law Number 6/2014, accessed from http://menpan.go.id/jdih/perundang-undangan/undang-undang.
36. Ronsmans C, Scott S, Qomariyah SN, Achadi E, Braunholtz D, Marshall T, et al. Professional assistance during birth and maternal mortality in two Indonesian districts. Bull World Health Organ. 2009;87(6):416-23 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=19565119
37. Rukhsana Ahmed, Din Syafruddin, Jeanne Poepoprodjo, Puji Budi Asih, Endang Sumiwi, Faragher B, et al. Intermittent Screening and Treatment or Intermittent Preventive Therapy for Control of Malaria in Pregnancy in Indonesia: 'STOPMIP' Study. DFID/MRC/Wellcome Trust Joint Global Health Trial Scheme. 2011.
38. Schantz-Dunn J, Nour NM. Malaria and pregnancy: a global health perspective. Rev Obstet Gynecol. 2009; 2(3):186-92 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=19826576
39. Soto EJ, Firth S, La Vincente S, Morgan A, Bauze A, Dayal P. Developing an Investment Case for Financing Equitable Progress towards MDGs 4 and 5 in the Asia Pacific Region. Phase 1: Strategic Report 2009.
40. Steketee RW, Nahlen BL, Parise ME, Menendez C. The burden of malaria in pregnancy in malaria-endemic areas. Am J Trop Med Hyg. 2001; 64(1-2 Suppl):28-35 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=11425175
41. Supargiyono S, Bretscher MT, Wijayanti MA, Sutanto I, Nugraheni D, Rozqie R, et al. Seasonal changes in the antibody responses against Plasmodium falciparum merozoite surface antigens in areas of differing malaria endemicity in Indonesia. Malar J. 2013;12:444 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citatio
98 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
n&list_uids=24321092
42. Titaley CR, Dibley MJ, Agho K, Roberts CL, Hall J. Determinants of neonatal mortality in Indonesia. BMC Public Health. 2008; 8:232 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=18613953
43. Titaley CR, Hunter CL, Dibley MJ, Heywood P. Why do some women still prefer traditional birth attendants and home delivery?: a qualitative study on delivery care services in West Java Province, Indonesia. BMC Pregnancy Childbirth. 2010; 10:43 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=20701762
44. Titaley CR, Hunter CL, Heywood P, Dibley MJ. Why don't some women attend antenatal and postnatal care services?: a qualitative study of community members' perspectives in Garut, Sukabumi and Ciamis districts of West Java Province, Indonesia. BMC Pregnancy Childbirth. 2010; 10:61 Accessed from: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&dopt=Citation&list_uids=20937146
45. UNICEF. Annual Report 2012 for Indonesia, EAPRO: Accessed from: http://www.unicef.org/about/annualreport/files/Indonesia_COAR_2012.pdf
46. UNICEF. Sharpening the equity focus: Selected Innovations and lessons learned from UNICEF-assisted programmes 2009-20102011.
47. UNICEF Indonesia. Integration of Malaria Control with Maternal and Child Health in Eastern Indonesia – Second Progress Report (1 June - 30 November 2011) 30 November 2011.
48. UNICEF Indonesia. ACHIEVE - Reducing Maternal and Neonatal Mortality in Indonesia: The Pathway to Achieving MDG 4&5 - Second Progress Report (1 May - 31 October 2011) 30 November 2011.
49. UNICEF Indonesia. Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP and ACHIEVE) - Third Progress Report (November 2011 - April 2012) May 2012.
50. UNICEF Indonesia. Integration of Malaria Control with Maternal and Child Health in Eastern Indonesia - First Progress Report (October 2010 - April 2011) 31 May 2011.
51. UNICEF Indonesia. Integration of Malaria Control with Maternal and Child Health in Eastern Indonesia - End of year report (1 January 2010 to 31 December 2010) 17 December 2010.
99 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
52. UNICEF Indonesia. Indonesia - Annual Report 2013.
53. UNICEF Indonesia. Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP and ACHIEVE) - Seventh Progress Report (November 2013 - April 2014) May 2014.
54. UNICEF Indonesia. Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP and ACHIEVE) - Fifth Progress Report (November 2012 - April 2013) May 2013.
55. UNICEF Indonesia. Maternal and Child Health and Integrated Malaria Control in Eastern Indonesia (MIP and ACHIEVE) - Sixth Progress Report (May 2013 - October 2013) November 2013.
56. UNICEF Indonesia. Joint Progress Report on Australia-UNICEF Education Assistance to Papua and West Papua and Striding towards Equity in Education in Papua and West Papua (STEP) - Second Progress Report (1 March 2011 - 31 March 2012) 20 April 2012.
57. UNICEF Indonesia. ACHIEVE - Reducing Maternal and Neonatal Mortality in Indonesia: The Pathway to Achieving MDG 4&5 - First Progress Report (October 2010 - April 2011) 31 May 2011.
58. UNICEF Indonesia. DEMOGRAPHY: Demographic profile, inequality characteristics and mortality in Papua and West Papua provinces. MICS Further Analysis Series. 2011. Accessed from:
59. UNICEF Indonesia. EDUCATION: Disparities in educational outcomes in West Papua and Papua. MICS Further Analysis Series. 2011. Accessed from:
60. UNICEF Indonesia. THE GAPS: Social Disparities in West Papua and Papua. MICS Further Analysis Series. 2011. Accessed from:
61. Issue Briefs: Maternal and Child Health [database on the Internet]. UNICEF Indonesia, 2012 [cited August 7, 2014]. Accessed from: http://www.unicef.org/indonesia/A5-_E_Issue_Brief_Maternal_REV.pdf.
62. USAID. Report on a joint evaluation of UNICEF's USAID-funded program to control malaria in pregnancy in Eastern Indonesia. Jakarta May 13, 2009.
63. USAID. How-To-Note: Preparing Evaluation Reports in Publisher. Number 1, Version 1.02012.
64. USAID. Investing in Indonesia: A stronger Indonesia advancing national and global development - Country Development Cooperation Strategy for Indonesia 2014 – 2018: USAID2013.
100 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
65. WHO. A Strategic Framework for Malaria Prevention and Control in Africa: Accessed from: http://www.who.int/malaria/publications/atoz/afr_mal_04_01/en/.
66. WHO Bulletin. Editorial. 2005 April; 83(4)
67. WHO Global Malaria Programme. Malaria: Global Fund proposal development (Round 11) - WHO Policy Brief 2011: Accessed from: http://www.who.int/Malaria/publications/atoz/Malaria_gf_proposal_dev_who_policy_brief_201106.pdf.
68. WHO Jakarta. Review of the National Malaria Control Programme of the Republic of Indonesia. 201; (September): Accessed from: http://apps.who.int/iris/bitstream/10665/94367/1/2012-08_SEARO.ReviewMalariaControl%20Prog.Indonesia%20201.pdf.
69. WHO. International Malaria profiles: Indonesia Accessed from: http://www.who.int/Malaria/publications/country-profiles/profile_idn_en.pdf.
70. Wickramasuriya. Malaria and Ankylostomiasis in the Pregnant Woman. Oxford.: Oxford University Press; 1937.
71. World Bank. Indonesia Economic Quarterly - Slower Growth; High Risks2013: Accessed from: http://www.worldbank.org/content/dam/Worldbank/document/EAP/Indonesia/IEQ-Dec13-ENGLISH.pdf.
72. World Malaria Report - Indonesia2013: Accessed from: http://www.who.int/malaria/publications/country-profiles/profile_idn_en.pdf?ua=1.
101 USAID-UNICEF Maternal and Child Health Integrated Malaria Control program in Eastern Indonesia
Statement regarding conflict of interest
No conflicts of interest exist among the investigators of this Mid-Term Evaluation
of the USAID-funded UNICEF Maternal and Child health and Integrated Malaria
Project in Eastern Indonesia.
Lucas Pinxten Team Leader
Siti Nurul Qomariyah
Irma Anintya Tasya