final mozambique race endline survey report 31march17 ... · race endline survey final report v...

ENDLINE SURVEY

FINAL REPORT

SAVE THE CHILDREN, MOZAMBIQUE

Prepared by ICF and Save the Children for WHO Rapid Access Expansion (RAcE) Program

March 2017

AUTHORS: Meghan Swor, Debra Prosnitz, Kirsten Zalisk, Tanya Gunther, Eleanor Hill, Jeanne Koepsell

RAcEEndlineSurveyFinalReport i

ACKNOWLEDGEMENTS

ICF and Save the Children would like to thank the Instituto Nacional de Saúde and the Mozambique Ministry of Health for their contributions to this work. We would also like to thank the Agente Polivalente Elementares (Mozambique’s community health workers) who work hard to provide services to caregivers and children in communities, and to the caregivers who give so much to ensure and improve the health of their children. This work was made possible by the World Health Organization through funding by the Canadian Government.

RAcEEndlineSurveyFinalReport ii

TABLE OF CONTENTS

ABBREVIATIONS ............................................................................................................................................................. iii

EXECUTIVE SUMMARY ................................................................................................................................................. iv

1 BACKGROUND ...................................................................................................................................................... 1 1.1 RAcE Program Goals and Objectives ....................................................................................................... 1 1.2 RAcE Mozambique Project Background .................................................................................................. 1 1.3 RAcE Mozambique Endline Survey Objectives ...................................................................................... 2

2 SURVEY METHODS ................................................................................................................................................ 3 2.1 Survey Implementation and Partnership .................................................................................................. 3 2.2 Survey Design ................................................................................................................................................. 3 2.3 Survey Questionnaire ................................................................................................................................... 5 2.4 Selection and Training of Survey Staff ...................................................................................................... 6 2.5 Data Collection ............................................................................................................................................. 6 2.6 Data Entry and Management ...................................................................................................................... 7 2.7 Data Analysis .................................................................................................................................................. 8 2.8 Survey Indicators ........................................................................................................................................... 8 2.9 Survey Limitations ......................................................................................................................................... 8

3 FINDINGS .................................................................................................................................................................. 9 3.1 Characteristics of Sick Children and Caregivers ................................................................................... 9 3.2 Decision-Making .......................................................................................................................................... 11 3.3 Caregiver Knowledge and Perception of APEs .................................................................................... 11 3.4 Care-Seeking ................................................................................................................................................ 12 3.5 Assessment ................................................................................................................................................... 14 3.6 Treatment Coverage .................................................................................................................................. 15 3.7 First Dose of Treatment and Counseling from APE .......................................................................... 17 3.8 Referral Adherence .................................................................................................................................... 18 3.9 Sick Child Follow-Up .................................................................................................................................. 18 3.10 Illness Management and Diagnostics by Sex ......................................................................................... 19

4 DISCUSSION .......................................................................................................................................................... 21 4.1 Implications ................................................................................................................................................... 23

Annex A. Survey Report by INS .................................................................................................................................. 24

Annex B. Endline Sample ............................................................................................................................................... 26

Annex C. Detailed Sampling Design ............................................................................................................................ 27

Annex D. Endline Survey Questionnaire ................................................................................................................... 28

Annex E. Endline Survey Training Schedule .............................................................................................................. 29

Annex F. Endline Survey Fieldwork Schedule ........................................................................................................... 34

Annex G. Details of Data Cleaning and Analysis ..................................................................................................... 37

Annex H. Key Indicators for Full Project Area ........................................................................................................ 39

Annex I. Summary of Findings from RAcE Mozambique Endline APE Survey ................................................... 42

RAcEEndlineSurveyFinalReport iii

ABBREVIATIONS

ACT artemisinin-based combination therapy

APE Agente Polivalente Elementar (community health worker)

CCM community case management

iCCM integrated community case management

INS Instituto Nacional de Saúde (National Institute of Health)

M&E Monitoring and evaluation

MISAU Ministério da Saúde (Ministry of Health)

MOH Ministry of Health

mRDT malaria rapid diagnostic test

ORS oral rehydration solution

PPS probability proportional to size

RDT rapid diagnostic test

RAcE Rapid Access Expansion

WHO World Health Organization

RAcEEndlineSurveyFinalReport iv

EXECUTIVE SUMMARY

Save the Children, in partnership with Malaria Consortium, implemented the Rapid Access Expansion (RAcE) program in four provinces in Mozambique—Inhambane, Manica, Nampula, and Zambezia—from 2013 to 2016. In October 2016 the Instituto Nacional de Saúde (INS, or National Institute of Health) conducted the RAcE endline survey, with technical assistance from ICF and logistical, administrative, and financial support from Save the Children. We compared baseline and endline data to assess changes in sick child care-seeking, assessment, and treatment coverage, as well as caregivers’ knowledge of childhood illnesses and perceptions of Agente Polivalente Elementar (APE, or community health worker) services and used the information to make inferences about project accomplishments.

The percentage of caregivers who know that an APE works in their community increased significantly from baseline (62 percent) to endline (93 percent) (p<0.001). Care-seeking from APEs increased significantly between baseline and endline, even though the overall level of care-seeking was high at baseline and remained so at endline. The percentage of children age 2-59 months who were sick in the two weeks preceding the survey and taken to a community case management (CCM)-trained APE as a first source of care overall more than doubled, from 23 percent at baseline to 57 percent at endline (p<0.001).

The proportion of sick children receiving appropriate treatment for integrated community case management (iCCM) illnesses did not change over the course of the project. The proportion of sick children receiving appropriate treatment for an iCCM illness from an APE, however, increased significantly, from 12 percent at baseline to 29 percent at endline (p<0.001). At endline, appropriate treatment by APEs accounted for more than half of all reported appropriate treatment [58.4 percent (28.8 percent of cases of illness received appropriate treatment from an APE/49.3 percent of cases of illness received appropriate treatment, from any source)] compared to about a quarter at baseline [26.0% (11.9%/45.7%)].

Appropriate treatment for diarrhea—treatment with both oral rehydration solution (ORS) and zinc—increased significantly, from 8 percent at baseline to 31 percent at endline. Among those cases that sought care from an APE, there was a nearly five-fold increase in treatment of diarrhea with both ORS and zinc by an APE, from 2 percent at baseline to 23 percent at endline, and treatment of cough with difficult or fast breathing with amoxicillin by an APE nearly doubled, from 17 percent at baseline to 33 percent at endline. There was no significant change in appropriate malaria treatment.

With the RAcE project, the supply chain management shifted from being led by Save the Children to being led by the Ministry of Health, resulting in greater challenges maintaining supply levels. The lack of a measured increase for appropriate fever treatment could be in part due to consistent and widespread stockouts of rapid diagnostic tests and artemisinin-based combination therapy (ACT), both of which need to be in stock for APEs to delivery appropriate treatment. Only 78 percent of APEs had a functional timer to use in counting a child’s respiratory rate at the time of the endline survey. Stock levels of both ORS and zinc were quite high at the time of the endline.

RAcEEndlineSurveyFinalReport v

Table 1. Changes in key indicators and end-of-project indicator estimates

Indicators highlighted in green had a statistically significant change from baseline to endline, determined by a p-value of less than 0.05 and no overlap in confidence intervals. Those determined to have statistically significant change over time based on p-values less than 0.05 but have overlapping confidence intervals are noted by red highlight.

Indicator Baseline Endline % Point

Change p-value

% (CI %) % (CI %)Caregiver Knowledge

1

Percentage of caregivers of children age 2–59 months who have been sick in the two weeks preceding the survey who are aware of the presence of the CCM-trained APE in their community

62.0 (45.4 – 76.2)

93.4 (83.3 – 97.6)

31.4% 0.0007

2

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who know the role of the CCM-trained APE in their community

49.2 (36.8 – 61.8)

69.1 (59.4 – 77.5)

19.9% 0.0142

3

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who know two or more signs of childhood illness that require immediate assessment by an appropriately trained provider

86.5 (81.8 – 90.2)

92.9 (87.3 – 96.1)

6.4% 0.0474

Caregiver perceptions of iCCM services

4

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who view CCM-trained APEs as trusted health care providers

82.9 (75.3 – 88.5)

78.2 (70.5 – 84.4)

-4.7% 0.3070

5

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who believe CCM-trained APEs provide quality services

74.8 (63.7 – 83.3)

76.7 (70.3 – 82.1)

1.9% 0.7217

6

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who found the CCM-trained APE at first visit

N/A 82.2

(74.0 – 88.3) N/A N/A

7

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who cite the CCM-trained APE as a convenient source of treatment

80.4 (70.2 – 87.7)

81.9 (72.4 – 88.6)

1.5% 0.8254

RAcEEndlineSurveyFinalReport vi


Change p-value

% (CI %) % (CI %)Sick Child Care-seeking

8

Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey for whom advice or treatment was sought from an appropriate provider

Overall 79.1

(73.3 – 84.0) 79.5

(73.2 – 84.7) 0.4% 0.8991

Malaria (ACT)** 84.1

(77.6 – 89.0) 77.5

(70.5 – 83.3) -6.6% 0.0982

Diarrhea (ORS and zinc) 74.2

(65.9 – 81.1) 78.1

(69.9 – 84.6) 3.9% 0.3570

Cough with difficult or fast breathing (amoxicillin)

79.1 (72.3 – 84.6)

83.1 (76.3 – 88.3)

4.0% 0.3452

9

Percentage of children age 2-59 months who were sick in two weeks preceding the survey taken to a CCM-trained APE as first source of care

Overall 23.1

(15.1 – 33.6) 57.0

(48.5 – 65.1) 33.9% 0.0000

Fever 24.4

(15.5 – 36.1) 54.1

(44.6 – 63.2) 29.7% 0.0003

Diarrhea 22.9

(14.5 – 34.2) 60.8

(51.5 – 69.3) 37.9% 0.0000


21.7 (14.0 – 32.2)

56.1 (47.4 – 64.4)

34.4% 0.0001

Sick Child Assessment

10

Percentage of children age 2-59 months with fever in the two weeks preceding the survey who had finger or heel stick

43.9 (35.4 – 52.8)

51.1 (43.9 – 58.3)

7.2% 0.2490

11

Percentage of children age 2-59 months for whom their caregiver received the results of the malaria diagnostic test of the children who had had finger or heel stick in the two weeks preceding the survey

84.9 (75.0 – 91.3)

93.6 (87.2 – 96.9)

8.7% 0.0603

12

Percentage of children age 2-59 months with cough and difficult or rapid breathing (suspected pneumonia) in the two weeks preceding the survey who had their respiratory rate counted to assess fast breathing

32.6 (25.7 – 40.4)

39.2 (29.6 – 49.7)

6.6% 0.2398

Sick Child Assessment by CHW

13

Percentage of children age 2-59 months with fever in the two weeks preceding the survey who had a finger or heel stick by an APE (among those who sought care from an APE)

19.1 (10.6 – 32.0)

51.2 (38.2 – 64.0)

32.1% 0.0008

RAcEEndlineSurveyFinalReport vii


Change p-value

% (CI %) % (CI %)

14

Percentage of children age 2-59 months for whom their caregiver received the results of the malaria diagnostic test of the children who had a finger or heel stick by an APE in the two weeks preceding the survey (among those who sought care from an APE)

100 96.6

(89.6 – 98.9) -3.4% 0.3907

15

Percentage of children age 2-59 months with cough and difficult or rapid breathing (suspected pneumonia) in the two weeks preceding the survey who had their respiratory rate counted to assess fast breathing by an APE (among those who sought care from an APE)

23.2 (12.2 – 39.8)

39.0 (27.1 – 52.3)

15.8% 0.0915

Sick Child Treatment

16

Percentage of children age 2-59 months who have been sick in two weeks preceding the survey who received appropriate treatment

Overall 45.7

(40.7 – 50.7) 49.3

(42.1 – 56.5) 3.6% 0.4168

Fever (Confirmed malaria – ACT same or next day with positive

blood test)

77.2 (66.5 – 85.2)

70.5 (59.4 – 79.5)

-6.7% 0.3652


(4.3 – 14.7) 31.1

(21.8 – 42.1) 23.0% 0.0001

Cough with difficult or fast breathing (Amoxicillin)

69.6 (61.1 – 76.9)

58.8 (50.9 – 66.2)

-10.8% 0.0558

17

Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey who received appropriate treatment from a CCM-trained APE

Overall 11.9

(7.4 – 18.5) 23.7

(18.5 – 29.7) 11.8% 0.0003

Fever (Confirmed malaria – ACT same or next day with positive

blood test)

23.5 (12.9 – 38.9)

34.2 (23.5 – 46.7)

10.7% 0.1736


(0.7 – 7.1) 22.6

(15.0 – 32.4) 20.4% 0.0000


17.0 (10.2 – 27.0)

33.1 (26.0 – 41.1)

16.1% 0.0164

18

Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey who received the first dose of treatment in the presence of an APE among those who received prescription medicines for a CCM condition in the two weeks preceding the survey

Overall 41.3

(29.7 – 53.9) 33.7

(24.5 – 44.5) -7.6% 0.3549

Fever 56.3

(36.8 – 73.9) 26.8

(16.5 – 40.4) -29.5% 0.0289

RAcEEndlineSurveyFinalReport viii


Change p-value

% (CI %) % (CI %)

Diarrhea 0.0 17.4

(7.0 – 37.1) 17.4% 0.4418


28.2 (21.0 – 36.7)

51.0 (37.4 – 64.5)

22.8% 0.0014

19

Percentage of sick children age 2-59 months for whom their caregivers received counseling on how to provide the treatment(s) among those who received prescription medicines for a CCM condition in the two weeks preceding the survey

Overall 96.7

(90.8 – 98.9) 96.0

(92.0 – 98.0) -0.7% 0.7481

Fever 95.8

(85.2 – 98.9) 98.8

(91.2 – 99.8) 3.0% 0.2781

Diarrhea 80.0

(22.8 – 98.2) 95.7

(86.4 – 98.7) 15.7% 0.1740

Cough with difficult or fast breathing

100 93.9

(85.6 – 97.5) -6.1% 0.1767

Sick Child Referral and Follow-up

20

Percentage of sick children age 2-59 who were referred in the two weeks preceding the survey whose caregiver adhered to referral advice

68.8 (54.2 – 80.3)

65.4 (55.4 – 74.2)

-3.4% 0.6689

21

Percentage of sick children age 2-59 months receiving treatment from an APE in the two weeks preceding the survey who received a follow-up visit from an APE

40.9 (25.0 – 58.9)

41.2 (32.1 – 50.9)

0.3% 0.9753

22

Percentage of sick children age 2-59 months receiving treatment from an APE in the two weeks preceding the survey whose caregiver followed-up with an APE

10.2 (4.1 – 23.1)

10.3 (6.1 – 16.9)

0.1% 0.9826

* Indicates that a figure is based on fewer than 10 cases ** Numerator: Cases of fever with a positive blood test who received ACT within 24 hours; Denominator: all cases of fever

RAcEMozambiqueEndlineSurveyFinalReport 1

1 BACKGROUND

1.1 RAcE Program Goals and Objectives

In 2013, the World Health Organization (WHO) launched the Rapid Access Expansion (RAcE) program in five sub-Saharan African countries—Democratic Republic of Congo, Malawi, Mozambique, Niger, and Nigeria. The goal of the program was to increase coverage of diagnostic, treatment, and referral services for malaria, pneumonia, and diarrhea to decrease overall mortality and the number of severe cases among children aged 2–59 months. The program would accomplish this goal through the following objectives:

Catalyze the scale-up of integrated community case management (iCCM) as an integral part of government-provided health services in sub-Saharan Africa.

Stimulate policy review and regulatory update in each country on disease case management.

Accelerate adaptation of supply management and surveillance systems to include services at the community level.

This effort came at a time when there was great momentum for iCCM at the country level and a high degree of interest among the global health community to understand how to best measure success and how to build country ownership and capacity to sustain iCCM interventions.

1.2 RAcE Mozambique Project Background

Save the Children, in partnership with Malaria Consortium, implemented the RAcE program in four provinces in Mozambique—Inhambane, Manica, Nampula, and Zambezia—from 2013 to 2016. The goal of the program was to reduce under-five mortality and morbidity by increasing access to iCCM services, improving the quality of iCCM services, strengthening links between iCCM services and communities, and advocating with the Ministry of Health (MOH) and partners to strengthen the national system to support, sustain, and scale up iCCM.

In Mozambique, iCCM is part of the Ministério da Saúde (MISAU, or Ministry of Health) Agente Polivalente Elementar (APE, or community health worker) program. The APE program started in 1978 and, after a period of decline, was revitalized starting in 2010. In 2014, the APE program became a department in the National Directorate of Public Health, which oversees planning, coordination, and monitoring of interventions. APEs, who live in the communities they service, provide iCCM services as well as conduct health promotion activities, preventive home visits, and malnutrition screening. The communities have populations between 500–2,000 residents or more and are located between 8 and 25 km from the nearest health facility.

The RAcE program began implementation in the four provinces in April 2013. Save the Children was the lead implementing partner, working closely with MISAU in Manica, Nampula, and Zambezia. In Inhambane, Malaria Consortium led the implementation, in collaboration with MISAU. As of September 2016, a total of 1,339 APEs were providing iCCM services in the four project provinces, covering a total population of approximately 4,196,074, including an estimated 719,444 children under five years of age.


The national program is expanding with the ongoing training of new APEs and the roll out of an additional package of interventions, introduced in October 2015, which includes family planning, distribution of vitamin A capsules for children aged 6–59 months, provision of Misoprostol for postpartum hemorrhage, and support for HIV/AIDS and tuberculosis treatment adherence.

The baseline household survey was conducted mid-November through mid-December 2013 by Save the Children, in collaboration with the Instituto Nacional de Saude (INS, or National Institute of Health) and with technical support from ICF.

1.3 RAcE Mozambique Endline Survey Objectives

The objective of the RAcE endline household survey was to assess care-seeking behavior for sick children, iCCM coverage, and caregiver knowledge, attitudes, and practices related to pneumonia, diarrhea, and malaria in the RAcE Mozambique intervention areas. We compared baseline and endline data to assess changes in sick child care-seeking, assessment, and treatment coverage as well as caregivers’ knowledge of childhood illnesses and perceptions of APE services, and used the information to make inferences about project accomplishments.


2 SURVEY METHODS

2.1 Survey Implementation and Partnership

INS conducted the RAcE endline survey, with technical assistance from ICF and logistical, administrative, and financial support from Save the Children. INS worked with ICF and Save the Children to finalize the questionnaire; led the training of enumerators, data entry operators, and supervisors; and provided oversight to the entire implementation process.

The survey protocol received ethical approval from ICF’s Institutional Review Board and from Mozambique’s Ministry of Health Bio Ethics Committee.

Annex A contains a complete list of the people involved in the survey and their roles.

2.2 Survey Design

Household survey: This was a cross-sectional cluster-based household survey, targeting primary caregivers of children aged 2–59 months who had recently been sick with diarrhea, fever, or fast breathing. All primary caregivers of children aged 2–59 months reported to have experienced diarrhea, fever, or cough with rapid breathing in the two weeks prior to interview were considered eligible for inclusion in the survey. ICF developed standardized sampling guidance for all RAcE projects, which was adapted for Save the Children Mozambique.

To be able to detect a 20 percent difference at 90 percent power with a two-tailed test and 95 percent confidence using cluster sampling, 263 cases were needed for each disease. ICF rounded up to 300 cases to ensure a consistent number of interviews per cluster and a slight increase in the precision of the coverage estimates.

The household survey used a 30x30 multi-stage cluster sampling methodology. At baseline, the target population comprised the RAcE project area: iCCM-eligible areas—more than 8 km and less than 25 km from a health facility, in Manica, Nampula, and Zambezia provinces.

At baseline, 30 clusters were selected using probability proportional to size (PPS). Inhambane was excluded from the baseline survey sample, because a household survey had been conducted there earlier in the year and there was concern about over-surveying those communities.

Prior to selection of enumeration areas, Save the Children and ICF worked with INS to develop the sampling frame, the list of all APEs (both active and inactive) and corresponding communities in the areas where RAcE was already being implemented. Where possible, the Instituto Nacional de Estatistica (National Institute of Statistics) provided population information about these enumeration areas.

The same clusters sampled at baseline were planned to be sampled at endline, plus an additional two clusters from Inhambane discussed further below. However, 9 of the 30 clusters were replaced using PPS, 4 due to security issues that made in the original clusters inaccessible and 5 because the clusters were no longer part of the project area because APEs had abandoned the sites sometime between 2012 and 2015.

During fieldwork, an additional five clusters were replaced by selecting the nearest accessible cluster from the sampling frame: three were misidentified in the sampling frame, and although they had APEs, they were not supported by Save the Children with RAcE funding; one was inaccessible due to conflict;


and one was replaced because upon arriving in the cluster the data collection team found that the APE serving that community had passed away the night before. All replacement sites were selected under the instruction of and with guidance from ICF and Save the Children.

Inhambane was included in the endline survey to get estimates for the entire RAcE project area, inclusive of all four provinces. Following discussion among project stakeholders, agreement was reached and supported by WHO, to get a project-wide estimate only. Per WHO’s request, a representative sample of Inhambane was not collected, because it did not have a baseline for comparison. To power the sample to obtain project wide estimates, we only needed to sample two clusters from Inhambane. These two clusters were selected using PPS. This sampling approach provides comparable estimates to baseline for the combined area of Manica, Nampula, and Zambezia, as well as endline estimates for the entire RAcE project area. Please see Annex B for a complete list of the 32 endline clusters (30 from the original 3 provinces and 2 from Inhambane).

Within each cluster, 10 interviews were conducted for each of the 3 illness modules—diarrhea, fever, and fast breathing—for a total of 30 interviews per cluster, or 320 interviews per each illness across the project area for the 32 clusters at endline (inclusive of Inhambane).

Within each cluster, the survey team randomly selected the first household for interview and proceeded to the household with its front door nearest to the front door of the current household until the team conducted 10 interviews for each illness.

At each household, the enumerator first determined whether an eligible child lived there. An eligible child was aged 2–59 months old and had been sick with diarrhea, fever, cough with rapid breathing, or any combination of the three illnesses in the two weeks preceding the survey.

If there was an eligible child in the household, the interviewer administered the questionnaire, including all applicable illness modules, to the caregiver of the eligible child. If more than one child was eligible, and they were sick with different illnesses, their caregiver was asked about each instance of illness. If there was more than one eligible child in the household for an illness, the interviewer randomly selected one of the eligible children and interviewed his or her caregiver. If multiple children in the same household were reported to have symptoms in the preceding two weeks but had different caregivers, interviewers could interview multiple caregivers, as long as not more than one child from each household was included for each iCCM condition.

Before survey implementation, based on experience with the baseline study, Save the Children expressed concerns about the difficulty in finding the requisite number of pneumonia cases per cluster. Therefore, if survey teams were not able to identify and survey the required number of cases per disease, the team visited an adjacent enumeration area using the same household selection methodology. In the 30 clusters in Manica, Nampula, and Zambezia, the sample included a total of 305 cases with cough with fast or difficult breathing, 308 cases of diarrhea, and 312 cases of fever. In the 32 clusters inclusive of Inhambane, the sample included a total of 326 cases with cough with fast or difficult breathing, 329 cases of diarrhea, and 341 cases of fever.

Details related to the household and respondent selection process are provided in Annex C.

APE survey: As part of the endline survey, Save the Children conducted a survey of APEs to assess the implementation strength and quality of the iCCM services they delivered. The APE survey was conducted with the endline household survey to assess care-seeking practices and treatment coverage


for iCCM conditions. The objective of the APE survey was to gain a better understanding of the APEs’ background characteristics, activity levels, and support and supervision received to help interpret the results of the household survey. The APEs serving the 32 clusters selected for the endline household survey formed the sample population for the APE survey, and all were interviewed.

2.3 Survey Questionnaire

ICF developed a standard household survey questionnaire for all RAcE grantees to use for their baseline surveys. Each grantee adapted the questionnaire to fit the iCCM program and country context, such as appropriate local terminology for community health workers, care-seeking locations, and treatment options. The same questionnaire used for the baseline survey was used at endline, with some additional questions added to the endline questionnaire: two questions were added to each of the illness modules to gather information about whether caregivers sought care for their sick child and whether they sought care from an APE.

The survey questionnaire contains seven modules: caregiver and household background information; caregivers’ knowledge of iCCM activities in their community; caregivers’ knowledge of childhood illness danger signs; household decision-making; and a module for each major childhood illness: fever, diarrhea, and fast breathing. In addition to collecting information about caregiver knowledge, care-seeking, and treatment coverage, the questionnaire collected standard Demographic and Health Survey data on household ownership of selected assets, materials used for housing construction, and types of water access and sanitation facilities, which ICF will analyze and use for the final evaluation.

Save the Children preferred mobile data collection over paper surveys. To avoid programming mistakes that complicated baseline survey analysis, ICF programmed the questionnaire using CommCare, and Save the Children uploaded this program onto the mobile devices (Samsung Galaxy 12 phones in Manica, Nampula, and Zambezia; iPad tablets in Inhambane).

The survey questionnaire was translated into Portuguese by a translator selected by ICF, and fine-tuned to Mozambican Portuguese by Save the Children staff. Because most households understand local languages rather than Portuguese, supervisors and enumerators, under the direction of INS staff, orally translated the questionnaire to each province’s local language during the enumerator and supervisor training. Under the direction of Save the Children and INS, the questionnaires were not translated into each local language in writing, because supervisors and enumerators do not necessarily read the local language.

Following the enumerator and supervisor training, the survey was pretested in each province in the local language. Questionnaire field testing was conducted in communities that were not in the survey sample. No adjustments were identified as being necessary to the questionnaire during field testing.

The APE survey questionnaire was developed based on tools used in the previous iCCM program funded by CIDA from 2009 to 2012 and from the more recent quality of care assessment conducted in January 2016. The questionnaire was administered to selected APEs in Portuguese using phones and tablets (same as the household survey questionnaire).

Annex D contains the survey household questionnaire.


2.4 Selection and Training of Survey Staff

Terms of reference for the enumerators and supervisors were modified from the baseline survey. Save the Children worked in collaboration with the Provincial Health Authorities to advertise the positions in the local papers and on the radio. INS indicated an individual in the Provincial Health Authorities to work with Save the Children on pre-selecting candidates, and the CVs for potential candidates were sent to INS for final selection.

More participants than were required to field the data collection teams were recruited from each province to participate in the training, to ensure high-quality teams with alternate enumerators available. A five-day training was held in Maputo September 27–October 1, 2016, led by INS with support from ICF and Save the Children. Some of the same INS and Save the Children staff who facilitated the baseline training also facilitated the endline training. INS supervisors from each of the four project provinces were also among the training facilitators. The endline training used the same format and structure as the baseline training, but it was conducted with all participants together in Maputo rather than as separate trainings in the provinces, as was done at baseline.1

The five-day training covered the following:

Overview of the RAcE project goals and activities

Fieldwork procedures

Overview of the roles and responsibilities of enumerators and supervisors

Review of the translated questionnaire and questions and the topics covered by the questionnaire, including consistency of the questions asked

Mobile phone data collection and practice on collecting data using mobile phones

Training participants took an exam after the second day of training, and based upon these results and their participation in the training, the project team selected the strongest participants to serve as team supervisors and the others to serve as enumerators. The training schedule is provided in Annex E.

2.5 Data Collection

Endline survey data collection took place October 9–15 in Inhambane, October 8–18 in Manica, October 10–20 in Nampula, and October 10–30 in Zambezia. A total of 12 survey teams (2 in Inhambane, 3 in Manica, 4 in Nampula, and 3 in Zambezia), each composed of 1 supervisor and 2 enumerators, conducted the data collection. Mobile devices were used to complete the survey, with smart phones (Samsung Galaxy 12) in Manica, Nampula, and Zambezia, and iPad tablets in Inhambane.

Written informed consent was obtained from each caregiver and APE prior to interview. Participation in the study was voluntary, and there was no penalty for non-participation. Survey respondents were not compensated for their time away from income-earning activities or daily duties for participating in the data collection. The questionnaire took approximately one hour to administer per caregiver interviewed.

1 Due to travel delays, the participants from Nampula arrived at the start of the second day of training. They spent additional time on second day getting an overview of material covered on the first day.


Quality control procedures during fieldwork included daily spot checks by supervisors, during which they observed at least one interview per enumerator per day and reviewed data entered into the mobile devices. At the end of each day, data were sent to a cloud server (CommCare) and reviewed by ICF staff, who collaborated with INS and Save the Children to address questions and follow up with the survey teams. However, many clusters did not have a mobile internet connection, so survey teams often needed to upload the data after they left the cluster area.

There were delays throughout fieldwork for a number of reasons. The distance between clusters within each province resulted in significant travel time. Often teams were not able to leave one cluster for the next in a timely manner because they first needed confirmation from ICF or INS that data from a completed cluster had been uploaded to the system. In some clusters, the survey team needed two to three days to complete the data collection. In addition, before starting data collection the survey teams had to present themselves to local authorities and then work with a community leader or guide to map the area before the household selection. These local guides were not always immediately available, thus delaying the start of fieldwork in some clusters. In Zambezia, the start date of the data collection was delayed by five days while the team waited to receive official credentials and permission to work in the province, which were provided by the Provincial Health Authorities.

In addition to various delays, and the challenge of long distances needed to travel between clusters, supervision was an issue. The survey supervisors assigned to Manica and Zambezia were unable to serve in these roles at the last moment, one due to illness and the other due to conflicts with the delayed survey schedule. The Manica survey team was consequently supervised remotely from Maputo and in Zambezia, the Inhambane INS supervisor relocated to supervise the team after completing the data collection in Inhambane.

The fieldwork schedule is provided in Annex F.

2.6 Data Entry and Management

Using mobile devices to capture data removed the additional step of data entry. Quality control checks were built into the mobile application to set allowable ranges and avoid the most common data entry errors. However, the application did allow enumerators to skip modules and questions without receiving an error report.

INS and Save the Children submitted the final dataset to ICF in Excel in separate spreadsheets for each module and for each province. Data were merged in Microsoft Excel and Stata.

The household survey dataset submitted to ICF had a number of errors that required extensive review, additional cleaning, and some reconstruction of data. This is due to two major errors: (1) some enumerators conducted interviews with caregivers who did not have a child who had been sick in the two weeks prior to the survey; these caregivers were asked the caregiver-specific modules but not the sick child modules even though they were not selected for the survey; and (2) many enumerators did not record the cluster number, household number, caregiver number, or child number correctly in each questionnaire module as they moved through the survey, meaning that ICF, in collaboration with INS, had to go back through the data and manually correct this information to ensure that questionnaire records were correctly linked.


2.7 Data Analysis

ICF analyzed the survey data using Stata v14 and Microsoft Excel. At the time of endline survey analysis, ICF also re-analyzed the baseline household survey and made key updates to correct a number of errors discovered in the baseline analysis to ensure comparability with endline data. Steps taken to clean and update endline survey data and key corrections made in the re-analysis of the baseline data are outlined in Annex G.

The analyst calculated point estimates and 95 percent confidence intervals for baseline and endline survey indicators accounting for cluster effects. To test for statistically significant changes between indicators at baseline and endline, a Pearson’s chi-squared test was used for binary and categorical variables and regression for continuous variables. Indicators with p-values less than 0.05 are determined to show a statistically significant change between baseline and endline.

Endline data are displayed disaggregated by a child’s sex and illness. For the comparison of indicators between baseline and endline, we only disaggregated data by sex if we found the differences between males and females to be statistically significant.

The initial analysis of the APE survey was conducted by ICF, which provided tables with point estimates and 95 percent confidence intervals for a set of indicators developed by Save the Children. Save the Children reviewed the tables and conducted additional analyses. The results of the APE survey are included in Annex I, and relevant aspects have been integrated into the main household survey results and discussion sections.

2.8 Survey Indicators

The household survey collected data on 18 key indicators related to caregiver knowledge of APEs and child illnesses; caregiver perceptions of APEs; and sick child care-seeking, assessment, treatment, referral adherence, and follow-up. The survey also collected information on household and caregiver characteristics and household decision-making.

The APE survey collected data on 16 indicators related to APE residency, functionality, medicine and diagnostics availability, supervision, service availability and activity levels, recording completeness, knowledge, and data display and use.

2.9 Survey Limitations

The survey provides estimates for the RAcE project area as a whole. The survey was not powered to provide province-level estimates. Resource limitations led RAcE stakeholders to agree that collecting an adequate sample for province-level estimates (four times the total sample size, to collect data on 300 cases of illness per disease, per province) was not feasible. However, the diversity of the RAcE provinces, and the fact that some RAcE provinces had different project implementation inputs and challenges, makes not having province-specific RAcE estimates a limitation in interpreting the changes in care-seeking and intervention coverage.

In addition, there are known potential biases and limitations with the indicators that assess caregiver recall of malaria diagnostic testing and coverage of appropriate treatment for children with fever and with cough with difficult or fast breathing. The potential biases and limitations of these indicators are further detailed in the findings section.


3 FINDINGS

The results presented in this report reflect only the 30 clusters from Manica, Nampula, and Zambezia where we had comparable baseline and endline data. A summary of results from the 32 cluster sample, representative of the full four-province project area, is presented in Annex H. Two additional factors should be acknowledged when interpreting the changes over time. First, iCCM was being implemented in Inhambane and Nampula provinces prior to the start of the RAcE project. The baseline survey thus reflects iCCM activity already occurring at that time. The endline survey sampled 17 clusters in Nampula. Second, RAcE household surveys used a sampling frame inclusive of only RAcE project areas. At baseline, the project area was defined as areas in which APEs were, or were planned to be, deployed during the project period. At endline, the project area was defined as areas in which APEs were deployed.

3.1 Characteristics of Sick Children and Caregivers

Table 2. Characteristics of sick children included in the survey

Characteristic Baseline EndlineSex* Male, % 50.5 47.6

Female, % 49.5 52.4 Age (months)* <12, % 22.7 24.0 6

12-23, % 24.4 23.2 24-35, % 21.8 20.5 36-47, % 18.3 19.2 48-59, % 12.8 13.1

Two week history of illness** Had fever, % 53.8 51.1

Had diarrhea, % 49.7 45.5 Had cough with fast

breathing, % 38.3 40.4

Average number of illnesses

1.4 1.4

Total number of sick children included in survey

579 696

Cases of illness Fever, n 271 307

Diarrhea, n 275 306 Cough with fast

breathing, n 230 296

Total number of sick child cases included in survey

776 909

* Age of 64 sick children and sex of 5 sick children missing at endline; age of 1 sick child missing at baseline. ** Fever information missing for 16 sick children, cough with fast or difficult breathing information missing for 1 child at baseline.


Table 3. Caregiver characteristics

Characteristic Baseline % Endline %Age (years)* 15-24 N/A 38.5

25-34 34.9 35-44 21.4 45-60 5.1 Mean age (years) 29.0

Education* None N/A 36.9 Primary, ≤ year 4 34.8 Primary, ≥ year 5 24.4 Secondary or higher 3.9

Marital status* Currently married or

living with partner 57.6 55.5

Not married but living with a partner

29.2 28.1 Not in union 13.2 16.4

Partner living with caregiver (among those in union) Yes 94.2 95.7 Total number of caregivers 519 652 * Age of 102 caregivers, education and relationship status of 5 caregivers at endline missing; marital status of 2 caregivers missing at baseline

Table 4. Reported distance and mode of transport to nearest health facility

Baseline Endline

% (CI%) % (CI%)

Distance to nearest facility

< 8 km 23.9

(13.7 - 38.4) 15.0

(8.4 - 25.2) 8-25 km

18.7 (12.0 - 27.9)

31.7 (20.9 - 45.1)

> 25 km 9.6

(3.3 - 25.1) 7.2

(3.0 - 16.5) Don’t know

47.8 (35.6 - 60.2)

46.1 (31.6 - 61.2)

Mean distance to nearest facility 13.7 km 14.5 km Number of caregivers 519 649 Mode of transport

Walk 81.4

(70.3 - 89.0) 79.8

(71.3 - 86.3) Motorbike/Taxi/Bus

9.7 (5.1 - 17.7)

11.1 (6.9 - 17.3)

Other 8.9

(4.3 - 17.5) 9.1

(5.3 - 15.1) Number of caregivers 516 649 Time to nearest facility (among those who go to the facility)

< 30 minutes

13.7 (6.7 - 26.1)

9.8 (4.7 - 19.3)

30–59 minutes

5.0 (2.7 - 9.0)

6.6 (2.8 - 14.6)

1–< 2 hours

38.5 (28.7 - 49.3)

16.7 (12.2 - 22.5)

2–< 3 hours

15.1 (10.6 - 20.9)

32.0 (23.9 - 41.3)

3 hours or more

15.1 (10.6 - 20.9)

32.0 (23.9 - 41.3)

Mean time to nearest facility 2 hours, 17 minutes 2 hours, 13 minutes Total number of caregivers 517 531


3.2 Decision-Making

Decision-making about both income and care-seeking was, at both baseline and endline, made predominantly by a caregiver’s spouse or partner. Approximately one-third of all caregivers reported making these households decisions jointly with their spouse or partner. As shown in Table 5, household decision-making about income and care-seeking did not change significantly over the course of the project period. However, at endline, among caregivers who sought care for their child aged 2–59 months who had been sick in the two weeks before the survey, slightly more than half (57 percent) made the decision to seek care jointly with their spouse or partner.

Table 5. Usual decision-maker in household about income and care-seeking

Decision-maker Income decisions

p-value Care-seeking decisions

p-value Baseline Endline Baseline Endline % (CI%) % (CI%) % (CI%) % (CI%)

Caregiver 5.1

(3.3 - 8.0) 7.4

(4.6 - 11.6) 7.4

(4.3 - 12.3) 13.3

(8.4 - 20.4) Caregiver’s husband or partner

71.7 (61.4 - 80.2)

67.5 (60.0 - 74.2)

68.8 (58.3 - 77.7)

65.4 (56.9 - 73.1)

Caregiver and partner jointly

22.5 (14.9 - 32.4)

23.8 (17.4 - 31.8)

0.8066 23.4

(15.5 - 33.7) 20.5

(15.4 - 26.8) 0.5636

Other 0.7

(0.2 - 3.0) 1.3

(0.5 - 3.1) 0.5

(0.1 - 1.9) 0.7

(0.2 - 2.4) Total number of caregivers

449 541

449 541

Table 6. Joint decision-making to seek care for sick child by illness

Decided to seek care jointly with partner/spouse

Baseline Endline Endline N % (CI %) % (CI %)

Overall N/A 56.3

(50.7 - 61.8) 751

Fever N/A 55.5

(48.8 - 61.9) 256

Diarrhea N/A 53.6

(46.5 - 60.6) 248

Cough with fast breathing N/A 59.9

(51.9 - 67.5) 247

N/A=not available

3.3 Caregiver Knowledge and Perception of APEs

The percentage of caregivers who know that an APE works in their community increased significantly from baseline (62 percent) to endline (93 percent) (p<0.001). There was a moderate increase in the percentage of caregivers who were aware of the APE in their community and were able to list at least two curative services provided by the APE. Of the many activities that APEs implement in communities, from community mobilization to assessment and treatment of iCCM illnesses, caregivers most noted malaria testing (54 percent) and malaria treatment (58 percent). Nearly all caregivers who know an APE is in their community knew where the APE was located at both baseline and endline.


Table 7. Caregiver knowledge of childhood illnesses

Caregiver knowledge Baseline Endline

p-value % (CI %) % (CI %)

Knows 2+ child illness signs 86.5 (81.8 - 90.2)

92.9 (87.3 - 96.1)

0.0474

Knows cause of malaria 67.4 (58.2 - 75.5)

75.9 (66.7 - 83.3)

0.1948

Knows fever is a sign of malaria 64.7 (57.7 - 71.2)

74.7 (66.8 - 81.2)

0.0348

Knows malaria treatment 72.6 (65.0 - 79.1)

86.5 (80.1 - 91.1)

0.0102

Total number of caregivers 519 644

Table 8. Caregiver knowledge of APE

Caregiver knowledge Baseline Endline


Knows APE works in community 62.0 (45.4 - 76.2)

93.4 (83.3 - 97.6)

0.0007

Total number of caregivers 518 649 Knows location of APE* 95.0

(90.6 - 97.4) 94.5

(90.1 - 97.0) 0.8075

Knows 2+ APE curative services 49.2 (36.8 - 61.8)

69.1 (59.4 - 77.5)

0.0142

Total number of caregivers 321 606 * 134 responses missing at endline

Table 9. Caregiver perceptions of iCCM APE

Caregiver perceptions Baseline Endline


View CCM-trained APEs as trusted health care providers

82.9 (75.3 - 88.5)

78.2 (70.5 - 84.4)

0.3070

Believe CCM-trained APEs provide quality services

74.8 (63.7 - 83.3)

76.7 (70.3 - 82.1)

0.7217

Found the CCM-trained APE at first visit (for all instances of care-seeking included in survey)*

N/A 82.2

(74.0 - 88.3) N/A

Cite the CCM-trained APE as a convenient source of treatment

80.4 (70.2 - 87.7)

81.9 (72.4 - 88.6)

0.8254

Total number of caregivers 321 606 N/A=not available * Denominator is 366 caregivers at endline; only those who sought care from an APE for at least one sick child are included

3.4 Care-Seeking

Care-seeking from APEs increased significantly between baseline and endline, although the overall level of care-seeking from an appropriate provider was high at baseline (79 percent) and remained high at endline (80 percent). The source of care-seeking changed substantially over the course of the project, shifting from public facilities as the location where most sought care (65 percent at baseline, 33 percent at endline) to APEs as the location where most sought care (38 percent at baseline, 70 percent at endline).

A similar shift took place in the first source of care. The percentage of cases of illness among children aged 2–59 months who were sick in the two weeks preceding the survey taken to an APE as a first source of care overall more than doubled, from 23 percent at baseline to 57 percent at endline (p<0.001). Among those who sought care, those who sought care from an APE first increased significantly over the course of the project, more than doubling overall, from 27 percent at baseline to


67 percent at endline. This pattern of increased care-seeking from APEs, overall and among those who sought any care, was consistent across fever, diarrhea, and fast breathing.

Among those who did not seek care at all (n=140 cases of illness), the majority did not seek care because either the condition was not serious (30 percent) or the condition passed (31 percent). Among those who sought care but did not seek care from an APE, the majority noted that they did not seek care from the APE because the APE was not available (33 percent) or the APE did not have medicines or supplies (19 percent). The APE survey results showed that although nearly all APEs (91 percent) resided in their communities, only 41 percent had also provided iCCM services at least 5 days per week, and only 37 percent had all essential iCCM medicines and supplies (see Annex I).2

Table 10. Source of care by illness

Illness

Sought care from appropriate provider* p-

value

APE was first source of care p-

value Baseline

N Endline

N Baseline Endline Baseline Endline% (CI %) % (CI %) % (CI %) % (CI %)

Overall 79.1

(73.3 - 84.0) 79.5

(73.2 - 84.7)0.8991

23.1 (15.1 - 33.6)

57.0 (48.5 - 65.1)

0.0000 776 909

Fever 84.1

(77.6 - 89.0) 77.5

(70.5 - 83.3)0.0982

24.4 (15.5 - 36.1)

54.1 (44.6 - 63.2)

0.0003 271 307

Diarrhea 74.2

(65.9 - 81.1) 78.1

(69.9 - 84.6)0.3570

22.9 (14.5 - 34.2)

60.8 (51.5 - 69.3)

0.0000 275 306


79.1 (72.3 - 84.6)

83.1 (76.3 - 88.3)

0.345221.7

(14.0 - 32.2)56.1

(47.4 - 64.4)0.0001 230 296

* Appropriate providers include hospital, private clinic, other health center, and APE.

Table 11. Care-seeking from APEs

Illness

APE was first source of care among those who sought any care p-

value Baseline N Endline N

Baseline Endline% (CI %) % (CI %)

Overall 27.2

(17.8 - 39.2) 67.4

(58.5 - 75.1) 0.0000 658 769

Fever 27.6

(17.3 - 41.1) 66.7

(55.8 - 76.0) 0.0001 239 249

Diarrhea 28.8

(18.4 - 42.0) 73.8

(65.0 - 81.1) 0.0000 219 252


25.0 (16.2 - 36.5)

61.9 (52.5 - 70.6)

0.0001 200 268

2 Artemether-lumefantrine (at least 1x6 or 2x6), amoxicillin, ORS, zinc, RDTs, and functional timer


Table 12. Cases of illness for which no care was sought

Illness Did not seek care

p-value

Sought care but not from APE

p-value Baseline Endline Baseline Endline % (CI %) % (CI %) % (CI %) % (CI %)

Overall 15.2

(11.2 - 20.4) 15.4

(12.5 - 18.9) 0.9266

62.0 (49.7 - 72.9)

30.8 (23.3 - 39.6)

0.0007

Fever 11.8

(7.8 - 17.6) 18.9

(14.5 - 24.2) 0.0119

45.2 (28.2 - 63.4)

31.7 (22.4 - 42.8)

0.2461

Diarrhea 20.4

(14.3 - 28.1) 17.7

(12.3 - 24.6) 0.4784

70.8 (57.7 - 81.1)

24.6 (17.7 - 33.1)

0.0000


13.0 (8.5 - 19.6)

9.5 (6.8 - 13.1)

0.2742 72.5

(60.5 - 81.9) 35.8

(27.2 - 45.5) 0.0001

Total number of sick child cases

776 909 658 769

3.5 Assessment

Caregiver recall of malaria diagnostic testing is poor, which could affect the malaria diagnosis and appropriate treatment indicators calculated. According to the Indicator Guide: Monitoring and Evaluating Integrated Community Case Management, “Studies have found poor sensitivity and specificity of maternal recall for malaria diagnostic tests (finger/heel stick). Consequently, the current recommendation is that household surveys track treatment coverage of fever and, where possible, supplement with data from service delivery assessment to better understand the proportion of suspected malaria cases that receive appropriate diagnosis and treatment.”3

Reported assessment of fever by APEs increased significantly over the course of the project. Among all cases of fever that were assessed by a provider, nearly all were assessed with a malaria rapid diagnostic test (mRDT)4 by either an APE or a nurse, but the percentage assessed by each shifted over the course of the project. At baseline, 73 percent of cases of fever tested were assessed by a nurse and 23 percent by an APE. At endline, 43 percent of fever cases tested were assessed by a nurse and 55 percent by an APE.

Among cases of fever managed by APEs, the percentage of cases for which the APE administered an mRDT increased significantly from baseline (19 percent) to endline (51 percent) p<0.001. Despite this 32 percentage point increase, the endline value of 51 percent is low and may be due to ongoing stockouts of malaria kits, which include rapid diagnostic tests (RDTs) and artemisinin-based combination therapy (ACT). The APE survey revealed that 75 percent of APEs had mRDTs in stock on the day of the survey, but that only 44 percent had at least one age formulation of first-line antimalarial drugs (Coartem; artemether-lumefantrine 1x6 or 2x6) in stock (see Annex I). Data on the availability of RDTs and ACTs at first-level health facilities were not collected at the time of the survey.

Assessment of cough with fast or difficult breathing by APEs did not increase over the project period.

3 Maternal and Child Health Integrated Program. (2013). Indicator Guide: Monitoring and Evaluating Integrated Community Case Management. 4 The survey questionnaire asked caregivers whether blood was drawn. For purposes of this report, we assume that any blood draw in a community and by an APE is an mRDT. However, at the health facility level it is possible that some children received a blood smear and microscopy test for malaria.


Table 13. Malaria assessment among children with fever

Fever assessment Cases managed by APE

p-value All cases

p-value Baseline Endline Baseline Endline % (CI %) % (CI %) % (CI %) % (CI %)

Child had blood drawn 19.1

(10.6 - 32.0) 51.2

(38.2 - 64.0) 0.0008

43.9 (35.4 - 52.8)

51.1 (43.9 - 58.3)

0.2490

Caregiver received result of blood test

100 96.6

(89.6 - 98.9) 0.3907

84.9 (75.0 - 91.3)

93.6 (87.2 - 96.9)

0.0603

Blood test positive for malaria

100 91.7

(75.6 - 97.5) 0.3211

92.9 (81.7 - 97.5)

91.0 (79.4 - 96.4)

0.7216

Received ACT after positive blood test, among those who had a positive blood test

50.0 (30.6 - 69.4)

56.6 (42.0 - 70.2)

0.5494 93.5

(85.8 - 97.1) 82.6

(68.8 - 91.1) 0.0626

Total number of fever cases

131 170 271 307

Table 14. Respiratory rate assessment

Respiratory rate assessment

Cases managed by APEp-value

All cases p-value Baseline Endline Baseline Endline

% (CI %) % (CI %) % (CI %) % (CI %)

Respiratory rate assessed 23.2

(12.2 - 39.8) 39.0

(27.1 - 52.3) 0.0915

32.6 (25.7 - 40.4)

39.2 (29.6 - 49.7)

0.2398

Total number of cough with difficult breathing cases

56 172 230 296

3.6 Treatment Coverage

Overall, appropriate treatment of iCCM illnesses did not increase over the course of the project. Appropriate treatment for diarrhea—treatment with both ORS and zinc—increased significantly by almost 23 percent, from only 8 percent at baseline to 31 percent at endline.

Of the cases of illness among children aged 2–59 months who sought care from any provider in the two weeks prior to the survey, the percentage who received appropriate treatment from the APE increased significantly, from 12 percent at baseline to 29 percent at endline (p<0.0001). Of those who sought care from an APE, the percentage who received appropriate treatment from the APE did not change significantly. Looking at appropriate treatment for each iCCM illness by an APE among only those cases that sought care from an APE, there was a slight but non-significant decrease in treatment of cough with difficult or fast breathing with amoxicillin by an APE. There was a significant (p<0.0001) and nearly five-fold increase in the treatment of diarrhea with ORS and zinc by an APE, from 8 percent at baseline to 36 percent at endline, but no difference in the percentage of cases of diarrhea among children whose caregiver continued fluids or feeding for the child during the illness. Although appropriate treatment with diarrhea improved over the course of the project, still only one-third of children with diarrhea taken to an APE received treatment with ORS and zinc as recommended. The APE survey results showed that most APEs had ORS (91 percent) and zinc (81 percent) in stock on the day of the survey, and most (75 percent for ORS and 63 percent for zinc) reported no stockouts in the past month. In comparison, 81 percent of APEs had amoxicillin in stock on the day of the survey, but only 53 percent reported having no stockouts of amoxicillin in the past month, highlighting that during the recall period of the survey quite a few APEs may have had no amoxicillin in stock. Stockouts of ACTs and RDTs were widespread, with only 28 percent of APEs reporting continuous stock of ACTs and 44 percent reporting


continuous stocks of RDTs during the month before the survey (refer to Annex I for more details). APEs must have both RDTs and ACTs in stock to provide appropriate treatment for fever; if RDTs are out of stock, the APEs are to refer to the health facility even if they have ACTs in stock; if they are out of ACTs but have RDTs, the APEs should test the fever and refer malaria positive cases to the nearest facility for treatment. Further analysis of the APE survey showed that only 41 percent (13 of 32) of APEs had both RDTs and ACTs in stock at the time of the survey, and only 19 percent (6 of 32) of APEs reported no stockouts of either RDTs or ACTs during the month before the survey.

Table 15. Treatment coverage

Received appropriate treatmentp-value

Baseline N

Endline N Illness (treatment)

Baseline Endline% (CI %) % (CI %)

Overall 45.7

(40.7 - 50.7) 49.3

(42.1 - 56.5) 0.4168 545 734

Confirmed malaria (ACT) 93.5

(85.8 - 97.1) 82.6

(68.8 - 91.1) 0.0626 92 132

Confirmed malaria (ACT within two days)

77.2 (66.5 - 85.2)

70.5 (59.4 - 79.5)

0.3652 92 132


(4.3 - 14.7) 31.1

(21.8 - 42.1) 0.0001 223 306


69.6 (61.1 - 76.9)

58.8 (50.9 - 66.2)

0.0558 230 296

Table 16. Appropriate treatment by APEs

Received appropriate treatment from

APE p-value

Baseline N

Endline N

Illness (treatment) Baseline Endline% (CI %) % (CI %)

Overall 11.9

(7.4 - 18.5) 28.8

(22.5 - 36.1) 0.0003 538 725

Confirmed malaria (ACT)* 25.9

(14.4 - 42.1) 38.2

(27.0 - 50.8) 0.1579 85 123

Confirmed malaria (ACT within two days)*

23.5 (12.9 - 38.9)

34.2 (23.5 - 46.7)

0.1736 85 123


(0.7 - 7.1) 22.6

(15.0 - 32.4) 0.0000 223 306


17.0 (10.2 - 27.0)

33.1 (26.0 - 41.1)

0.0164 230 296

* Denominator for received appropriate treatment from APE for confirmed malaria indicators is restricted to children aged 6–59 months. APE protocol is to refer children under five months who have fever.

Table 17. Sought care from an APE and received appropriate treatment by APE

Received appropriate treatment

from APE among those who sought care from APE p-value

Baseline N

Endline N

Illness (treatment) Baseline Endline % (CI %) % (CI %)

Overall 38.9

(31.7 - 46.5) 46.5

(37.9 - 55.4) 0.0958 162 445

Confirmed malaria (ACT within 2 days) 44.2

(25.4 - 64.8) 50.6

(35.8 - 65.3) 0.5605 43 83


(2.7 - 20.8) 35.8

(24.5 - 48.9) 0.0011 64 190


70.9 (55.3 - 82.8)

56.4 (46.5 - 65.8)

0.1181 55 172

* Denominators for received appropriate treatment from APE for confirmed malaria indicators is restricted to children aged 6–59 months.


Table 18. Continued fluids and feeding during diarrhea episode

Continued feeding and fluids

Baseline Endlinep-value

% (CI %) % (CI %)

Continued fluids 43.3

(36.1 - 50.7) 38.7

(29.9 - 48.3) 0.3962

Continued feeding N/A N/A Total number of sick children 275 297 N/A=not available

3.7 First Dose of Treatment and Counseling from APE

APE treatment protocols indicate that the first dose of treatment should be provided by the APE at the time of assessment and that the APE should counsel the caregiver on how to provide treatment to his or her child. However, according to caregivers surveyed, less than half of cases treated by an APE received the first dose of the treatment in the presence of the APE. There were no significant changes overall over the course of the project. However, there was a significant increase in the percentage of cases of cough with difficult or fast breathing that received the first dose of amoxicillin from the APE (28 percent at baseline to 51 percent at endline). The percentage of cases of confirmed malaria that received the first dose of ACT in presence of the APE dropped significantly (p<0.05), from 56 percent at baseline to only 27 percent at endline.

Counseling by APEs on how to provide treatment was high at baseline and remained high at endline.

Table 19. First dose of treatment from APE

Illness (treatment)

First dose received in presence of APE

p-value Baseline

N Endline

N Baseline Endline% (CI %) % (CI %)

Overall 41.3

(29.7 - 53.9) 33.7

(24.5 - 44.5) 0.3549 92 249


(36.8 - 73.9) 26.8

(16.5 - 40.4) 0.0289 48 82

Diarrhea (ORS) 26.2

(17.3 - 37.8) 28.8

(18.6 - 41.7) 0.7322 61 153

Diarrhea (zinc) 0.0 35.0

(21.3 - 51.7) 0.2039 5 80

Diarrhea (ORS and zinc) 0.0 17.4

(7.0 - 37.1) 0.4418 5 69


28.2 (21.0 - 36.7)

51.0 (37.4 - 64.5)

0.0014 39 98

Table 20. Counseling on treatment administration by APE

Illness (treatment)

Counseled on treatment administration

p-value Baseline

N Endline


Overall 96.7

(90.8 - 98.9) 96.0

(92.0 - 98.0) 0.7481 92 249


(85.2 - 98.9) 98.8

(91.2 - 99.8) 0.2781 48 82

Diarrhea (ORS) 95.1

(85.5 - 98.4) 96.1

(91.9 - 98.2) 0.6876 61 153

Diarrhea (zinc) 80.0

(23.0 - 98.2) 98.8

(89.9 - 99.9) 0.0196 5 80


Illness (treatment)

Counseled on treatment administration

p-value Baseline

N Endline



(22.8 - 98.2) 95.7

(86.4 - 98.7) 0.1740 5 69


100 93.9

(85.6 - 97.5) 0.1767 39 98

3.8 Referral Adherence

The levels of reported referrals from an APE increased markedly from 26 percent at baseline to 48 percent at endline. Caregiver’s adherence to a referral provided by an APE was reported at moderate levels at baseline (69 percent overall for all illnesses) and remained similar at endline (65 percent overall for all illnesses). For those who did not adhere to the referral, the most common reason given for not adhering was because the facility was too far (45 percent at baseline, 42 percent at endline), followed by the child improving (35 percent at baseline, 45 percent at endline). Other reasons reported included having no money, having no transport, having no time, and thinking that the illness was not serious.

Table 21. Adherence to APE referral

Illness Baseline Endline

p-value Baseline

N Endline

N % (CI %) % (CI %)

Overall 68.8

(54.2 - 80.3) 65.4

(55.4 - 74.2) 0.6689 64 257

Fever 80.0

(62.1 - 90.7) 70.6

(57.0 - 81.3) 0.3466 25 68

Diarrhea 75.0

(39.1 - 93.4) 60.4

(46.4 - 72.9) 0.4266 12 91


55.6 (33.4 - 75.7)

66.3 (52.5 - 77.9)

0.3401 27 98

3.9 Sick Child Follow-Up

According to the iCCM protocol, APEs are trained to counsel mothers to return for follow-up within three days. The percentage of cases of all illnesses for which mothers returned for follow-up with the APE with their child did not change significantly over the course of the project. The number of cases for which the mothers followed up, however, was low (7 at baseline, 55 at endline). Among the small number of caregivers who did return for follow-up with an APE, the percentage of those who followed up within three days increased from 0 percent at baseline to 22 percentage at endline. The percentage of cases of all illnesses for which APEs followed up with the child did not change over the course of the project. Among cases of fever, there was a significant increase in APE follow-up, from 15 percent at baseline to 38 percent at endline. However, it is important to note that the cases of illness captured in this survey are those that occurred at any time in the previous two weeks. Therefore, some cases of illness captured in the survey were not yet be eligible for follow-up by either the mother or the APE.


Table 22. Caregiver follow-up with APE

Condition Baseline Endline

p-value Baseline N Endline N % (CI %) % (CI %)

Overall 10.2

(4.1 - 23.1) 10.3

(6.1 - 16.9) 0.9826 137 532

Fever* 10.0

(2.5 - 32.8) 9.4

(4.7 - 17.9) 0.9350 20 170

Diarrhea 9.5

(3.3 - 24.6) 11.1

(6.1 - 19.1) 0.8018 63 190


11.1 (4.8 - 23.5)

10.5 (5.1 - 20.4)

0.9156 54 172

*112 responses missing for fever at baseline

Table 23. APE follow-up with sick child

Illness Baseline Endline

p-value Baseline N Endline N % (CI %) % (CI %)

Overall 40.9

(25.0 - 58.9) 41.2

(32.1 - 50.9) 0.9753 137 532

Fever* 15.0

(5.2 - 36.1) 37.7

(26.9 - 49.8) 0.0453 20 170

Diarrhea 47.6

(30.9 - 64.9) 38.4

(28.7 - 49.2) 0.3417 63 190


42.6 (23.0 - 64.9)

47.7 (37.2 - 58.4)

0.6505 54 172

*112 responses missing for fever at baseline

3.10 Illness Management and Diagnostics by Sex

There were no notable differences in appropriate assessment or treatment of iCCM illnesses between boys and girls.

Table 24. Fever management and treatment administered

Sex** Confirmed malaria treatment*

Number of children with positive RDT

Any antimalarial ACT ACT within 24 Hours % (CI %) % (CI %) % (CI %)

Overall 87.1

(76.1 - 93.5) 82.6

(68.8 - 91.1) 70.5

(59.4 - 79.5) 132

Male 87.1

(74.8 - 93.9) 82.9

(65.4 - 92.5) 71.4

(56.6 - 82.8) 70

Female 86.9

(73.0 - 94.2) 82.0

(68.0 - 90.7) 68.9

(56.9 - 78.7) 61

€ Among to those who sought care from a hospital, private clinic, other health center, or APE. ** Sex is missing for one child with fever.

Table 25. Fever diagnostics

Sex** Had blood taken from

finger or heel Among those who had blood taken Number of children

With fever Were given results Test result positive

Overall 51.1

(43.9 - 58.3) 93.6

(87.2 - 96.9) 91.0

(79.4 - 96.4) 307

Male 55.9

(46.0 - 65.3) 93.8

(87.0 - 97.2) 92.1

(77.2 - 97.6) 145

Female 46.6

(38.0 - 55.4) 93.3

(78.1 - 98.2) 87.1

(73.2 - 94.4) 161

** Sex is missing for one child with fever.


Table 26. Diarrhea management by provider and treatment administered

Sex Sought any advice or treatment

Sought treatment from an

appropriate provider€

Sought treatment

from an APE

Sought treatment

from an APE as first choice

Given more than usual to drink*

TreatmentTreated

with ORS AND zinc

Number of children

with diarrhea

ORS Homemade

fluid Zinc

Overall 82.4

(75.4 - 87.7) 77.5

(70.5 - 83.3) 62.1

(52.9 - 70.5) 60.8

(51.5 - 69.3) 38.7

(29.9 - 48.3) 69.9

(62.5 - 76.5) 25.5

(17.9 - 35.0) 35.0

(25.2 - 46.3)31.1

(21.8 - 42.1) 306

Male 83.0

(74.6 - 89.0) 78.9

(68.2 - 86.7) 65.3

(54.3 - 74.9) 64.0

(53.2 - 73.5) 44.7

(33.3 - 56.6) 66.7

(57.1 - 75.0) 19.7

(12.0 - 30.7) 36.7

(25.3 - 49.8)33.3

(23.2 - 45.3) 147

Female 81.8

(73.3 - 88.0) 77.4

(68.3 - 84.4) 59.1

(49.0 - 68.5) 57.9

(47.8 - 67.3) 33.3

(24.4 - 43.7) 73.0

(63.7 - 80.6) 30.8

(21.3 - 42.3) 33.3

(23.0 - 45.6)28.9

(19.0 - 41.5) 159

€ Refers to those who sought care from a hospital, private clinic, other health center, or APE. * Nine missing responses

Table 27. Cough with difficult breathing management by provider and treatment

Sex** Sought any advice or treatment

Sought treatment from an appropriate

provider€

Sought treatment from

an APE

Sought treatment from an APE as first

choice

Assessed for rapid breathing

Treatment Number of children

with cough and fast or

difficult breathing

Any antibiotic Amoxicillin

Overall 90.5

(86.9 - 93.2) 83.1

(76.3 - 88.3) 58.1

(49.2 - 66.5) 56.1

(47.4 - 64.4) 39.2

(29.6 - 49.7) 60.33

(52.28 - 67.85) 58.8

(50.9 - 66.2) 296

Male 90.0

(83.5 - 94.1) 83.9

(74.2 - 90.4) 57.7

(47.5 - 67.3) 55.4

(44.7 - 65.6) 41.5

(30.6 - 53.4) 63.9

(53.1 - 73.4) 62.3

(51.3 - 72.2) 130

Female 90.7

(85.3 - 94.3) 82.7

(74.0 - 89.0) 58.0

(47.6 - 67.8) 56.2

(46.2 - 65.7) 37.7

(26.3 - 50.6) 55.6

(45.3 - 65.4) 54.9

(45.0 - 64.5) 162

€ Refers to those who sought care from a hospital, private clinic, other health center, or APE. ** Sex is missing for four children with cough and fast or difficult breathing.


4 DISCUSSION

The revitalization of the APE program has increased the presence and scope of APEs in communities over the course of the RAcE project period. At the time of the baseline, iCCM services were available in most districts in Nampula and Inhambane provinces and were introduced in Manica and Zambezia provinces through RAcE. During the RAcE project, caregivers’ knowledge of APEs and care-seeking from APEs, specifically for iCCM services, increased. The percentage of caregivers who know that an APE works in their community increased significantly from baseline (62 percent) to endline (93 percent) p<0.001. Care-seeking from APEs increased significantly between baseline and endline, even though the overall level of care-seeking was high at baseline and remained high at endline. The percentage of children aged 2–59 months who were sick in the two weeks preceding the survey and were taken to a CCM-trained APE as a first source of care overall more than doubled, from 23 percent at baseline to 57 percent at endline (p<0.001). This pattern of increased care-seeking from APEs was consistent across fever, diarrhea, and fast breathing. As care-seeking from APEs increased, there was a concurrent shift in the source of care-seeking, from public facilities to APEs. At baseline, the majority of caregivers (65 percent) sought care from public facilities, and only 38 percent sought care from APEs; at endline, this flipped to most caregivers seeking care from APEs (69 percent) and only 34 percent seeking care from public facilities. Considering the overall increase in care-seeking, this suggests that deployment of APEs to provide iCCM services has indeed extended health services to those who might not have previously sought care. Among those who did seek care but did not seek care from an APE, the majority noted that they did not seek care from the APE because the APE was not available (33 percent) or the APE did not have medicines or supplies (19 percent).

These achievements suggest an overall increase in access to care in RAcE project areas. This is commendable, particularly given that episodic political violence rendered some project areas inaccessible at points during project implementation. This episodic violence also made some project areas inaccessible during both baseline and endline surveys, consequently skewing the results of the surveys to be more reflective of areas that likely had better or more consistent provision of services by APEs.

Significant improvements were found in assessment of cases of illness by APEs and in appropriately managing cases of illnesses. Reported assessment of illness by APEs increased significantly for both fever and cough with difficult or fast breathing over the course of the project.

Assessment of fever and diagnosis of malaria, in particular, significantly improved over the course of the project. Among those who sought care from an APE, the percentage of children aged 2–59 months with fever in the two weeks preceding the survey who were given an RDT (had a finger or heel stick) by an APE more than doubled from baseline to endline, from 19 percent at baseline to 51 percent at endline (p<0.05). Despite this 32 percentage point increase, the endline value of 51 percent is low, and this is most likely due to ongoing stockouts of malaria kits, which include RDTs. Data collected from RAcE teams in the provinces (source Serviço Distrital de Saúde, Mulher e Acção Social) post survey showed that stockouts of the malaria kits were registered in 6 districts in August, in 11 districts in September, and in 9 districts in October. In addition, the APE survey noted widespread stock disruptions; only 19 percent of APEs had continuous stock of both RDTs and ACTs the month before the survey (covering the 2-week recall period of the household survey).


The percentage of sick children receiving appropriate treatment for iCCM illnesses did not change over the course of the project. The percentage of sick children receiving appropriate treatment for an iCCM illness from an APE, however, increased significantly, from 12 percent at baseline to 29 percent at endline (p<0.001). At endline, appropriate treatment by APEs accounted for over half of all reported appropriate treatment [58.4 percent (28.8 percent of cases of illness received appropriate treatment from an APE/49.3 percent of cases of illness received appropriate treatment, from any source)] compared to about a quarter at baseline [26.0% (11.9%/45.7%)].

Appropriate treatment for diarrhea—treatment with both ORS and zinc—increased significantly by almost 23 percent, from 8 percent at baseline to 31 percent at endline. Among those cases that sought care from an APE, there was a nearly five-fold increase in treatment of diarrhea with both ORS and zinc by an APE, from 8 percent at baseline to 36 percent at endline. The increased availability of zinc among APEs between baseline and endline can help explain the improvement. There was no significant change in appropriate malaria treatment or in appropriate treatment for cough with difficult or fast breathing. The lack of a measured increase for appropriate fever treatment could be in part due to consistent and widespread stockouts of RDTs and ACTs, both of which need to be in stock for the APE to delivery appropriate treatment. It is also important to consider that caregiver recall of RDT testing is poor and that this could affect the results of malaria diagnosis and appropriate treatment. Caregivers can better recall treatment of any fever. Stockouts of amoxicillin, reported by 47 percent of APEs during the month before the survey, may have also contributed. Treatment of cough with difficult or fast breathing must be interpreted carefully. Pneumonia treatment, for which this indicator is a proxy, is globally recognized to have validity issues because this diagnosis of presumptive pneumonia is often inaccurate in comparison with clinical diagnosis of pneumonia at health facilities. Therefore, the number of cases of cough with difficult or fast breathing is likely an overestimate of actual clinical pneumonia cases, and the percentage of these treated with amoxicillin can, and should, reasonably not be 100 percent.5

The relatively low coverage of treatment with ORS and zinc among diarrhea cases managed by APEs (36 percent) is concerning. Stock levels of both ORS and zinc were quite high at the time of the endline, so lack of stock cannot explain all missed cases. A quality of care survey conducted in January 2016 found that APEs failed to identify and treat many cases of diarrhea when compared to a gold standard examiner. In cases where children presented with other iCCM conditions, such as fever or cough and fast-breathing, the APE may have overlooked the diarrhea.

Although the increases in appropriate assessment and treatment coverage between baseline and endline are positive, the values observed at endline are lower than expected in a well-supported program in which APEs were well-supplied and supervised. An endline survey conducted in 2012 for the CIDA-funded iCCM program6 in three intervention districts of Nampula (Angoche, Erati, and Monapo) found that 68 percent of children aged 6–59 months who sought care first from an APE received an RDT, compared to 51 percent of all cases managed by an APE in the RAcE endline survey, and that

5 Campbell H, el Arifeen S, Hazir T, O'Kelly J, Bryce J, Rudan I, et al. (2013). Measuring coverage in MNCH: Challenges in monitoring the proportion of young children with pneumonia who receive antibiotic treatment. PLoS Med 10(5): e1001421. doi:10.1371/journal.pmed.1001421 6 CIDA 2012 endline survey and RAcE surveys are not directly comparable because they used different methodologies and questionnaires.


98 percent of RDT+ cases received ACTs, compared to 57 percent in RAcE endline. An APE survey carried out alongside the 2012 CIDA endline household survey found that most APEs had RDTs (87 percent) and ACTs (80 percent) in stock, whereas the current endline APE survey found widespread shortages. With the RAcE project, the supply chain management shifted from being led by Save the Children to being led by the MOH, resulting in greater challenges maintaining supply levels. Similar decreases were seen between the 2012 CIDA endline and the 2016 RAcE endline for assessment of respiratory rate by APEs. In the 2012 CIDA endline survey, 60 percent of children presenting first to an APE had their respiratory rate counted, compared to 39 percent of the cases of cough with difficult or fast breathing managed by APE in the RAcE endline survey. Only 78 percent of APEs had a functional timer at the time of the RAcE endline survey, which may have contributed to the lower levels observed in 2016. Provision of first dose and levels of follow-up with the APE were also higher in the 2012 CIDA endline survey, with 64 percent receiving first dose of ACT, compared to 27 percent receiving first dose of ACT in the 2016 RAcE endline. Also in the 2012 CIDA endline, 70 percent of fever cases treated by APE returned to the APE for follow-up, compared to 9 percent in 2016 RAcE endline. These differences cannot be explained by differences in stock levels and suggest that greater reinforcement of the protocols through supervision is required.

Given that the population served by APEs includes individuals who are already far from a health facility, it is not surprising to see the level of referral adherence unchanged over the course of the project, from 69 percent at baseline to 65 percent at endline. The primary reason caregivers did not adhere to the APE referral was because the facility was too far, emphasizing larger issues of poor access to care for many in Mozambique living far from health facilities, including many who live father than even the 25km reach of the APEs.

4.1 Implications

The RAcE project has completed implementation in Mozambique, and the MOH will continue to implement iCCM on its own. The findings of the household survey and APE survey highlight the critical importance of ensuring that APEs and health facilities are adequately supported to provide case management. The comparison of RAcE baseline and endline survey results demonstrates that caregivers seek care from APEs for iCCM conditions and that access to appropriate treatment in these remote areas is greatly improved. However, the iCCM program cannot exist without commodities, and the widespread stockouts recorded during the APE survey, particularly of antimalarial drugs and RDTs, help explain why 2016 RAcE endline survey results for assessment and treatment practices were weaker than those observed in the 2012 CIDA endline survey, when APEs were well-supplied. The handover of supply chain management responsibilities to the MOH under RAcE resulted in pervasive disruptions in stock of key iCCM commodities. Moving forward, the MOH and other partners will need to come together to find ways to sustain supplies to APEs or risk further declines in program performance.


ANNEX A. PEOPLE INVOLVED IN THE SURVEY

The following table provides a list of people involved in the survey and their roles.

Persons Involved Role

Ministry of Health and National Health Institute Dr Mbofana Principal Investigador Dr Chaquisse Coordination and Oversight Dr Jose Braz Chidassicua Coordination and Oversight Dr Acacio Sabonete Trainer and Data Quality Control Sergio Chicumbe Trainer Clementina Jacares Macondzo Provincial Survey Team Supervisor Nampula

Arminda Ubisse Provincial Survey Team Supervisor Manica Hesia Chilengue Provincial Survey Team Supervisor Inhambane and Zambezia Amilcar Magaço Provincial Survey Team Supervisor Zambezia Save the Children Marla Smith Oversight and Technical Eleanor Hill Coordination Lazaro Mondlane Logistics and Administration Fatima Ibo Logistics and Administration Malumbila Basilio Logistics and Administration Paula Nhambirre Logistics and Administration Tanya Gunther Technical Jeanne Koepsall Technical Ezequiel Barreto Technical (Mobile Devices for APE survey) Malaria Consortium Sonia Casimiro Trigo Logistics and Administration Helen Counihan Technical Ana Cristina Castel - Branco Technical ICF Meghan Swor Technical Debra Prosnitz Technical Kirsten Zalisk Technical


Data Collection Teams

Province Team Name Role

Nampula A 1 Maria Anabela Uaraca Supervisor

Benilto Marcolino Germano Enumerator

Manuel Jacinto Paulino Enumerator

Nampula B 2 Jeirezinho O.A. Calisto Supervisor

Alvaro Calisto E. Orlando Enumerator

Mequeza Abdul Carimo Enumerator

Nampula C 3 Sérgio Vieira Supervisor

Mamudo Assane Sualehe Enumerator

Joaquina Fernando Enumerator

Nampula D 4 Heraclito S. Bras Cassimo Supervisor

Salmata Braimo Selemane Enumerator

Abubacar Abilio Paconeta Enumerator

Zambézia A 5 Zarina Jose Cussinho Supervisor

Baciao Dinis Enumerator

Carlos Marcelo Neves Enumerator

Zambézia B 6 Antonio Cipriano Calia Supervisor

Amilcar Salvador Mole Enumerator

Vania Salvador Enumerator

Zambézia C 7 Anelita Emano Assane Supervisor

Rosario da Silva Valia Enumerator

Valdmiro Albino Xavier Enumerator

Manica A 8 Albertina Lucinda J Manuel Supervisor

Mequelina Alfredo Alice Enumerator

Marcelo Brito Fernandes Enumerator

Manica B 9 Silvia Damião Supervisor

Anastância Samuel Naene Enumerator

António Simões Julio Enumerator

10 Verónica Caetano Nhancalize Supervisor

Manica C Ana Rita Faustino Boane Enumerator

Paulino Jaime Gimo Enumerator

Inhambane A 11 Merton Andre Mueeombe Supervisor

Fatima Herminio Matandalasse Enumerator

Jorge Mario Binguane Enumerator

Inhambane B 12 Natercia Martins Langa Supervisor

Samuel Saranga Enumerator

Nace do Crescencio Rungo Enumerator


ANNEX B. ENDLINE SAMPLE

*-denotes districts where iCCM had been rolled out at baseline

Province District Community (Cluster)

Nampula

Angoche* Siretene/ Bairro Mutolo, replaced by Ntupo- Nparame due to death of APE Zoro/ Quarteirão E

Erati* Zuleme OU Zulume Malema* Murrumbo/ Nivatha

Memba

Regulo Napita/ Chefe Nivaca replaced by community Nehequene due to death of APE Reg Puejequele, no APE in this community so was replaced by Mangane Sirissa

Mogincual* Chapueia Mogovolas* Nanhotho Moma* Cotocuane/ Reg Machangula Mossuril Namuco Muecate Napacala/ Chinatane Murrupula* Muarrapaz

Nacaroa* Cabo Juliano Mavalane

Rapale Tchaine Ribaue* Namarepo

Zambezia

Alto-Molocue Yeheiya Nauela/ Milevane (community is called Guilherme)

Gile Namerico/ Uataria (ou Uaturia) Puampuela replaced with Quirule due to absence of APE in Puampuela

Inhassunge Muterreno/ Mussangane

Lugela Comone, replaced with adjacent community Mpemula due to security situation

Milange Muriamuendo Manica Vanduzi Dongo

Gondola

Nhamagoa Amatongas/ Chipangara Mussangaze/ Nhoane Gojombe Manhere

Inhambane Panda* Polana Zavala* Nhambele


ANNEX C. DETAILED SAMPLING DESIGN

Within each cluster, the survey team selected the first household for interview using the selection by subdivision approach.7 The survey team proceeded from one household to the next by visiting the household with its front door nearest to the front door of the current household, until the team conducted 10 interviews for each illness.

In all surveyed clusters in Manica and Nampula the survey teams found adequate cases of each illness. In Inhambane and Zambezia, there were some clusters (the exact number was not specified by INS) in which, after visiting all selected households, the survey team did not find 10 cases of each illness, so the team proceeded to the nearest enumeration area to find the number of illness cases still needed.

At each household, the enumerator first determined whether an eligible child lived there. An eligible child was aged 2–59 months and had been sick with diarrhea, fever, cough with rapid breathing, or any combination of the three illnesses in the two weeks preceding the survey.

The enumerator administered the questionnaire, including all applicable illness modules, to the caregiver of the eligible child. If more than one child was eligible, and they were sick with different illnesses, their caregiver was asked about each instance of illness. If the children had different caregivers, each caregiver was administered a separate questionnaire and answered questions about only his or her eligible child or children. If there was more than one eligible child in the household for an illness, the interviewer randomly selected one of the eligible children and interviewed his or her caregiver.

7 Selection by subdivision instructions: Go to the population center of the sample area and identify four quadrants. Assign each quadrant a number. Write the numbers on separate slips of paper and put them in a container. Randomly select one of the slips of paper. Go to the place that equally divides the selected quadrant’s population in half. Randomly select which of the two ways to proceed, and repeat this step until you have a manageable set of households (for example, 30). Count the households in the area. Use a random number table to select the first household for interview.


ANNEX D. ENDLINE SURVEY QUESTIONNAIRE

See attachment.


ANNEX E. ENDLINE SURVEY TRAINING SCHEDULE

Agenda de Formação do Estudo de Base do Projecto de RAcE-Mozambique

Maputo Mozambique

Centro de Reciclagem, 27 de Setembro-1 de Outubro

27 de Setembro: Dia 1 Hora Sessão e Objectivos Metodologia/Material Facilitador8:00-8:30 Introduções

Apresentar todos os membros da equipe do inquérito

Introduzir projecto RAcE e objectivos da pesquisa

Detalhes administrativos Provide all members of the

survey team Enter RAcE project and

objectives of research Administrative Details

MISAU/INS/SC

8:30-8:45 Abertura de Treino Training Opening

8:45-9:15 Desenho do Estudo Study Design INS 9:15-9:45 Aspectos éticos e Política de

Salvaguarda da Criança Ethical aspects and the Protection of the Child Policy

-Ficha de salvaguarda da criança ICF/SC

9:45-10:00 Pausa Lanche 10:00-10:45 Responsabilidades dos inquiridores

e supervisores Responsibilities of enumerators and supervisors

ICF

10:45-11:30 Procedimentos do trabalho de campo field work procedures

ICF

11:30-12:15 Seleção dos agregados familiares Selection of households

ICF

12:15-13:00 Seleção dos entrevistados Selection of respondents

Fichas de Controlo de Módulos de Doença

SC

13:00-14:00 Almoço

14:00-16:00 Prática da selecção dos agregados familiares e dos entrevistados

Practice of selection of households and respondents

INS + ICF + SC

16:00-16:45 Apresentação dos telemoveis e como usá-los

Manutenção Aplicação no telemóvel

Ezequiel Barreto


para randomização

Presentation of mobile phones and how to use them

• Maintenance

• Application on the phone to randomisation

16:45-17:00 Avaliação das sessões

Evaluation of sessions

28 de Setembro: Dia 2

8:00-8:30 Revisão dos pontos-chave de Dia 1

Review of the Day key points 1

1 supervisor, 1 inquiridor

8:30-10:00 Seleção dos agregados familiares e dos entrevistados

Selection of households and respondents

Cenários e Situações hipotéticas INS, com dois

supervisores

10:00-10:15 Pausa Lanche

10:15-11:00 Técnicas de entrevista

interview techniques

Os participantes trabalham em conjunto para elaborar uma lista de todos os aspectos importantes e técnicas para implementar uma entrevista. Depois, dois supervisores fazem dois dramatizações (boas e maus) e os participantes a identificar e discutir os pontos fortes e áreas de melhoria.

Participants work together to draw up a list of all the important aspects and techniques to implement an interview. Then two supervisors make two dramatizations (good and bad) and participants to identify and discuss the strengths and areas for improvement.

ICF

11:00- 12:00 Como Iniciar o questionário

Apresentações no agregado

Consentimento informado Prática no telemóvel

Presentations on aggregate

• Informed consent

• Practice on your phone

Ezequiel + ICF


12:00-13:00 Revisão do Módulos do questionário (no telemóvel):

Identificação das crianças Antecedentes do(a)

cuidador(a)

Quiz Modules Review (on the phone):

• Identification of children

• Background to the (a) caregiver (a)

Para cada perguntas, deve-se ter:

Consenso de como dizer em língua local

Fazer revisão das respostas para cada pergunta e as respostas em língua local

For each question, you should have:

• Consensus as saying in local language

• Do review of the responses to each question and answers in local language

INS

13:00-14:00 Almoço

14:00-14:45 Prática notelemóvel (primeiros módulos)

Como revisar o questionário preenchido

Practice on your phone (first modules)

• Reviewing the completed questionnaire

Em pares Ezequiel

14:45-16:00 Revisão 2 do Módulos do questionário (no telemóvel):

Tomada de decisões Conhecimento dos APE

por parte do cuidador(a) Conhecimento dos

cuidadores

Revision 2 of the questionnaire modules (on the phone):

• Decision-making

• Knowledge of the EPAs by the caregiver (a)

• Knowledge of caregivers

SC

16:00-16:30 Prática no telemóvel

Practice on your phone

Ezequiel

16:30-17:15 Simulação (em grupos de 3: 1 supervisor, 1 inquiridor, 1entrevistado; cada grupo implementa um/dois módulos; anota observações/desafios para partilhar com o grupo)

Os voluntários fazem uma simulação de uma entrevista (a partir de chegar a casa hipotética, e o grupo identifica e discute os pontos fortes e áreas de melhoria)

ICF


Simulation (in groups of 3: 1 supervisor, 1 inquirer, one respondent; each group implements one / two modules; notes comments / challenges to share with the group)



8:00-8:30 Revisão dos pontos-chave de Dia2 1 supervisor, 1 inquiridor

8:30-9:30 Revisão 3 do Módulos do questionário:

Diarreia

INS

9:30-10:00 Prática no telemóvel Ezequiel



Febre

ICF

11:15-11:45 Prática no telemóvel Ezequiel


Tosse com respiração rápida/ dificuldades de respiração

Prática no telemóvel

SC

13:00-14:00 Almoço

14:00-14:45 Revisão de documentos de apoio Cábula de seleccionar criança, Ficha de Controlo para inquiridores e supervisores, álbum de Medicamentos, Tabela de Idade das Crianças em Meses, Ficha de Listagem dos AF; anotação de observações pelos inquiridores

SC

14:45-16:15 Simulação (em grupos de 3: 1 supervisor, 1 inquiridor, 1entrevistado; cada grupo implementa um questionário inteiro

Supervisores praticam a observação com o Lista de Verificação para Controlo da Qualidade e dar retroinformação)

ICF

16:15-17:00 Como responder a cenários difíceis Trabalho em grupos INS




8:00-8:30 Revisão dos pontos-chave de Dia 3 1 supervisor, 1 inquiridor

8:30-9:00 Preparações para a prática no campo

SC

9:00-16:00 Prática Em equipas, realizar a selecção das famílias, a selecção dos entrevistados, o consentimento informado e entrevistas. Cada entrevistador vai ter pelo menos uma oportunidade de fazer a entrevista. Supervisores vão avaliar a implementação dos inquiridores com um formulário padronizado.

16:00-17:00 Revisão da prática no campo Cada supervisor e inquiridor compartilhará uma lição aprendida ou um desafio confrontada

INS +ICF

17:00-17:15 Avaliação do dia

1 de Outubro: Dia 5

8:00-10:00 Revisão das áreas de dificuldade durante a prática no campo

INS


10:15-11:00 Teste Rapido de aprendizagem

11:00-11:45 Como enviar os questonários preenchidos

Ezequiel

11:45-12:30 Simulação ICF

12:30-13:30 ‘Troubleshooting’ (Resolução de problemas) no telemóvel

Ezequiel

13:30-14:30 Almoço

14:30-15:30 Selecção dos inquiridores de Quelimane e designações dos equipas de campo e planificação do trabalho de campo (e logística)

SC/INS

15:30-16:00 Tempo livre para conhecer os membros do equipe, fazer perguntas/prática no telemóvel

16:00-17:00 Jogo de conhecimentos ICF

17:00-17:15 Avaliação do dia e da formação


ANNEX F. ENDLINE SURVEY FIELDWORK SCHEDULE

Manica

Equipe 1

Equipe 2

Equipa 3

Equipas 1,2, 3 Pre teste

SUPERVISOR: Silvia; INQUIRIDOR: Antonio e Anastacia

SUPERVISOR:Albertina; INQUIRIDOR: Miquelima, Marcelo

SUPERVISOR: Veronica; INQUIRIDOR: Ana Rita, Paulino

WED THU FRI SAT SUN MON TUE WED THU FRI SAT SUN MON TUES

DISTRITO POSTO

ADMINISTRATIVO COMUNIDADE5‐Oct 6‐Oct 7‐Oct 8‐Oct 9‐Oct 10‐Oct 11‐Oct 12‐Oct 13‐Oct 14‐Oct 15‐Oct 16‐Oct 17‐Oct 18‐Oct

Gondola CAFUMPE CAFUMPE

Vanduzi MATSINHO DONGO

Gondola INCHOPE MANHERE

Gondola CHIPINDAUMWE NHAMAGOA

Gondola AMATONGAS MBIA MBOGUE

Gondola MACATE MUSSANGANHAZE NHOANE

Gondola MARERA GOJOMBE


Nampula

Distritos

Datas e locais 10/10/201

6 11/10/2016 12/10/2016 2016‐13‐10 2016‐14‐10 2016‐15‐

10 2016‐16‐

10 2016‐17‐

10 2016‐18‐

10 2016‐19‐

10 2016/20/1

0

Moma Viagem à Moma Cotocuane Cotocuane

Angoche Ntupo –Naprume

Ntupo –Naprume Zoro Zoro Zoro Zoro

Mogovolas

Viagem à Mogovola

s Nanhoth

o Nanhoth

o

Erati Viagem à Erati Zulume Zulume

Nacaroa Cabo Juliano Cabo Juliano Mavalane Mavalane

Muecate Viagem à Muecate Napacala Napacala

Malema Viagem à Malema Murrumbo Murrumbo

Ribaue Namarrepo Namarrepo

Murrupula

Muarrapaz

Muarrapaz

Rapale Viagem à Rapale Tchaiane Tchaiane

Memba Viagem à Memba

Comunidade de Nehequene

Comunidade de Nehequene

Comunidade de Mangane

Comunidade de Mangane Sirissa Sirissa

Mossuril Namuco Namuco

Mogincual Chapueia


Zambezia

Inhambane

Equipa Inquiridores Supervisor Distritos 9/10/2016 10/10/2016 11/10/2016 12/10/2016 13/10/2016 14/10/2016 15/10/2016 16/10/21016 17/10/2016 18/10/2016 19/10/2016 20/10/2016 21/10/2016 22/10/2016 23/10/20106 24/10/21016 25/10/2016 26/10/2016 27/10/2016 28/10/2016 29/10/2016 30/10/2016

5Baciao Dinis & Carlos Marcelo Neves

Zarina Jose CussinhoAlto Molocue e

InhassungePre test

Travel

preperations Travel

Alto Molocue/

Yeheiya

Alto Molocue/

Yeheiya

Alto Molocue/

Yeheiya

Alto Molocue/

Yeheiya Rest day

Ferry Boat

under repair,

team waiting

to cross river

for Inhassunge

Ferry Boat under

repair, team

waiting to cross

river for

Inhassunge

Inhassunge/

Mussangano

Inhassunge/

Mussangano

Inhassunge/

Mussangano

Inhassunge/

Mussangano

Inhassunge/

Mussangano

Inhassunge/

Mussangano

Inhassunge/

Mussangano and

travel to

Quelimane

6Amilcar Salvador Mole &

Vania SalvadorAntonio Cipriano Calia

Alto Molocue e

GilePre test

Travel

preperations Travel Gile/Uataria Gile/Uataria Gile/Uataria Gile/Uataria Rest day

Gile/

Puampuela

Gile/

Puampuela

(substituted by

Quirule)

Gile/Quirule Gile/Quirule Gile/QuiruleGile/Quirule &

Gile/Uaturia

Travel to Alto

Malocue/

Guilherme

Alto Malocué/

Guilherme

Alto Malocué/

Guilherme

Alto Malocué/

Guilherme

Alto

Malocué/

Guilherme

Alto Malocué/

Guilherme

Alto

Malocué/

Guilherme

Travel to

Quelimane

7Rosario da Silva Valia & Valdimiro Albino Xavier

Anelita Assane Milange &

LugelaPre test

Travel

preperations Travel

Lugela/ Comone

(substituted with

Mpemula)

Lugela/

Mpemula

Lugela/

Mpemula

Lugela/

MpemulaRest day

Travel to

Milange

Milange/

Muriamuendo

Milange/

Muriamuendo

Milange/

Muriamuendo

Milange/

Muriamuendo

Milange/

Muriamuendo

Milange/

Muriamuendo

Milange/

Muriamuendo

Milange/

Muriamuendo

Team waiting

for data check

Team

waiting for

data check

Travel to

Quelimane

Equipa Inquiridores Supervisor Distritos 7/10/2016 8/10/2016 9/10/2016 10/11/2016 11/10/2016 12/10/2016 13/10/2016 14/10/2016 15/10/2016

11Fatima Herminio

Matandalasse & Jorge

Mario Binguane

Merton Andre Mueeombe

Panda & Zavala Pre testPanda/

Polana

Panda/

Polana

Panda/

Polana

Panda/

Polana &

Travel to

Zavala

Zavala/

Nhamble

Zavala/

Nhamble &

return to Polana

to re do

interviews with

errors

Panda/

Polana

Travel to

Inhambane

12Samuel Saranga & Nace do Crescencio Rungo

Natercia Martins Langa Panda & Zavala Pre testPanda/

Polana

Panda/

Polana

Travel to

Zavala

Zavala/

Nhamble

Zavala/

Nhamble

Zavala/

Nhamble

Zavala/

Nhamble

Travel to

Inhambane


ANNEX G. DETAILS OF DATA CLEANING AND ANALYSIS

The dataset submitted to ICF had a number of errors that required extensive review, additional cleaning, and some reconstruction of data. This is due to two major errors in the field. First, some enumerators conducted interviews with caregivers who did not have a child who had been sick in the two weeks prior to the survey; these caregivers were asked the caregiver-specific modules but not the sick child modules despite not being selected for the survey. Second, many enumerators did not record the cluster number, household number, caregiver number, or child number correctly in each questionnaire module as they moved through the survey, meaning that ICF had to go back through and manually correct these to ensure that questionnaire records were correctly linked. The following is a summary of steps taken to clean the RAcE Mozambique endline survey data:8

Manually added sex, age, and two-week illness history information for children missing this information, when available.

Manually removed records in the Excel dataset that did not fall within valid data collection dates (e.g., practice interviews).

Manually updated incorrect cluster numbers, household numbers, child line numbers, and caregiver numbers in the Excel dataset, across all modules.

Manually deleted duplicate records. Corrected the child line numbers in Module 1, the listing of selected children, to align with their

line numbers in the household child roster. This was done in part manually in the Excel dataset and in part via a Stata RAcE Moz Endline Cleaning do file.

Corrected child line numbers in the diarrhea, fever, and fast breathing modules to align with the line number in Module 1. This was done in part manually in the Excel dataset and in part via a Stata RAcE Moz Endline Cleaning do file.

Dropped records for which there were no sick children included in the survey.

The following is a summary of errors and corrections made to baseline survey data during comparative analyses at endline:

In 2014, caregiver knowledge indicators were calculated using a denominator of 625, which included caregivers without a sick child included in the survey. In 2017, revising these data for comparison with endline data, any caregivers without sick children included in the survey were excluded. This reduced the baseline denominator to 527. For the caregiver knowledge of CCM-trained APE role indicator, the denominator was changed to 327 because only caregivers who reported having an APE in their community were asked the associated survey questions.

For the first source of care indicators, caregivers who sought care from an APE were incorrectly included in the numerator in the 2014 analysis, whether or not they sought care from an APE first. This was corrected in 2017 by removing caregivers who did not seek care from an APE first from the numerator.

8 A log of all of the manual updates is contained in “Mozambique endline survey dataset update log 12.21.2016.xlsx.” A list of modules available by cluster and household in the final cleaned dataset is contained in “Mozambique Cluster Cleaning Workbook 12-16-2016.xlsx.”


In 2014, the fever treatment coverage indicators did not account for the “prompt” component—the same or next day—in the calculation. In 2017, this indicator was recalculated to include only those who also sought care “promptly” or “within same or next day.”

In 2014, 53 caregivers for whom zinc treatment information was missing were included in the denominators for diarrhea treatment coverage. These caregivers were omitted in the updated analysis in 2017.

In 2014, the first dose of treatment from an APE indicator was calculated with all sick children included in the survey as the denominator. This was corrected in 2017 by including only sick children who received treatment from an APE in the denominator.

In 2017, the sick-child follow-up indicator was corrected to specify whether an APE visited the child for follow-up regardless of timeline. In 2014, the calculation included any follow-up within three days.


ANNEX H. KEY INDICATORS FOR FULL PROJECT AREA

The following are key indicators for the sample of 32 clusters representing the full project area across all four provinces: Inhambane, Manica, Nampula, and Zambezia.

Indicator Endline% (CI %)

End of Project Sample: Inhambane, Manica, Nampula, ZambeziaCaregiver Knowledge

1

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who are aware of the presence of the CCM-trained APE in their community

93.9 (84.4 - 97.7)

2

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who know the role of the CCM-trained APE in their community

68.1 (59.0 - 76.1)

3

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who know two or more signs of childhood illness that require immediate assessment by an appropriately trained provider

92.4 (87.3 - 95.5)

Caregiver perceptions of iCCM services

4

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who view CCM-trained APEs as trusted health care providers

78.4 (71.1 - 84.2)

5

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who believe CCM-trained APEs provide quality services

77.4 (71.4 - 82.5)

6

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who found the CCM-trained APE at first visit

83.0 (75.1 - 88.8)

7

Percentage of caregivers of children age 2-59 months who have been sick in the two weeks preceding the survey who cite the CCM-trained APE as a convenient source of treatment

81.7 (73.1 - 88.0)

Sick Child Care-seeking

8

Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey for whom advice or treatment was sought from an appropriate provider

Overall78.8

(72.8 - 83.7)

Fever78.0

(71.5 - 83.3)

Diarrhea76.5

(68.2 - 83.1)

Fast breathing82.0

(75.3 - 87.2)

9

Percentage of children age 2-59 months who were sick in two weeks preceding the survey taken to a CCM-trained APE as first source of care

Overall56.5

(48.5 - 64.2)


Fever55.1

(46.2 - 63.6)

Diarrhea59.3

(50.1 - 67.9)

Fast breathing55.2

(47.0 - 63.1) Sick Child Assessment

10 Percentage of children age 2-59 months with fever in the two weeks preceding the survey who had finger or heel stick

55.0 (41.8 - 67.6)

11

Percentage of children age 2-59 months for whom their caregiver received the results of the malaria diagnostic test of the children who had had finger or heel stick in the two weeks preceding the survey

93.9 (88.2 - 96.9)

12

Percentage of children age 2-59 months with cough and difficult or rapid breathing (suspected pneumonia) in the two weeks preceding the survey who had their respiratory rate counted to assess fast breathing

38.2 (29.1 - 48.1)

Sick Child Assessment by APE

13

Percentage of children age 2-59 months with fever in the two weeks preceding the survey who had a finger or heel stick by an APE (among those who sought care from an APE)

55.0 (41.8 - 67.6)

14

Percentage of children age 2-59 months for whom their caregiver received the results of the malaria diagnostic test of the children who had a finger or heel stick by an APE in the two weeks preceding the survey (among those who sought care from an APE)

96.2 (90.5 - 98.5)

15

Percentage of children age 2-59 months with cough and difficult or rapid breathing (suspected pneumonia) in the two weeks preceding the survey who had their respiratory rate counted to assess fast breathing by a CHW (among those who sought care from an APE)

39.2 (27.9 - 51.8)

Sick Child Treatment

16

Percentage of children age 2-59 months who have been sick in two weeks preceding the survey who received appropriate treatment

Overall49.6

(42.9 - 56.3)

Malaria (ACT)*72.1

(61.9 - 80.5)

Diarrhea (ORS and Zinc)31.2

(22.3 - 41.7)

Fast breathing (Amoxicillin)58.0

(50.7 - 65.1)

17

Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey who received appropriate treatment from a CCM-trained APE

Overall29.2

(23.0 - 36.2)

Fever36.2

(25.8 - 48.1)

Diarrhea22.9

(15.5 - 32.5)


Fast breathing32.5

(25.8 - 40.0)

18

Percentage of children age 2-59 months who have been sick in the two weeks preceding the survey who received the first dose of treatment in the presence of an APE among those who received prescription medicines for a CCM condition in the two weeks preceding the survey

Overall32.2

(23.5 - 42.3)

Fever26.1

(16.6 - 38.6)

Diarrhea16.0

(6.3 - 35.0)

Fast breathing49.5

(36.5 - 62.6)

19

Percentage of sick children age 2-59 months for whom their caregivers received counseling on how to provide the treatment(s) among those who received prescription medicines for a CCM condition in the two weeks preceding the survey

Overall96.3

(92.5 - 98.2)

Fever98.9

(92.0 - 99.9)

Diarrhea96.0

(87.3 - 98.8)

Fast breathing94.2

(86.2 - 97.7) Sick Child Referral and Follow-up

20 Percentage of sick children age 2-59 who were referred in the two weeks preceding the survey whose caregiver adhered to referral advice

65.6 (55.7 - 74.4)

21

Percentage of sick children age 2-59 months receiving treatment from an APE in the two weeks preceding the survey who received a follow-up visit from an APE according to country protocol

41.0 (32.5 - 50.2)

22

Percentage of sick children age 2-59 months receiving treatment from an APE in the two weeks preceding the survey whose caregiver followed-up with an APE

9.7 (5.7 - 16.0)

*Numerator: Cases of fever with a positive blood test who received ACT within 24 hours; Denominator: cases of fever with a positive blood test.


ANNEX I. SUMMARY OF FINDINGS FROM RAcE MOZAMBIQUE ENDLINE APE SURVEY

INTRODUCTION As part of the endline evaluation, Save the Children conducted a survey of APEs to assess the implementation strength and quality of the iCCM services they delivered. The APE survey was conducted with the endline household survey to assess care-seeking practices and treatment coverage for iCCM conditions. The objective of the APE survey was to gain a better understanding of the APEs’ background characteristics, activity levels, and support and supervision received to help interpret the results of the coverage survey. The APEs serving the 32 clusters selected for the endline household survey formed the sample population for the APE survey, and all were interviewed. Data collection was carried out during October–November 2016. RESULTS A detailed indicator table is provided in Table A. The main findings are summarized below. APE profile: Most sampled APEs were male (78 percent), and the majority had seven or more years of education. About two-thirds of APEs were aged 40 years and younger. As a result of the household survey sampling design, the APEs interviewed were primarily from Nampula province, with only small numbers sampled in Inhambane, Manica, and Zambezia. APE catchment areas: The majority of APEs (91 percent) resided in their catchment areas. The main modes of transportation were walking (59 percent) or bicycle (31 percent), and most APEs reported that it took one or more hours to reach the nearest health post. Catchment areas were geographically large, with 34 percent of APEs reporting traveling more than 2 hours and 41 percent travelling 1–2 hours to reach the farthest house within their catchment areas. Supervision: Most APEs (84 percent) reported receiving a supervisory visit in the last 3 months. District coordinators9 and health facility supervisors were the most frequently mentioned providers of supervision. Among APEs who reported supervision, the majority reported that the supervisors used a supervision checklist and that records and kit and medical supply availability were reviewed. A smaller percentage of APEs mentioned that the supervisor had observed sick child care or talked with village leaders. About two-thirds of APEs (68 percent) reported receiving clinical supervision in which sick child care was observed in the last 3 months. Availability of iCCM medicines and diagnostics: Only 44 percent of APEs had at least one age formulation of first-line antimalarial drugs (Coartem; artemether-lumefantrine 1x6 or 2x6) in stock on the day of observation. Most APEs were observed to have amoxicillin (81 percent), ORS (91 percent), zinc (81 percent), and paracetamol (81 percent) in stock. Three-quarters (75 percent) of APEs had RDTs, and 78 percent had a functional stopwatch. Overall, only 19 percent of APEs had all iCCM medicines and supplies available on the day of the survey, and 37 percent had all essential iCCM medicines and supplies.10 Stockouts were common, particularly for artemether-lumefantrine, RDTs, and amoxicillin. Only 28 percent of APEs reported that they had continuous stock of artemether-lumefantrine (1x6 or 2x6), 44 percent for RDTs, and 53 percent for amoxicillin in the month before the

9 There were both RAcE district supervisors and MOH district supervisors providing supervision at time of the study, but the questionnaire did not capture information to distinguish between the two. 10 Artemether-lumefantrine (at least 1x6 or 2x6), amoxicillin, ORS, zinc, RDTs, and functional timer


survey. Overall, only 16 percent reported no stockouts of essential iCCM medicines and RDTs in the past month. APE functionality: All APEs had provided iCCM services in the past month and reported providing iCCM services at least two days per week. However, only 41 percent met a stricter definition of “functionality,” in which they resided in their catchment areas, provided iCCM services in the last month, and reported providing iCCM at least 5 days per week. Activity levels and record-keeping: All APEs reported providing iCCM services at least 2 days per week, and nearly half (47 percent) reported providing iCCM 5 or more days per week. Register reviews indicated that in the last month, APEs treated an average of 16 malaria cases (range 0–70), 7.9 diarrhea cases (range 0–28), and 7.7 pneumonia cases (range 0–35). Overall, APEs treated an average of 31.6 iCCM cases per month (median 32.0; range 0–109). The majority (71 percent) of APEs had referred one or more sick children for danger signs in the last month. Completeness of recording of cases in APE registers was lacking, with 77 percent of APEs recording RDT results for the 5 most recent fever cases, and 71 percent with respiratory rates recorded for the 5 most recent cough cases. APE knowledge of danger signs and iCCM protocols: APE knowledge of danger signs for which they should refer a sick child was low, with only 13 percent able to cite all 4 of the most serious or general signs (vomiting everything, convulsions, not conscious/lethargy, and not able to drink/breastfeed). Three-quarters of the APEs (75 percent) said that they would count breaths per minute to assess for fast-breathing, and 69 percent said that they would ask whether the child had difficult or rapid breathing. About 53 percent said that they would ask about cough, and 19 percent would check for fever. All but two APEs (94 percent) correctly indicated that they would provide amoxicillin to a child determined to have pneumonia, and 91 percent gave the correct response for duration of treatment of malaria with Coartem (3 days). APE data use: Nearly 60percent of APEs (19 of 32) reported being trained in data use, and 47 percent were observed to have completed charts with data filled out for the past month. About 13 percent (4 of 32) had the chart books but did not fill them in. IMPLICATIONS These findings have several implications for the iCCM program in Mozambique:

District coordinators were the most frequently mentioned provider of supervision; however it was not possible to distinguish between RAcE-funded district coordinators and MOH district coordinators. As the RAcE program transitions activities to the MOH, the RAcE-funded district coordinator position will be discontinued, but the MOH district coordinators will continue. Further efforts will be needed to strengthen the capacity of health facilities to provide regular supervision for APEs. The MOH has already revised the policy to recommend quarterly supervision (previously supervision was monthly) to help make supervision more feasible; however, the lack of financial resources for transport and the distance to some remote APEs may continue to present challenges to reaching all APEs with routine supervision. Save the Children purchased 40 new motorcycles for supervision visits and will donate the existing motorcycles to the MOH. UNICEF also purchased 250 motorcycles for the APE program. Issues will continue to be the following: (1) fuel, maintenance, and per diem costs for the supervision visits; (2) prioritization of the motorcycles for the supervision visits when there is such a transportation need in the health facilities (3) time for the supervisions by the health facility supervisors because there is still a lack of medical staff at health facilities.


APEs cannot provide quality iCCM services without adequate and continuous stocks of medicines. Less than half of APEs had any artemether-lumefantrine available on the day of the survey, and nearly three-quarters reported stockouts of artemether-lumefantrine in the past month. Stockouts of RDTs and other essential iCCM medicines were also common. Medicines are still provided to APEs based on a kit system, in which supplies are not determined based on consumption. Mozambique has faced widespread shortages of antimalarial drugs and has struggled to maintain supplies at first-level health facilities. APE stocks have frequently been compromised. Systemic strengthening of the supply chain is urgently needed in Mozambique to avoid ongoing stockouts at the APE level, and there is a need to manage consumption data.

Most APEs were living in their catchment areas and providing iCCM on a regular basis, with nearly half providing iCCM at least five days per week. Activity levels were relatively high, with APEs treating more than 30 iCCM cases per month. Completeness of registers for recording of RDT and respiratory assessment results could be improved through reinforcement during supervision and data use training.

The knowledge of general danger signs and of assessment for pneumonia cases was markedly lower than in previous surveys of APEs. Only 13 percent of APEs interviewed could cite all 4 general danger signs, compared to 97 percent of APEs interviewed as part of the endline survey for the CIDA-funded iCCM program in Nampula in 2012. Reasons for these lower knowledge results are unclear. Under the previous program, supervision levels were higher and often provided through Save the Children staff with clinical training, and they may have focused more on reinforcing clinical skills. A quality of care study conducted in January 2016 found that only 25 percent of sick children assessed by APEs were assessed for all 4 general danger signs, providing further evidence of knowledge and practice gaps for general danger signs.

Table A. Summary of indicators for iCCM service delivery by APEs (n=32) Domain Indicator ResultResidency % of CCM-trained APEs residing in their catchment area 91% (73–97%) Functionality % of CCM-trained APEs who are functional (MOH definition):

i) have provided iCCM services in past month ii) report operating village clinic for at least two days/week

100%

% of CCM-trained APEs who are functional (stricter definition): i) reside in their catchment area ii) have provided CCM services in the past month iii) reports operating village clinic for at least five days per week

41% (25–59%)

Medicine and diagnostics availability

% of APEs with all key CCM medicines and diagnostics in stock on day of assessment (artemether-lumefantrine, amoxicillin, ORS, zinc, RDTs, timer)

38% (20–55%)

Artemether-lumefantrine (1x6 or 2x6) 44% RDTs 75% Amoxicillin 81% ORS 91% Zinc 81% Timer 78% Paracetamol 81% Graphic notebook (data use) 56%

% of APEs with all medicines and diagnostics in stock on the day of assessment (all above medicines and supplies)

19% (8–37%)

% of APEs reporting no stockouts of essential iCCM supplies lasting seven days or more in the month before the survey (artemether-lumefantrine 1x6 and 2x6, RDTs, amoxicillin, ORS, zinc)

16% (6–37%)

Artemether-lumefantrine (1x6 and 2x6) 28% RDTs 44% Amoxicillin 53% ORS 75% Zinc 63%


Domain Indicator ResultSupervision % of APEs who received at least one supervision session during the prior

three months during which registers were reviewed 84% (66–94%)

I. Reviewed treatment records 81% II. Checked medical supply quantities 75%

III. Used supervisory checklist 78% IV. Observed care/treatment for sick child 63% V. Talked to village leaders 47% VI. Talked to you about clinical activities 72%

% of APEs who received at least one mentorship session during the prior three months with observation of case management (clinical supervision)

69% (50–83%)

Service availability and activity levels

% of APEs who report typically operating their village clinic: i) <2 days/week 0% ii) 2+ days/week 100% iii) 5+ days per week 47% Number of days APEs report operating village clinic per week Mean: 4.5

Median: 4 Number of sick child cases treated in the last one month Mean: 31.6

Median: 32.0 Range: 0–109

I. Malaria cases Mean: 16.0 Range: 0-70

II. Suspected pneumonia cases Mean: 7.7 Range: 0-35

III. Diarrhea cases Mean: 7.9 Range: 0-28

% of APEs who referred one or more sick child cases for danger signs in the last 1 month

71% (52–85%)

Recording completeness

% of APEs with complete recording for use of RDTs for five most recent cases presenting with fever

77% (58–89%)

% of APEs with complete recording of RR for five most recent cases presenting with cough/difficult breathing

71% (52–85%)

APE knowledge

% of APEs who can cite all four general danger signs 13% I. Convulsions 84%

II. Lethargic or unconscious 22% III. Cannot eat or drink 63% IV. Vomits everything they eat or drink 66%

% of APEs who know how to classify a case of pneumonia (all four steps) 9% I. Ask if child has cough 53%

II. Count child's respiratory rate 75% III. Check if child has fever 19% IV. Ask if child has difficult or rapid breathing 69%

% of APEs who can cite the correct duration of treatment for artemether-lumefantrine 91% (73–97%)

Data display and use

% of APEs who were trained on the data use package 59% (41–76%) % of APEs trained in the data use package (n=19) who had template filled with data for last one month 79%

final mozambique race endline survey report 31march17 ... · race endline survey final report v...

Documents