end of project evaluation of health behaviour … · at the end of the ec-funded health behaviour...

1

END-OF-PROJECT EVALUATION OF HEALTH BEHAVIOUR CHANGE COMMUNICATION PROJECT IN CAMBODIA

2011

2

Evaluators:

Name: Eng Peou Title: Managing Director, Market Strategy & Development Co., Ltd (MSD) Cambodia

Phone: +855-12-675-266 Email: [email protected]

Name: Pavithra Ram Title: Public & Social Research, Taylor Nelson Sofres (TNS) Vietnam

Phone: +84-4-3987-7031 E-mail: [email protected]

This evaluation was funded by the European Commission and UNICEF and conducted in 2011, at the end of the EC-funded Health Behaviour Change Communication project. Its purpose is to assess the initiative’s performance and document good practices and successes, generate evidence-based lessons and recommendations, and further strengthen ongoing efforts, new initiatives (including possible programme replication) and expansion in the area of behaviour change communication. The views expressed in this publication do not necessarily reflect the views of the European Commission or UNICEF.

mailto:[email protected]

mailto:[email protected]

3

ACKNOWLEDGEMENTS The evaluation team is grateful for the participation of the European Commission; current and former staff of the National Centre for Health Promotion, National Maternal and Child Health Centre, and sub-national level stakeholders (provincial health departments, provincial health promotion units, operational districts, and health centre staff, including midwives); commune chiefs; commune committees for women and children; village health volunteers and women from the community who participated. Additionally, the team is also thankful to the Reproductive and Child Health Alliance (RACHA), MEDiCAM and BBC World Service Trust representatives who contributed their opinions and experiences.

4

LIST OF ACRONYMS ABCP advanced behaviour change communication for practitioners

ANC antenatal care

BBC WST British Broadcasting Corporation World Service Trust

BCC behaviour change communication

BCC-TWG Behaviour Change Communication Technical Working Group

CAS Cambodia Anthropometrics Survey

C-IMCI Community Integrated Management of Childhood Illness

CDHS Cambodia Demographic and Health Survey

CHV community health volunteer

DHPU district health promotion unit

EC European Commission

HSSP Health Sector Support Project

IDI in-depth interview

IEC information, education and communication

IMC integrated modular course

KAP knowledge, attitudes and practices

MCH maternal and child health

MDG Millennium Development Goal

MEDiCAM umbrella organization for NGOs working in child survival in Cambodia

M&E monitoring and evaluation

MoH Ministry of Health

MPA minimum package of activities

NCHP National Centre for Health Promotion

OD operational district

PBCI provider behaviour change intervention

PHD provincial health department

PHPU provincial health promotion unit

PSC project steering committee

UNFPA United Nations Population Fund

UNICEF United Nations Children’s Fund

VHSG village health support group

VHV village health volunteers

WHO World Health Organization

5

EXECUTIVE SUMMARY BACKGROUND The importance of changing the behaviour of both health providers and their clients was recognized and highlighted in Cambodia’s first Health Sector Support Project (HSSP I) 2003–2007, which aimed to improve the sector’s capacity and performance through measures targeting the poor, particularly in rural areas, and lessening the impact of infectious diseases and malnutrition. Three strategies directly related to behaviour change communication (BCC) were emphasized in the HSSP I:

1. Improve attitudes of health providers sector-wide to effectively communicate with consumers, especially regarding the needs of the poor, through sensitization and enhanced interpersonal communication skills

2. Empower consumers, especially women, to interact with other stakeholders in the development of quality health services through mass media and interpersonal communication

3. Promote healthy lifestyles and appropriate health-seeking behaviours through advocating for healthy environments and implementing counselling and behavioural change activities

The idea of a health BCC project was initiated in March 2003 after a team of consultants recommended that the EC’s support to the health sector in Cambodia focus on strengthening BCC. In November 2003, an EC formulation mission laid the foundations for the project, proposing that it be located within the existing structures of the Ministry of Health’s (MoH) National Centre for Health Promotion (NCHP), with provincial health promotion units (PHPUs) as direct implementers. The project sought to strengthen the capacity of national, provincial and district health care providers to deliver effective health promotion and education services by improving skills, attitudes and behaviours with clients. In this way, the project could substantially improve the overall quality of health services, while also building awareness among the public of key health care practices, especially those affecting mothers and children. The EC subsequently signed an agreement with the United Nation’s Children’s Fund (UNICEF) in December 2004 to execute the BCC project and provide technical assistance and financial support to NCHP. UNICEF was the co-financing partner with the EC, based on its child health implementation structure, which is an important BCC component.

The Health BCC project, jointly supported by the EC and UNICEF, was initiated on 1 January 2005 and had a total budget of 5.7 million euros over a six-year period. Originally, the project duration was five years, ending 31 December 2009. A one-year no-cost extension was provided (based on recommendations from the 2008 Midterm Report) for the project to transition from EC funding to other support and to allow results to be consolidated. As the initiative was nationwide and involved various partners at the national and sub-national levels, implementation of activities was guided by five entities:

6

Project Steering Committee: In 2005, the BCC Project Steering Committee (PSC) was established. A high-level committee, the PSC was chaired by the secretary of MoH with regular participation of representatives from the BBC World Service Trust (BBC WST), MEDiCAM (an umbrella organization for NGOs working in child survival in Cambodia), the EC delegation, United Nations Population Fund (UNFPA), UNICEF and World Health Organization (WHO). The committee’s main purpose was to provide policy guidance, facilitate coordination among all institutions and groups involved, and advise NCHP in implementing and harmonizing planned activities.

BCC Technical Working Group: With representatives from MEDiCAM, MoH, the Ministry of Rural Development, UNFPA and WHO, the key role of the BCC Technical Working Group (BCC-TWG) was to review the existing national BCC policy, revise as necessary, and advocate for its nationwide use and implementation; consult with national programmes and other health partners to coordinate BCC activities, including joint fundraising and pooling of resources for national campaigns; position NCHP to act as a technical advisory body on BCC and build the health workforce’s capacity on good BCC practices; monitor the quality of BCC materials and campaigns produced in Cambodia to ensure accordance with national BCC policy, protocol and guidelines; support the establishment of provincial BCC forums to coordinate BCC activities at the provincial level; and participate in key activities pertaining to the revision and finalization of the Community Integrated Management of Childhood Illness (C-IMCI) training package and related BCC materials.

NCHP working teams, including teams for monitoring and evaluation (M&E), information services, training, protocols and guidelines, and management.

BCC forums were established across provinces to serve as a focal point for information sharing, planning and organizing activities, including coordination of BCC activities, BCC-related trainings, health promotion campaigns (to plan and implement province- and community-based BCC activities on a wide variety of health issues, including breastfeeding, diarrhoea, dengue, food hygiene and vitamin A supplementation) and fundraising events. Forum members included provincial health departments (PHDs), PHPUs, NGOs, operational districts (ODs), health centres, village health volunteers (VHVs), commune councils and village chiefs.

UNICEF staff members, including a technical officer, assistant project officers and project assistants from the communications and child survival teams, provided technical support and financial management.

EVALUATION OBJECTIVES, METHODOLOGY AND FINDINGS

Evaluation objectives

The main goals of the evaluation were:

1. To assess the project’s performance using standard evaluation criteria of relevance/appropriateness, effectiveness, efficiency, impact (potential) and sustainability

2. To document good practices and successes, to generate evidence-based lessons learned and recommendations, and to guide the way forward toward further

7

strengthening ongoing efforts, new initiatives (including possible programme replication) and BCC expansion

It is intended that the primary users of the evaluation will be MoH (in particular NCHP), the EC and UNICEF. In addition, the findings of the antenatal care (ANC) BCC project evaluation will be shared with a broader group of interested communication and development partners, including NGOs.

Evaluation methodology

The assessment was conducted to ensure participation of stakeholders at the national, provincial, district and village levels. Methodology comprised three phases:

1. The desk review phase included comprehensive assessments of all project documents, log frames, EC monitoring reports, UNICEF annual progress reports, work plans, midterm reviews, a training needs assessment, a baseline survey on community health volunteers (CHVs) and data from the Cambodia Anthropometrics Survey (CAS) 2008 and the Cambodia Demographic and Health Survey 2005.

2. During the national-level phase, key people were interviewed from NCHP, EC, UNICEF and the PSC. These interviews built understanding of the relevance of the BCC project in the Cambodian context, the effectiveness of implementing the project, the challenges of working with multiple partners, and perspectives on sustainability.

3. During the provincial-level phase, interviews were conducted with representatives of PHDs, PHPUs, ODs, maternal and child health (MCH), district health promotion units (DHPUs), health centre staff and village health support groups (VHSGs). A baseline survey on key family practices was conducted among community and village health volunteers to obtain end-line data on knowledge, attitudes and practices (KAP) in terms of the 12 key family practices1. Some additional questions about the volunteers’ role in the BCC project were also included. Paired interviews were conducted with community members (women with children) to understand their health-seeking behaviours, triggers and barriers.

Summary of evaluation findings

The evaluation’s overall conclusion is that the Health BCC project was relevant to the context and largely effective in contributing to Cambodia’s health promotion needs. The project was successful in meeting its objective as defined by the national-level log frame: strengthening national BCC policymaking and NCHP’s capacity to support BCC at the national and provincial level. At the provincial levels, BCC training was provided to PHPUs and health providers. Health centre staff and VHSGs were also trained on counselling and interpersonal communication.

1 The 12 key family practices are: 1. Exclusive breastfeeding for the first six months of life; 2. Begin complementary

feeding at six months with continued breastfeeding until 2 years or more; 3. Provide children with foods rich of meat, vegetables and fruit (iron-, iodine- and vitamin A-rich foods) everyday and give vitamin A every six months; 4. Use sanitary toilet or bury stool and wash hands with soap or ash before cooking, eating, feeding infants and after using the toilet (do the same for your child); 5. Bring your child to have full immunization in the first year; 6. Sleep under an insecticide-treated net to prevent malaria, especially for children and pregnant women in areas with high incidents of malaria; 7. Ensure healthy mental and social development for the child; 8. Increase feeding children when they are sick and continue feeding them more for a week after they have recovered; 9. Home care of sick children (diarrhoea, cough/cold, fever); 10. Recognize danger signs that need treatment and care at health centre; 11. Follow trained health worker’s advice on treatment of child illnesses, follow-up and referral for health care services for some dangerous diseases; and 12. Pregnant women should have four ANC visits and safe delivery and continue to have postnatal care.

8

Though the evaluation was unable to quantify the extent to which NCHP-developed BCC tools and resources were used, it showed that most provincial-level health providers found the training and IEC materials to be useful. The strongest evidence of the project’s success can be seen in the planning and implementation of the nationwide ANC BCC campaign that was launched in 2009. Other notable achievements include revision and dissemination of the national BCC policy, establishment of BCC forums in all provinces and the roll-out of C-IMCI training across provinces. Despite these accomplishments, the project also faced several key challenges. The fact that NCHP had no budgetary oversight resulted in decreased ownership of the project, especially from a long-term point of view as NCHP staff perceived the project to be time-bound and operating with limited funding. Second, and most importantly, NCHP’s long-term role and MoH’s vision for the institution as a pivotal point for health promotion remains unclear. Apart from these challenges, other issues pertained to staff turnover, low staff motivation due to low remuneration, etc. While the BCC resources developed during the initiative have wide applicability, the sustainability and expansion of the project’s BCC activities are doubtful due to the lack of a long-term approach for health promotion in Cambodia and necessary government funds. At the provincial level, while the benefits of health provider and provincial staff training can be sustained, expansion of the activities will be difficult. It is therefore imperative for NCHP to expand its donor base and leverage additional resources. Overall, BCC is a strategic undertaking – a vision of a process of social transformation – and dealing with it as a project has been complicated. Despite the complex nature of the initiative itself, including partnerships between bilateral and multilateral organizations and an institute within a sector, the project can be largely considered a success.

Project relevance

Overall, the project addressed Cambodia’s BCC health sector needs as set forth in the sector strategic plan and related policies. Behavioural change at service delivery and usage levels is essential for progress towards health-related Millennium Development Goals (MDGs). While three BCC strategies were given prominence in HSSP I, in the second HSSP, BCC was not mentioned explicitly.

A key aspect that the project covered included building service providers’ knowledge and interpersonal skills (including counselling), thus establishing a first step towards encouraging health-seeking behaviours in the community. However, it is difficult to determine if the NCHP capacities developed under the project will continue to be relevant as vertical health programmes within MoH departments continue to be involved in BCC activities, thus diminishing NCHP’s role as a focal point for health promotion. It is imperative for NCHP to begin establishing itself as the key institution for BCC activities in health.

9

Project efficiency

In general, the national-level coordination mechanism was not the most efficient due to its complex structure and hierarchies. The fact that NCHP had to wait for MoH approval to pursue key strategies, coupled with the institution’s limited capacities, proved to be a hindrance when developing a long-term vision of NCHP as a key institution for health promotion. Though the PSC was formed to provide policy guidance, facilitate coordination among institutions and groups, and advise NCHP on implementation and organizational activities, interviews with PSC members revealed that the entity largely served as a platform for project updates. PSC members not among the key partners (EC, UNICEF, NCHP) did not play an important role in lending expertise to the project, other than providing advice/feedback on key messages. The PSC was also considered too big to be effective. Utilization of funds was a concern during the five-year project, with only 30 per cent of funds employed at the end of the third year. However, by 2008, the fourth year of the project, 80 per cent of funds were utilized as activities were accelerated and accomplished. A one-year no-cost extension was provided to allow for the utilization of funds and ensure sustainability of key BCC activities.

Project effectiveness

During the project, roll-out of the nationwide ANC BCC campaign was a key achievement in Cambodia’s health BCC. The campaign was highly successful in achieving its behavioural goal of increasing the proportion of women who access early ANC (<8 weeks), with a significant increase from less than 6 per cent at baseline to over 35 per cent post-campaign. As the ANC BCC campaign was the first practical application of the BCC policy, protocol and guidelines, this is truly a notable achievement and indicates the campaign’s potential impact and sustainability.

The national BCC policy was reviewed and revised by NCHP, endorsed by MoH and widely disseminated, including protocols for developing BCC materials and guidelines. This key achievement supports the sustainability of BCC activities, as these guidelines provide a foundation for the planning and implementation of health promotion campaigns/programmes outside the scope of this project, as evidenced in the ongoing complementary feeding programme. The project was successful in meeting its objective as defined by the national-level log frame: strengthening national BCC policymaking and the capacity of NCHP to support BCC at the national and provincial level. At the provincial level, BCC-related training was provided to PHPUs and health providers, while health centre staff and VHSGs received counselling and interpersonal communication training. As stated previously, though the evaluation did not demonstrate the extent to which NCHP-developed BCC tools and resources were used, most health providers found the training and IEC materials to be useful. Training modules, especially on C-IMCI, client/provider rights and interpersonal communication, were instrumental in changing care providers’ attitudes at health centres. Interviews with community members revealed an attitudinal change among care providers at health centres. As key agents of change and influence in the community, the impact of VHSGs was also important, as they were able to reach people who do not use health centres. Through

10

door-to-door counselling and interpersonal communication, the groups shared knowledge on key family practices, which was well received by the community. While the BCC project effectively met its project objective, it failed to influence collaboration between MoH structures and among other line ministries, local authorities, NGOs, civil society organizations and various national and sub-national players. Vertical departments within MoH continue to exist, each implementing their own BCC activities. While the PSC and BCC-TWG aimed to include various national-level stakeholders, those who were not key members were largely uninvolved.

Project impact

Based on the assessment of the ANC campaign and interviews with community members during the evaluation, it was found that the project did result in overall better health-seeking behaviours. The BCC forums were extremely important in providing a platform to discuss and spread awareness about health issues at the community level. One example is the successful way in which the ANC campaign was implemented in Kampot province through the BCC forums, as Kampot was not considered a key intervention province for the campaign’s interpersonal communication component. Improved service delivery at health centres in the intervention provinces was another key aspect addressed during this project. Following training on counselling and interpersonal communication, attitudes of service providers at health centres changed for the better, resulting in better service delivery and thereby positively impacting overall health-seeking behaviours.

Project sustainability

Overall, the sustainability of the project at the national level remains questionable due to the absence of a long-term approach to health promotion in Cambodia, the uncertainty of NCHP’s future as the focal point for behaviour change and health promotion, and the low remuneration of MoH staff.

The future of provincial-level activities also remains a concern despite substantial capacity building. On a positive note, there is great ownership of activities at the provincial and lower levels. The BCC policies, guidelines and communication tools developed under this project, including all training and campaign materials, are very pertinent. They have been employed for the ANC campaign and are now being used for the complementary feeding program jointly implemented by MoH’s NCHP and National Maternal and Child Health Centre. It is hoped they will be implemented for future BCC campaigns and programmes. It is encouraging to note that NCHP has been active in writing proposals to seek funds from other sources, such as the Global Fund to Fight AIDS, Tuberculosis and Malaria. Though this proposal was not accepted, the fact that NCHP applied is a positive indication of its capabilities and intentions to find additional funding. In 2009, an additional US$350,000 from the MDG Spanish Fund and the second HSSP was made available for priority BCC activities in 2010.

RECOMMENDATIONS

11

Based on the evaluation, the following recommendations have been made for future activities:

1. NCHP should proactively collaborate with other vertical health programmes to integrate and coordinate BCC activities. This would require the institution to promote itself as the focal point for health promotion. As NCHP has the experience of working on two large BCC campaigns (ANC and complementary feeding), it should be able to promote itself as a capable, professional entity equipped to implement BCC campaigns.

2. NCHP should continue to seek other sources of funding by actively writing proposals for different funds. The evaluation noted that NCHP had already begun this process by writing a proposal to the Global Fund. Though the funds were not granted, NCHP views the experience as a lesson learned in improving future proposals.

3. Given that NCHP has a role in fundraising, it should consider to build capacities of its

staff in proposal writing and results-based reporting in order to create successful proposals.

4. While the primary role of NCHP should be health promotion, the evaluation found that NCHP’s capacity as a lead institution in development and management of social research is high. Partners who worked with the institution on research projects have been impressed with staff abilities. Though this is outside the realm of health promotion, the NCHP skill base built over the last 10 years should be utilized. NCHP should therefore continue to nurture a separate research team for health development and promotion.

5. If an organizational restructure is not possible in the future, the NCHP team structure under the BCC project should be implemented institution-wide (i.e. teams for M&E, information services, protocols and guidelines, training and management), as the departments are more relevant than the current divisions (primary health care, tobacco and health, etc.).

6. NCHP should continue to monitor and supervise activities under the Health BCC project to ensure that activities do not dissipate at the sub-national levels. So far, sub-national involvement has been strong in most aspects and it is imperative to ensure this continues. However, as funding under this project is over, it is necessary that NCHP include M&E (such as training, home visits) in its annual work plans.

7. The BCC forums have been very effective and should be continued in as many

provinces as possible. It is recommended that this activity be integrated into PHPUs’ annual work plans and budgets. Pooled funds should be used to support the forums, as some provinces have already done.

8. Refresher trainings should be conducted, particularly among district, health centre and village staff. As these activities are no longer supported by the project and therefore require funding, trainings should be integrated into regular activities on a less formal/theoretical level and on a more on-site level.

9. When IEC materials are developed or used for other activities, the main materials used at health centres should employ large pictures. The project demonstrated that text-heavy

12

brochures have a smaller impact, as only those who can read benefit from them, and are only effective when complemented by other materials.

Recommendations for future projects involving multiple partners:

10. Ensure that similar projects in the future have a smaller and more effective PSC that includes people with relevant expertise. PSC should also be involved more in the decision-making process.

11. Ensure that all project partners are on board with relevant roles and profiles in the similar projects involving multiple partners in the future.

12. In the future, responsibilities of the key implementing partner should include some budget management so that financial management capacities for a large-scale project are built. More reporting structures may be needed, especially when involving financial reporting between bilateral organizations and government departments.

13

TABLE OF CONTENTS I. INTRODUCTION 1.1 Background and context 1.2 Objectives and log frame 1.3 Project management arrangements 1.4 Monitoring and evaluation

II. EVALUATION 2.1 Evaluation purpose 2.2 Intended users of the evaluation 2.3 Evaluation methodology 2.4 Evaluation team 2.5 Evaluation limitations III. EVALUATION FINDINGS 3.1 Project relevance 3.2 Project efficiency 3.3 Project effectiveness 3.4 Project impact 3.5 Project sustainability IV. CONCLUSIONS AND RECOMMENDATIONS 4.1 Conclusions 4.2 Recommendations ANNEXES ANNEX 1: LOG FRAME OF HEALTH BCC PROJECT 2007 ANNEX 2: LOG FRAME OF HEALTH BCC PROJECT 2010 ANNEX 3: TOR ANNEX 4: Discussion Guides ANNEX 5: FINDINGS FROM THE VHV END-LINE SURVEY ON 12 KEY FAMILY PRACTICES

14

I. INTRODUCTION

1.1 Background and context 1.2 Objectives and log frame 1.3 Project management arrangements 1.4 Monitoring and evaluation

15

I. INTRODUCTION 1.1 Background and context In 2002 Cambodia’s Ministry of Health identified behaviour change communication as one among a set of key strategies to improve the health of women and children in the country. The European Commission and UNICEF therefore jointly established ‘The Health Behaviour Change Communication (BCC) Project’ providing technical assistance and 5 million Euros to strengthen the national health promotion capacity between 2005 and 2009. The implementing partner was Cambodia’s National Center for Health Promotion (NCHP) with 60 staff based in the capital Phnom Penh, supported by Provincial Health Promotion Units (PHPU) in each of the 24 provinces. While nationwide in scope, the BCC Project was in particular focused on demonstrating good BCC practices in seven of the least developed provinces. Activities were implemented in close collaboration with Provincial Maternal and Child Health Managers and Supervisors. The idea of a health BCC project was initiated in March 2003 after a team of consultants recommended that the EC’s support to the health sector in Cambodia focus on strengthening BCC. In November 2003, an EC formulation mission laid the foundations for the BCC project, recommending that it be carried out within the existing structures of MoH’s NCHP and with PHPUs as direct implementers. The project sought to strengthen the capacity of national, provincial and district health care providers to deliver effective health promotion and education services by improving skills, attitudes and behaviours with clients. In this way, the project expected to substantially improve the overall quality of Cambodia’s health services, as well as build greater awareness among the public of key health care practices, especially those affecting mothers and children. The EC signed an agreement with UNICEF in December 2004 to execute the BCC project and provide technical assistance and financial support to NCHP. UNICEF co-financed the initiative due to its expertise in child health, an important BCC component, and the fact that it was working closely with UNFPA on reproductive health2.

This financial partnership between the EC and UNICEF was expected to increase the project’s effectiveness and result in a more effective, efficient use of resources through pooled funding and better donor coordination, thereby further supporting MoH’s sector-wide management approach.

The Health BCC project, jointly supported by the EC and UNICEF, was initiated on 1 January 2005 and had a total budget of 5.7 million euros over a six-year period. The original project duration was five years, ending 31 December 2009. Based on recommendations from the Midterm Report 2008, a one-year no-cost extension was provided for the project to transition from EC funding to other support and to allow achievements and results to be consolidated. The importance of changing the behaviour of both health providers and their clients was recognized and highlighted in the first HSSP 2003–20073. The HSSP I stated4:

2Midterm Report 2008.

3As BCC does not have the same prominence in the second HSSP (2008–2015), it must be noted that the Health

BCC project was initiated based on HSSP 1.

16

- More attention to behavioural change is required to improve household health practices

of consumers and caregivers with regard to nutrition, the use of preventive measures and healthy lifestyles.

- More efforts are needed to promote important behaviours such as immunization, contraceptive use, breastfeeding, oral rehydration therapy, the use of trained providers for delivery and treatment of childhood infections, and means of protection against major infectious diseases, such as condom use and bed nets for vector control.

- At all service delivery levels, poor communication between providers and consumers is a major obstacle in promoting family health.

- Poor quality within the public sector – from the perspective of both professionals and consumers – has deterred effective utilization of basic health services.

- Limited resources, poor dissemination of clinical standards and limitations in technical and counselling skills have affected client trust in providers.

Three strategies directly relating to BCC were given prominence in HSSP I.

Figure 1: Strategies related to BCC as described in HSSP I

1.2 Objectives of the Health BCC project and log frame The overall objective of the project was to promote healthier lifestyles, better health-seeking behaviours and improved delivery of primary health care services to reduce poverty in Cambodia.

These objectives were to be achieved by improving the capacity of NCHP and PHPUs in their role as advocates, communicators and change agents for service providers, who were to promote healthy practices among families through four strategies (Figure 2). Figure 2: Four key strategies of the Health BCC project

4HSSP 2003–2007.

Outcome: Appropriate practices and healthy lifestyles as a result of informed decisions, especially by women

Change for the better the attitudes of health providers sector-wide to effectively communicate with consumers, especially regarding the needs of the poor, through sensitization and building interpersonal communication skills

Empower consumers, especially women, to interact with other stakeholders in developing quality health services through mass media and interpersonal communication

Promote healthy lifestyles and appropriate health-seeking behaviour through advocating for healthy environments and implementing counselling and behavioural change activities

17

Though a logical framework using SMART5 indicators was not finalized in 2006, it was identified as a priority and a revised version by NCHP and UNICEF was submitted with the 2007 annual work plan. According to a March 2009 EC monitoring mission report, “The project is using a logical framework that was last updated in 2007 to strengthen the clear, objectively verifiable indicators. Overall, the log frame is of reasonable quality.” The logical framework was further revised in 2010 to include additional verification indicators. Both the 2007 and 2010 logical framework documents are included in the Annex while Table 1 provides a summary of the log frame.

Table 1: Snapshot of the key indicators of the log frame Intervention logic Objectively verifiable indicators Project purpose

The improvement of national, provincial and district health providers’ delivery of effective BCC in Cambodia

- The national BCC policymaking capacity is strengthened - The capacity of NCHP to support implementation of BCC services at national and provincial levels is strengthened - 50% of PHPUs and other key stakeholders use the BCC tools and resources developed by NCHP to implement BCC activities at national or provincial levels - 50% of district-level primary health care providers in seven provinces (Kampong Speu, Prey Veng, Svay Rieng, Stung Treng,

5 Specific, measurable, achievable, realistic and time-bound

- Strengthen national BCC policymaking

- Build the capacity of NCHP in providing the following services:

•Information services on BCC

•Training of BCC specialists

•Development of protocols to standardize the production of BCC materials

•Development of guidelines to ensure the quality of BCC

•Evaluation of BCC activities

- Build the capacity of PHPUs in all of Cambodia's 24 provinces to implement BCC activities, as well as of officials/staff in other ministries and NGOs that implement health-related BCC activities in their respective areas

- Implement intensive BCC activities in five selected provinces (later increased to seven) to provide a model for good BCC practices

18

Mondolkiri, Kampong Thom, Oddar Meanchey) have improved attitudes and skills to engage in quality health education and BCC with consumers, especially the poor - 50% of target population in seven provinces reached with NCHP- and PHPU-supported BCC campaigns and activities

Overall objective

Healthier lifestyles, better health-seeking behaviours and improved delivery of aspects of primary health care services, contributing to the reduction of poverty of the Cambodian population

To contribute to the reduction of infant mortality, maternal mortality, household expenditure on health and other health-related indicators as specified in the MoH Sector Strategic Plan

Expected results

1. National BCC policy revised and distributed to key stakeholders

1. National BCC policy revised and endorsed by the government and key stakeholders

2. NCHP has capacity to provide four key BCC services

2. NCHP acquires knowledge and skills in providing four key BCC services

a) Provision of information on BCC activities for health in Cambodia

a) BCC forums established in 24 provinces and used for information sharing

b) Specialist BCC training modules for national-, provincial- and district-level providers

b) Integrated modular course (IMC) developed and pretested. Minimum package of activities (MPA) Module 7 on health education for health workers revised to include training on BCC. C-IMCI course for VHSGs revised to include BCC training

c) Development and distribution of protocols for BCC materials production; development and distribution of BCC quality guidelines

c) Protocol for development of BCC materials and guidelines for BCC implementation developed and distributed to 24 PHPUs and key BCC stakeholders

d) M&E of BCC activities

d) System for M&E of NCHP and PHPU BCC activities developed and in use. Monitoring tools developed and used by NCHP and PHPU to monitor BCC progress and achievement

3. PHPUs in demonstration provinces have capacity to plan, implement and monitor BCC at provincial level, including training and supervision of health providers

3. All PHPUs in seven provinces have participated in training and supervision of health providers on advanced BCC for practitioners (ABCP), MPA, C-IMCI, use of protocol for BCC materials production and guidelines for BCC activities, and M&E reports to be sent to NCHP M&E teams

4. Health providers of selected districts in demonstration provinces have improved BCC skills to promote behaviour change

NCHP-supported BCC/health promotion campaigns and VHSGs of selected districts have participated in and passed revised C-IMCI training

1.3 Project management arrangements As the Health BCC project was nationwide and involved various national and sub-national partners, the implementation of project activities was guided by five entities:

19

Project Steering Committee In 2005, the high-level BCC PSC was established with the EC delegation; MEDiCAM; UNFPA; UNICEF; WHO; MEDiCAM; MoH departments of preventive medicine, human resources, and planning; and NCHP, also the secretariat, as members. The secretary of MoH acted as chair. BBC WST regularly participated and other stakeholders were invited as necessary.

The committee met twice annually (within the first half of the year and in December) to:

provide policy guidance to the project

facilitate coordination among all institutions and groups involved in the project

advise NCHP in its implementation and coordination of planned activities During the meetings, committee members reviewed the project’s mid-year status and progress, discussed bottlenecks, agreed on collective means to overcome them, and approved the work plan for the following year. The PSC met a total of 11 times (most recently on 3 September 2010); meeting highlights are presented in Table 2.

Table 2: Snapshot of key conclusions of PSC meetings, 2005–2010 Year Key conclusions

2005 The annual operational plan/BCC annual action plan was agreed on in principle. The official work plan was to be signed by the EC, UNICEF, NCHP and MoH after completion of a detailed narrative. Decentralization of IEC production was to be considered in coordination with NCHP monitoring and its protocols, guidelines and generic messages and after provincial staff capacity was built. It was agreed that NCHP should be a lead agency on BCC. NCHP staff capacity was to be strengthened.

2006 Participants were informed of progress and lessons learned and the direction of NCHP’s annual 2007 work and operational plans. NCHP received several constructive comments from meeting attendees. Key points were explained, clarified and solved during the meeting. A procedure to accelerate budget disbursement was proposed and accepted by Mr. Rodney Hatfield, UNICEF Representative. All participants appreciated the results of the meeting.

2007 Working relations between UNICEF, EC and NCHP were discussed to reduce bureaucratic obstacles. It was suggested that an external midterm report be organized to examine project achievements and ways for the project to be redirected so as to support NCHP’s long-term development as a sustainable institution.

2008 It was agreed to revise the project budget and the 2008 annual work plan to include provision of supplies and BCC materials to enable all 24 PHPUs to expand BCC activities in 2009. Participants reaffirmed that NCHP is the key implementing agency of the project and encouraged ownership of project activities and action to achieve project goals. Resources and materials developed with project support would be finalized and disseminated by end 2008. Year-to-date project progress was reviewed and discussed and the proposed 2009 annual work plan was approved.

2009 Participants discussed accelerating the C-IMCI package roll-out; strengthening NCHP’s M&E function with regard to BCC activities, including documentation of lessons learned and good practices in order to inform future communication efforts; finalizing the NCHP Strategic Development Plan 2011–2015; and proceeding with the request for a no-cost extension of the Health BCC project until 31 December 2010.

2010 Progress made in 2009 was reviewed in order to approve the proposed 2010 annual work plan. The meeting highlighted 2009 BCC achievements, particularly those related to the national ANC communication campaign. It was also noted that the final draft of the NCHP Strategic Plan was ready for external consultation with development partners before finalization.

20

BCC Technical Working Group

In 2005, the BCC project reactivated the BCC-TWG, under NCHP guidance. The working group’s terms of reference were revised. Roles and functions were defined as follows:

- Review the existing national BCC policy for further revision if necessary and advocate for its nationwide use and implementation

- Consult with national programmes and other partners working in health to strengthen coordination of BCC activities, including possibilities for joint fundraising and pooling of resources for nationwide BCC campaigns

- Position NCHP to act as a technical advisory body on BCC and build capacity of the health workforce on good BCC practices

- Monitor the quality of BCC materials and campaigns produced in Cambodia to ensure accordance with national BCC policy, protocol and guidelines

- Support the establishment of provincial BCC forums in collaboration with PHDs to coordinate BCC activities at the provincial level

The working group also participated in key activities relating to the revision and finalization of the C-IMCI training package and related BCC materials. BCC-TWG members included representatives from MEDiCAM, the Ministry of Rural Development, MoH, UNFPA and WHO.

Table 3: BCC-TWG meetings Year

2005 BCC-TWG reactivated. 2006 BCC-TWG convened four times and was instrumental in revising the group’s terms of reference,

finalizing the 2006 annual work plan, developing guidelines for BCC forums, providing advice on major workshops on topics such as policy review, launching the BCC website and monitoring the progress of BCC project plans.

2007 The BCC-TWG reviewed and organized dissemination of protocol and guidelines for BCC activities and provided technical input for BCC forum activities and coordination. Information sharing between the BCC-TWG and BCC forums was strengthened and used to provide technical support to PHPUs and provincial-level BCC partners. The working group advocated for the use of BCC policy and guidelines to other key stakeholders at the national and provincial level.

2008 BCC-TWG met five times in 2008 to discuss the project and its challenges with two midterm review consultants; discuss how key partner organizations and stakeholders could contribute and collaborate on provision of the ABCP training package; review the final draft guidelines for BCC activities for health and the protocol for development and distribution of BCC materials; share input and ideas on BCC project progress and challenges with a monitoring mission from the EC; and review project progress for 2008 and contribute to the review and finalization of the draft 2009 annual work plan.

2009 BCC-TWG met twice in 2009. During the first meeting, first quarter project progress was reviewed and key issues to be presented to the PSC were identified. The discussion focused on ANC campaign achievements, constraints and lessons learned; BCC forum outputs and good practices; and the early draft of NCHP’s strategic plan. A subsequent meeting reviewed progress of the first three quarters against annual objectives and PSC recommendations. Preparations were made for the annual review and planning meeting between NCHP and PHPUs.

NCHP working teams

To efficiently implement BCC activities, the NCHP formed technical working teams in August 2005, assigning NCHP staff to work in units based on their technical areas.

21

Table 4: NCHP and the BCC project NCHP Internal Organization BCC Project Organization

Primary Health Care M&E

Tobacco or Health Environmental Health and Hygiene Information Services Health Promotion and Training Training Health Communications and IEC Protocols and Guidelines Management Team Management Team

In the first quarter of 2006, the roles and specific functions of these teams were defined more clearly:

Figure 3: NCHP Technical Teams

BCC forums

For efficient implementation of BCC interventions in the provinces, BCC forums were established in each province to serve as an avenue for information sharing, planning and organizing the following activities:

•oversee information provision of health BCC activities in Cambodia to BCC stakeholders Information Services

(IS)

•specialize in development and provision of BCC trainings for national-, provincial- and district-level providers, BCC managers and implementers Training

•develop standard protocols and guidelines to ensure the quality of BCC-related activities and disseminate these guidelines for nationwide application by all BCC actors

Protocol and Guidelines (P&G)

•conduct BCC-related research and monitoring and evaluation of BCC activities Monitoring and

Evaluation (M&E)

•provide overall coordination of NCHP's BCC activities and supervise the five work teams

Management

22

- coordination of BCC activities - BCC-related trainings - health promotion campaigns (to plan and implement province- and community-based

BCC activities on a wide variety of health issues, including breastfeeding, diarrhoea, dengue, food hygiene and vitamin A)

- fundraising events

Members of the BCC forums included6:

- PHD: director and deputy director - PHPU: secretariat - national programmes (malaria, nutrition, immunization): one representative each - health NGOs: two representatives - OD: two representatives - health centre staff: two representatives - VHV: two representatives - private health sector: two representatives - commune council: two representatives - village chief: two representatives

These forums were established in five BCC demonstration provinces as a priority in 2006 and then expanded to all provinces. In June 2008, MoH endorsed guidelines for BCC forums and by the end of 2009 BCC forums had been established and were functioning in all 24 provinces. The forums provide PHPUs with a mechanism to effectively plan, coordinate, implement and monitor BCC activities in collaboration with key stakeholders at the provincial level.

Figure 4: Process of BCC forums

6Certain member categories are present in some provinces but not in others as membership formation is up to the

discretion of individual PHD and PHPU teams.

Pla

tfo

rm fo

r id

en

tify

ing

pri

ori

tie

s

The BCC forums provide the PHDs with a mechanism to effectively plan, coordinate, implement and monitor BCC activities in support of priority health issues in the province.

Pro

vin

cia

l-le

ve

l a

ctio

n p

lan

Following identification of the priority health or nutrition issue to be addressed through communication, PHPUs collaborate with key provincial-level stakeholders to develop a communication action matrix plan, which includes a description of communication activities, responsible people, timelines and budgets. Im

ple

me

nta

tio

n o

f th

e p

lan

Following the forum, PHPU staff coordinates and monitors implementation of the plan at provincial, district, health centre and community levels, with support from local authorities.

23

UNICEF team

Initially, to support NCHP in implementing the BCC project, two UNICEF staff members – one international project manager officer and one national assistant project officer – worked full-time with NCHP. In the first quarter of 2006, a project officer from UNICEF’s communications team mobilized NCHP in developing avian influenza communication materials to be presented as a joint effort of NCHP/UN. During the reporting period, four assistant project officers and five project assistants from UNICEF’s Child Survival Programme provided technical and operational support on all activities conducted at national and provincial levels. In addition, the head of child survival provided oversight on project management and implementation and on liaising with high-level MoH officials and representatives of other health development partners. In total, two UNICEF staff based in Phnom Penh provided full-time support to the project while 11 staff provided part-time support at national and provincial levels7.

After the full-time key technical person from UNICEF left in 2009, he was replaced briefly by a consultant. From 2009, UNICEF’s child survival and communication sections provided technical support to NCHP and managed financial processes with NCHP and the EC.

1.4 Monitoring and Evaluation Monitoring of project progress at the national level

The EC monitored the project annually and the results of their reports, including observations and recommendations, were shared with NCHP, UNICEF and the PSC. UNICEF provided a full narrative annual report to the EC to accompany the funding request for the following year. A midterm review was conducted in 2008 to examine project achievements; identify difficulties, bottlenecks and solutions to address them; consider performance, support and management issues both within NCHP and in its relationships with UNICEF and the EC; gather views and experience of NCHP and of stakeholders, including EC, UNICEF and other donors and NGOs that have, or have had, a relationship with NCHP; review NCHP’s position as an intrinsic part of MoH in Cambodia; understand the view that NCHP has of itself, its mission and its plans for the future; and review the project’s potential sustainability in terms of both the topics delineated in the project and BCC methodology within NCHP8.

Support for monitoring of BCC activities With technical support from a training institution, the Protocol and Guidelines team outlined the various steps to be followed in the development and implementation of quality BCC activities, including BCC campaign monitoring and evaluation of BCC interventions. These draft guidelines were first shared with BCC partners through two consultative workshops in 2006. In December 2006, 24 PHPUs pretested the revised guidelines in their respective provinces. The training institution, in collaboration with NCHP’s M&E team, developed guidelines for a mechanism to monitor BCC-related training activities, resulting in a database and questionnaire. In 2007, after further refinement, the M&E team used this database to monitor selected BCC training activities.

7 UNICEF Annual Report, 2006.

8Midterm Review, 2008.

24

In 2009, the ABCP training package was offered to 17 PHPU staff, one NCHP representative, one representative of MoH’s Preventive Medicine Department and three NGO representatives. This course improved staff capacity in the planning, implementation and monitoring of BCC activities.

In addition, two NCHP staff – a deputy director and the technical bureau chief – attended the Results-Based M&E course offered by IMA International in Brighton, UK.

Provincial-level monitoring and supervision Feedback meetings were held every six months for PHPU staff on issues related to monitoring and supervision, including use of BCC checklists (MPA 7, client/provider rights, provider behaviour change intervention (PBCI) and C-IMCI). These meetings were held to respond to PSC recommendations with the goal of strengthening PHPUs’ monitoring and supervision capabilities. In 2009, between two and four BCC forums were organized in each province, with the main goal of mobilizing a coordinated BCC response by a wide set of stakeholders to a priority health or nutrition issue. This was considered an important outcome and a significant step towards coordinating the implementation and monitoring of provincial-level activities. In 2009, NCHP also organized two workshops with the participation of all PHPUs to discuss strengths and weaknesses of the BCC forums, to document lessons learned and to develop recommendations improving the organization, follow-up implementation and M&E of the BCC forums.

25

II. EVALUATION

2.1 Evaluation purpose 2.2 Intended users of the evaluation 2.3 Evaluation methodology 2.4 Evaluation questions 2.5 Evaluation team 2.6 Evaluation limitations

26

II. EVALUATION

2.1 Evaluation purpose

An independent evaluation at the end of the project was planned as part of the initial health BCC proposal. The specific objectives of the evaluation were as follows:

To assess the project’s performance using standard evaluation criteria of relevance/appropriateness, effectiveness, efficiency, impact (potential) and sustainability

To document good practices and successes, to generate evidence-based lessons and recommendations, and to guide the way forward toward strengthening ongoing efforts, new initiatives (including possible programme replication) and expansion of BCC

The evaluation referred to four main activities (as described in Figure 2) in order to assess the initiative’s degree of achievement and performance: strengthen national BCC policymaking; build NCHP capacity in five key areas; build PHPU capacity across the 24 provinces to implement BCC activities, as well as of officials/staff in other ministries and NGOs that implement health-related BCC activities in their respective areas; and implement intensive BCC activities in five provinces (later increased to seven) to provide a model for good BCC practices.

Based on a specification brief delivered by NCHP and UNICEF (refer to the Annex), TNS Vietnam, an organization specializing in public and social research, and its Cambodian partner, MSD, were contracted to conduct this evaluation.

2.2 Intended users of the evaluation

Primary users of this evaluation will be MoH (in particular NCHP), EC and UNICEF. In addition, findings will be shared with a broader group of interested partners, including NGOs, working in the area of communication and development.

It is hoped that the good practices and lessons learned will be used by MoH and relevant stakeholders for the development, implementation and monitoring of other health BCC activities and that the evaluation results will inform future communication programme design and implementation efforts undertaken by MoH and interested partners.

2.3 Evaluation methodology

The methodology used is illustrated in Figure 5. Its goal was to ensure participation of all stakeholders at the national, provincial, district and village level.

27

Figure 5: Evaluation methodology

Desk review

The purpose of the desk review was to obtain a thorough understanding of the Health BCC project through existing documentation. It was also conducted to establish a comprehensive overview of the various partners and activities involved in the project. On commissioning the study, detailed discussions were held with UNICEF representatives who advised that a thorough review of the following documents was essential:

- EC monitoring reports - annual progress reports - UNICEF annual reports - log frame - work plans - midterm reviews

•EU monitoring reports

•UNICEF annual progress reports and annual reports

• log frame, annual work plans

•midterm review, training needs assessment, baseline survey (CHVs)

•CAS 2008, CDHS 2005 and 2010

Desk review

•NCHP

•EU

•UNICEF

•PSC

National-level interviews

•PHPU

•PHD Interviews with provincial staff

•OD

•health centre staff

•VHSG

•commune chief/CCWC members involved in BCC forums

Interviews with sub-national staff

•women with children under 2 years old Interviews with community

members

28

- training needs assessment - baseline survey, CHVs - CAS 2008, CDHS 2005 and 2010

National-level interviews At the national level, key interviewees represented:

- NCHP (former and current staff) - EC delegates - UNICEF - PSC members (RACHA, MEDiCAM, BBC WST)

The main purpose of these interviews was to understand the project’s relevance in the Cambodian context, the effectiveness of implementing the project, the challenges of working with multiple partners, and perspectives on the project’s sustainability. The evaluation methodology used was in-depth interviews, which provided a confidential, one-on-one environment in which perceptions, attitudes and experiences could be thoroughly understood and key areas of improvement identified for future campaigns. Attitudes and perspectives on the project’s quality may not have been openly discussed in a group environment. Stakeholders to be interviewed (Table 5) were discussed with NCHP and UNICEF prior to the evaluation. It was also proposed to interview the secretary of MoH, His Excellency Professor Eng Huot, but this interview could not take place due to the professor’s busy schedule. Interviews with former deputy directors of NCHP could not be conducted as the evaluation team was unsuccessful in establishing contact with them.

29

Table 5: Sample size–national level

Provincial-level interviews

Figure 6 (on the next page) shows the provinces covered during the evaluation, which represent five of the seven good practice demonstration provinces9. Two additional provinces, where UNICEF does not have staff, were also selected in order to observe if there were any differences in implementation of Health BCCs.

9 The selected provinces with UNICEF staff presence were Ratanakiri, Stung Treng, Svay Rieng, Mondolkiri,

Kampong Speu, Prey Veng, and Oddar Meanchey.

Organization Name Title 1 EC Mr. Christian

Provoost Attaché, Head of Unit, Health, Education and Social Development

2 Current NCHP staff

Dr. Khun Sokrin Director of NCHP 3 Dr. Sin Sovann Technical Bureau Chief 4 Dr. Ouk Sokun Technical Bureau Deputy Chief 5 Dr. Ya Kim Sean Head of IEC 6 Dr. Viratey Kim Head of M&E 7 Former NCHP

staff Dr. Lim Thai Pheang Former Director of NCHP

8 National Maternal and Child Health Centre

Dr. Koum Kanal Director

9 UNICEF Ms. Isabelle Austin Deputy Representative

10 Ms. Viorica Berdaga Chief of Child Survival

11 Ms. Ung Vanny Health Education Officer

12 Dr. Tan Try Communication for Development Specialist

13 Mr. Marc Vergara Chief of Communication

14 Ms. Bossadine Uy M&E Officer

15 Mr. Tomas Jensen Health Education Specialist

16 MEDiCAM Mr. Sin Somony Executive Director

17 RACHA Ms. Chan Ketsana Child Health Team Leader

19 BBC WST Mr. Charles Hamilton Director/Project Coordinator

30

Figure 6: Geographic coverage of evaluation

Representatives from the following provincial-level bodies were interviewed: PHPUs: PHPUs are responsible for promoting health in their respective province and report to the PHD and NCHP. As part of the Health BCC project, PHPU staff underwent training for capacity building. Hence it was imperative to interview them in detail about the importance of the project and the impact it has had on their capacity to plan, monitor and implement. They were also asked about the impact of the BCC project on the capacities of their staff (at the OD- and health-centre levels).

PHDs: PHD staff were interviewed to assess the project’s impact on staff capacity at the OD- and health-centre level. The departments are also responsible for budget allocation at the provincial level, so it was important to get their perspective on the sustainability of some project activities, such as BCC forums. Interviews were conducted with both the PHD chief/vice-chief and the PHD MCH since the 12 key family practices and C-IMCI that the Health BCC project focused on relate to maternal and child health.

MAIN PROVINCES

Stung Treng

Svay Rieng

Mondolkiri

Kampong Speu

Prey Veng

CONTROL PROVINCES

Kratie

Sihanoukville

31

Table 6: Sample size–provincial level

District-level interviews

At the district level, both OD-DHPU and OD-MCH were interviewed, as one is in charge of health promotion and the other for health care services. ODs are important in the implementation of the BCC project as they are responsible for training and monitoring health centres. In-depth interviews using semi-structured interview guides were used.

Table 7: Sample size–district level

Good practice demonstration provinces

OD-DHPU OD-MCH TOTAL

Kampong Speu 1 1 2 Stung Treng 1 1 2 Mondolkiri 1 1 2 Prey Veng 1 1 2 Svay Rieng 1 1 2 5 5 10 Control provinces

Kratie 1 1 Sihanoukville 1 1 2 2

TOTAL 12

Commune-level interviews

At the commune level, the following people were interviewed:


PHD chief/ vice-chief

PHD MCH PHPU chief/ vice-chief

TOTAL

Kampong Speu 1 1 1 3 Stung Treng 1 1 1 3 Mondolkiri 1 1 1 3 Prey Veng 1 1 1 3 Svay Rieng 1 1 1 3 5 5 5 15 Control provinces Kratie 1 1 1 3 Sihanoukville 1 1 1 3 2 2 2 6

TOTAL 7 7 7 21

32

Health centre staff: The health centres were the key beneficiaries of the Health BCC project as the project focused on improving skills, attitudes and behaviour with clients in order to improve overall quality and delivery of primary health care/health care services. Both the health chief and midwife of two health centres were interviewed. The health centre chief was interviewed to get his/her overall perspective on the project and its impact on health providers, challenges, further needs, etc. Midwives were interviewed as they were key beneficiaries of the C-IMCI, ANC and MCH activities. Midwives are also seen as leaders in communities and capacity building at that level was an important aspect of the project. Table 8: Sample size–health centre level


Health chief Midwife TOTAL

Kampong Speu 2 2 4 Stung Treng 2 2 4 Mondolkiri 2 2 4 Prey Veng 2 2 4 Svay Rieng 2 2 4 10 10 20 Control provinces

Kratie 1 1 2 Sihanoukville 1 1 2 2 2 4

TOTAL 24

VHVs: VHVs are instrumental in interpersonal communication with the community. VHVs/CHVs underwent relevant training in order to improve their communication skills and thereby spread awareness of health-related issues and to encourage community members to practice positive health-related behaviours. A baseline survey on the 12 key family practices was conducted among the volunteers. It was therefore proposed to administer the same questionnaire to get end-line data on the KAP of the VHVs/CHVs with regard to the family practices. In addition, interviews with the volunteers allowed the evaluation to gauge which outreach activities they were aware of and which of the following required further information:

- the BCC project - IEC materials - home visits - impact on the community

Since the end-line survey was not a key objective of the current evaluation, it was not possible to conduct it with same baseline sample size. Therefore, while the sample sizes allowed data comparisons between the baseline and current evaluation, it must be noted that the results may not be statistically significant/comparable.

33

Table 9: Sample size–VHSGs


VHSGs

Kampong Speu 20 Stung Treng 20 Mondolkiri 20 Prey Veng 20 Svay Rieng 20 100 Control provinces Kratie 15 Sihanoukville 15 30

TOTAL 130

CHV/BCC forum members were assessed in paired interviews, with BCC forum members who participated in at least two forums. The paired interviews helped illustrate different perspectives and challenges faced by people in the same role. A semi-structured discussion guide was used to get information regarding:

- awareness of the BCC project - relevance and impact of BCC forums - training - impact of BCC on the community

Table 10: Sample size–BCC forum members


Paired interview

Number of commune chiefs/commune committee for women and children members or other

members of BCC forums per interview

Kampong Speu 1 2

Stung Treng 1 2

Mondolkiri 1 2

Prey Veng 1 2

Svay Rieng 1 2

5 10

Control provinces

Kratie 1 2

Sihanoukville 1 2

2 4

TOTAL 7 14

In order for interviewees to provide relevant insight, respondents in provincial-, district-, commune- and village-level interviews had to have at least two years’ experience in their current position. In-depth interviews using semi-structured interview guides were conducted.

34

Community-level interviews Given that the main focus of the Health BCC project was health promotion pertaining to 12 key family practices and C-IMCI, it was proposed to conduct interviews with community members, specifically with mothers of children under 2 years old, who had visited a health centre at least once in the last six months (criteria used to avoid recall bias). The main objective was to gain insight into their most recent experience with health care (e.g. what was the attitude of the health care provider, what information was given, what more did the patient expect from the health centre). Interviews were recommended over observations at health centres to avoid acted behaviour, whereby a health worker may behave differently than he/she normally would due to the presence of the interviewer. Paired interviews helped secure different perspectives (respondent homogeneity in economic and education levels was ensured). Table 11: Sample size–Interviews with community members


Paired interviews Number of women per interview

Kampong Speu 1 2

Stung Treng 1 2

Mondolkiri 1 2

Prey Veng 1 2

Svay Rieng 1 2

5 10

Control provinces

Kratie 1 2

Sihanoukville 1 2

2 4

TOTAL 7 12

35

Figure 6: Snapshot of target respondent groups covered during the evaluation

2.4 Evaluation team

A unit specializing in public and social research of TNS Vietnam, a market research and consulting organization, and its local Cambodian partner, MSD, conducted the evaluation. Desk review and analysis, interviews with UNICEF, EC, PSC and NCHP representatives were conducted by TNS.

The TNS team, along with support from UNICEF, familiarized the MSD teams with the project and discussion guides prior to fieldwork in the provinces. Interviews with NMCH, current and former staff of NCHP, and the provincial-, district-, and commune-level stakeholders, VHSGs and community members were conducted by MSD in Khmer.

Over 230 people were interviewed in this

evaluation

Evaluation respondents

National level (19)

Provincial level (PHD and PHPU)

(21)

District level (OD-MCH and OD-

DHPU) (12)

Commune level

VHSGs (130)

BCC forum members

(14)

Community members

(14)

Health centre

(chief and midwife)

(24)

36

2.5 Evaluation limitations Limitations of the evaluation include:

Though it was proposed to conduct interviews with the Stung Treng OD, during the evaluation the OD-MCH was unavailable and the OD-DHPU position was vacant.

Some sub-national level staff did not have an opinion on key questions such as limitations of the project, challenges faced and sustainability.

The evaluation cannot state to what extent the project contributed to increasing caregivers/families’ knowledge and practices in the area of health, particularly ANC, C-IMCI, community care of newborns, and breastfeeding. As a large survey among community members was not conducted, it is difficult to estimate the coverage of caregivers/families with related BCC interventions and its impact on them.

The team did not interview PHPUs in all 24 provinces to evaluate capacity building in implementing BCC activities.

37

III. EVALUATION FINDINGS

3.1 Project relevance and appropriateness 3.2 Project efficiency 3.3 Project effectiveness 3.4 Project impact 3.5 Project sustainability

38

III. EVALUATION FINDINGS

3.1 Project relevance and appropriateness

Overall, the Health BCC project was very important in supporting Cambodia’s BCC-related health sector needs as set forth in the sector strategic plan and related policies. Behavioural change in service delivery and usage levels is essential for the country’s progress towards health-related MDGs.

In the first HSSP, the Cambodian government recognized that the country’s health profile was among the lowest in the region, in particular related to the MDGs and the areas of underweight children; high maternal, infant and under-five mortality; low modern contraceptive use; and high rates of both HIV and malaria, among other health issues. The importance of changing the behaviour of both health providers and their clients was recognized and highlighted in the first HSSP 2003–2007 as a priority area10. Through its overall objective of promoting a healthier lifestyle, better health-seeking behaviours and improved delivery of primary health care services in order to reduce poverty in Cambodia, the health BCC initiative responded to the demand and needs of the disadvantaged and underserved young children and families. This was the project’s overall objective, to be achieved by improving the capacity of national, provincial and district health providers to deliver effective BCC in Cambodia via revised national BCC policy, relevant training modules, BCC guidelines and training at national and provincial level. The project was particularly relevant at the policy level. While a BCC policy existed before the commencement of this project, under the health BCC initiative it was reviewed and decided that revisions were necessary to make it realistic and practical as the reference for all BCC actors. NCHP finalized the policy, MoH endorsed it and it was disseminated at the national and provincial levels. Likewise, all BCC-related protocol and guidelines were finalized, endorsed by MoH and distributed at national and sub-national levels. The second HSSP (2008–2015) was drafted after the project had already commenced. In HSSP II, while prevention of communicable and non-communicable diseases and maternal and child health is still a major priority, BCC does not have the prominence of the earlier plan. Nonetheless, the Health BCC project continued to remain relevant as a step towards prevention of diseases and better health-seeking behaviours and service delivery. The project was also in line with the EC’s Country Strategy for Cambodia, which aims to support long-term improvement of basic health of the population, poverty reduction and meeting the needs of the poorest. However, it is no longer EC’s vision to support national ownership through projects such as this, but rather to turn to budgetary support to sector-wise approaches related to a national strategy. Communication for development remains a core function for UNICEF. A key aspect of the BCC project was to build the knowledge and interpersonal skills (including counselling) of service providers, thus embarking on a first step towards encouraging health-seeking behaviours among the community.

10

Described in greater detail in the report’s first section.

39

HSSP I noted that at all service delivery levels, poor communication between providers and consumers is a major obstacle in promoting family health. The Health BCC project was relevant in this context as it employed various trainings to improve the skills of implementers at all levels.

Initially, the EC identified that the project’s relevance was justified by the prevailing need to improve the capacity of NCHP and PHPUs in delivering effective BCC, and in particular the capacity to plan, implement, evaluate and coordinate BCC for health. NCHP itself also identified

a lack of capacity (as mentioned in the midterm review report11)12. Planning, implementing and monitoring BCC campaigns at the provincial and district levels and improving counselling skills and knowledge at the health centre and VHSG levels were key requirements that the BCC project addressed. A training needs assessment13 was conducted to identify needs among health promotion practitioners and the skill gaps in expected BCC competencies. The communication tools developed under this project, including all training materials,

guidelines and materials, are pertinent to effective communication concepts and theories. They were used for the ANC campaign and are now in use for the complementary feeding programme. It is hoped they will be employed for future BCC campaigns and programmes.

However, it is difficult to determine if the capacities of the NCHP developed in the framework of the project will remain relevant for the following reasons:

a. NCHP has a semi-autonomous role within MoH. While it is specifically devoted to behaviour change and health promotion, long-term plans are unclear. Under the Health

11

The fact that the NCHP considers itself to be lacking in expertise was a surprising result, as it has been the recipient of considerable donor finance. For instance, a five-year AusAid-funded project (1997–2001) assisted the NCHP with refurbishing its building and providing essential equipment. 12

During the current evaluation, it was found that NCHP did not think there was a great need for capacity building at

the national level, citing staff’s ability to plan and implement BCC activities as already high (especially as they had received training and experience under an AusAID-funded project). 13

This assessment was developed by NCHP with support from UNICEF Cambodia and the EC through the Health BCC project, July 2006. The survey was conducted by Domrei Research and Consulting and the report is available with UNICEF and NCHP.

One aspect of the BCC project was interpersonal skills training: The BCC project helped us a lot in changing behaviour. Before, our behaviour was not good enough because we didn’t communicate with patients well. However, now staff at the hospital have changed their behaviour a lot, especially the communication between doctors and clients is much better and they are not arrogant or unfriendly any more.

– PHD

The training needs assessment identified that at the national level, training was essential for relevant NCHP staff in M&E, health research and IEC/BCC material design. At the sub-national level, training was required on BCC planning, M&E and training of trainers. At the health-centre level, training was needed for simple BCC planning procedures that consider health centre constraints and for training of trainers, including simple guidelines on preparing training sessions, with short checklists of things to do, materials to request or prepare and concrete examples.

40

MoH

EU

NCHP

UNICEF

BCC project, NCHP did develop a vision for the future, but UNICEF and the EC considered this outlook limited. NCHP was limited in developing a long-term vision because of its limited capacity on one hand and the fact that is still functions under the mandate of MoH (and is therefore unable to develop its own vision without MoH endorsement).

b. Ideally, NCHP is supposed to be the lead agency in BCC, but in practice, BCC activities go through strong vertical health programmes (HIV/AIDS, TB, malaria) and other MoH departments 14.

c. NCHP organizational structure and management was complicated by a parallel BCC project structure and it is uncertain how this will develop now that the Health BCC project is over.

d. As highlighted in the midterm review, it is unclear how NCHP sees its role in health promotion15. This was apparent when conflicts arose regarding the production of BCC materials – NCHP expected and wanted to carry out some of these activities internally, while UNICEF expected material production to be outsourced. A key outcome of the BCC project’s capacity-building activities should be NCHP’s ability to identify communication priorities (for MoH); plan a comprehensive, nationwide communication plan; coordinate implementation, especially through the national structure and with other MoH departments, including the technical department and service delivery departments; and M&E. In terms of design of BCC materials, outsourcing is encouraged and it is expected that NCHP will supervise and provide technical inputs to design and communication messages and work with the agency. However, NCHP believes it has the capacity for material production and views outsourcing as a lost opportunity.

Given the above points, capacities built at the national and provincial levels were and are being demonstrated during the ANC campaign and complementary feeding programme. As PHPUs are the bridge between NCHP and sub-national/commune-level implementers, strengthening their competencies also impacts districts and health centres.

3.2 Project efficiency In general, the national-level coordination mechanism was not the most efficient due to the complexity of its structure and hierarchies.

As described earlier, the Health BCC project was co-financed by the EC and UNICEF (with EC being the donor of a large proportion of the finances). The main role of EC in the project was its role as the donor; UNICEF’s key function was to manage the project by providing technical assistance and financial support to NCHP while reporting to EC about progress. NCHP itself was supposed to be the key implementing partner in the project. Some of the national-level implementing mechanisms in place were the setting

14

EC monitoring report. 15

Should it be an overarching commissioning organization that organizes training, provides advice and guidance, steering, coordination and monitoring through a range of partners, from within the ministry and outside, to provide effective and efficient BCC and health promotion? Or should it be a provider of high-quality BCC resources, training and consultation?

Figure 7: Key partners of the project

41

up of the PSC and the BCC-TWG. However, key issues that

arose were:

a. Ambiguous role of NCHP within MoH As described in an earlier section, NCHP has a semi-autonomous role within MoH. However, there are institutional issues between NCHP and MoH. This was reflected in the fact that some key documents created by NCHP and UNICEF were not immediately endorsed by MoH, thus resulting in delays in planning and implementation.

The fact that NCHP could not develop a long-term vision for itself in terms of health promotion also reflects that as a government institution, NCHP is held back by bureaucratic/political issues and any long-term mandate needs

to be endorsed by MoH.

b. Complicated parallel structures within NCHP to implement the Health BCC project Instead of integrating the Health BCC project into NHCP’s existing organizational structure, separate teams were formed according to their respective technical areas (Table 4). According to an EC monitoring report, these teams remained semi-detached from the official organizational structure of NCHP, which is made up of its management, an administration bureau covering units for primary health care, environmental hygiene and tobacco or health, and an administration bureau. In line with the BCC midterm review report, the EC monitoring mission recommended NCHP organizational restructuring and repositioning as a genuine BCC centre. However, the proposal for restructuring was not supported by MoH due to the ministry’s limited decision-making capacity over the NCHP structure. In Cambodia, such decisions are taken by the Council of Ministers and require extensive preparation and prolonged negotiations. It was revealed during the evaluation that NCHP itself did not see the purpose and benefit of such an organizational restructuring (especially as a part of the Health BCC project). Further, all documents mentioned that reorganization was necessary but there was no elaboration on what this would entail.

c. Unclear role of the PSC

The PSC was formed to provide policy guidance to the project, facilitate coordination among all institutions and groups involved and advise NCHP on implementation and coordination of planned activities.

Still, the project is not as efficient as could be and this stems partly from the project design, partly due to the nature of the implementing agencies and the lack of support and commitment of MoH. – EC Monitoring Report, 2008

For steering committees to work better, they need to be smaller, more manageable. The more people involved, the less they are engaged. – Member of PSC

42

However, interviews with PSC members revealed that the committee was primarily a platform where progress updates were given. Members of the PSC who were not among the key partners (EC, UNICEF, NCHP) did not have an important role in lending their expertise to the project, except when it came to giving advice/feedback on key messages. The PSC was also considered to be too big to be efficient. It was shown that members of the PSC sometimes spent more time finding small faults instead of examining the bigger picture and providing more technically sound advice and specific expertise. One member said that if the committee had had more leverage and the right to veto things, its role might have been more efficient. While such a committee would be difficult to develop in the Cambodian context wherein bureaucratic systems need to be followed, it is recommended that in the future, PSC members have more defined roles and more accountability to the project.

d. Lack of consistency in institutional memory

The EC noted that project input in the form of human resources from NCHP was not always provided or available on time to implement activities. Crucial staff left the institution without being replaced16. The issue of staff turnover was raised by other members of the PSC, as well as by UNICEF and EC. However, during the evaluation it was found that NCHP itself did not consider internal staff turnover to be an issue. According to key NCHP staff, only two employees left during the period of the project (apart from a change in leadership). The key person responsible for full-time technical assistance within UNICEF also left the project in March 2009. While this gap was filled by other UNICEF staff, there was no full-time person to replace the technical advisor. A consultant was

eventually hired, but this was short-lived as NCHP was unhappy with the consultant’s inputs and role. The EC monitoring report also noted that the BCC-TWG, including MoH heads of departments, donors and NGOs, was not providing adequate support to NCHP (reviewing, facilitating and advising on BCC activities) as the participating organizations were not always committed to attending meetings and sent different participants, thus impacting project support.

Overall, the coordination between the EC, NCHP and UNICEF was not always smooth and impacted the effectiveness and efficiency of the project. Institutional differences and divergence in understanding project objectives were key issues, as different partners had different expectations.

While the EC’s choice of UNICEF as an implementing agency was strategically effective, there was a misalignment between the two organizations’ internal procedures. UNICEF’s standard approach to work planning and budget management is different to the way this project was

16

EC monitoring report 2007.

To improve efficiency of NCHP in the long term, thorough management support and an organizational overhaul is urgently needed as well as UNICEF gradually stepping back as manager of the BCC project. – EC monitoring report

43

managed. Likewise, UNICEF also had to adhere to UN recruitment rules, thus leading to a delay in actually starting the project with a full-time technical assistant.

Due to the agreement formally signed with the principal funder, the EC, the BCC project functioned separately within the UNICEF organization, with its own staff. However, as BCC is a core strategy of the UNICEF country programme, it was difficult for UNICEF to apply the project in a useful way to the entire country programme. These issues were discussed with EC missions and during the initial stages. Efforts were made to include both the EC framework as well as UNICEF’s own management systems in order to take advantage of work planning systems.

NCHP found EC as a donor to be very supportive and encouraging. However, the institution raised issues it faced with UNICEF with EC representatives, thus treating EC as a mediator. This was not the role of the EC, and while the EC tried to help iron out some key issues (especially during participation in the PSC), EC representatives were of the opinion that if NCHP wanted UNICEF to align itself with the institution’s mandate, a clearer, MoH-endorsed strategy was needed.

Several factors impacted the smooth partnership between UNICEF and NCHP:

a. Difference in understanding project objectives and expectations

Overall, UNICEF and the EC saw the Health BCC project as one that would help build NCHP’s capacities as a pivotal organization in health promotion. However, NCHP’s vision was not a long-term one and it perceived the project as time-bound and as a source of income. This led to differences in expectations in terms of support. While UNICEF was expected to provide technical support, NCHP expected support in terms of project management. Thus, the technical advisor’s role strayed from the project’s original concept in providing both technical assistance and being involved in daily NCHP management. However, it was recognized by the EC and UNICEF that separating the two aspects was difficult, especially because he was posted as full-time staff within NCHP. While his

accountability was to UNICEF, NCHP expected the advisor to report to the institution. This issue was raised in the midterm review as well.

EC and UNICEF saw the

objectives as they are written in

the document – strategic

support to capacity building of

an institution for this institution

to deliver better. NCHP itself

saw this as a project – thus

expectations were that of

project management, which

would be limited to the project

itself.

EC and UNICEF relationship: Found it

interesting that it was a partnership-type

relationship in terms of strategic

discussions as well as programme

management and programme reviews. It

was interesting because very often with

donors there is a partnership but not

close (reporting relationship, programme

discussion, etc). With EC, it was a close

and constant partnership that involved

very productive discussions. – UNICEF

44

Once the full-time technical advisor left the project, NCHP noted that it was unsure of the appropriate contact person within UNICEF. As there was no full-time advisor after 2009, UNICEF internally redistributed project activities between teams.

b. Lack of ownership and of NCHP budget oversight

A key issue that impacted the relationship between NCHP and UNICEF was budgetary. UNICEF’s accountability with financing was directly with the EC, an arrangement that NCHP did not always appreciate.

As a development partner, UNICEF typically tries to support national programmes and their own work plans, making efforts to position itself as a partner that contributes to the health sector’s annual operation plan. During the evaluation, it was found that the EC supported the decision to make UNICEF the main partner in charge of finances, in efforts to help NCHP control budgets and increase transparency. However, according to NCHP, the EC did not assess the institution’s capacity to manage finances before electing UNICEF as the financial controller.

NCHP sought this technical and financial accountability in order to boost their credibility and credentials.

Overall, NCHP did not feel that UNICEF treated it as an equal partner.

Even though NCHP was involved in all stages of the project, including planning, the perception was that UNICEF permission was required to take action. This is an unfortunate finding, since efforts were made from the project inception to ensure all partners were equally involved in all stages of the project.

With regard to the financial management of the project, some key issues related to project efficiency were:

a. Utilization of funds was a concern in the first few years of the five-year project, when only 30 per cent of funds had been used at the end of three years. However, by 2008, the fourth year of the project, 80 per cent of the funds were utilized as project activities were accelerated and accomplished. By the end of 2009, 4,600,100 euros of the allotted

5,706,00017

had been spent, thus leaving a balance of 1,105,900 euros18.

Taking into consideration the project’s slow start and to allow enough time for utilization of remaining funds and to ensure sustainability of BCC activities, a no-cost extension was recommended in the midterm review. This also had the added value of synchronizing the BCC project with UNICEF’s country programme cycle (2006–2010). The no-cost extension was conditional on:

- Availability of sufficient balance budget at the end of 2009 - Development and endorsement of NCHP’s Organizational Development Strategy - Good progress of the ANC BCC campaign - Good progress rolling-out C-IMCI - Improved M&E and documentation of BCC activities by NCHP

17

Both EC (€5,000,000) and UNICEF’s (€706,000) fund committed to the project. 18

As per the UNICEF 2009 Annual Report. There are discrepancies between the amount in the annual report and the June 2009 PSC meeting minutes.

45

The no-cost extension was granted along with an approved budget reallocation19.

b. As the funding was in euros and the spending was in US dollars, fluctuations in currency rates sometimes resulted in discrepancies in the allocated and spent budgets. This was specifically an issue in 2008 as noted in UNICEF’s annual report20.

c. UNICEF managed allocation of funds at different levels while budgetary planning was

done along with NCHP. Before the midterm review, UNICEF and the EC decided to concentrate financial and technical support within the provincial units, thus keeping in line with the capacity-building objective. This was further recommended in the midterm review and implemented from 2008 onwards. This was considered a positive step as improving capacities at the provincial level would result in a wider impact in the community. However, the evaluation failed to capture if this was a truly efficient way of working and if provincial staff benefited from this change in budgetary allocation compared to the years prior to 2008.

d. A key cause of contention throughout the project was related to remuneration and staff per diems.

Performance-based incentives were included in the project in addition to government salaries, which were recognized as being low. NCHP staff working on the project received the agreed UN rate for category B, which was around US$130 per month, to achieve the planned results of the BCC project activities. At the provincial level, vice-provincial health directors were paid US$50 per month, PHPU staff were paid US$40 per month and finance officers were paid US$30. However, NCHP was not entirely satisfied with the performance-based incentives offered, even though these were based on government policy. The midterm review also noted that the performance-based salary offered by the project was insufficient to motivate staff. NCHP staff was also dissatisfied with the per diem rates on projects.

The discontinuation of incentives after the project and the slow pace of public sector reform could impact the sustainability of the project, especially at the provincial levels.

3.3 Project effectiveness Overall, the project met its purposes and expected outcome of enhancing health providers’ responsiveness to and communication with clients in order to improve health-seeking behaviours and promote healthier lifestyles, as defined in the log frame. This section further describes the effectiveness of the project against the expected results (as per the log frame) at the national and sub-national levels.

19

According to the 2009 UNICEF Annual Report, the proposed budget revision was based on the fact that most of the

estimated balance was coming from the “human resources” budget heading. In addition, analysis of planned allocations for 2005-2010 suggested that, if budget revision were not carried out, some headings would exceed the allowed 15 per cent deviation. 20

During 2008, UNICEF project management routinely monitored estimated expenditures against fluctuations in the US dollar-euro exchange rate. This involved continuous assessment of budget implications in terms of the 2008 annual work plan against a possible need to cancel activities and/or adjust planned inputs/budgets, particularly for supplies. This action resulted in adjustments that prevented exceeding agreed budget headings within the maximum allowance of 15 per cent.

46

a. Effectiveness in revising BCC policy

A major activity under the project was the development of a national BCC policy. The national BCC policy was reviewed and revised under the leadership of NCHP and the BCC-TWG.

A BCC policy review workshop was conducted under the leadership of BCC-TWG and NCHP and presided over by the Secretary of State of the MoH, His Excellency Professor Eng Hout, and 95 professionals representing MoH, WHO, UNFPA, NGOs, PHDs, PHPUs and VHVs.

The policy was revised to ensure that it was in line with views and realities of BCC stakeholders at the national, sub-national

and community levels and was endorsed by MoH. It

was widely disseminated so that all major BCC stakeholders could use it as a reference to advocate for planning and coordination of BCC activities. The revision included protocol for development of BCC materials and guidelines for BCC implementation. Following a workshop with the BCC-TWG in 2006, it was agreed that BCC guidelines that covered all aspects of the BCC process – planning, implementation and monitoring – were to be developed into a practical tool for BCC stakeholders. With technical support from the training institution, GFA Consulting Group, the Protocol and Guidelines team outlined the various steps in the development and implementation of quality BCC activities, which were first shared with BCC partners through two consultative workshops in 2006, pretested by 24 PHPUs and finally reviewed and disseminated by the BCC-TWG. The development of national policy, guidelines and protocols is one of the project’s key achievements and will support the sustainability of BCC activities, as these guidelines can assist with the planning and implementation of health promotion campaigns/programmes beyond the scope of this project (as

BCC stakeholders at the national BCC policy review, 2006; ©UNICEF.

The BCC guidelines cover principles and standards for BCC research, development of behaviour change results and indicators, audience segmentation, selection of communication channels, material development and production, BCC campaign monitoring, and evaluation of BCC interventions.

47

has been demonstrated in the ANC campaign and the ongoing complementary feeding programme).

b. Effectiveness of capacity building

A number of activities were covered by capacity building activities. Computer skills: As reported in the EC monitoring report in 2006, one of the project’s setbacks was the lack of computer skills at NCHP. An IT consultancy firm was appointed to assist the NCHP Information Services team for half a year in developing a BCC website, basic IT skills and BCC content management. Despite the availability of the training opportunities, team members were not always available for training as many of them were also part of NCHP’s Hygiene Promotion team. This can be considered a lost opportunity for NCHP. The training team, with support of the training institution, developed an IMC on BCC. The course comprised of six modules covering BCC-related concepts, research, planning, message and material development, health promotion in practice, and M&E of BCC interventions. NCHP collaborated with the training institute to conduct the training of trainers and select and train the first group of trainees. Training of BCC specialists: To strengthen the technical capacity of NCHP staff in health promotion and BCC, four long-term fellows pursued a master’s degree in public health. Two NCHP staff – a deputy director and the chief of technical bureau – attended the Results-Based M&E course offered by IMA International in Brighton, UK. Two NCHP staff participated in a two-week training at Kasetsart University in Bangkok to improve their IEC development skills. However, not all staff members trained as a part of the project returned to NCHP, instead joining organizations that offered them better opportunities. During the evaluation, it was shown that some NCHP members did not feel that their capacities had grown during this project. ABCP training package: Six NCHP staff participated in a study tour and workshop to finalize the ABCP curriculum, which included BCC research, planning, material development, pretesting and M&E. This curriculum was then pretested and finalized by NCHP and provided to all technical staff. The theory and skills learned through this training was put to practical use while developing and planning the ANC campaign. C-IMCI: In collaboration with MoH’s Communicable Disease Control Department and UNICEF, NCHP revised the C-IMCI training curriculum for VHSGs. The training package included nine training modules for trainers, health care workers, interpersonal communication materials and aids for VHSG, implementation guidelines and monitoring and supervision tools. The C-IMCI package ensured a standardized tool in the area of health promotion and community-based provision of essential services by VHSGs. Revision of the C-IMCI continued in 2008 and was endorsed by MoH in 2009. Based on the revised curriculum and developed materials, the C-IMCI monitoring tool used by NCHP’s M&E team and the Communicable Disease Control Department was updated. C-IMCI training was pretested and rolled-out in seven UNICEF-supported provinces, including five good practice demonstration provinces.

48

PBCI: One of the most effective training activities, PBCI focused on improving the behaviour of health care providers, including training on counselling, patient interaction, client rights etc. MPA Module 7: NCHP revised the MPA Module 7 on health education, health promotion and BCC for health centre staff to include interpersonal communication skills. It was used to train health centre staff on basic health education and health promotion practices.

KAP in 12 key family practices: Two research consultants supported the M&E team in planning, conducting and reporting a nationwide KAP survey on the 12 key family practices for health. The survey design and protocol were endorsed by key national and international stakeholders in health, including the national programmes of MoH, WHO and international and national NGOs. Eighty staff of NCHP and PHPUs were trained on survey methodologies and data collection and the trained staff collected quantitative and qualitative data nationwide at the household level, covering 3,000 respondents.

One key family practice for health, ANC, was promoted as a nationwide campaign. The campaign demonstrated the benefits of BCC planning and implementation training at various levels. Messages and materials for the campaign were developed in consultation with key national and provincial implementation partners. Notable results included successful pretesting of messages and materials; media planning of the nationwide media campaign; interpersonal communication training developed and provided to select health centre staff and VHSGs in each of the five demonstration provinces prior to campaign implementation; development of BCC materials, including flipcharts; M&E activities and documentation of results.

At the provincial level, training was provided to PHPUs and health providers. Training on counselling and interpersonal communication was also given to health centre staff and VHSGs. Though the evaluation was unable to quantify the extent to which BCC tools and resources developed by NCHP were used, it was revealed that most health providers found the training and IEC materials to be useful. Training modules, especially on C-IMCI, client rights/providers’ rights and interpersonal communication, were instrumental in changing attitudes of the health providers at health centres.

The capacity-building activities improved service providers’ knowledge and practices to a great extent at all levels, especially among health centre staff and village health workers. PBCI, including counselling and interpersonal communication, was thought to be the most relevant training for health centre staff. Provincial- and district-level staff agree that the attitudes of health centre staff have changed for the better, which has resulted in encouraging more people to access health centres.

The capacity of NCHP in research has been commended by other partners, such as BBC WST, who partner with NCHP on various nationwide surveys.

Yes, all trainings are useful, but if I were to choose I would say client rights/provider rights. Why do we do this training first? Because community and service providers do not understand their duties and rights. So, we want them to know about their rights clearly. – PHPU

49

During the survey with VHSGs, it was revealed that the most useful trainings according to village volunteers were C-IMCI and hygiene. The VHSGs tended to view the sessions most directly related to community health as the most useful. Most VHSGs reported that trainings were very useful because they helped improve knowledge about diseases and health problems. The knowledge of VHSGs improved to some extent due to the trainings – KAP differences between baseline and end-line is found in the Annex.

Table 12: Usefulness of training, VHSGs Kampong

Speu Mondolkiri Prey

Veng Stung Treng Svay

Rieng Control Provinces

C-IMCI Module 3 80 80 70 75 80 73.3

Hygiene (C-IMCI Module 7)

65 35 55 30 45 56.7

Immunization (C-IMCI Module 6)

10 15 25 10 35 30

Malaria/Dengue (C-IMCI Module 10)

15 60 50 60 30 30

Tuberculosis 40 15 50 30 40 30

Breastfeeding/ Complementary Feeding (C-IMCI Module 4)

5 10 15 10 15 23.3

Micronutrient (C-IMCI Module 5)

0 0 20 5 35 20

Home care of sick children (C-IMCI Module 8)

5 20 15 30 5 16.7

Diarrhoea 30 25 20 20 5 16.7

Basic Health Education (C-IMCI Module 2)

20 10 0 25 20 13.3

Bird flu 20 20 25 10 15 13.3

HIV 5 25 30 10 25 13.3

BFCI–Baby-Friendly Community Initiative

0 0 0 5 0 3.3

Birth spacing 10 0 0 5 0 3.3

Total number 20 20 20 20 20 30

Most VHSGs also agreed that they applied lessons learned during the training to a large extent.

50

Table 13: Extent of training application

Kampong Speu

Mondolkiri Prey Veng

Stung Treng Svay Rieng

Control Province

Overall

To a large extent

60 65 55 60 90 70

To some extent

40 35 45 40 10 30

Not at all 0 0 0 0 0 0

Total number

20 20 20 20 20 30 130

The majority of VHSGs believed they were able to widely apply what they learned because of the clarity of information provided during the trainings. They reported that trainers made lessons easy to understand by using flash cards, pictures, other technical documents and practice games. According to the VHSGs, community members also trust and follow their advice, making it easier to convey the trainings’ messages. However, it has been identified both by VHSG themselves and provincial and district staff that refresher trainings are essential. A lack of financial support to do so has been identified as a key constraint. Apart from continued training sessions, supervision of activities has also been identified as a way to ensure that capacities built are sustained and training imparted is being implemented. Documentation at health centres, which is a key way to measure the outputs of the project, was considered to have room for improvement. A key reason for lower levels of documentation at the health centres is a lack of resources. A key achievement in health BCC in Cambodia that is attributable to the BCC project is the roll-out of the nationwide ANC BCC campaign.

First, we have to provide continued training and some incentive to our health workers, if possible. For example, some regions have much support from organizations that support health workers if they can complete the clinic card for IMCI counselling, so this can be a good incentive for health workers and they can provide good counselling. IMCI counselling is very important because it can help child caregivers learn how to take care children. Training courses should be provided and meetings between health centres and VHSGs should be conducted more often or monthly. Before, at some places, health workers and VHSGs didn’t meet and exchange ideas. Therefore, we really want meeting to take place, but support is needed. – OD

51

Overall, the ANC BCC campaign was highly successful in achieving its behavioural goal of increasing the proportion of women to access early ANC (<8 weeks): there was a significant increase from less than 6 per cent at baseline to over 35 per cent post-campaign. As the ANC BCC campaign was the first practical application of the BCC policy, protocol and guidelines, this is truly a notable achievement and indicates the potential impact and sustainability of the campaign. Similarly, the planning and implementation of the complementary feeding programme also demonstrates the effectiveness of the BCC project.

Document completion at health centres is not good enough because they have a lot of responsibilities and they have staff shortages. If health centres are well staffed and have skilled midwives, their work goes smoothly. These are the difficulties and obstacles encountered. – DHPU In order to improve service providers, I think we should make plans to supervise health centre staff and provide revision sessions for health workers in order to help them understand clearly. We have to try to encourage them and follow-up with them. Supervision is a good practice because it can be an encouragement for them and they can ask teachers from the provincial level if they have any questions. – PHPU

52

Chart 1: Proportion of women with early ANC

3.4 Project impact One of the key positive long-term outcome of the Health BCC project on health promotion in Cambodia has been the creation of an adequate BCC policy framework. As described in an earlier section, the BCC policy was completely revamped under the project to make it more applicable for key BCC stakeholders. NCHP reviewed and revised the policy with inputs from various BCC stakeholders, thus strengthening capacities at the national level in policy development. However, it is difficult to assess if capacity building at the national level will have a long-term impact, given high rates of staff turnover. National-level staff (to some extent), and especially sub-national staff, who were trained reported positive behaviour changes. There is also evidence of improved delivery of aspects of primary health care that is linked to the activities and training under the project. According to NCHP, the training received under the BCC project augmented its already existing capacities to plan and implement BCC projects, such as past experience with the AusAID-funded project. At the sub-national level, the impact of the BCC project on provincial staff in terms of strengthening/improving approaches to health BCC was felt to some extent. However, while provincial-level staff felt that their BCC implementation skills had improved, their ability to more widely plan and expand BCC activities was still seen as limited, mainly due to budgetary constraints.

2008 2009 2010

ANC <8 weeks 6.3 36.2 31

0

20

40

60

80

100P

erc

en

tag

e

ANC <8 weeks

53

Improved service delivery at health centres was a key issue addressed by this project. Along with training on counselling and interpersonal communication, attitudes of service providers at health centres have changed for the better, resulting in better service delivery and thereby impacting overall health-seeking behaviours. Training on communication skills seemed to have the greatest impact on health staff.

The BCC forums were extremely important in providing a platform to discuss and spread awareness about health issues in communities. One example of the forums’ impact was the way in which the ANC campaign was implemented in Kampot province, which was not an intervention province for interpersonal communication. The forums also contributed to ownership at the provincial level. However, not all provinces chose to continue these forums by

looking for funding and including them in annual plans. The project contributed to the overall objective of better health-seeking behaviour. Interviews with community members during the evaluation revealed that participants, especially women, were more open to accessing the health centre for prenatal care, birth and other general health issues. This is because of two key reasons. First, BCC messages on ANC and other health practices were delivered through village health workers, BCC forums and BCC campaigns. Second, community members are more satisfied

The BCC project helped us a lot in changing behaviour. Before, our behaviour was not good enough because we didn’t communicate with patients well. However, now staff at the hospital has changed their behaviour a lot. The communication between doctors and clients is much better and doctors are not arrogant or unfriendly anymore. Supervision after the training courses also helped. Moreover, we sacrificed our time to follow-up and monitor our health workers. But we no longer have any support for those activities. – PHD In the past, hospital staff didn’t talk much and didn’t take good care of people who came to get served. After the training, they work better. It is not excellent yet, but we may say it’s good. – PHPU Before, I didn’t know much and explain things to clients clearly. However, after I participated in the training courses, I can provide much better explanations. I understand better than I did before. After I the training courses, I corrected my mistakes and now understand how to communicate with clients. – Health centre staff

Now there are a lot of people who come to use our services. In the community, there are a lot people who know us and give us value. For the people that I provide health education, I can reach at least 50 per cent to 80 per cent of them. – Midwife Before, women gave birth at home, but now they deliver at the health centre. There were only 30 or 40 pregnant women who visited the health centre for ANC in the past, but now there are hundreds of them because they understand the danger signs. – Midwife

54

with service delivery at the health centres and thus are more willing to go to them. The assessment of the ANC campaign also demonstrated the impact of the Health BCC project on MCH in Cambodia. In January 2009, UNICEF and NCHP launched the “ANC Within the First Month after Missing a Period” campaign. Its aim was to increase the number of women receiving ANC at a health centre within the first weeks of pregnancy. Through proper ANC, women received key basic health services vital to their well-being and that of their babies. The impact of the campaign was visible with the increasing number of women who sought early ANC. Analysis of CDHS data from 2005 and 2010 revealed that the maternal mortality ratio for Cambodia in 2010 was 206 per 100,000 live births, which was markedly lower than the figure reported in 2005 (472). Likewise, comparing other indicators in maternal health care revealed that prenatal care and health-seeking behaviours improved remarkably from 2005 to 2010. The proportion of women who delivered in a health facility was more than two times higher in 2010 over the five-year period, with an increase of 34 percentage points. The proportion of women who sought ANC from a skilled health provider increased by 20 percentage points from 2005 and 2010, and in 2010 close to 9 out of 10 women reported that they sought ANC from a health provider.

These trends can be attributed to a large extent to the Health BCC project and the ANC campaign.

Antenatal care provided byskilled provider

Delivered in a health facilityDelivery assistance by skilled

provider

2000 38 10 32

2005 69 22 44

2010 89 54 71

0

20

40

60

80

100

Perc

en

tag

e

Trends in maternal health care-CDHS data

What the women said: I like going to the health centre because the staff member there is very friendly and polite and he also explains how to use medicine. She (the VHV) goes to talk to people at their homes. She has two members; sometimes she calls for meetings and raises important matters and issues for women, such as pregnancy, and asks them to have ANC. She also tells women needing contraceptives what to use. Previously, health centre staff came to work late because they were busy with their business. Now whenever we go to the centre, we are able to meet them.

55

Assessing the project’s long-term impact is difficult as it depends on several factors, including the priority given to health promotion by the government and the ability of NCHP to be recognized as a centre for excellence in BCC through alliances with partners, donors and vertical health programmes.

3.5 Project sustainability Overall, the sustainability of the project remains questionable at the national level due to the lack of a long-term approach to health promotion in Cambodia, the uncertainty of NCHP’s future as the focal point for behaviour change and health promotion, and the low remuneration for MoH staff.

As this report has noted, while health promotion was an important strategy in HSSP I, its prominence was diminished in HSSP II. There is currently no long-term approach to health promotion in Cambodia. Given this, government funds allocated to health promotion are likely to be insufficient, if at all. Funds allocated by

MoH through the health sector pooling fund will not be sufficient to cover expected BCC expenditures.

Further, despite the funds spent on building NCHP capacities as the key institute for health promotion, separate verticals in different MoH departments continue to exist, some of which implement their own behaviour change strategies. This renders the role of NCHP redundant. For the project to be sustainable, MoH and its departments need to consider NCHP as a key partner in implementing health promotion campaigns. With the experience from the ANC campaign and the complementary feeding programme, NCHP now has greater professional capacity to support the planning and implementation of a national and provincial BCC campaign. These skills should be tapped and allow NCHP’s experience with health promotion projects/campaigns to grow.

As noted in one of the EC monitoring reports, another factor that impacts the project’s sustainability is the relatively low scale of remuneration in Cambodian public service, which potentially hinders maintaining adequate levels of staff capacity and motivation. This was also noted in the midterm review as a reason for staff turnover. Sustainability of activities at the provincial level also remains a concern, even though capacities have been built to a great extent. There is also great ownership of activities at the provincial and lower levels.

Sometimes in workshops and

seminars they don’t even mention

BCC as a priority. The amount of

money also reflects how much it is

a national priority. The government

allocated US$250,000 this year for

NCHP/health promotion. While this

is a step, it is a very small amount

against the needs of health

promotion in this country. – EC

We continue to provide counselling support at health centres. We have to give more counselling to pregnant women and all people who come to use health centre services. What we cannot continue is to provide more trainings in the village or refresh what they have learned. However, the knowledge that health workers have will continue to be applied to their work. – DHPU

56

While the training that has already been provided will continue to be effective (assuming no staff change), financial constraints prevent the expansion of BCC activities involving training. Despite the fact that M&E tools have been designed, current provincial M&E skills need improvement. Without adequate funding, the sustainability of continuous monitoring is doubtful. It is encouraging to note that at the provincial level, efforts are being made find support from other organizations to continue BCC activities. The BCC forums, one of the key components of the Health BCC project at the sub-national levels, were being supported by the project. Even though funding has stopped, these forums continue to be implemented in some provinces. The BCC guidelines, policies and training materials developed as a part of the Health BCC project are to some extent sustainable and can and should be used as frameworks for other health promotion projects and campaigns. This was further demonstrated by the national ANC BCC campaign that was developed collaboratively by NCHP, NMCH, PHPUs and other health partners, which was the first practical application of the BCC policy, protocol and guidelines. The ongoing complementary feeding programme also uses these guidelines and is being implemented by NCHP in collaboration with NMCH, UNICEF and other partners and is funded by the MDG Spanish Fund.

Expanding donor base and leveraging additional resources. It is encouraging to note that the NCHP has been active in writing proposals to seek funds from other sources, such as the Global Fund. While the EC monitoring report in 2009 noted, “Other external funds are neither likely to be obtained, without NCHP reaching higher levels of professionalism in BCC and independence as an organization, and promoting itself likewise,” the current NCHP director is very aware of the importance of marketing the NCHP as an institution capable of providing BCC-related services.

In 2009, overall, an additional US$350,000 from the MDG Spanish Fund and the second HSSP was made available for priority BCC activities in 2010. Yet, as noted in UNICEF’s Annual Report 2009, several more years of support will be required in order for the BCC framework created with project support to be internalized into Cambodia’s health sector.

Frankly, if we don’t have money, the activities won’t amount to much because we need funds to travel and provide oversight. Moreover, we need per diems for daily spending during supervision. – OD-MCH

I have to try to continue the activities and find more support from other organizations. If it is available and we have funds, we can continue all activities. – PHPU

Yes, we have held the BCC forum by ourselves. We depended on national guidance only the first and

the second time and after that we did it on our own with support from pooled funds. – PHPU

57

IV. CONCLUSIONS AND

RECOMMENDATIONS

4.1 Conclusions

4.2 Recommendations

58

Summative Evaluation Overall conclusion The evaluation’s overall conclusion is that the Health BCC project, jointly supported by the EC and UNICEF and implemented by NCHP-MoH, was relevant to the context and largely effective in contributing to Cambodia’s health promotion needs.

IV. CONCLUSIONS AND RECOMMENDATIONS 4.1 Conclusions 1. The project was effective in meeting its national-

level purposes i.e. strengthening NCHP’s national BCC policymaking and capacity building to support implementation of BCC services at national and provincial levels.

2. The best evidence of the project’s success is seen

in the planning and implementation of the nationwide ANC BCC campaign that was launched in 2009. Other notable achievements include revision and dissemination of the national BCC policy, establishing BCC forums in all provinces and the roll-out of training C-IMCI across the country.

3. While the BCC resources developed as a part of the project are sustainable, sustainability

and expansion of the activities is doubtful due to the lack of a long-term approach for health promotion in the country and of government funds. At the provincial level, while the effect of health provider and provincial staff training can be sustained, it will be difficult to expand the activities.

4. Overall, the Health BCC project was very important in addressing the BCC-related health

sector needs and policies set forth in the sector strategic plan, as behavioural change at the service delivery and usage levels is essential for Cambodia’s progression towards health-related MDGs. While three strategies directly relating to BCC were given prominence in HSSP I, in HSSP II BCC is not mentioned explicitly.

5. Capacity building to increase knowledge and interpersonal skills (including counselling) of service providers were key issues addressed by the project, thereby meeting the demands and needs of service providers as a first step towards encouraging health-seeking behaviours among the community.

6. The communication tools developed under this project, including all training materials,

guidelines and materials, are pertinent to effective communication concepts and theories and were used for the ANC campaign and the ongoing complementary feeding programme. It is hoped they will be used for future BCC campaigns and programmes.

7. However, it is difficult to determine if the NCHP capacities developed by the project will

remain relevant as vertical health programmes within MoH continue to implement BCC activities, thereby diminishing NCHP’s role as a focal point for health promotion. It is imperative for the institute to begin positioning itself as the key institution for health BCC activities.

8. In general, the national coordination mechanism was not the most effective due to the

complexity of structure and hierarchies. NCHP was required to wait for MoH endorsements on key strategies. That, coupled with the fact that capacities within NCHP were limited,

59

proved to be a hindrance when developing NCHP’s long-term vision as a key institution for health promotion.

9. The PSC, which was formed to provide policy guidance to the project, facilitate coordination

among all institutions and groups involved, and advise NCHP on implementation and coordination of planned activities, was simply a platform where progress updates were given. PSC members who were not among the lead partners (EC, UNICEF, NCHP) did not have an important role in lending their expertise except when it came to giving advice/feedback on the key messages.

10. A key achievement in Cambodia’s health BCC profile that is attributable to the BCC project

is the roll-out of the nationwide ANC BCC campaign. The campaign was overall highly successful in achieving its behavioural goal of increasing the proportion of women who access early ANC (<8 weeks), with a significant increase from less than 6 per cent at baseline to over 35 per cent post-campaign. As the ANC BCC campaign was the first practical application of the BCC policy, protocol and guidelines, this is truly a notable achievement and indicates the project’s potential impact and sustainability.

11. The Health BCC project was effective in meeting the purposes defined in the log frame at

the national level: strengthening national BCC policymaking and NCHP capacity building to support implementation of BCC services at national and provincial levels.

12. While the BCC project was effective in meeting its project purposes, it failed to influence

collaboration among various MoH structures and among other line ministries, local authorities, NGOs, civil society organizations and other national and sub-national players. Vertical MoH departments continue to exist, implementing their own BCC activities. The PSC and BCC-TWG aimed to include various national-level stakeholders, however it was achieved in a marginal way.

13. It is concluded that the project was able to improve the overall health-seeking behaviours.

Evidence of this proceeds from the ANC campaign and from the interviews help with community members.

14. Within the ANC campaign it was found that the BCC forums were extremely important in

providing a platform to discuss and spread awareness about health issues at the community level.

15. Attitudes of health centre service providers have changed for the better, resulting in better

service delivery and health-seeking behaviours as a result of the project. 16. The project’s national-level sustainability remains questionable due to the lack of a long-

term approach to health promotion in Cambodia. This is based on the fact that NCHP’s long-term role as the focal point for behaviour change and health promotion seems to be uncertain. In addition, low remuneration of MoH staff contributes to high turnover.

17. The BCC guidelines, policies and training materials developed as a part of the Health BCC project are to some extent relevant to the context and should be used as frameworks for other health promotion projects and campaigns.

60

4.2. Recommendations Based on the evaluation, the following recommendations have been made for future activities:

1. NCHP should proactively seek collaboration with other vertical health programmes to integrate and coordinate BCC activities. This would require NCHP to promote itself as the focal point for health promotion. As NCHP currently has the experience of working on two large BCC campaigns under this health project (ANC and complementary feeding), it should be able to promote itself as having the knowledge and expertise to implement BCC campaigns.

2. NCHP should continue to seek other sources of funding by actively writing proposals. The evaluation noted that NCHP had already begun this process by writing a proposal to the Global Fund. Though the funds were not granted, NCHP views the experience as an opportunity to improve future proposals.

3. Given that NCHP has a role in fundraising it should consider building capacities of its staff in proposal writing and results-based reporting in order to create successful proposals.

4. While the key role of NCHP should be health promotion, the evaluation found that the institution’s capacities as a lead institution in development and management of social research is high and partners who have worked with them on research projects have been impressed with their abilities. Even though this is not NCHP’s key focus, the capacities it has built in this regard for over 10 years should not be overlooked. It is recommended that NCHP continue to nurture a separate team for health development/promotion.

5. If an organizational restructure as recommended by the project is not possible in future, it is recommended that the NCHP team structure used under the BCC project is implemented institution-wide (i.e. teams for M&E, information services, protocols and guildelines, training and management), as the departments are more relevant than current divisions, such as primary health care, tobacco and health, etc.

6. NCHP should continue to monitor and supervise activities under the Health BCC project to ensure that activities do not diminish at sub-national levels. So far, sub-national capacities have been strong in most aspects. It is therefore imperative to ensure that this continues. NCHP should include M&E of the project-related activities (training, home visits) in its annual work plans.

7. The BCC forums have been very effective and it is essential that they continue in as

many provinces as possible. It is recommended that they be integrated into PHPU annual work plans and budgets and that pooled funds are used to continue the forums. Some provinces have already done so.

8. Refresher trainings should be conducted, especially among district, health centre and village staff. As obtaining funds for this is a key issue, it is recommended that trainings be integrated into regular activities on a less formal/theoretical level and on a more on-site level.

61

9. If IEC materials are being developed or used for other activities at health centres, they should be in the form of large pictures. Small brochures with text are only effective if they are complemented by other materials and only benefit those who can read.

Recommendations for future projects involving multiple partners:

10. Ensure that similar projects in the future have a smaller and more effective PSC that includes people with relevant expertise. PSC should also be involved more in the decision-making process.

11. Ensure that all project partners are on board with relevant roles and profiles in the similar projects involving multiple partners in the future.

12. In the future, responsibilities of the key implementing partner should include some budget management so that financial management capacities for a large-scale project are built. More reporting structures may be needed, especially when involving financial reporting between bilateral organizations and government departments.

62

ANNEX

63

ANNEX 1: LOG FRAME OF HEALTH BCC PROJECT 2007

Intervention logic Objectively verifiable indicators

Means of verification

Assumptions and risks

Overall objective

Healthier lifestyle, better health-seeking behaviour and improved delivery of aspects of primary health care services, contributing to the reduction of poverty of the Cambodian population

To contribute to the reduction of infant mortality, maternal mortality, household expenditure on health and other health-related indicators as specified in the MoH Sector Strategic Plan

Sources as specified in the MoH Strategic Plan

Project purpose

The improvement of the capacity of national, provincial and district health providers to deliver effective BCC in Cambodia

[long-term outcome indicators:] 1. The national BCC

policymaking capacity is strengthened

2. The capacity of NCHP

to support implementation of BCC services at national and provincial levels is strengthened

3. 50% of PHPUs and

other key stakeholders use the BCC tools and resources developed by NCHP to implement BCC activities at the national or provincial level

4. 50% of primary health

care providers at district level in five provinces (Kampong Speu, Prey Veng, Svay Rieng, Stung Treng and Mondolkiri) have improved attitudes and skills to engage in quality health education and BCC with consumers, especially the poor.

5. 50% of target group

population in five provinces reached with NCHP- and PHPU-supported BCC campaigns and activities

Observation studies on provider behaviour/exit interviews with patients Project monitoring reports from provinces and of NCHP Baseline KAP study End-line KAP survey

a) Continuing economic growth and political stability b) Continued government commitment to the Health Sector Strategic Plan and improved health services c) Continuity of funds (domestic/ international) for health sector d) Government funding of PHDs and public providers becomes more regular and closer to the official budget e) Salaries of the health workforce will be paid regularly and will rise f) Willingness of vertical programmes to integrate activities with NCHP

Expected results

1. National BCC policy revised and distributed to key stakeholders 2. The NCHP has capacity to provide 5 key BCC services: a)

1. National BCC policy revised and endorsed by the government and key stakeholders by November 2007 2. a) BCC website online

- NCHP annual report - Revised national BCC policy for health - BCC Working Group Report

(a) Local structures (volunteers, committees) will come or remain committed to improving health

64




provision of information on BCC activities for health in Cambodia; b) specialist BCC training modules for national, provincial and district providers; c) development and distribution of protocols for BCC materials production; d) development and distribution of BCC Quality guidelines; e) M&E of BCC activities 3. PHPUs in five demonstration provinces have capacity to plan, implement and monitor BCC at provincial level, including training and supervision of health providers 4. Health providers of selected district in five demonstration provinces have improved BCC skills to promote behaviour change

with information by August 2006, and used as a resource by five PHPUs and key BCC stakeholders to improve BCC activities by December 2007. BCC newsletter produced and distributed to 24 PHPUs and key BCC stakeholders min. every 6 months from November 2007. BCC forums established in 24 provinces and used for information sharing min every 6 months by June 2007, b) IMC developed and pretested by November 2007. MPA Module 7 on health education for health workers revised to include training on BCC by June 2007. C-IMCI course for VHSGs revised to include training on BCC by April 2008, c) protocol for development of BCC materials and guideline for BCC implementation developed and distributed to 24 PHPUs and key BCC stakeholders by November 2007, d) S=system for M&E of NCHP and PHPU BCC activities developed and in use to produce and distribute quarterly monitoring reports by April 2008 and annual evaluation reports by December 2008. 3. All PHPUs in 5 provinces have participated in IMC training and passed the exam, and have been trained in provision and supervision of MPA Module 7 to health workers and Community IMCI to VHSGs. PHPUs use BCC website, protocol for BCC materials production and guidelines for BCC activities, and monitor and share reports with NCHP M&E team every quarter by December 2007. 4. 50% of health providers of selected districts have

- Minutes of BCC Forum meetings - Report on training modules - Provincial reports - Evaluation report on use of protocol and guidelines - Evaluation report on demonstration provinces

and health care (b) Willingness of donors to make NCHP an active partner in funded projects (c) Willingness of donors and projects with health promotion activities to coordinate with NCHP

65




participated in and passed revised MPA Module 7 and PBCI training by December 2008. 50% of selected health providers at district level have participated in NCHP-supported BCC/health promotion campaigns. Number of demonstration BCC programmes implemented, evaluated and disseminated. 50% of VHSGs of selected districts have participated in and passed revised Community IMCI training by December 2008.

Result 1 The national BCC policy making is strengthened

Means

Human Resources

Equipment and supplies Office supplies/project costs Other direct costs Administrative recovery costs Contingencies

Costs

€2,916,452

€1,397,574 €145,000

€850,350

€371,656

€24,968

Activities

1. Support to the BCC working group secretariat 2. Support policy seminars for BCC working group 3. Dissemination of policy decisions

Result 2 The NCHP is strengthened, and able to provide five key BCC services

Activities 1. Management training (including management, financial planning, organizational theory and quality assurance) 2. Advocacy training 3. Advocacy/marketing plan developed and applied 4. Organization of national BCC conferences 5. Production/distribution of an annual report 6. Expanding and maintenance of database to include research, evaluations and materials 7. Establishment of website for NHCP and maintenance/update 8. BCC forum meetings every 6 months, annually at provincial level 9. Fellowships (long and short term) 10. Training needs assessment (including review of Module 7) 11. Design of BCC modular courses as well as feasibility study and costing, delivery and review

66




12. Training of trainers 13. Course materials development 14. In-house training for NHCP staff on BCC materials protocols 15. Set up BCC materials team to develop BCC messages and materials 16. Set up BCC forum establishment and support group 17. Prepare, test, review, finalize, disseminate and follow-up on use protocols for BCC materials production 18. Set up Quality BCC Guidelines team drawing upon members of different NCHP Units 19. Prepare guidelines, organize a consultation workshop, finalize, disseminate and follow-up

Result 3 The capacity of PHPUs to support BCC is strengthened

Activities 1. Procurement of equipment for all provinces 2. Procurement of BCC materials by PHPUs with their enhanced budget

Result 4 The practices of health care providers are improved by demonstrating good practice BCC model in five selected provinces

Activities 1. Workshop on research and evaluation methodology 2. Identify target groups and health topics 3. Needs assessment 4. Upgrading of PHPU offices (5 provinces) 5. Design and development of BCC programmes for meeting specific needs of 5 provinces 6. Implement programmes 7. Monitor/evaluate BCC programmes 8. Dissemination and follow-up

67

ANNEX 2: LOG FRAME OF HEALTH BCC PROJECT 2010




Overall objective

Healthier lifestyle, better health-seeking behaviour and improved delivery of aspects of primary health care services, contributing to the reduction of poverty of the Cambodian population

To contribute to the reduction of infant mortality, maternal mortality, household expenditure on health and other health-related indicators as specified in MoH Sector Strategic Plan

Sources as specified in MoH Strategic Plan

Project purpose

The improvement of the capacity of national, provincial and district health providers to deliver effective BCC in Cambodia

[long-term outcome indicators:] 6. The national BCC

policymaking capacity is strengthened

7. The capacity of NCHP

to support implementation of BCC services at national and provincial level is strengthened

8. 50% of PHPUs and

other key stakeholders use the BCC tools and resources developed by NCHP to implement BCC activities at national or provincial level

9. 50% of primary health

care providers at district level in five provinces (Kampong Speu, Prey Veng, Svay Rieng, Stung Treng and Mondolkiri) have improved attitudes and skills to engage in quality health education and BCC with consumers, especially the poor

10. 50% of target group

population in five provinces reached with NCHP- and PHPU-supported BCC campaigns and activities

Observation studies on provider behaviour/ exit interviews with patients Project monitoring reports from provinces and of NCHP Baseline KAP study End-line KAP survey

a) Continuing economic growth and political stability b) Continued government commitment to the Health Sector Strategic Plan and improved health services c) Continuity of funds (domestic/ international) for health sector d) Government funding of the PHDs and public providers becomes more regular and closer to the official budget e) Salaries of the health workforce will be paid regularly and will rise f) Willingness of vertical programmes to integrate activities with NCHP

Expected results

1. National BCC policy revised and distributed to key stakeholders 2. The NCHP has capacity to provide 5 key BCC services: a) provision of information on

1. National BCC policy revised and endorsed by the government and key stakeholders by November 2007 2. a) BCC website online with information by August

- NCHP annual report - Revised national BCC policy for health - BCC Working Group Report - Minutes of BCC

(a) Local structures (volunteers, committees) will come or remain committed to improving health and health care

68




BCC activities for health in Cambodia; b) specialist BCC training modules for national, provincial and district level providers; c) development and distribution of protocols for BCC materials production; d) development and distribution of BCC Quality guidelines; e) M&E of BCC activities 3. PHPUs in five demonstration provinces have capacity to plan, implement and monitor BCC at provincial level, including training and supervision of health providers 4. Health providers of selected districts in five demonstration provinces have improved BCC skills to promote behaviour change

2006, and used as a resource by five PHPUs and key BCC stakeholders to improve BCC activities by December 2007. BCC newsletter produced and distributed to 24 PHPUs and key BCC stakeholders min. every 6 months from November 2007. BCC forums established in 24 provinces and used for information sharing min every 6 months by June 2007; b) IMC developed and pretested by November 2007. MPA Module 7 on health education for health workers revised to include training on BCC by June 2007. C-IMCI course for VHSGs revised to include training on BCC by April 2008; c) protocol for development of BCC materials and guidelines for BCC implementation developed and distributed to 24 PHPUs and key BCC stakeholders by November 2007; d) system for M&E of NCHP and PHPU BCC activities developed and in use to produce and distribute quarterly monitoring reports by April 2008 and annual evaluation reports by December 2008 3. All PHPUs in 5 provinces have participated in IMC training and passed the exam, and have been trained in provision and supervision of MPA Module 7 to health workers and Community IMCI to VHSGs. PHPUs use BCC website, protocol for BCC materials production and guidelines for BCC activities, and monitor and share reports with NCHP M&E team every quarter by December 2007 4. 50% of health providers of selected districts have participated in and passed revised MPA Module 7 and PBCI training by December

Forum meetings - Report on training modules - Provincial reports - Evaluation report on use of protocol and guidelines - Evaluation report on demonstration provinces

(b) Willingness of donors to make NCHP an active partner in funded projects (c) Willingness of donors and projects with health promotion activities to coordinate with NCHP

69




2008. 50% of selected health providers at district level have participated in NCHP-supported BCC/health promotion campaigns. Number of demonstration BCC programmes implemented, evaluated and disseminated. 50% of VHSGs of selected districts have participated in and passed revised C-IMCI training by December 2008

Result 1 The national BCC policy making is strengthened

Means

Human resources

Equipment and supplies Office supplies/project costs Other direct costs Administrative recovery costs Contingencies

Costs

€2,916,452

€1,397,574 €145,000

€850,350

€371,656

€24,968

Activities

1. Support to the BCC working group secretariat 2. Support policy seminars for BCC working group 3. Dissemination of policy decisions

Result 2 The NCHP is strengthened, and able to provide five key BCC services

Activities 1. Management training (including management, financial planning, organizational theory and quality assurance) 2. Advocacy training 3. Advocacy/marketing plan developed and applied 4. Organization of national BCC conferences 5. Production/distribution of an annual report 6. Expanding and maintenance of database to include research, evaluations and materials 7. Establishment of web site for NHCP and maintenance/update 8. BCC Forum meetings every 6 months, and annually at provincial level 9. Fellowships (long and short term) 10. Training needs assessment (including review of Module 7) 11. Design of BCC modular courses as well as feasibility study and costing, delivery and review 12. Training of trainers 13. Course materials

70




development 14. In-house training for NHCP staff on BCC materials protocols 15. Set up BCC materials team to develop BCC messages and materials 16. Set up BCC forum establishment and support group 17. Prepare, test, review, finalize, disseminate and follow-up on use protocols for BCC materials production 18. Set up Quality BCC Guidelines team drawing upon members of different NCHP Units 19. Prepare guidelines, organize a consultation workshop, finalize, disseminate and follow-up

Result 3 The capacity of PHPUs to support BCC is strengthened

Activities 1. Procurement of equipment for all provinces 2. Procurement of BCC materials by PHPUs with enhanced budget

Result 4 The practices of health care providers are improved by demonstrating good practice BCC model in five selected provinces

Activities 1. Workshop on research and evaluation methodology 2. Identify target groups and health topics 3. Needs assessment 4. Upgrading of PHP Unit offices (5 provinces) 5. Design and development of BCC programmes for meeting specific needs of 5 provinces 6. Implement programmes 7. Monitor/evaluate BCC programmes 8. Dissemination and follow-up

71

ANNEX 3: TOR UNICEF Cambodia

TERMS OF REFERENCE

End-of-project evaluation of the EU/UNICEF supported

Health Behavior Change Communication project 2005-2010

Requesting Section: Child Survival

1. Background:

The Health Behaviour Change Communication (BCC) Project, jointly supported by the European Union and UNICEF, was initiated on 1 January 2005, with a total value of 5,6 millions Euro. The Project is implemented by the National Center for Health Promotion (NCHP) and aims to promote healthier lifestyles, better health-seeking behaviour and improved delivery of Primary Health Care Services to contribute to reducing poverty in Cambodia. Project support focuses on contributing to the achievement of this overall objective through capacity strengthening of the NCHP and 24 Provincial Health Promotion Units (PHPUs), including the seven “good practice

demonstration” provinces21

, to deliver effective BCC. The detailed project description can be found in the Proposal for the Joint Co-financing Partnership between EU and UNICEF. Additional information on results is contained in the Health BCC Project’s Log Frame.

Originally planned for a period of five years, from 2005 to 2009, the project was extended to 31 December 2010 following a request for no-cost extension from UNICEF. The no-cost extension was recommended by the EU Monitoring Missions conducted in 2008 and 2009 and the Health BCC Project Steering Committee (PSC). The main purposes of the no-cost extension are: (1) based on recommendations of the project Mid-Term Review, to provide the opportunity to the NCHP and the 24 PHPUs to consolidate the current BCC achievements and (2) to help ensure a smooth transition of activities initiated with EU funding to other sources of support, primarily from the national budget and the second Health Sector Support Programme, thus leading to longer term programmatic and financial sustainability of the Health BCC project.

The first three years of the Health BCC project were complex and challenging due to a difficult set of organizational, management and financial issues affecting the project’s progress and coordination between the EU, UNICEF and NCHP. Despite accelerated implementation in 2008, at 80 per cent of its lifetime (31 December 2008) almost one third of the project budget was still unspent. This was, however, a considerable improvement compared to the situation in 2007 when only 35 per cent of the budget was utilized.

2009 was a turning year for the project. It was a year of significant achievements in terms of improved NCHP and PHPU capacities to implement and monitor communication actions and in terms of behavioural results achieved in the areas of BCC promotion. Financial implementation has also improved with 81 per cent of the project budget being utilized by 31 December 2009.

Another important development was the ability of NCHP with UNICEF support to mobilise additional resources for implementation of BCC activities at the national and sub-national level and, thus, to ensure geographical expansion of various BCC activities and to improve programmatic and financial sustainability of the project beyond the EU/UNICEF support. Overall, an additional US$ 265,000 was made available for 2010 for priority BCC activities from the MDG Spanish Fund and the second Health Sector Support Programme.

Progress against original objectives is further documented in the Mid-Term Review, Annual Reports and BCC monitoring reports.

2. Evaluation purpose:

An independent evaluation at the end of the project was planned as part of the Health BCC proposal. The specific objectives of the evaluation are as follows:

a) To assess the project’s performance using standard evaluation criteria of relevance/appropriateness, effectiveness, efficiency, impact (potential) and sustainability.

b) To document good practices and successes, to generate evidence based lessons and recommendations, and to guide the way forward to further strengthen on-going efforts, new initiatives including possible programme replication and scaling up in the area of behaviour change communication.

21

The seven good practice demonstration provinces are Kampong Speu, Kampong Thom, Mondol Kiri, Oddar Meanchey, Prey Veng, Stung

Treng, Svay Rieng

72

The results of the evaluation, including good practices, lessons learned and recommendations will be used by the Ministry of Health, EU, UNICEF and other partners working in the area of behaviour change communication to inform future programming in related areas.

3. Evaluation focus and purpose:

The evaluation will assess programme achievement, performance and effectiveness against the BCC project objectives to contribute to poverty reduction in Cambodia through “healthier lifestyle, better health-seeking behaviour and improved delivery of Primary Health Care services” and to the project purpose "to improve the capacity of national, provincial and district health care providers to deliver BCC activities effectively".

In order to assess degrees of achievement and performance, the evaluation will refer to the four main sets of activities:

1) Strengthening national BCC policy making

2) Capacity building of the National Centre for Health Promotion (NCHP) of the MOH in providing the following services:

a) Information services on BCC

b) Training of BCC specialists

c) Development of protocols to standardize the production of BCC materials

d) Development of guidelines to assure the quality of BCC

e) Evaluation of BCC activities

3) Capacity building of the Provincial Health Promotion Units of all 24 provinces to implement BCC activities

4) Intensive implementation of BCC activities in five (later agreed 7) selected provinces to demonstrate a model for good practices of BCC

More specifically, the evaluation will provide evidence-based analysis in order to answer the following questions:

Project relevance /appropriateness

How relevant and appropriate was the project design for:

Responding to the Cambodia health sector needs related to behaviour change communication set forth in the sector strategic plan and related policies?

Advancing/accelerating Cambodia progress towards health-related MDGs?

Meeting the demand and needs of service providers?

Responding to the demand and needs of the disadvantaged and less reached young children and families?

Are developed capacities of NCHP/PHPUs and communication tools developed in the framework of the project relevant for the current health care context in Cambodia?

Project effectiveness

What are the major achievements in the area of Health BCC in Cambodia that are attributable to the EU/UNICEF supported project; which strategies have yielded good results?

What is the project contribution to capacity-building of Health BCC policy makers, NCHP management/professional staff, PHPU staff? What do the new capacities consist of and how they are being used, including for BCC policy/programme development?

To what extent did the project contribute to increasing service providers’ knowledge and practices in the area of interpersonal communication and counselling, their responsiveness to patient’s needs, etc.? What is the estimated coverage of service providers with these interventions? Is there any evidence of increased knowledge and improved service providers’ practices after those interventions (e. g. follow-up visits)?

To what extent did the project contribute to increasing caregivers/families’ knowledge and practices in the area of health, particularly ANC, Community IMCI, community care of new borns, breastfeeding ? What is the estimated coverage of caregivers/families’ with related BCC interventions? Is there any evidence of improved caregivers/family knowledge and practices after those interventions (e. g. improved care-seeking, better home care)?

What were the main constraints to capacity development, both institutional and individual? What lessons and recommendations can be drawn for the future?

73

To what extent and in what ways did the project influence collaboration amongst various MoH structures and among MoH, other line ministries, local authorities, NGOs, CSOs and others at the national and sub-national levels?

Are there any particular good practices and lessons learned? If yes, what are those?

Project efficiency

How systematically have the funds been allocated/utilised at each level (across administrative levels and programme strategies/activities) to realise project objectives? If there were delays/deviations in fund allocation/utilisation, how were these justified and what are the implications for attaining programme objectives? What lessons and recommendations can be drawn for future behaviour change related development programmes

How effectively have the project resources been used to deliver outputs in a timely fashion in order to achieve project objectives? To what extent were funds used in a cost efficient manner in order to optimise project outcomes?

How effective and efficient were the coordination mechanisms at the national level?

How successful was the coordination between EU, UNICEF and NCHP? To what extent did the project influence internal structures, processes and goals within NCHP and UNICEF? (for the last question, if UNICEF does not want to include UNICEF in the assessment, NCHP proposes to delete this question form ToR)

Sustainability (country level)

The evaluation will examine institutional, technical/programmatic and financial sustainability and explore possible opportunities for expanding and disseminating successes, good practices and lessons learned to inform future BCC efforts in Cambodia:

Will the National Center for Health Promotion be able to sustain the implementation of activities without direct EU/UNICEF technical/financial support? What is the evidence regarding the level of programme and financial sustainability of the policies, training programmes, monitoring and supervision activities, communication plans, as well as the results achieved in the framework of the Health BCC project?

Will the Provincial Health Promotion Units be able to sustain the implementation of activities without direct EU/UNICEF technical/financial support? What is the evidence regarding the level of programme and financial sustainability of the communication, training, monitoring and supervision tools and results achieved in the framework of the Health BCC project?

What level of progress has been achieved to build the sustainability of the policies, training programmes, monitoring and supervision activities, communication plans, as we as the results achieved in the framework of the BCC project ?

What are the issues and options related to the feasibility for replication and expansion of key approaches developed by/through the BCC project?

Programme Impact (outcomes / potential impact)

The Health BCC project has been implemented for 5.5 years and the project activities’ intensity was thought to be sufficient to produce changes at the outcome level. The evaluation will address to the best extent possible the following questions some of which relate to outcome level changes and potential impact:

To what extent did the project contribute to creating an adequate BCC policy framework and to strengthening MoH capacities for BCC policy development?

What is the evidence regarding the results of capacity building efforts of NCHP and PHPUs supported by the Health BCC Project? How significantly has the project contributed to strengthening/improving approaches to Health BC communication?

To what extent did the project contribute to development and strengthening of partnerships at the national and sub-national level in support of Health BC Communication?

To what extent did the project contribute to improved knowledge and practices of caregivers and families in the areas targeted by the project (ex. antenatal care utilization)?

What is the evidence regarding the effectiveness/outcomes of the Health Behaviour Change Communication activities implemented in the framework of the project?

To what extent did the Health BCC project develop national and sub-national commitment to and ownership of activities developed in the framework of the project. Is there any evidence of increased budgetary

74

allocations? What are the success factors and lessons learned? Where/if this has not occurred fully, what are the constraints and consequent lessons for the future?

Intended users and uses of the evaluation:

Primary users of the evaluation will the Ministry of Health (including NCHP), EU and UNICEF. In addition, the findings of the Health BCC Project evaluation will be shared with broader group of interested partners, including NGOs, working in the area of communication and development.

Good practices and lessons learned will be used by the MoH for the development, implementation and monitoring of the Health BCC activities.

Also, the evaluation results will inform future communication programme design and implementation efforts undertaken by the MoH and interested partners.

Guiding principles

The evaluation process will adhere to the United Nations norms and standards and ethical guidelines for evaluation available at: http://www.uneval.org/currentwork/currentworklist.jsp?currentworkid=100&doc_cat_source_id=2&doc_source_id=100

_____________________________________________________________________________________

4. Programme Area and Specific Project Involved:

Programme areas: Child Survival

Project involved: Health Behavior Change Communication

Activity 2: Support to monitoring and evaluation of BCC activities

_____________________________________________________________________________________

5. Work Assignments/TOR (evaluation approach and methods):

1. Based on the evaluation focus, objectives and intended uses, design a detailed methodology for the evaluation, including evaluation plan and analytical framework for the evaluation report. Among others, the evaluation methodology will detail evaluation methods, with particular attention to qualitative methodologies; information to be collected with description of data collection methods, data collection instruments and information sources; types of data analysis.

2. Given the proposed scope of the evaluation, a multitude of methods will be used combining desk review, interviews with key informants, field observation visits, and focus group discussions among others:

Desk review of the relevant programme documents and secondary data. Programme documents will be prepared and provided by UNICEF and NCHP to the evaluator. Among others these include the original project document, the log-frame, annual project reports, EU monitoring reports, mid-term review report, ANC BCC campaign evaluation report, as well as BCC related policies, training packages, monitoring and supervision tools, communication strategies developed in the framework of the project.

For outcome/potential impact evaluation, additional sources of information are household based surveys, such as 2005 CDHS, 2008 CAS and 2009 CSES, as well as Health Information System data. A number of approaches and sources might also be employed, such as (a) coverage of health workers with training and supervision activities from the progress reports and (b) findings of the interviews and focus group discussions with health workers covered by training courses and (c) patients’ assessment of the counseling or observation of 1-2 consultations.

Interviews with key informants at the national and sub-national level, including members of the Health BCC Steering Committee, NCHP and UNICEF staff involved in project management and operations, National Programmes and Central MoH Departments involved in the BCC related activities, PHD managers and PHPU staff from selected areas.

Field observation and focus group discussions with health service providers and project participants/ beneficiaries (ex. pregnant women for ANC campaign). No major survey at the level of beneficiaries is envisaged.

3. Prepare the evaluation report in accordance with the agreed analytical framework. Among others the report will include the description of the Background, Analytical Framework, Methodology used, Bibliography/list of sources, Evaluation results, including Lessons Learned and Good Practices; Conclusions; and Recommendations. A

75

summary of the evaluation in Word and in Power Point will also be prepared. All the reports and presentations will make full use of visual materials (photos relevant to the project, graphs, etc.)

4. Facilitate the dissemination of findings and recommendations among key stakeholders.

Note on the process:

At the design stage, the contractor will work in close consultation with UNICEF M&E section, EU and MoH. Before finalization, the proposed methodology and tools will be shared with UNICEF, EU and MoH/NCHP for inputs/comments. The feedback from the stakeholders will be used to finalize the evaluation report and summary documents.

The evaluation report will also be shared for comments with the MoH, EU, UNICEF. The feedback will be addressed in the final version of the report.

6. Qualifications/Competencies Required:

The evaluation is planned to be conducted by an institution or by a registered consultancy group/firm. It is recommended that the team consists of at least three experts (a leader and 2 technical experts). One of the experts or the team leader should have significant research/evaluation background in the area of health communications. Another will have substantial experience in the area of qualitative research design and data analysis (both from quantitative and qualitative studies). The exact division of work will be decided by the institution/team, however the team leader will have the responsibility for all negotiations, decisions and deliverables. The technical work will be divided between the team leader and the team members.

The qualifications and experience required for the members of the evaluation team are as follows:

At least 5 years of experience in research/evaluation/data analysis; experience in both qualitative and quantitative research techniques is required.

Extensive expertise and experience in designing and implementing evaluations in development programming is required.

Knowledge of institutional issues related to development programming (including funding, administration, EU development assistance, the role of the UN system, partnerships, human rights, sustainable development issues)

Work experience in developing countries is required; previous working experience is Cambodia is an asset;

Demonstrated skill in conducting evaluations of communication (or related) programmes is an asset;

Strong analytical and report writing skills;

Good communication, advocacy and people skills. Ability to communicate with various stakeholders. Ability to express concisely and clearly ideas and concepts in written and oral form.

Language proficiency: English (mandatory), Khmer (an asset).

The qualifications and experience required for the institution/consultancy firm include:

Experience with programme evaluations of a similar nature, demonstrated by a list of past and current clients, with particular emphasis on UN agencies and NGOs.

Valid Government business license, NGO or institutional registration to operate in Cambodia.

_____________________________________________________________________________________

7. Estimated Duration of Contract (Dates and period):

The evaluation is expected to commence in September and to be completed by 15 December 2010.

_____________________________________________________________________________________

8. Deliverables with due date: (please mentioned here the number of working day and total budget for the contract)

No. Description of deliverables Timeline for submission

76

1. Draft methodology for evaluation, including evaluation plan/tools, and analytical framework for the evaluation report (for comments to MoH, EU, UNICEF)

Within 2 weeks of signing of the contract

2. Final methodology for evaluation, including evaluation plan/tools, and analytical framework for the evaluation report

Within 4 weeks of signing of the contract

3. Draft evaluation report (for comments to MoH, EU, UNICEF) By 22 November

4. Final evaluation report By 15 December

_____________________________________________________________________________________

9. Payment Schedule (Please link to deliverables to the extent possible):

No. Instalment linked to deliverable Payment value

1. Upon signing the contract 20%

2. Final methodology for evaluation 30%

3. Final evaluation report 50%

_____________________________________________________________________________________

10. Official Travel Involved (specify if yes):

The contractor will travel to a mix of provinces reached by the project (demonstration provinces, focus and other provinces, etc...). UNICEF will facilitate travel organization, logistics and contacts with sub-national partners.

_____________________________________________________________________________________

11. Contract Supervisor:

The evaluation will be supervised by the UNICEF Monitoring and Evaluation Specialist.

_____________________________________________________________________________________

12. Type of Supervision that will be provided:

The contract supervisor will ensure that the contractor has detailed orientation and briefing of the ToR at the beginning of the consultancy; is introduced to the relevant MoH, EU and UNICEF staff involved in the project. The supervisor and the contractor will discuss the deliverables on regular basis during the consultancy.

Supervisor is responsible to monitor the on-time delivery of the expected outputs/results and will certify, jointly with the EU, the deliverables upon their completion; evaluate the contractor’s work and follow up on the payment.

_____________________________________________________________________________________

13. Contractor’s Work Place/Any facilities to be provided by office:

Not envisaged

_____________________________________________________________________________________

14. Nature of ‘Penalty Clause’ to be Stipulated in Contract:

Payment: All payments are subject to satisfactory completion and submission of due deliverables, certification, and evaluation of the work done by the contract supervisor. Performance indicators: Contractors’ performance will be

evaluated against the following criteria: meeting ToR specification, timeliness and quality of deliverables.

_____________________________________________________________________________________

77

ANNEX 4: Discussion Guides

Evaluation of the Health BCC Project

Discussion Guide – IDI – PHD (Chief or vice-chief in charge of health promotion)

INTRODUCTION Objective: This section is the ‘warm up’ section to help the moderator understand the respondent’s role as a PHD.

A Province:

B Name of the respondent:

C Designation/Title of respondent:

D

Place of interview:

E

Date:

Hello, I’m.............[Insert interviewer name] from MSD, which is a consultancy company. MSD, along with TNS (also a global consultancy company), has been contracted by UNICEF to conduct the final evaluation of the Health Behaviour Change Communication (BCC) project. As the PHD is one of the key provincial-level stakeholders in this project, your opinion is very valuable to us. Your information will be kept strictly confidential and will only be used for analysis purposes, along with many other interviews. Thank you very much for taking the time to participate in this interview.

F How long have you been in this position in the PHD?

years

Instruction: Only those who have been in the position for at least 2 years have been recruited for the interview.

G Could you please tell me something about your role as a PHD?

What are your roles and responsibilities as PHD? Are you satisfied/dissatisfied with it? Why? Why not?

ABOUT BCC PROJECT Objective: The aim of this section is to find out the respondent’s awareness and understanding of the BCC project.

A Do you know about the BCC project?

Yes

No

Not sure

B If yes, can you please explain what it is?

Instruction: If the respondent does not know about the BCC project, moderator to probe. If the respondent does not know even after probing, please continue to the next section.

C Do you know the objectives of the BCC project?

Yes

No

Not sure

D Can you please elaborate on the objectives of the BCC?

Instruction: If the respondent does not know about the objectives of the BCC project, interviewer to probe.

PLANNING Objective: This section aims to understand the planning process with regard to the Health BCC project and if the project has led to an increase in capacity in planning of health promotion activities.

A Do you think your staff has better capacity to plan health promotion activities now?

Yes to a great extent

Yes to some extent

Not really

B Why do you say so? Please give examples etc.

78

C What is your role in planning health promotion activities in your area?

D Who are the different stakeholders/partners you interact with in planning BCC activities? Instruction: Moderator to probe, not prompt. Only after the respondent mentions all partners, mention the ones he has left out and ask ‘”what about…”

NCHP - What is the level of interaction?

- Is there any overlap in your role? How is your role different from PHPU etc?

- Do you think any improvement is needed to make collaboration with other partners better? Why? Why not?

PHPU

OD

health centre

VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder (NCHP, PHPU, OD, etc) listed above.

E What are some of the main challenges faced in planning activities?

- lack of human resources - lack of finances - lack of capacity - other?

IMPLEMENTATION Objective: This section aims to understand how BCC activites are implemented and the various stakeholders and their roles in implementation and challenges faced. It also aims to understand if the Health BCC project has led to an increase in capacity of staff at various levels with regard to implementing health promotion activites.

A Do you think your staff has better capacity to implement

activities now?


Yes to some extent

Not really


C What is your role in implementing health promotion activities in your area?

D Who are the different stakeholders/partners you interact with in implementing

BCC activities? Instruction: If all aspects of planning, implementation and M&E have been covered in the previous question, do not repeat the question..




PHPU

OD

Health centre

VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder (NCHP, PHPU, OD etc) listed above

E What are some of the main challenges faced in implementing activities?

- lack of human resources - lack of training - lack of finances - lack of capacity - other

INTERPERSONAL/COUNSELLING SKILLS Objective: This section aims to understand how interpersonal and couselling skills of health workers have improved since the implementation of the Health BCC project.

A Can you please explain what interpersonal skills are?

B Did the project contribute to strengthening the interpersonal skills of the health workers?


Yes to some extent

Not really

C Why do you say so? Please give examples of how their skills have improved.

79

D If no, why do you think their skills have not improved? What is required for their skills to improve?

Instruction: Moderator to probe for different aspects such as training and monitoring. Moderator to probe by asking, “what else”, “anything else” etc.

E Have you seen any change in the last 2-3 years in the health centre in your area in terms of:

Service delivery Yes to a large extent Yes to some extent Not really

Behaviour of health workers/staff to clients

Yes to a large extent Yes to some extent Not really

Interpersonal skills of health workers/staff


Any other changes? Yes to a large extent Yes to some extent Not really

F Why do you say so? What kind of changes do you see? Please give examples.

G What more should be done to improve the quality of various aspects of service delivery, behaviour, etc of health staff?

BCC FORUMS Objective: This section aims to understand the relevance and sustainability of the BCC forums.

A Have you heard or been involved in a BCC forum?

Yes No

Instruction: If the respondent does not know what BCC forums are, moderator to probe. If the respondent does not know even after probing, please proceed to the next section.

B Can you tell me more about the BCC forums?

Instruction: Moderator to ask, “What are these forums’’; “How often are these forums held’”; ”Who plans and organizes these forums”?

C How useful are these BCC forums?

Very useful

Somewhat useful

Not useful

D Why do you say so?

E How important is it to continue having these BCC forums?

Very important

Important

A little important

Not so important

F Why do you say so?

G Do you think it will be possible to continue organizing the BCC forums?

Yes

No

Not sure

H If yes, why do you say so? Where will the funding for the BCC forums come from? Where are you mobilizing funds for the BCC forums from?

I If no/not sure, why not? What are the main challenges in organizing the BCC forums?

J What support will you need from the central level for this?

K Have you included BCC forums in your annual work plans?

Yes

No

Not sure

80

L How do you prioritize this activity?

IEC MATERIALS Objective: The key objective of this section is to understand the different types of IEC materials that are available for health promotion and if these materials are sufficient and effective.

A What IEC materials are available for health promotion in your area?

B Do you think these materials are adequate in terms of:

Number of materials available

- Why do you say so?

- What do you do in case you need more materials?

- When you ask for more IEC materials, is this request met promptly?

- How are these materials distributed to the health centres and VHSG?

- Do you think any improvement is needed in distribution of IEC materials? Any suggestions?

- Of all the IEC materials, which ones do you think are most effective?

Content of materials

TRAINING Objective: As a key component of the Health BCC project was capacity building through training, this section aims to check if the respondent has been trained under the project and the respondent’s opinion on the relevance and effectiveness of the training.

A Have you received any training as part of the BCC project?

Yes

No

Not sure

B If yes: When By whom What did you learn from the training?

C If yes, what training have you received?

MPA Module 7 Basic health education (C-IMCI Module 2) Hygiene (C-IMCI Module 7) Home care of sick children (C-IMCI Module 8) Early childhood development (C-IMCI Module 9) PBCI Client/provider rights ABCP BCC protocol for IEC development BCC guidelines Others

D To what extent do you think learning from the training can be applied?

To a large extent

To some extent

Not at all

E Why do you say so?

F Which modules did you think were particularly useful and why?

G Which modules did you think were not so useful? Why?

H Have you/other PHD staff been a trainer and trained any OD/health centre staff?

Yes

No

I What recommendations can you make to help improve the

81

training courses so that they are more applicable and helpful to your work?

VHSG & HOME VISITS Objective: This section aims to understand the implementation of the Health BCC project at the grassroots level (i.e. VHSG home visits) and challenges faced.

A

How important are the home visits made by the VHSG?

Very important

Somewhat important

Not so important

Not at all important

B

Why do you say so?

C

What is the quality of the home visits conducted by VHSGs for the care of mothers and newborns?

Excellent

Good

Needs improvement

Poor

D

Why do you say so?

E How do you promote and ensure good quality of home visits by VHSGs in your area?

-through monthly meetings -through review of records and registers -through field visits -other?

F What is the status of home visits in your PHD/OD area (proportion of VHSGs making regular and timely home visits)?

G What are the main challenges faced in continuing home visits?

- What do you think are the essential conditions to improve and sustain home visits in your PHD/OD area?

- More training required? Monetary? Others?

H Did you receive any support from an external agency to strengthen and promote home visits by VHSGs?

- If yes, name them.

- What was the nature of support received?

IMPACT ON COMMUNITY Objective: The key objective of this section is to get the respondent’s opinion as to what the impact of the Health BCC project has been on the community.

A

Do you see a change between the current and past situation in the last 1-2 years in your HF area in terms of service utilization and community behaviour change in health?

Yes to a large extent

Yes to some extent

Not really

B

If yes, what are the changes?

C

What do you think are the factors that have contributed to this change?

D

If there have been no changes, what do you think are the barriers/factors that have prevented change?

E

What more do you think should be done to promote better health-seeking behaviour among the community?

F

What are the current challenges faced in promoting behaviour change among the community?

82

MONITORING & SUPERVISION Objective: This section aims to understand how monitoring is done at various levels and if the Health BCC project contributed to increased capacity and skills in M&E.

A Do you think your staff has better capacity to monitor and supervise project activities?


Yes to some extent

Not really

B In what way?

C What more do you think is required to further improve their M&E skills?

D How and in what way do you as PHD supervise health promotion activities at OD-/health-centre level? Please give examples.

E How and in what way does the PHPU supervise health promotion activities at OD-/health-centre level? Please give examples.

F How and what does the OD supervise?

- health centres

- VHSG

G Do you monitor the status of home visits made by VHSGs in your supervisory area?

Yes - What key aspects do you review? - How frequently? - What are your sources of information for assessing

the status of home contact in your HF area?

No Why not?

H Do you think the monitoring and supervision from the central level is adequate?

Yes Why do you say so? What kind of monitoring is done from the central level?

No

I What more should be done in monitoring and supervision of activities?

OVERALL IMPACT, SUSTAINABILITY, LESSONS LEARNED Objective: The objective of this section is to get concluding remarks on the BCC project as a whole – which aspects were useful, support received and lessons learned. It also captures the respondent’s opinion on sustainability of the project and activities.

A What elements of the BCC project did you find most useful? Why do you say so?

B What elements of the BCC project did you find less useful?

C Why do you say so?

D Do you get enough support from NCHP for health promotion activities?

Yes, more than enough

Yes, to some extent but could use more support

No support at all

E

Why do you say so?

F Do you get enough support from other partners? Which partners?

NGOs

PHPU

OD

District/Commune chief

Health workers/VHSG

Others

Instruction: Please ask the above question for each stakeholder – probe for each category. Wait for respondent to respond and then probe for partners he has not listed.

G

Why do you say so?

83

H

What more support would you need? From whom?

I

What would be required to expand the BCC activities to other areas in your province?

J

Is there any documentation of good practices and lessons learned available?

K

Do you think that now that EC/UNICEF have stopped financial and technical assistance, the capacity of provincial level staff is adequate enough to continue implementing the activities?

Yes definitely

- Why do you think so? - What more do you think is

required for the activities to continue?

Yes to some extent

Not at all

Not sure

L Do you have any other recommendations for the BCC project? Any improvements/lessons learned for the future?

***Thank respondent and close the discussion***

84

Evaluation of the Health BCC Project Discussion Guide – IDI – PHPU

INTRODUCTION Objective: This is the ‘warm up’ section to help the moderator understand the respondent’s role as a PHPU.

A Province:



D

Place of interview:

E

Date:

Hello, I’m.............[Insert interviewer name] from MSD, which is a consultancy company. MSD, along with TNS (also a global consultancy company), has been contracted by UNICEF to conduct the final evaluation of the Health Behaviour Change Communication (BCC) project. As the PHPU is one of the key provincial-level stakeholders in this project, your opinion is very valuable to us. Your information will be kept strictly confidential and only be used for analysis purposes, along with many other interviews. Thank you very much for participating in this interview.

F How long have you been in this position in the PHPU?

years

Instruction: This should have been a screening criteria wherein only those people who have been in the position for at least 2 years have been recruited for the interview.

G Could you please tell me something about your role as a PHPU?

What are your roles and responsibilities as a PHPU? Are you satisfied/dissatisfied with it? Why? Why not?



Yes

No

Not sure


Instruction: If the respondent does not know about the BCC project, moderator to probe. If the respondent does not know even after probing, please go to the next section.


Yes

No

Not sure

D Can you please elaborate more on the objectives of the BCC?



A Do you think the PHPU has better capacity to plan now?


Yes to some extent

Not really



D Who are the different stakeholders/partners you

NCHP - What is the level of interaction? PHD

85

interact with in planning BCC

activities? Instruction: Moderator to probe, not prompt. Only after the respondent mentions all partners, mention the ones he has left out and ask “what about…”?

OD - Is there any overlap in your role? How is your role different from PHD, etc?


Health centre

VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder (NCHP, PHD, OD, etc) listed above


- lack of human resources - lack of finances etc?

IMPLEMENTATION Objective: This section aims to understand how BCC activites are implemented and the various stakeholders and their roles in implementation and challenges faced. It also aims to understand if the Health BCC project has led to an increase in capacity of staff at various levels with regard to implementing health promotion activities.

A Do you think the PHPU has better capacity to implement

activities now?


Yes to some extent

Not really



D Who are the different stakeholders/partners you interact with in implementing BCC activities? Instruction: If all aspects of planning, implementation and M&E have been covered in the previous question, do not repeat the question.


- Is there any overlap in your role? How is your role different from PHD, etc?


PHD

OD

Health centre

VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder (NCHP, PHD OD, etc) listed above.


- lack of human resources? - lack of training? - lack of finances etc?

INTERPERSONAL/COUNSELLING SKILLS Objective: This section aims to understand how interpersonal and counselling skills of health workers have improved since the implementation of the Health BCC project.


B Did the project contribute to strengthening the interpersonal skills of the health workers?


Yes to some extent

Not really



Instruction: Moderator to probe for different aspects, such as training, monitoring. Moderator to probe by asking, “what else”, “anything else”, etc.

E Have you seen any change in the last 2-3 years in the health centre in your area in terms of:


86



Interpersonal skills’ of health workers/staff







Yes No



Instruction: Moderator to ask, “what are these forums’’, “how often are these forums held”, “who plans and organizes these forums”.


Very useful

Somewhat useful

Not useful


E How important is it to continue having these BCC forums

Very important

Important

A little important

Not so important


G Do you think it will be possible to continue organizing the BCC forums

Yes

No

Not sure

H If yes, why do you say so? Where will the funding for the BCC forums come from? Where are you mobilizing funds for the BCC forums?


J What support will you need from the central level for this?

K Have you included BCC forums in your annual work plans?

Yes

No

Not sure

L How do you prioritize this activity?

87








- How are these materials distributed to the health centres and VHSG?





A Have you received any training regarding community care of mothers and newborns?

Yes

No

Not sure

B If yes: When? By whom? What did you learn from the training?

C If yes, what other training have you received?

MPA Module 7 Basic health education (C-IMCI Module 2) Hygiene (C-IMCI Module 7) Home care of sick children (C-IMCI Module 8) Early childhood development (C-IMCI Module 9) PBCI Client/provider right ABCP BCC protocol for IEC development BCC guideline Others

D

To what extent do you think learning from the training can be applied?

To a large extent

To some extent

Not at all




H Have you/other PHPU staff been a trainer and trained any OD/health centre staff/VHSG?

Yes

No

I What recommendations can you make to help improve the training courses so that they are more applicable and helpful to

88

your work?


A


very important

Somewhat important

Not so important


B

Why do you say so?

C


Excellent

Good

Needs improvement

Poor

D

Why do you say so?

E How do you promote and ensure good quality visits to pregnant and post-partum women by VHSGs in your OD area?


F

What is the status of home visits in your area (proportion of VHSGs making regular and timely home visits)?

G What are the main challenged faced in continuing home visits?

- What do you think are the essential conditions to improve and sustain home visits in your area?






A

Do you see a change between the current and past situation in the last 1-2 years in your HF area in terms of service utilization and community behaviour during pregnancy, childbirth and in the post partum period?


Yes to some extent

Not really

B


C


D


E

What more do you think should be done to promote better health seeking behaviour among the community?

F


89




Yes to some extent

Not really

B In what way?


D How do you as PHPU supervise health promotion activities at OD level? Please give examples.

E How does the PHD supervise health promotion activities at OD level? Please give examples.

F How does the OD supervise - health centres

- VHSG

G Do you monitor the status of home visits made by VHSGs in your supervisory area?

Yes - what key aspects do you review - how frequently - what are your sources of information for assessing


No Why not?



No

I What more should be done in monitoring and supervision of activities?





D Do you get enough support from NCHP for health promotion activities



No support at all

E

Why do you say so?

F Do you get enough support from PHD for health promotion activities



No support at all

G

Why do you say so?

H Do you get enough support from OD for health promotion activities



No support at all

I

Why do you say so?

J Do you get enough support NGOs

90

from other partners? Which partners?


Health workers/VHSG

Others

Instruction: Please ask the above question for each stakeholderprobe for each category. Iat for respondent to respond and then probe for partners he has not listed.

K

Why do you say so?

L


M


N


O


Yes definitely



Yes to some extent

Not at all

Not sure

P Do you have any other recommendations for the BCC project? Any improvements/lessons learned for the future?

91

Evaluation of the Health BCC Project Discussion Guide – IDI – OD (DHPU and MCH)

INTRODUCTION Objective: This is the ‘warm up’ section to help the moderator understand the respondent’s role as a OD.

A Province:



D

Place of interview:

E

Date:

Hello, I’m.............[Insert interviewer name] from MSD, which is a consultancy company. MSD, along with TNS (also a global consultancy company), has been contracted by UNICEF to conduct the final evaluation of the Health Behaviour Change Communication (BCC) project. As the OD is one of the key provincial-level stakeholders in this project, your opinion is very valuable to us. Your information will be kept strictly confidential and only be used for analysis purposes, along with many other interviews. Thank you very much for participating in this interview.

F How long have you been in this position in the OD?

years


G Could you please tell me something about your role as an OD?

What are your roles and responsibilities as an OD? Are you satisfied/dissatisfied with it? Why? Why not?



Yes

No

Not sure




Yes

No

Not sure


Instruction: If the respondent does not know about the BCC project, moderator to probe.


A Do you think your staff has better capacity to plan now?


Yes to some extent

Not really



D Who are the different NCHP - What is the level of

92

stakeholders/partners you interact with in planning BCC activities? Instruction: Moderator to probe, not prompt. Only after the respondent mentions all partners, mention the ones he has left out and ask “what about…”?

PHPU interaction? - Is there any overlap in your

role? How is your role different from PHD, etc?


PHD

Health centre

VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder (NCHP, PHPU, OD, etc) listed above


- lack of human resources? - lack of finances etc?



activities now?


Yes to some extent

Not really







PHPU

PHD

Health centre

VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder NCHP, PHPU, PHD etc listed above


- lack of human resources - lack of training - lack of finances etc



B

Did the project contribute to strengthening the counselling skills of the health workers?


Yes to some extent

Not really




E Have you seen any change in Service delivery Yes to a large extent

93

the last 2-3 years in the health centre in your area in terms of:

Yes to some extent Not really










Yes No





Very useful

Somewhat useful

Not useful



Very important

Important

A little important

Not so important



Yes

No

Not sure

H If yes, why do you say so?


J What support will you need from the PHD and central level for this?






- What do you do in case you need more materials? Content of materials

94


C How often do you distribute IEC materials to the health centres?

Monthly - How are these materials distributed to the health centres and VHSG?

Quarterly

Every 6 months

Annually

Only when the health centre requests them

Never

D How are IEC materials stored at the OD?

E Have you ever faced any problems in distributing IEC materials to the health centres in your area?

Yes always - If yes, what kind of problems do you face?

- What do you suggest to improve this? Yes sometimes

No

F What recommendations do you have for making the use and distribution process of IEC materials easier? Do you think any improvement is needed in distribution of IEC materials? Any suggestions?

F Of all the IEC materials, which ones do you think are most effective?


A Have you received any training regarding community care of mothers and newborns?

Yes

No

Not sure


C If yes, what other training have you received?

MPA Module 7 Basic Health Education (C-IMCI Module 2) Hygiene (C-IMCI Module 7) Home care of sick children (C-IMCI Module 8) Early childhood development (C-IMCI Module 9) PBCI Client/provider rights ABCP BCC protocol for IEC development BCC guideline Others


To a large extent

To some extent

Not at all




H Have you/other OD staff been a Yes

95

trainer and trained any health centre staff/VHSG?

No

I What recommendations can you make to help improve the training courses so that they are more applicable and helpful to your work?


A

How important are the home visits made by the VHSGs?

Very important

Somewhat important

Not so important


B

Why do you say so?

C


Excellent

Good

Needs improvement

Poor

D

Why do you say so?

E How do you promote and ensure good quality visits to pregnant and post-partum women by VHSGs in your OD area?


F

What is the status of home visits in your OD area (proportion of VHSGs making regular and timely home visits)?


- What do you think are the essential conditions to improve and sustain home visits in your PHD/OD area?






A

Do you see a change between the current and past situation in the last 1-2 years in your HF area in terms of service utilization and community behaviour during pregnancy, childbirth and in the post-partum period?


Yes to some extent

Not really

B


C


D


E

What more do you think should be done to promote better health seeking behaviour among the community?

96

F





Yes to some extent

Not really

B In what way?


D How does the PHD supervise health promotion activities at OD level? Please give examples.

E How does the OD supervise: - health centres

- VHSGs

F Do you monitor the status of home visits made by VHSGs in your supervisory area?

Yes - What key aspects do you review? - How frequently? - What are your sources of information for assessing


No Why not?

G What more should be done in monitoring and supervision of activities?








No support at all

E

Why do you say so?

F Do you get enough support from PHD for health promotion activities?



No support at all

G

Why do you say so?

H Do you get enough support from PHPU for health promotion activities?



No support at all

I

Why do you say so?

J Do you get enough support from other partners? Which partners?

NGOs


Health workers/VHSG

97

Others

Instruction: Please ask the above question for each stakeholder; probe for each category. Wait for respondent to respond and then probe for partners he has not listed.

K

Why do you say so?

L


M


N


O


Yes definitely



Yes to some extent

Not at all

Not sure

P Do you have any other recommendations for the BCC project? Any improvements/lessons learned for the future?


98

Evaluation of the Health BCC Project Discussion Guide – IDI – Health Centre Chief

INTRODUCTION Objective: This is the ‘warm up’ section to help the moderator understand the respondent’s role as a health centre chief.

A Province:



Health centre chief

Others

D

Place of interview:

E

Date:

Hello, I’m.............[Insert interviewer name] from MSD, which is a consultancy company. MSD, along with TNS (also a global consultancy company), has been contracted by UNICEF to conduct the final evaluation of the Health Behaviour Change Communication (BCC) project. As the health centre staff is one of the key stakeholders in this project, your opinion is very valuable to us. Your information will be kept strictly confidential and will only be used for analysis purposes, along many other interviews. Thank you very much for participating in this interview.

F How long have you been in this position in the health centre?

years


G Could you please tell me something about your role as health centre chief___________?

What are your roles and responsibilities as health centre chief__________? Are you satisfied/dissatisfied with it? Why? Why not?

H Can you tell me: How many communes are covered by this health centre? How many villages are covered by this health centre? How many staff work in this health centre? How many village health volunteers are attached to this health centre?



Yes

No

Not sure




Yes

No

Not sure



99


A Do you think your staff has better capacity to plan health promotion activities now?


Yes to some extent

Not really



D Who are the different stakeholders/partners you interact with in planning health promotion activities? Instruction: Moderator to probe, not prompt. Only after the respondent mentions all partners, mention the ones he has left out and ask “what about…”

OD - What is the level of interaction?

- Is there any overlap in your role? How is your role different from OD etc?


VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder (OD, VHSG, etc) listed above.





activities now?


Yes to some extent

Not really




health promotion activities Instruction: If all aspects of planning, implementation and M&E have been covered in the previous question, do not repeat the question.


- Is there any overlap in your role? How is your role different from OD, etc?


VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder (OD, VHSG, NGO etc)



100


A Have you received any training as part of the project?

Yes

No

Not sure



MPA Module 7 Basic health education (C-IMCI Module 2) Hygiene (C-IMCI Module 7) Home care of sick children (C-IMCI Module 8) Early childhood development (C-IMCI Module 9) PBCI Client/provider rights ABCP BCC protocol for IEC development BCC guideline Others


To a large extent

To some extent

Not at all




H Which modules according to you were easy to understand?

I Which modules were not so easy to understand?

- Why not? What aspects were difficult? - What do you suggest we can do to make it easier to understand?

J Have you/other health centre staff been a trainer and trained any VHSG?

Yes

No

K What recommendations can you make to:

- improve the training courses so that they are more applicable and helpful to your work?

- improve training courses for VHSG?

- improve refresher trainings? Are they sufficient? How often do you think refresher training is required?



B Do you think the BCC project has contributed to the interpersonal skills of your health centre staff/midwives?


Yes to some extent

Not really



E What new things do you think the health workers have learned as a result of the BCC project?

101

F Have you seen any change in the last 2-3 years in the health centre in your area in terms of:







G Why do you say so? What kind of changes do you see? Please give examples (ask with regard to each aspect – service delivery; behaviour; knowledge and others)

H What more should be done to improve the quality of various aspects of service delivery, behaviour, etc of health staff?

I Imagine there is a pregnant woman who has come to the health centre. Please explain what you will say and how will

you offer your counselling skills.

ANC package Interviewer to probe: What else, anything else?

Delivery at health centre Interviewer to probe: What else, anything else?

What do you do to make the woman feel comfortable when giving her advice? How do you speak to her?

Moderator to probe: Anything you do in particular to make her feel comfortable, body language, etc?

J In general, what are the main topics of health education your health centre staff/midwives give to members of the community? And how do you do it (through outreach activities, meetings, health education session, home visit, other)?

Breastfeeding Micronutrients Home care for sick children, fever, diarrhoea and ARI Hygiene, water and sanitation Malaria/dengue fever Early child care and mental development Immunization

K Which of these were explained specifically during the BCC project training?


A What IEC materials are available for health promotion in your health centre?







C How often do you receive IEC materials at the health centre?

Monthly - How are these materials distributed to the VHSG?

Quarterly

Every 6 months

Annually

Only when we request for it

Never

D How are IEC materials displayed and stored at the health centre?

102

E Have you ever faced any problems in displaying IEC materials in your health centre?



No


G Of all the IEC materials, which ones do you think are most effective?

- What is easy for people to understand?

- What materials are attractive/catch people’s attention?

H Which IEC materials are not effective?

- Why not?


A


Very important

Somewhat important

Not so important


B

Why do you say so?

C


Excellent

Good

Needs improvement

Poor

D

Why do you say so?

E

Has the BCC project helped improve the skills of VHSGs?


Yes to some extent

Not really

F

Why do you say so?

- If yes, how? Please give examples.

- If no, what more do you think needs to be done to improve the skills of VHSGs?

G How do you promote and ensure good quality of home visits by VHSGs who are attached to your health centre?


H

What is the proportion of VHSGs making regular and timely home visits?

I What are the main challenges faced in continuing home visits?

- What do you think are the essential conditions to improve and sustain home visits in your areas?


J Did you receive any support from an external agency to strengthen and promote home visits by VHSGs?




A Do you know about BCC forums?

Yes No


103


Instruction: Moderator to ask, “what are these forums’’, “how often are these forums held”, “who plans and organizes these forums”, “what do they discuss”.


Very useful

Somewhat useful

Not useful



Very important

Important

A little important

Not so important

F What is your role in the BCC forum?


Yes

No

Not sure



J What support do you think is needed to be able to organize the BCC forums?




Yes to some extent

Not really

B In what way?

C What more do you think is required to further improve their monitoring skills?

D How and in what way does the health centre supervise health promotion activities at village/commune level? Please give examples.

E How often do you supervise health promotion activities in the village/commune per year?

- 4 times or more per year

- 2-3 times per year

- 1 time per year

- Never

F How useful is this supervision? Very useful Somewhat useful Not useful

G Why do you say so?

H What are the main challenges?

I Do you have a monitoring tool/checklist for monitoring of VHSG’s works?

- Yes

- No

- Don’t know

Instruction: Moderator to probe about monitoring tools/checklist

J Do you use the monitoring tools/checklist?

- Yes always

- Yes sometimes

- Yes rarely

- No

104

K Do you find the monitoring tools very useful?

- Yes to a great extent

- Yes to some extent

- No

L Why do you say so? What suggestions do you have to improve it?

M What are the main challenges faced in monitoring of activities at the village/commune level?

Instruction: Moderator to probe by asking, “what else?”

N

Who supervises activities of the health centre and how?

Yes to some extent

O

Do you monitor the works of VHSGs at the community in your area?

Not really - What kind of VHSG work do you monitor? (home visits, health education sessions, support outreach services, etc.)?

- How frequently?

No Why not?

P What more should be done in monitoring and supervision of activities:

- in terms of provincial- and district-level monitoring of health centres?

- In terms of health centre montoring the VHSG?


A

Do you see a change between the current and past situation in the last 1-2 years in your area in terms of service utilization and community behaviour change in health?


Yes to some extent

Not really

B

If yes, what are the changes? Ask specifically for antenatal care; childbirth and post-partum care.

C


D If there have been no changes or there have been some changes, what do you think are the barriers/factors that have prevented change?

E


F









No support at all

No direct support from NCHP but through provincial level

105

E

Why do you say so?




No support at all

G

Why do you say so?




No support at all

No direct support from PHPU but through district level

I

Why do you say so?

J Do you get enough support from OD for health promotion activities?



No support at all

K

Why do you say so?

L Do you get enough support from other partners?

NGOs Yes, more than enough

District/Commune chief Yes, to some extent but could use more support

VHSG/VHSG

Others No support at all

M

If yes, what kind of support do you get?

N


O


P


Q

Do you think that now that EC/UNICEF have stopped financial and technical assistance, the capacity of staff at health centre is adequate enough to continue implementing the activities?

Yes definitely



Yes to some extent

Not at all

Not sure

R Do you have any other recommendations for the BCC project? Any improvements/lessons learned for the future?


106

Evaluation of the Health BCC Project Discussion Guide – IDI – Health Centre Staff & Midwife

INTRODUCTION Objective: This is the ‘warm up’ section to help the moderator understand the respondent’s role as health centre staff/midwife.

A Province:



Midwife

Health workers

Others

D

Place of interview:

E

Date:

Hello, I’m.............[Insert interviewer name] from MSD, which is a consultancy company. MSD, along with TNS (also a global consultancy company), has been contracted by UNICEF to conduct the final evaluation of the Health Behaviour Change Communication (BCC) project. As the health centre staff is one of the key stakeholders in this project, your opinion is very valuable to us. Your information will be kept strictly confidential and will only be used for analysis purposes, along many other interviews. Thank you very much for participating in this interview.

F How long have you been in this position in the health centre?

years


G Could you please tell me something about your role as health centre staff/midwife___________?

What are your roles and responsibilities as health centre staff/midwife__________? Are you satisfied/dissatisfied with it? Why? Why not?

H Can you tell me: How many communes are covered by this health centre? How many villages are covered by this health centre? How many staff work in this health centre? How many VHVs are attached to this health centre?



Yes

No

Not sure




Yes

No

Not sure



107


A Do you think you have better capacity to plan health promotion activities now?


Yes to some extent

Not really



D Who are the different stakeholders/partners you interact with in planning health promotion activities? Instruction: Moderator to probe, not prompt. Only after the respondent mentions all partners, mention the ones he has left out and ask, “what about…”?


- Is there any overlap in your role? How is your role different from health centre chief, OD etc?


VHSG

NGOs

CWCC

Commune chief

Others





A Do you think you have better capacity to implement activities

now?


Yes to some extent

Not really




health promotion activities? Instruction: If all aspects of planning, implementation and M&E have been covered in the previous question, do not repeat the question.


- Is there any overlap in your role? How is your role different from health centre chief, OD etc?


VHSG

NGOs

CWCC

Commune chief

Others




108



Yes

No

Not sure



MPA Module 7 Basic health education (C-IMCI Module 2) Hygiene (C-IMCI Module 7) Home care of sick children (C-IMCI Module 8) Early childhood development (C-IMCI Module 9) PBCI Client/provider rights ABCP BCC protocol for IEC development BCC guideline Others


To a large extent

To some extent

Not at all




H Which modules according to you were easy to understand?

I Which modules were not so easy to understand?

- Why not? What aspects were difficult? - What do you suggest we can do to make it easier to understand?

J Have you been a trainer and trained any VHSG?

Yes

No

K What recommendations can you make to:

- improve the training courses so that they are more applicable and helpful to your work?

- improve training courses for VHSGs?

- improve refresher trainings? Are they sufficient? How often do you think refresher training is required?



B Do you think the BCC project has contributed to your counselling skills?


Yes to some extent

Not really

C Why do you say so? Please give examples of how your skills have improved

D If no, why do you think your skills have not improved? What is required for your skills to improve?

109

E What new things do you think you have learned as a result of the BCC project?

F Have you seen any change in the last 2-3 years in the health centre in your area in terms of:


Your behaviour towards clients Yes to a large extent Yes to some extent Not really

Your BCC knowledge Yes to a large extent Yes to some extent Not really


G Why do you say so? What kind of changes do you see? Please give examples (ask with regard to each aspect – service delivery; behaviour; knowledge and others)

H What more should be done to improve the quality of various aspects of your service delivery, behaviour, etc?

I Imagine there is a pregnant woman who has come to the health centre. Please explain what will you say to her and how will use your counselling skills.

antenatal care package Interviewer to probe: What else, anything else?

delivery at health centre Interviewer to probe: What else, anything else?

post-partum care Interviewer to probe: What else, anything else?

birth spacing Interviewer to probe: What else, anything else?

What all do you do to make the woman feel comfortable when giving her advice? How do you speak to her?

Moderator to probe: Anything you do in particular to make her feel comfortable, body language, etc?

J In general, what are the main topics of health education you offer to members of the community? And how do you do it (through outreach activities, meetings, health education sessions, home visits, other)?

Breastfeeding Micronutrients Home care for sick children, fever, diarrhoea and ARI Hygiene, water and sanitation Malaria/dengue fever Early child care & mental development Immunization

K Which of these were explained specifically during the BCC project training?


A What IEC materials are available for health promotion in your health centre?







C How often do you receive IEC materials at the health centre?

Monthly - How are these materials distributed to the VHSG?

Quarterly

Every 6 months

Annually

110

Only when we request for it

Never

D How are IEC materials displayed and stored at the health centre?

E Have you ever faced any problems in displaying IEC materials in your health centre?



No


G Of all the IEC materials, which ones do you think are most effective?

- What is easy for people to understand?

- What materials are attractive/catch people’s attention?

H Which IEC materials are not effective?

- Why not?


A


Very important

Somewhat important

Not so important


B

Why do you say so?

C


Excellent

Good

Needs improvement

Poor

D

Why do you say so?

E

Has the BCC project helped improve the skills of VHSGs?


Yes to some extent

Not really

F

Why do you say so?

- If yes, how? Please give examples

- If no, what more do you think needs to be done to improve the skills of VHSGs?

G How do you promote and ensure good quality of home visits by VHSGs who are attached to your health centre?


H

What is the proportion of VHSGs making regular and timely home visits?

I What are the main challenges faced in continuing home visits?

- What do you think are the essential conditions to improve and sustain home visits in your areas?


J Did you receive any support from an external agency to strengthen and promote home visits by VHSGs?



111


A Do you know about BCC forums?

Yes No



Instruction: Moderator to ask, “what are these forums’’, “how often are these forums held”, “who plans and organizes these forums”, “what do they discuss”.


Very useful

Somewhat useful

Not useful



Very important

Important

A little important

Not so important

F What is your role in the BCC forum?


Yes

No

Not sure



J What support do you think is needed to be able to organize the BCC forums?




Yes to some extent

Not really

B In what way?

C What more do you think is required to further improve their monitoring skills?

D How and in what way does the health centre supervise health promotion activities at village/commune level? Please give examples.

E How often do you supervise health promotion activities in the village/commune per year?

- 4 times or more per year

- 2-3 times per year

- 1 time per year

- Never

F How useful is this supervision? Very useful Somewhat useful Not useful

G Why do you say so?

H What are the main challenges?

I Do you have a monitoring tool/checklist for monitoring VHSG work?

- Yes

- No

- Don’t know

112

Instruction: Moderator to probe about monitoring tools/checklist

J Do you use the monitoring tools/checklist?

- Yes always

- Yes sometimes

- Yes rarely

- No

K Do you find the monitoring tools very useful?

- Yes to a great extent

- Yes to some extent

- No

L Why do you say so? What suggestions do you have to improve it?

M What are the main challenges faced in monitoring of activities at the village/commune level?

Instruction: Moderator to probe by asking, “what else?”.

N

Who supervises activities of the health centre and how?

Yes to some extent

O

Do you monitor the works of VHSGs at the community in your area?

Not really - What kind of VHSG works do you monitor? (home visit,s health education sessions, support outreach services, etc.)

- How frequently?

No Why not?

P What more should be done in monitoring and supervision of activities?

- in terms of provincial- and district-level monitoring at health centres - in terms of health centres monitoring the VHSG


A

Do you see a change between the current and past situation in the last 1-2 years in your area in terms of service utilization and community behaviour change in health?


Yes to some extent

Not really

B

If yes, what are the changes? Ask specifically for antenatal care; childbirth and post-partum care.

C


D If there have been no changes or there have been some changes, what do you think are the barriers/factors that have prevented change?

E


F






113




No support at all

No direct support from NCHP but through provincial level

E

Why do you say so?




No support at all

G

Why do you say so?




No support at all

No direct support from PHPU but through district level

I

Why do you say so?

J Do you get enough support from OD for health promotion activities?



No support at all

K

Why do you say so?

L Do you get enough support from other partners?

NGOs Yes, more than enough

District/Commune chief Yes, to some extent but could use more support

VHSG/VHSG

Others No support at all

M

If yes, what kind of support do you get?

N


O


P


Q

Do you think that now that EC/UNICEF have stopped financial and technical assistance, the capacity of staff at health centre is adequate enough to continue implementing the activities?

Yes definitely



Yes to some extent

Not at all

Not sure

R Do you have any other recommendations for the BCC project? Any improvements/lessons learned for the future?


114

Evaluation of the Health BCC Project Discussion Guide – IDI – BCC Forum Member

INTRODUCTION

A Province:



Commune chief

Village chief

CWCC member

Others (specify)

D

Place of interview:

E

Date:

Hello, I’m.............[Insert interviewer name] from MSD, which is a consultancy company. MSD, along with TNS (also a global consultancy company), has been contracted by UNICEF to conduct the final evaluation of the Health Behaviour Change Communication (BCC) project. As the <respondent title>, your opinion is very valuable to us. Your information will be kept strictly confidential and will only be used for analysis purposes, along many other interviews. Thank you very much for participating in this interview.

F How long have you been in this position as <respondent title>?

years


G Could you please tell me something about your role as a <respondent title>?

What are your roles and responsibilities? Are you satisfied/dissatisfied with it? Why? Why not?

BCC FORUMS

A How many BCC forums have you participated in?

<Record number>



C What is your role in the BCC forums?

D How useful are these BCC forums?

Very useful

Somewhat useful

Not useful


F How important is it to continue having these BCC forums?

Very important

Important

A little important

Not so important

G How do you think the BCC forums have helped the community members in your commune/village? Please give examples?

H Does the village chief/commune chief/CCWC talk about health issues or issues agreed upon during the BCC forums in relevant meetings? What did you do as a result of the BCC forum?

IMPACT ON COMMUNITY

A Do you see a change between the current and past situation in the last 1-2 years in your area


115

in terms of service utilization and community behaviour change in health?

Yes to some extent

Not really

B


C

What do you think are the factors that have contributed to this change? Why do you say so?

D If there have been no changes, what do you think are the barriers/factors that have prevented change? Why do you say so?

E What more do you think should be done to promote better health-seeking behaviour among the community? Why do you say so?

F What are the current challenges faced in promoting behaviour change among the community? Why do you say so?


116

Evaluation of the Health BCC Project Discussion Guide – IDI – National Level IDIs

This DG will be used for all national-level interviews with adjustments as required during the course of the interview.

INTRODUCTION Objective: This is the ‘warm up’ section to help the moderator understand the respondent’s role as

<designation of respondent>

A Province: Not applicable for national-level interviews



D

Place of interview:

E

Date:

Hello, I’m.............[Insert interviewer name] from MSD, which is a consultancy company. MSD, along with TNS (also a global consultancy company), has been contracted by UNICEF to conduct the final evaluation of the Health Behaviour Change Communication (BCC) project. As one of the key stakeholders in this project, your opinion is very valuable to us. Your information will be kept strictly confidential and will only be used for analysis purposes, along many other interviews. Thank you very much for participating in this interview.

F How long have you been in this position as <designation of respondent>?

years

G Could you please tell me something about your role as <designation of respondent>?

What are your roles and responsibilities as <designation of respondent>? Are you satisfied/dissatisfied with it? Why? Why not?



Yes

No

Not sure


Instruction: If the respondent does not know about the BCC project, moderator to probe.


Yes

No

Not sure



E Why was the Health BCC project important for Cambodia?

PLANNING AND POLICYMAKING Objective: This section aims to understand the planning process with regard to the Health BCC project and if the project has led to an increase in capacity in planning of health promotion activities.

A To what extent do you think the Health BCC project funded by


117

the EC has improved the NCHP’s capacity to plan health promotion activities?

Yes to some extent

Not really

B Can you explain why you say so?

If they say improved to a great extent or some extent, ask: In what way do you feel the NCHP’s capacity in planning has been improved? If they say it has not improved: Why do you think capacity has not improved? Can you describe the capacity of NCHP before the Health BCC project and what it is now?

C What is your role in planning health promotion and BCC activities?

D Who are the different stakeholders/partners you interact with in planning BCC activities? Instruction: Moderator to probe, not prompt. Only after the respondent mentions all partners, mention the ones he has left out and ask “what about…”.

MOH - What is the level of interaction?



NCHP

EC

UNICEF

PSC

Other organzations: UNFPA, WHO, RACHA, RAAC, BBC World Service Trust

NGOs

Provincial and district level (PHD, PHPU, OD)

Others

Instruction: Please ask the above questions for each stakeholder listed above

E Can you tell me more about the PSC?

Who all were a part of it? How did the PSC work? What was the role of the committee? What were the advantages of having a committee? What were the challenges faced in working with the committee?

F Can you tell me more about working with so many partners?

How was this useful? What were the challenegs in working with so many partners? What would you recommend for future programmes based on your experience in working with different partners on a project?

G How successful was the coordination between NCHP, UNICEF and EC?

Very successful Somewhat successful Problems faced

Can you explain more about the coordination? Why do you say so (about successful cooridination)? Please give examples

H To what extent did the project influence collaboration between NCHP, MOH, NGOs, etc?

To a great extent To some extent Not at all

Can you explain more about the coordination? Why do you say so? Please give examples

I What are some of the main challenges faced in planning activities?


J In terms of human resources, Yes to a large extent

118

was there turnover of people involved in the health BCC project?

Yes to some extent

No

K If yes: Did this cause a problem? What kind of problems?

Was there any transfer of knowledge/handover from the original person to the new person?

If yes, what kind of handover took place (training etc?)?

If no, why not?

What would you recommend for the future based on your experience from this project?



activities now?

At the national level?


Yes to some extent

Not really

Why do you say so? Please give examples. Can you describe the capacity before the Health BCC project and what it is now?

At the provincial level?


Yes to some extent

Not really

Why do you say so? Please give examples. Can you describe the capacity before the Health BCC project and what it is now?

C What is your role in implementing health promotion/BCC activities?


MCH - What is the level of interaction?

- Is there any overlap in your role? How is your role different from PHPU, etc?


NCHP

PSC

PHPU

OD

Health centre

VHSG

NGOs

CWCC

Commune chief

Others

Instruction: Please ask the above questions for each stakeholder (NCHP, PHPU, OD, etc) listed above



119

BCC SKILLS Objective: This section aims to understand how interpersonal and counselling skills of health workers have improved since the implementation of the Health BCC project.

A Did the project contribute to strengthening the BCC skills of the health workers?


Yes to some extent

Not really

B Why do you say so? Please give examples of how their skills have improved.

C If no, why do you think their skills have not improved? What is required for their skills to improve?


D Has there been any change in the country (especially the seven good practice demonstration provinces) the last 2-3 years in terms of:




BCC knowledge of health workers/staff



E Why do you say so? What kind of changes do you see? Please give examples.

F What more should be done to improve the quality of various aspects of service delivery, behaviour, etc of health staff?


A According to you, how useful are these BCC forums?

Very useful

Somewhat useful

Not useful

B Why do you say so?

C How important is it to continue having these BCC forums?

Very important

Important

A little important

Not so important


E Do you think it will be possible to continue organizing the BCC forums?

Yes

No

Not sure

F If yes, why do you say so? Where will the funding for the BCC forums come from? Where can the PHD/PHPU mobilize funds for the BCC forums?

G If no/not sure, why not? What are the main challenges in organizing the BCC forums?

H What support will you be able to provide to the provincial level for this?

I What support will you need from MoH for this?

120


A What IEC materials are available for health promotion?






- How are these materials distributed to the health centres and VHSGs?






Yes

No

Not sure




To a large extent

To some extent

Not at all




H Have you trained provincial and other staff?

Yes

No

I On what topics did you conduct the training?

- What challenges did you face while training?

J What recommendations can you make to help improve the training courses so that they are more applicable and helpful to your work?


A How important are the home visits made by the VHSGs?

Very important

Somewhat important

121

Not so important


B

Why do you say so?

C


Excellent

Good

Needs improvement

Poor

D

Why do you say so?

E How can the PHD/PHPU promote and ensure good quality of home visits by VHSGs in their area?



- What do you think are the essential conditions to improve and sustain home visits


H What further support do you think the provincial level requires to be able to continue to strengthen and sustain home visits by VHSG?

- What support can NCHP provide?

- What support can MoH provide?

- What support can international donors provide?


A

Do you see a change between the current and past situation in the last 1-2 years (especially in the seven provinces) in terms of service utilization and community behaviour change in health?


Yes to some extent

Not really

B


C


D


E


F





Yes to some extent

Not really

B In what way?


122

D What kind of M&E and supervision does NCHP do in the Health BCC project? Please give examples.

E What kind of M&E and supervision does MoH do in the Health BCC project? Please give examples.

F What kind of M&E and supervision does the PSC/other national-level stakeholders do in the Health BCC project? Please give examples.

G What kind of M&E and supervision does MoH do? Please give examples.



No

I What more should be done in monitoring and supervision of activities at the national level?

J How and in what way does the PHD supervise health promotion activities at the OD/health-centre level? Please give examples.

K How and in what way does the PHPU supervise health promotion activities at the OD/health-centre level? Please give examples.

L How and in what way does the OD supervise”

- health centres

- VHSGs

M - What more should be done in monitoring and supervision of activities at the sub-national level?





D Do you get enough support from MoH for health promotion activities?



No support at all

E

Why do you say so?

F Do you get enough support from other partners?

NGOs

PSC

Other international donors

Provincial and district-level staff

UNICEF

Others

G

Why do you say so?

H


I

What would be required to expand the BCC activities to other areas?

J


123

K To what extent were the recommendations given after the midterm review implemented?


- Why do you say so? - Please give examples

Yes to some extent

Not at all

Not sure

L

Do you think that now that EC/UNICEF have stopped financial assistance, the

capacity of provincial-level staff is adequate enough to continue implementing the activities?

Yes definitely



Yes to some extent

Not at all

Not sure

M

Do you think that now that EC/UNICEF have stopped technical assistance, the

capacity of provincial-level staff is adequate enough to continue implementing the activities?

Yes definitely



Yes to some extent

Not at all

Not sure

N Do you have any other recommendations for the BCC project? Any improvements/lessons learned for the future?


124

Women with children who are under 2 years old and have accessed the health centre/VHSG at least once in the last 6 months

SCREENING AND INTRODUCTION Objective: This section allows the interviewer to screen the reposndent to see if she is eligible for the interview. This is also the the ‘warm up’ section.

A Province:

B Commune

C Village

D

Name of the respondents in the paired interview:

Respondent 1 Respondent 2

C Do you have children Yes Yes

No No Terminate the interview. Please note that both women in the paired interview should have children

D How many children do you have?

E How old are your children (ask about all children)?

If she does not have any children less than 2 years old, do not continue the interview. Please note that both women in the paired interview should have children aged under 2 years.

F Have you ever been to the health centre?

Yes Yes

No No Terminate the interview. Please note that both women in the paired interview should have visited the health centre.

G When did you last go to the health centre?

Less than 6 months ago

Less than 6 months ago

More than 6 months ago

More than 6 months ago

Terminate the interview. Please note that both women in the paired interview should have visited the health centre less than 6 months ago.

I Has a VHSG ever come to your house for a home visit or have you ever received any information from the VHSG?

Never Never Terminate the interview. Please note that both women in the paired interview should have had VHSGs visit them.

Yes less than 6 months ago

Yes less than 6 months ago

Yes, more than 6 months ago

Yes, more than 6 months ago

J Place of interview:

K

Date:

125

L Interviewer to record the number of interviews terminated because of screening criteria.

Hello, I’m.............[Insert interviewer name] from MSD, which is a consultancy company. We are trying to understand more about health and related issues in your province. Your information will be kept strictly confidential and will only be used for analysis purposes, along many other interviews. Thank you very much for participating in this interview.

I’ve got lots of things to ask you today about matters that affect you and your life. I am really interested in your ideas and what you think to help us develop potential services that suit your needs and for people such as yourselves. It will take about two hours in total. Please let me know if you need to take a break at any time.

Please answer all questions honestly and remember there are no right or wrong answers.

We are only interested in your opinion and your thoughts and not that of other people.

Instruction: Moderator to address both women in paired interview while asking these questions. If one respondent is more dominant, please ensure that you ask the other respondent the questions as well and let her know that her opinion/experience is also important.

Health Centre Objectives: This section aims to find out the perception of women about the quality of services available at the health centre.

A You said you went to the health centre recently. For what purpose did you go there?

Instruction: Probe for reasons. Ask if it was for herself or family members, why, etc.

B Who attended to you when you went to the health centre?

Midwife Health centre staff Others

C Who recommended that you go to the health centre?

- Other women - VHSG - Commune chief - Midwife - My relatives (husband, mother, etc) - Others

Instruction: Probe, do not prompt.

D Were you satisfied with your visit to the health centre?


Yes to some extent

Not really

E Why do you say so? Give me reasons why you say you were satisfied/dissatisfied.

F Can you tell me what they asked and did at the health centre? Imagine I am coming to the health centre for <quote reason given by respondent for visiting health centre>. What was the procedure? What did they say to you? What did they do?

G Were you satisfied with: the health centre staff’s behaviour Yes to a great extent Yes to some extent No Why do you say so? What more do you require/need?

the services provided at the health centre

Yes to a great extent Yes to some extent No Why do you say so? What more do you require/need?

information/advice given to you Yes to a great extent Yes to some extent

126

No Why do you say so? What more do you require/need?

H Did you see any information –posters, brochures, etc – at the health centre?

Yes No Do not remember

I If yes, what were they about?

J Did the health centre staff show you any materials/information, like a poster, brochure, etc?

Yes No Do not remember

K If yes, what was it about?

L Did you have to pay a fee when you went to the health centre?

Yes No

- How much? - Do you think this is reasonable? Why? Why not?

N Do you find anything different going to the health centre now and than, say, around two years ago?

Yes to a great extent Yes to some extent Cannot see the difference

If yes, what is the main difference? How does the health centre now compared to before?

O On a scale of 0 to 5, if you were to give marks to the health centre for the following, how much would you give (0 being lowest and 5 being highest)?

Distance of the health centre from home Environment of health centre Attitude of staff/friendliness Knowledge of staff Fee Information/IEC materials seen and received at the health centre Medical equipment and supplies Others

Instruction; For each point, ask “why do you say so? Please give examples” after the woman responds

P Why do you think some people do not go to the health centre?

Q Imagine there is a new lady in your village and she needs to visit the health centre. What will you tell her about it?

Instruction: Ask about health services, staff attitudes, friendliness, etc

VILLAGE HEALTH SUPPORT GROUP (VHSG) Objectives: This section aims to find out the perception of women about the VHSG.

A You said that you were visited by a VHSG. Was it a home visit or a group discussion/meeting?

Home visit: how many times have they come to your home? When? Why? Group discussion/meeting

What did they say to you? Did you find it useful? Why? Why not?

B What do you think is the role of the VHSG?

C Do you listen to what the VHSG tells you?

Yes to a large extent Yes to some extent No


127

E Can you list a few things that you have learned recently (in the last 1-2 years) based on what the VHSG or health centre staff have told you?

ACCESS TO MCH care Objectives: This section aims to find out the attitudes and practices of women with regard to ANC and MCH.

You said you have a child who is <age> years old. Let me take you back to that special day when you found out you were pregnant with your child. Instruction: Moderator to use a culturally appropriate way of saying this.

A What did you do when you were pregnant with your child?

- How did you realize you were pregnant? - Did you seek ANC treatment? - If yes, how many months pregnant were you when you sought ANC

treatment? - If no, why not?

B Did the midwife/health centre staff tell you/do anything when you were pregnant?

- What advice did you receive from the midwife? - Did they show you any materials about ANC? If yes, what did they

show you? What were the messages? - Did you see any other communcation materials/messages about

ANC? If yes, what did you see? Where did you see it? What was it about?

- Did you receive anything else from the health centre?

C Were you satisfied with the ANC services you received?

- Yes to a great extent - Yes to some extent - Not really


E Did the VHSG tell you anything when you were pregnant? What did they tell you?

- What advice did you receive from the VHSG? - Did the VHSG show you any materials about ANC? If yes what did

they show you? What were the messages?

F Where did you deliver your baby?

- At home with the help of a traditional midwife

- Why did you not opt for the presence of a midwife/skilled birth attendant?

- At home in the presence of a skilled birth attendent

- At the health centre - Why did you have your delivery at the health centre?

- To what extent do you think you were persuaded by the VHSG to have your delivery at the health centre (to a great extent; to some extent; not really)?

- To what extent do you think you were persuaded by the midwife to have your delivery at the health centre (to a great extent; to some extent; not really)?

- Hospital

- Others

G Did the midwife/health centre staff tell you/do anything after you delivered your baby? What did they say?

What advice did they give you? What advice did they give you for your baby?

H Did the VHSG tell you/do anything after you delivered your baby? What did they

What advice did they give you? What advice did they give you for your baby?

128

say?


A Have you seen or heard any messages or information on:

ANC Breastfeeding Malaria Hand-washing Complementary feeding

- If yes, where did you see this message/ who told you (list all sources, media, people etc)?

- What was the message about? (ask for each option)


B Could you understand the message?


Instruction: Ask for each option

Yes to some extent

No

C Did you find the messages to be useful?


Instruction: Ask for each option

Yes to some extent

No

Imagine you are speaking to your friend who is pregnant for the first time:

- What all will you tell her about pregnancy? - What will you tell her about ANC? - What will you tell her about delivery of the baby?

For each point, ask “what else”, “anything else”.

FLASHCARD

129

Imagine she has given birth to a baby: - What will you tell her about feeding of the newborn baby? - In case her baby has diarrohea, what advice will you give her? - What will you tell her about nutrition for her children? - What advice will you give her related to good hygiene?

For each point, ask “what else”, “anything else”. SHOWCARD


130

ANNEX 5: FINDINGS FROM THE VHV END-LINE SURVEY ON 12 KEY FAMILY PRACTICES

CONTENTS 1. Basic information on VHVs .................................................................................................................................... 131

2. Breastfeeding ......................................................................................................................................................... 138

3. Complementary feeding ......................................................................................................................................... 140

4. Vitamin A deficiency ............................................................................................................................................... 142

5. Iodized salt ............................................................................................................................................................. 145

6. Hygiene and safe water ......................................................................................................................................... 146

7. Childhood illness .................................................................................................................................................... 151

8. Antenatal care ........................................................................................................................................................ 161

9. Malaria ................................................................................................................................................................... 165

131

1. Basic information on VHVs

Table 1: Per cent sex distribution of CHV by domain

Kampong

Speu Mondolkiri Prey Veng Stung Treng Svay Rieng

Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Male 46.2 45 47.3 55 49.4 55 35.7 40 57.1 85 47.3 33.3

Female 52.3 55 47.3 45 50.6 45 57.1 60 42.9 15 52.7 66.7

Missing 1.5 0 5.5 0 0 0 7.1 0 0 0 0 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 2: Percentage of CHV who attended school by domain

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Attended school 96.9 100 100 95 100 100 92.9 100 98.6 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 3: Percentage distribution of CHV by type and domain

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

VHV 66.2 95 92.7 95 20.5 100 50 100 18.6 95 71.6 96.7

VHSG 33.8 5 7.3 5 79.5 0 50 0 81.4 5 28.4 3.3

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Table 4: Percentage of CHV reported to collaborate with health centre and PHPU by domain

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Collaboration with local health centre

98.5 100 78.2 100 96.4 100 97.6 100 100 100 94.6 100

Collaboration with PHPU

63.1 85 23.6 85 61.4 80 59.5 95 70 85 59.5 96.7

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

132

Table 5: Percentage of CHVs receiving technical support from ministries or non-governmental entities

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

MOH 72.3 45 69.1 75 56.6 45 47.6 70 85.7 50 79.7 56.7

UNICEF 40 60 0 10 31.3 25 69 55 48.6 40 13.5 13.3

RHAC 24.6 70 0 15 9.6 10 2.4 0 11.4 30 13.5 50

RACHA 3.1 10 0 0 0 90 2.4 0 1.4 0 10.8 6.7

WORLD VISION 13.8 0 0 0 2.4 0 0 0 0 0 9.5 0

PFD 0 0 0 0 1.2 0 0 0 0 0 5.4 3.3

CARE 0 0 0 0 1.2 0 0 0 5.7 0 4.1 0

MoWA 1.5 0 9.1 0 1.2 0 2.4 0 2.9 0 2.7 3.3

MoRD 0 0 0 0 2.4 0 2.4 0 0 0 2.7 0

IRC 3.1 5 5.5 10 1.2 0 2.4 0 0 0 2.7 0

HKI 0 0 0 0 0 0 0 0 0 0 2.7 0

YWAM 0 0 0 0 1.2 0 0 10 0 0 0 0

SAVE THE CHILDREN

0 0 0 0 0 0 0 0 0 0 0 0

NOMAD 0 0 0 15 0 0 0 0 0 0 0 0

CRS 0 0 0 0 0 0 0 0 0 0 0 0

Rain water for Cambodia

. 25 . 0 . 0 . 0 . 0 . 0

Kratie Women's Welfare Association (KWWA)

. 0 . 0 . 0 . 0 . 0 . 10

National Property Association (NAPA)

. 20 . 0 . 0 . 0 . 0 . 0

Health centre . 0 . 5 . 0 . 10 . 5 . 0

Health Unlimited (HU)

. 0 . 60 . 0 . 0 . 0 . 0

Name unknown . 0 . 10 . 5 . 10 . 5 . 16.7

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130 )

Table 6: Percentage of CHV who received any training and type of training received

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Received training

98.5 100 87.3 100 96.4 100 100 100 98.6 100 98.6 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

133

Type of training

Tuberculosis 62.5 65 27.1 35 55 90 54.8 70 56.5 85 56.2 90

Vaccinations 46.9 55 2.1 30 40.5 25 40.5 40 47.8 50 47.9 33.3

HIV/AIDS 46.9 45 4.2 40 42.5 75 38.1 35 72.5 60 46.6 53.3

Birth spacing 37.5 45 8.3 45 27.5 25 16.7 65 30.4 15 35.6 63.3

Vitamin A 29.7 20 0 10 5 5 7.1 15 5.8 15 31.5 20

Polio vaccination

17.2 . 0 . 18.8 . 9.5 . 42 . 30.1 .

Antenatal care 32.8 80 4.2 70 43.8 70 26.2 85 18.8 65 26 66.7

Avian influenza 6.3 35 8.3 25 12.5 50 7.1 35 34.8 30 26 36.7

Diarrhoea treatment

18.8 60 4.2 75 20 30 26.2 80 11.6 35 21.9 50

Dengue 7.8 15 0 30 22.5 30 42.9 25 5.8 10 19.2 30

Basic health Education

23.4 5 4.2 5 20 15 42.9 20 2.8 10 13.7 6.7

danger signs Pregnancy

14.1 15 2.1 15 22.5 25 4.8 10 1.4 20 9.6 13.3

VHV or VHSG Training

14.1 0 12.5 0 8.8 5 7.1 0 7.2 0 9.6 0

reproductive health

4.7 15 2.1 25 6.3 10 7.1 25 7.2 15 8.2 26.7

Birth preparedness

3.1 0 0 15 3.8 5 2.4 15 14.5 5 6.8 3.3

Nutrition 1.6 . 0 . 0 . 9.5 . 7.2 . 6.8 .

Training – DK Name

12.5 0 0 0 1.3 0 4.8 0 0 0 5.5 0

Malaria prevention

6.3 10 68.8 35 0 5 28.6 70 0 0 4.1 33.3

Eye care 0 . 0 . 3.8 . 2.4 . 5.8 . 4.1 .

Community health promotion

7.8 0 8.3 0 6.3 0 35.7 5 0 0 2.7 6.7

Breastfeeding 14.1 0 0 0 26.3 5 21.4 0 33.3 0 2.7 0

Road to health 0 0 0 0 3.8 0 0 0 0 5 1.4 3.3

Hygiene . 20 . 5 . 15 . 5 . 10 . 36.6

Vaccination of measles

. 0 . 0 . 5 . 0 . 20 . 0

Care for a child who has pneumonia

. 5 . 0 . 15 . 0 . 0 . 0

Bird flu . 5 . 0 . 0 . 0 . 0 . 0

STD . 0 . 5 . 0 . 0 . 0 . 0

Good food . 0 . 0 . 0 . 0 . 5 . 3.3

Safe food . 0 . 0 . 0 . 0 . 5 . 0

Liver . 0 . 0 . 0 . 0 . 5 . 0

Vaccination against tetanus for pregnant women

. 0 . 0 . 5 . 0 . 0 . 0

Total number 64 20 48 20 80 20 42 20 69 20 73 30

2006 (n=376); 2011 (n=130)

134

Table 7: Percentage of CHVs who receive financial support and source of financial support

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Receive financial support

95.4 90 45.5 90 86.7 95 95.2 95 85.7 60 91.9 73.3

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Source of financial support

RHAC 25.8 70 0 10 12.5 30 2.5 0 8.3 15 16.2 33.3

RACHA 1.6 5 0 0 0 60 2.5 0 3.3 0 13.2 3.3

UNICEF 38.7 45 0 10 36.1 10 75 60 50 25 11.8 13.3

WORLD VISION 12.9 0 0 0 2.8 0 0 0 0 0 11.8 0

HEALTH CENTER

12.9 0 16 10 2.8 10 12.5 25 1.7 5 5.9 0

CARE 0 0 0 0 2.8 0 0 0 3.3 5 4.4 0

HKI 0 0 0 0 0 0 0 0 0 0 2.9 0

PFD 0 0 0 0 1.4 0 0 0 0 0 2.9 0

IRC 1.6 5 12 5 1.4 0 15 0 0 0 1.5 0

CRS 0 0 0 0 0 0 0 0 0 0 0 0

NOMAD 0 0 0 10 0 0 0 0 0 0 0 0

SAVE THE CHILDREN

1.6 5 0 0 0 5 0 0 0 0 0 0

YWAM 0 0 0 0 1.4 0 0 10 0 0 0 0

Health Unlimited . 0 . 65 . 0 . 0 . 0 . 0

Kratie Women's Welfare Association (KWWA)

. 0 . 0 . 0 . 0 . 0 . 10

Name unknown . 0 . 10 . 5 . 10 . 5 . 13.3

Total number 62 20 25 20 72 20 40 20 60 20 68 30

2006 (n=327); 2011 (n=130)

135

Table 8: Percentage CHV reported involvement in community mobilization on specific issues and other related activities

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Helping to mobilize the community for

ANC visits 90.8 100 45.5 90 94 95 92.9 100 98.6 100 98.6 100

Child immunization vitamin A for children

87.7 100 18.2 100 82.8 100 85.7 100 95.7 100 95.9 100

Tetanus toxoid 84.6 100 20 100 94 100 88.1 100 91.4 100 89.2 100

Child de–worming

80 100 9.1 90 79.5 100 83.3 100 92.9 100 76.7 100

Post-partum Vitamin A

69.2 100 9.1 80 73.5 90 78.6 95 85.7 100 75.7 96.7

Iodized salt 67.7 65 20 60 72.3 75 88.1 80 92.9 100 67.6 73.3

Insecticide-rreated bed nets

16.9 55 63.6 95 8.4 45 78.6 85 2.9 60 14.9 66.7

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Other CHV Activities

Health centre referral

23.1 55 16.4 55 30.1 36.8 7.1 25 51.4 70 27 42.9

Hygiene education

10.8 70 16.4 70 16.9 73.7 38.1 75 17.1 60 18.9 71.4

Information on pregnant women

21.5 10 5.5 10 24.1 10.5 11.9 20 10 10 17.6 7.1

Dengue diagnosis

4.6 5 5.5 5 2.4 0 14.3 5 2.9 10 16.2 0

Follow-up Tb patients

7.7 20 7.3 10 4.8 10.5 4.8 5 28.6 20 10.8 14.3

Treat mosquito larvae

9.2 0 0 5 1.2 0 9.5 10 4.3 0 10.8 10.7

Education in child illness

9.2 15 0 15 13.3 26.3 9.5 10 4.3 10 6.8 14.3

Diarrhoea treatment

1.5 35 1.8 45 3.6 5.3 11.9 20 2.9 20 4.1 25

Malaria . 0 . 10 . 5 . 10 . 0 . 0

Advertisement to have six type of vaccination

. 0 . 0 . 5 . 5 . 0 . 0

Explain the pregnant women to go to health centre

. 5 . 0 . 0 . 0 . 5 . 0

Cure TB . 5 . 0 . 0 . 0 . 0 . 6.7

Don't know/nothing else

56.9 10 60 0 47 10 45.2 5 11.4 5 36.5 3.3

Total number 65 20 55 20 83 19 42 20 70 20 74 28

2006 (n=389); 2011 (n=127)

136

Table 9: CHVs usage of mobile phone, accessing email on internet

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Use mobile phone

9.2 75 14.5 90 15.7 90 9.5 60 14.3 95 24.3 73.3

Use internet 1.5 0 1.8 0 1.2 0 2.4 0 0 0 1.4 3.3

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Table 10: Roles/responsibilities of CHVs and their satisfaction in these roles

Kampong

Speu Mondolki

ri Prey Veng

Stung Treng

Svay Rieng

Control Province

Overall

Roles and responsibilities of CHVs

Tell community about various health issues

95 90 75 85 90 76.7

Home visits 50 60 40 70 35 50

Attend meetings at health centre

50 65 45 60 35 50

Attend trainings 50 55 50 70 25 46.7

Ensure community members visit health centre when required (e.g. make pregnant women go for ANC check up)

25 10 35 0 30 26.7

Distribute IEC materials 0 15 20 25 5 20

Follow-up visits 0 5 0 5 0 6.7

Tell them about hygiene 5 0 10 0 0 3.3

Attend BCC forums 0 0 0 0 0 0

Total number 20 20 20 20 20 30 130

Satisfaction of CHV's role

Very satisfied 75 75 85 75 95 86.7

Somewhat satisfied 10 15 15 25 5 10

Neither satisfied nor dissatisfied

15 5 0 0 0 3.3

Somewhat dissatisfied 0 0 0 0 0 0

Very dissatisfied 0 5 0 0 0 0

Total per cent 100 100 100 100 100 100

Total number 20 20 20 20 20 30 130

137

Table 11: Percentage of CHV responses on the reasons why they are satisfy with their roles or not

Kampong

Speu Mondolki

ri Prey Veng

Stung Treng

Svay Rieng

Control Province

Overall

Satisfaction

Improve my knowledge 60 90 60 65 55 73.3

Helps me communicate well with the community

45 25 30 55 45 46.7

Helps my people through the knowledge that I got from training

40 30 45 25 45 36.7

I like helping people by spreading knowledge about health

30 20 35 20 25 33.3

I like helping my community

35 45 35 50 30 30

Many people know me 10 15 5 10 15 26.7

Many people respect me 20 5 0 15 15 10

I receive some money as incentive

10 20 15 10 5 6.7

Can help the society/reduce poverty

5 0 10 0 15 3.3

Total number 20 20 20 20 20 30 130

Dissatisfaction

Do not have enough time to do my work

0 0 5 0 0 3.3

Lack of transportation 0 5 5 0 0 0

Do not get any incentives/money

0 5 5 0 0 0

Need to use my own money for transportation

0 5 0 0 0 0

I have received too many trainings for different issues so do not remember

0 0 0 5 0 0

The villagers do not listen to me

0 0 0 0 0 0

Total number 20 20 20 20 20 30 130

138

2. Breastfeeding

Table 12: Per cent distribution of CHV's knowledge on optional initiation of breastfeeding practices

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

In the first hour 27.7 30 27.3 15 51.8 40 47.6 10 34.3 30 47.3 5

1 to 2 hours after birth

43.1 5 10.9 0 32.5 0 40.5 0 50 5 23 5

2 to 23 hour after birth

24.6 0 36.4 5 12 5 9.5 0 12.9 0 17.6 0

One day after birth

4.6 0 21.8 5 3.6 0 2.4 0 2.9 0 9.5 0

Do not know 0 0 1.8 0 0 0 0 0 0 0 1.4 0

Missing 0 65 1.8 75 0 55 0 90 0 65 1.4 90

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 13: Per cent distribution of CHV's knowledge on optimal colostrum feeding

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Give all to newborn child

95.4 100 45.5 90 94 100 100 95 94.3 100 96.5 100

Squeeze to let milk begin

1.5 0 43.6 10 3.6 0 0 5 2.9 0 9.5 0

Don't know 3.1 0 10.9 0 2.4 0 0 0 2.9 0 4.1 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 14: Percentage response of CHV's knowledge of appropriate liquids to give to newborns

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Breast milk 92.3 100 63.6 100 67.5 100 95.2 100 97.1 100 78.4 100

Plain water 7.7 0 23.6 0 8.4 0 2.4 0 7.1 0 9.5 0

Sugar/honey water

0 0 23.6 0 2.4 0 2.4 0 2.9 0 2.7 0

Juice/coconut water

1.5 0 3.6 0 2.4 0 0 0 0 0 1.4 0

Infant formula 1.5 0 25.5 0 0 0 0 0 1.4 0 1.4 3.3

Water with sugar/salt

0 0 0 0 0 0 0 0 0 0 0 0

Herbal tea 1.5 0 0 0 0 0 0 0 0 0 0 0

Don't know 1.5 0 1.8 0 18.1 0 0 0 0 0 8.1 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

139

Table 15: Percentage response of CHV's knowledge of optimal breastfeeding practices

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Children under 6 months need water with breast milk

18.5 10 67.3 5 19.3 5 7.1 10 15.7 10 10.8 13.3

Do not breastfeed when child sick

3.1 0 36.4 0 18.1 0 4.8 0 4.3 10 14.9 6.7

Do not breastfeed when mother sick

30.8 0 50.9 35 28.9 25 2.4 10 34.3 30 24.3 26.7

Hot breast milk after vigorous activity can make child sick

49.2 85 76.4 80 61.4 95 40.5 65 64.3 65 52.7 93.3

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

140

3. Complementary feeding

Table 16: Per cent distribution of CHV's knowledge of optimal age at initiation of feeding rice porridge

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Less than 5 months

4.6 0 29.1 5 4.8 0 0 0 5.7 0 8.1 6.7

6 to 7 months 90.8 95 58.2 85 86.7 100 100 95 88.6 100 85.1 93.3

8 to 11 months 1.5 0 7.3 5 2.4 0 0 0 2.9 0 2.7 0

More than 12 months

3.1 5 5.5 0 2.4 0 0 5 1.4 0 4.1 0

Do not know 0 0 0 5 2.4 0 0 0 0 0 0 0

Missing 0 0 0 0 1.2 0 0 0 1.4 0 0 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 17: Per cent distribution of CHV's knowledge of optimal age at initiation of feeding bananas

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Less than 5 months

4.6 0 9.1 0 0 0 0 0 5.7 0 4.1 0

6 to 7 months 78.5 80 25.5 75 67.5 80 76.2 65 64.3 50 70.3 70

8 to 11 months 6.2 15 16.4 15 8.4 10 14.3 15 14.3 40 9.5 13.3

More than 12 months

3.1 5 32.7 5 7.2 5 7.1 20 10 10 9.5 13.3

Do not know 7.7 0 16.4 5 16.9 5 2.4 0 4.3 0 6.8 3.3

Missing 0 0 0 0 0 0 0 0 1.4 0 0 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Table 18: Per cent distribution of CHV's knowledge of optimal age at initiation of feeding other fruits (not bananas)

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Less than 5 months

4.6 0 5.5 0 1.2 0 0 0 2.9 0 1.4 0

6 to 7 months 72.3 55 16.4 30 51.8 75 35.7 45 42.9 15 60.8 56.7

8 to 11 months 4.6 10 10.9 20 6 15 26.2 35 17.1 30 6.8 16.7

More than 12 months

12.3 35 52.7 45 28.9 10 23.8 20 37.1 55 23 26.7

Do not know 6.2 0 14.5 5 10.8 0 14.3 0 0 0 8.1 0

Missing 0 0 0 0 1.2 0 0 0 0 0 0 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

141

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 19: Per cent distribution of CHV's knowledge of optimal age at initiation of feeding foods from animal sources

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Less than 5 months

4.6 0 5.5 0 0 0 0 0 2.9 0 1.4 0

6 to 7 months 70.8 65 16.4 45 66.3 80 76.2 60 52.9 55 70.3 70

8 to 11 months 9.2 10 16.4 10 6 10 9.5 25 18.6 20 6.8 3.3

More than 12 months

10.8 25 50.9 40 21.7 10 9.5 10 24.3 25 18.9 26.7

Do not know 4.6 0 10.9 5 6 0 4.8 5 1.4 0 2.7 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Table 20: Per cent distribution of CHV's knowledge of optimal age at initiation of feeding foods with oil or fats

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Less than 5 months

4.6 0 3.6 0 0 0 0 0 1.4 0 1.4 3.3

6 to 7 months 66.2 60 14.5 55 53 80 61.9 60 50 70 54.1 60

8 to 11 months 3.1 5 3.6 5 3.6 10 7.1 20 5.7 5 4.1 6.7

More than 12 months

18.5 35 61.8 35 27.7 10 16.7 20 40 25 20.3 30

Do not know 7.7 0 14.5 5 14.5 0 14.3 0 2.9 0 20.3 0

Missing 0 0 1.8 0 1.2 0 0 0 0 0 0 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

142

4. Vitamin A deficiency

Table 21: Percentage of CHV responses on signs of vitamin A deficiency

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Child has night blindness

53.8 20 25.5 10 44.6 45 59.5 40 45.7 25 64.9 40

Anaemia 27.7 80 16.4 80 41 70 52.4 85 41.4 90 39.2 86.7

Child sick often 3.1 10 3.6 30 22.9 10 7.1 10 18.6 30 12.2 16.7

White spots on eyes

0 10 0 0 3.6 0 4.8 0 2.9 5 8.1 16.7

Swollen stomach

1.5 5 1.8 0 4.8 5 0 0 10 10 6.8 3.3

Child shorter than normal

3.1 15 1.8 5 2.4 25 7.1 10 4.3 15 4.1 10

Goiter 1.5 0 0 0 0 0 0 0 0 0 0 0

Thin . 5 . 15 . 20 . 25 . 10 . 33.3

No power . 25 . 20 . 5 . 15 . 10 . 16.7

Numb leg and hand

. 0 . 5 . 5 . 10 . 5 . 13.3

Yellow eyes . 10 . 0 . 0 . 5 . 0 . 0

Wrinkled skin . 0 . 0 . 5 . 0 . 15 . 0

Eat less . 0 . 5 . 5 . 0 . 0 . 3.3

Not active . 0 . 0 . 5 . 0 . 5 . 0

Pale eyes . 5 . 0 . 5 . 0 . 0 . 0

Don't know 30.8 0 60 0 22.9 0 9.5 0 2.9 0 24.3 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

143

Table 22: Percentage of CHV response on how to prevent vitamin A deficiency

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Eat vitamin A -rich Foods

21.5 60 18.2 50 34.9 50 64.3 65 74.3 40 40.5 50

Get vitamin A supplement every 6 months

47.7 50 14.5 50 27.7 60 26.2 65 45.7 55 39.2 70

Grow vitamin A- rich fruits and vegetables for child

13.8 75 14.5 35 33.7 85 23.8 55 11.4 60 35.1 60

Give colostrum 0 0 1.8 5 2.4 15 2.4 5 5.7 20 8.1 10

Only give breastmilk for first 6 months

0 10 1.8 5 8.4 0 4.8 5 10 10 5.4 3.3

Feed oil or fats with meal

1.5 0 0 10 1.2 0 0 10 1.4 0 4.1 10

Take to health centre when sick

3.1 0 0 25 4.8 5 2.4 20 2.9 5 1.4 6.7

Don't know 33.8 0 58.2 10 18.1 0 19 0 2.9 0 17.6 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

144

Table 23: Percentage of CHV responses on foods rich in vitamin A

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Dark green leafy vegetables

56.9 100 34.5 90 71.1 85 23.8 85 80 85 75.7 93.3

Ripe papaya 26.2 25 16.4 45 37.3 30 50 40 41.4 65 43.2 40

Pumpkin 36.9 35 20 15 41 15 45.2 35 35.7 35 39.2 20

Yellow fleshed fruit and vegetables

32.3 25 21.8 35 26.5 35 57.1 45 57.1 40 36.5 56.7

Ripe mango 15.4 10 3.6 45 27.7 45 38.1 45 25.7 50 32.4 46.7

Banana 10.8 30 14.5 40 32.5 45 42.9 40 44.3 40 25.7 26.7

Liver 6.2 5 3.6 0 7.2 10 16.7 20 7.1 10 23 26.7

Carrot 18.5 15 7.3 0 15.7 15 28.6 5 17.1 10 23 10

Fish 13.8 50 1.8 20 15.7 40 21.4 15 25.7 55 14.9 46.7

Pineapple 0 0 1.8 0 2.4 5 2.4 10 5.7 0 4.1 0

Breast milk 0 5 0 0 3.6 0 4.8 5 2.9 0 4.1 3.3

Sugar cane 0 10 0 0 0 10 2.4 10 0 15 2.7 13.3

Lime 0 5 1.8 0 1.2 0 4.8 5 1.4 0 1.4 0

Rice 0 0 0 0 1.2 0 0 5 0 0 0 0

Egg . 45 . 15 . 20 . 30 . 40 . 26.7

Meat . 40 . 15 . 20 . 15 . 25 . 20

Potatoes . 20 . 5 . 5 . 10 . 10 . 10

Oil . 0 . 10 . 0 . 0 . 5 . 0

Durian . 0 . 5 . 0 . 0 . 0 . 3.3

Orange . 0 . 5 . 0 . 0 . 0 . 3.3

Don't know 9.2 0 47.3 10 4.8 0 4.8 0 2.9 0 5.4 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

145

5. Iodized salt

Table 24: Percentage of CHVs reporting to use iodized salt for cooking in their household and the importance of salt iodization

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Use iodized salt in household for cooking

73.8 95 56.4 85 69.9 95 90.5 100 70 100 71.6 90

Total number 65 20 55 20 83 20 42 20 70 20 74 30

The importance of iodized salt

Prevent goiter 69.2 65 52.7 55 65.1 75 85.7 70 58.6 95 58.1 66.7

Cleverness 40 75 25.5 55 53 65 57.1 75 50 90 47.3 70

Prevent cretinism

0 10 0 5 7.2 10 4.8 20 4.3 0 17.6 16.7

Prevent stunting 15.4 5 9.1 0 8.4 5 31 10 5.7 20 16.2 0

Healthy/grow up well

4.6 5 1.8 5 4.8 5 0 15 22.9 20 9.5 6.7

Good vision 4.6 0 3.6 0 3.6 5 14.3 10 2.9 0 2.7 3.3

Prevent low birthweight

0 0 0 0 3.6 0 0 10 0 5 2.7 3.3

Healthy skin 0 10 1.8 0 0 0 0 5 0 10 0 0

Prevent cancer . 0 . 0 . 0 . 5 . 0 . 3.3

Prevent diarrhoea

. 0 . 0 . 5 . 0 . 0 . 3.3

Polio . 10 . 0 . 0 . 5 . 0 . 6.7

Don't know 10.8 0 29.1 20 7.2 0 2.4 5 1.4 0 16.2 6.7

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

146

6. Hygiene and safe water

Table 25: Percentage of CHV response on the importance of hand-washing and when to wash hands

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Importance of hand-washing

To prevent disease

90.8 100 87.3 90 95.2 100 90.5 100 64.3 100 91.9 100

To wash away dirt

4.6 55 12.7 30 19.3 55 7.1 35 27.1 95 21.6 43.3

Nice appearance

24.6 20 5.5 10 6 20 0 5 11.4 10 16.2 6.7

Hygiene 9.2 . 7.5 . 0 . 7.1 . 37.1 . 6.8 .

To clean smelly hands

6.2 5 5.5 0 0 5 4.8 0 1.4 0 2.7 3.3

Healthy . 0 . 5 . 0 . 0 . 0 . 0

Don't know 0 0 0 0 0 0 0 0 0 0 1.4 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

When should caregivers wash hands

Before feeding child

46.2 40 47.3 90 62.7 85 47.6 60 68.8 85 77 80

Before eating 43.1 40 60 20 39.8 25 61.9 35 58.6 25 29.7 50

After defecation 24.6 75 9.1 30 24.1 55 69 85 11.4 75 23 30

Before food preparation

6.2 50 16.4 25 9.6 40 28.6 35 7.1 20 21.6 20

After work in field

24.6 25 5.5 5 28.9 0 9.5 20 17.1 10 14.9 10

After food preparation

3.1 10 9.1 10 4.8 20 16.7 20 4.3 25 12.2 3.3

After cleaning child who defecated

3.1 40 3.6 35 2.4 45 19 45 2.9 25 9.5 30

After eating 12.3 0 7.3 30 1.2 10 11.9 10 4.3 5 4.1 16.7

After dispose child faeces

1.5 10 1.8 15 0 5 2.4 5 1.4 5 1.4 13.3

Before breastfeeding

. 0 . 10 . 0 . 0 . 0 . 20

Before touch baby

. 20 . 5 . 0 . 15 . 15 . 13.3

Before go to bed . 0 . 0 . 0 . 0 . 5 . 0

Don't know 0 0 1.8 0 1.2 0 0 0 0 0 0 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

147

Table 26: Percentage of CHV responses on symptoms of parasite infection in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Bloated abdomen

81.5 75 76.4 75 79.5 45 78.6 80 81.4 70 78.4 63.3

Skinny thin child 61.5 50 49.1 70 51.8 40 33.3 50 64.3 60 60.8 70

Anaemia 43.1 65 27.3 45 28.9 35 45.2 40 31.4 40 47.3 40

Persistent diarrhoea

9.2 10 16.4 5 28.9 15 16.7 10 21.4 15 24.3 6.7

Vomiting 4.6 10 5.5 15 18.1 30 11.9 15 25.7 20 10.8 13.3

Cough 0 0 3.6 10 7.2 10 2.4 5 10 5 9.5 3.3

Stomachache 1.5 10 5.5 10 3.6 5 0 0 7.1 5 9.5 0

Eats a lot but hungry/weak

7.7 5 0 0 4.8 0 7.1 15 21.4 0 6.8 3.3

Loss of appetite 0 0 16.4 15 10.8 10 4.8 10 8.6 10 5.4 16.7

Fever 3.1 5 0 10 6 15 9.5 5 5.7 10 2.7 10

Swollen limbs 0 0 5.5 0 4.8 0 9.5 0 1.4 0 2.7 0

Cough up worms

1.5 0 0 0 0 0 14.3 0 0 5 1.4 0

Blood in stool 1.5 0 0 0 0 0 0 0 0 0 0 0

Rash 0 5 0 0 0 10 0 5 0 10 0 0

Loose runny stools

0 20 0 20 0 30 0 0 0 5 0 26.7

Worm in stool 0 0 0 0 0 0 0 0 0 0 0 3.3

Runny nose 0 0 0 5 0 0 0 0 0 0 0 0

Headache 0 0 0 0 0 0 0 5 0 0 0 0

Chest in drawing

0 5 0 0 0 5 0 0 0 0 0 6.7

Sunken eyes . 5 . 10 . 5 . 0 . 5 . 0

No power . 5 . 5 . 0 . 5 . 10 . 10

Don't know 4.6 5 9.1 0 6 5 9.5 5 2.9 5 2.7 6.7

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

148

Table 27: Percentage of CHV responses on how to prevent parasite infection in young children

Kampong

Speu Mondulkiri Prey Veng Stung Treng Svay Rieng

Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Prepare food hygienically

27.7 10 49.1 20 36.1 30 40.5 5 44.3 35 40.5 13.3

Drink boiled/pure water

33.8 50 36.4 50 34.9 45 54.8 50 31.4 70 35.1 50

Wear shoes/sandals

32.3 20 14.5 20 12 15 66.7 45 27.1 30 32.4 43.3

Wash hands before eating

27.7 15 23.6 35 13.3 20 16.7 45 28.6 45 24.3 30

Don't let play in dirt

27.7 20 14.5 20 19.3 25 14.3 10 14.3 20 21.6 13.3

Eat a healthy diet

12.3 25 12.7 15 31.3 25 4.8 10 18.6 20 17.6 23.3

Wash hands before food preparation

3.1 10 14.5 30 8.4 25 14.3 35 2.9 15 13.5 20

Use toilet/ latrine, not field

9.2 0 1.8 5 1.2 0 21.4 5 0 5 8.1 3.3

Take medicine 7.7 5 1.8 0 1.2 5 4.8 0 28.6 0 8.1 13.3

Clean area around house

3.1 5 3.6 0 3.6 0 14.3 0 1.4 5 6.8 6.7

Cut nails short 3.1 0 0 10 0 5 14.3 5 0 0 2.7 0

Don't feed child meat/fish

1.5 0 3.6 0 0 0 0 0 7.1 0 2.7 6.7

Wear a mask to cover nose and mouth

0 0 0 5 0 0 0 0 0 0 0 0

Cook well . 10 . 10 . 5 . 20 . 15 . 6.7

Using soap while having bath

. 10 . 0 . 0 . 0 . 5 . 3.3

Don't eat raw vegetables

. 5 . 0 . 5 . 5 . 0 . 0

Don't know 16.9 10 12.7 0 24.1 5 7.1 10 2.9 0 12.2 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

149

Table 28: Percentage of CHV responses on how to treat parasite infection in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Medicine – don't know name

21.5 60 40 25 67.5 10 47.6 20 60 20 43.2 36.7

Mebendazole 16.9 0 9.1 10 7.2 0 26.2 5 7.1 15 25.7 3.3

Send to health clinic

46.2 75 36.4 70 34.9 95 16.7 85 41.4 83 21.6 83.3

Albendazole 0 0 0 0 0 0 14.3 0 0 0 2.7 0

Oral dehydration solution

0 5 0 0 0 10 0 5 0 10 0 0

ORYO . 0 . 0 . 0 . 0 . 0 . 3.3

Chili . 0 . 0 . 0 . 0 . 5 . 0

Herb . 0 . 0 . 0 . 5 . 0 . 0

Mebang . 0 . 5 . 0 . 0 . 0 . 0

Benda 500 . 0 . 0 . 0 . 0 . 0 . 3.3

Don't know 20 0 14.5 5 0 0 7.1 5 2.9 0 9.5 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

150

Table 29: Percentage of CHV responses on how to make water safer to drink

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Boil 100 100 96.4 100 96.4 100 100 100 95.7 100 100 100

Use water filter 9.2 85 23.6 90 13.3 90 21.4 60 41.4 85 27 73.3

Add bleach/chlorine

0 10 5.5 10 3.6 15 7.1 0 2.9 15 4.1 0

Let it stand/settle

1.5 0 3.6 0 8.4 0 2.4 0 27 0 4.1 0

Strain through cloth

1.5 0 3.6 0 25.3 0 0 0 0 0 2.7 0

White alum 0 0 0 0 1.2 0 2.4 0 0 0 0 0

Solar disinfection

0 15 0 15 2.4 25 0 15 0 10 0 6.7

Put albeit . 0 . 0 . 5 . 0 . 0 . 0

Herb use in the water to kill the disease

. 0 . 5 . 0 . 0 . 0 . 0

Don't know 0 0 0 0 0 0 0 0 0 0 0 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

151

7. Childhood illness

Table 30: Percentage of CHV responses on causes of diarrhea in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Eating dirty food 73.8 90 56.4 70 65.1 60 61.9 60 71.4 45 63.5 70

Drinking dirty water

64.6 90 58.2 90 61.4 90 88.1 85 57.1 60 62.2 80

Eating food touched by flies

33.8 45 10.9 40 19.3 45 38.1 40 12.9 50 41.9 33.3

Fever 16.9 . 9.1 . 7.2 . 26.2 . 7.1 . 25.7 .

Not washing hands before food preparation

0 30 14.5 15 13.3 30 11.9 25 7.1 30 23 26.7

Not washing fruits or raw vegetables before eating

9.2 25 9.1 50 13.3 45 31 50 31.4 25 23 36.7

Not washing after defecating

3.1 20 1.8 10 0 25 11.9 15 2.9 25 6.8 23.3

Not washing hands after cleaning child's stool

1.5 10 1.8 15 1.2 10 4.8 15 0 15 5.4 3.3

Lack of hygiene/don't wash hands before eating

7.7 0 0 5 0 0 7.1 15 18.6 0 4.1 3.3

Hot breast milk or milk from sick mother

3.1 . 1.8 . 2.4 . 0 . 4.3 . 2.7 .

Bottle feeding 1.5 0 0 0 3.6 0 0 0 1.4 0 0 0

Eat raw vegetables

. 5 . 5 . 10 . 5 . 0 . 0

Untidy defecation

. 0 . 10 . 0 . 5 . 0 . 3.3

Don't know 1.5 0 1.8 0 4.8 0 2.4 0 0 0 4.1 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

152

Table 31: Percentage of CHV responses on symptoms of diarrhoea in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Loose runny stools

58.5 60 41.8 65 51.8 55 73.8 75 57.1 60 50 70

Fever 21.5 15 23.6 45 26.5 50 26.2 25 40 40 25.7 33.3

Dehydration–Sunken eyes

41.5 55 14.5 40 38.6 35 33.3 35 17.1 55 24.3 26.7

Tired/exhausted 9.2 30 3.6 10 1.2 20 1.2 10 8.6 25 13.5 16.7

Vomiting 1.5 20 14.5 20 6 15 6 15 5.7 10 8.1 20

Wrinkled/dry skin

7.7 20 0 25 3.6 15 3.6 5 11.4 35 8.1 23.3

Cough 1.5 0 0 5 1.2 10 1.2 5 0 5 5.4 6.7

Blood in stool 1.5 5 0 0 2.4 0 2.4 5 12.9 5 5.4 3.3

Bloated abdomen

9.2 5 10.9 10 4.8 0 8.4 25 4.3 5 5.4 13.3

Loss of appetite 3.1 15 10.9 15 14.3 20 12 20 1.4 5 4.1 10

Lost weight 1.5 0 7.3 5 0 0 2.4 5 7.1 5 4.1 10

Fast/rapid breathing

1.5 0 0 0 0 0 2.4 0 4.3 25 1.4 3.3

Runny nose 1.5 0 0 0 0 0 0 0 0 0 1.4 0

Stomachache 13.8 0 14.5 5 0 5 0 5 4.3 0 1.4 0

Difficult breathing

0 5 0 0 0 10 0 0 0 5 0 3.3

Chest drawn in 0 15 0 5 0 5 0 5 0 25 0 20

Rash 0 5 0 5 0 5 0 5 0 5 0 3.3

Cadaverous . 25 . 20 . 5 . 10 . 20 . 13.3

Not active . 0 . 5 . 25 . 5 . 5 . 3.3

Don't know 4.6 0 5.5 0 11.9 0 8.4 0 5.7 0 5.4 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

153

Table 32: Percentage of CHV responses on how to prevent diarrhoea in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Drink only boiled/purified water

75.4 90 67.3 90 78.3 90 81 90 72.9 75 74.3 76.7

Eat well-cooked food

53.8 25 49.1 45 59 45 59.5 50 74.3 50 59.5 50


16.9 40 16.4 40 22.9 50 31 55 30 65 32.4 33.3

Protect food from flies

16.9 45 16.4 50 7.2 30 33.3 30 17.1 25 31.1 30


3.1 10 7.3 10 2.4 45 11.9 25 2.9 10 17.6 10

Use toilet/latrine 10.8 5 5.5 20 4.8 5 21.4 15 4.3 0 10.8 23.3

Clean around house

3.1 0 5.5 0 9.6 0 14.3 0 11.4 0 8.1 13.3

Do not eat dirty/moldy food

9.2 0 10.9 10 8.4 0 14.3 0 4.3 10 6.4 3.3

Cut nails short 3.1 0 0 0 1.2 0 2.4 0 1.4 0 2.7 6.7

Dispose of child faeces safety

0 10 0 25 3.6 0 0 0 0 5 2.7 3.3

Wash hands after clean child who defecated

4.6 15 5.5 0 3.6 5 9.5 15 2.9 15 2.7 10

Eat a healthy diet

0 20 0 15 4.8 35 2.4 15 1.4 10 2.7 26.7

Wear shoes/sandals

3.1 0 1.8 5 2.4 0 4.8 0 1.4 0 1.4 3.3

Wear a mask to cover nose and mouth

0 5 0 0 0 0 0 0 0 0 0 3.3

Don't bottle feed 0 0 0 0 1.2 0 0 5 0 0 0 3.3

Don't know 6.2 0 5.5 0 8.4 0 0 0 2.9 0 6.8 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

154

Table 33: Percentage of CHV responses on how to treat diarrhoea in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

ORS 75.4 70 14.5 80 43.4 65 66.7 50 55.7 65 54.1 76.7

Send to health clinic

33.8 70 52.7 65 47 70 28.6 70 61.4 75 35.1 70

Drink boiled/purified water

6.2 0 3.6 5 3.6 0 0 0 5.7 0 17.6 0

Medicine–don't know name

6.2 25 32.7 10 16.9 15 16.7 10 24.3 15 16.2 6.7

Continue breastfeeding

6.2 0 0 10 16.9 0 11.9 0 8.6 15 16.2 3.3

Coconut juice 10.8 0 0 0 2.4 5 2.4 0 5.7 0 13.5 10

Boiled rice water 3.1 0 0 0 6 0 11.9 0 5.7 0 10.8 0

Cotrimaxozole 1.5 0 5.5 0 1.2 0 4.8 0 5.7 0 8.1 0

Tetracycline 1.5 0 3.6 0 2.4 0 0 0 10 0 8.1 3.3

Paracetamol 0 0 1.8 0 0 0 0 0 0 0 2.7 0

Intravenous drip 1.5 0 1.8 0 7.2 5 7.1 0 1.4 0 2.7 3.3

Aspirin 0 5 0 0 0 0 0 0 0 0 0 0

Safe water with sugar and salt

0 0 0 0 0 5 0 15 0 5 0 0

Zn drug . 10 . 0 . 5 . 0 . 0 . 10

ORYO . 5 . 0 . 5 . 0 . 0 . 3.3

Drink much more water

. 5 . 0 . 20 . 0 . 0 . 0

Herb . 0 . 0 . 0 . 5 . 0 . 0

Don't know 3.1 0 7.3 0 0 0 7.1 0 0 0 2.7 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

155

Table 34: Percentage of CHV responses on signs of severe dehydration in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Skin slow to unfold

53.8 75 16.4 70 45.8 60 76.2 65 51.4 80 45.9 70

Sunken eyes 47.7 50 25.5 50 28.9 25 59.5 55 30 55 37.8 36.7

Lethargic–not active

13.8 85 38.2 75 37.3 55 66.7 65 40 75 35.1 70

Few or no tears 1.5 5 0 0 4.8 5 2.4 0 2.9 0 20.3 6.7

Drowsy 12.3 5 9.1 0 22.9 5 26.2 0 5.7 0 17.6 3.3

Lost weight/ cadaverous

6.2 5 14.5 20 2.4 20 0 20 22.9 15 12.2 13.3

Mouth or tongue dry

9.2 15 1.8 15 4.8 10 0 15 2.9 45 9.5 6.7

Loss of appetite 1.5 0 1.8 15 6 20 2.4 20 7.1 0 8.1 0

Thirsty but unable to drink

12.3 5 7.3 10 14.5 0 9.5 10 12.9 5 6.8 0

Grumpy–poor temper

0 0 1.8 0 3.6 0 2.4 0 0 5 4.1 0

Little/no Urination

0 0 0 0 0 0 0 0 0 0 1.4 3.3

Sunken chest . 0 . 5 . 0 . 0 . 5 . 3.3

Fever . 0 . 5 . 5 . 5 . 0 . 0

Rapid breathing . 0 . 5 . 0 . 5 . 5 . 0

Don't know 21.5 0 25.5 5 19.3 0 2.4 0 4.3 0 16.2 3.3

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

156

Table 35: Percentage of CHV responses on symptoms of ARI in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Fast/rapid breathing

33.8 45 10.9 35 48.2 25 38.1 50 34.3 55 55.4 50

Difficulty breathing

21.5 45 25.5 35 44.6 40 47.6 50 32.9 40 50 53.3

Cough 29.2 30 36.4 20 38.6 40 23.8 30 38.6 30 37.8 20

Fever 38.5 50 21.8 55 51.8 65 35.7 35 32.9 60 31.1 36.7

Chest drawn in 53.8 65 1.8 30 28.9 60 16.7 50 25.7 60 28.4 33.3

Noisy breathing 23.1 25 7.3 20 19.3 30 33.3 10 25.7 30 28.4 36.7

Runny nose 4.6 0 9.1 5 3.6 5 4.8 5 2.9 5 6.8 0

Loss of appetite 3.1 0 3.6 10 6 5 0 10 2.9 20 5.4 16.7

Vomiting 6.2 0 1.8 10 0 15 4.8 5 0 5 4.1 6.7

Convulsions 0 0 0 0 1.2 0 0 5 4.3 0 1.4 3.3

Loose, runny stool

0 0 0 5 0 0 0 0 0 0 0 0

Blood in stool 0 0 0 0 0 0 0 0 0 0 0 3.3

Rash 0 0 0 0 0 5 0 0 0 0 0 3.3

Headache 0 0 0 0 0 0 0 0 0 10 0 3.3

Bloated abdomen

0 5 0 5 0 10 0 5 0 0 0 3.3

No power . 5 . 0 . 15 . 0 . 10 . 0

Don't know 20 0 40 20 10.8 0 21.4 10 20 0 10.8 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

157

Table 36: Percentage of CHV responses on how to prevent ARI in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Eat a healthy diet

1.5 20 7.3 0 12 20 2.4 5 4.3 20 16.2 20

Clean area around house

15.4 0 1.8 0 6 5 9.5 5 1.4 0 10.8 0

Only drink safe water

4.6 30 9.1 20 8.4 15 26.2 25 8.6 15 9.5 20

Protect from dew

1.5 5 1.8 35 18.1 20 23.8 45 1.4 5 6.8 16.7

Wear a mask 0 5 1.8 10 0 0 7.1 10 1.4 0 5.4 0


1.5 0 3.6 0 1.2 0 2.4 0 1.4 25 5.4 16.7


0 0 1.8 0 2.4 5 0 0 0 5 2.7 0

Use toilet/latrine 0 0 1.8 0 0 0 0 5 0 5 1.4 6.7

Ventilation for cooking area

1.5 5 0 0 1.2 5 2.4 0 2.9 0 1.4 6.7

Wear shoes/sandals

0 10 0 20 0 20 2.4 15 0 0 0 16.7

Anti–pneumonia vaccination

. 5 . 0 . 15 . 0 . 5 . 0

Not allow children to play or soak in water too long

. 5 . 5 . 0 . 10 . 5 . 0

Don't know/forget

72.3 40 70.9 55 55.4 45 50 40 62.9 35 52.7 40

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Table 37: Percentage of CHV responses on treatment of ARI in young children

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Send child to health centre

75.4 100 47.3 95 85.5 95 47.6 100 85.7 100 78.4 100

Antibiotics–don't know name

0 10 10.9 5 3.6 10 21.4 15 2.9 10 5.4 3.3

Amoxiciline 1.5 0 5.5 0 2.4 0 2.4 0 2.9 0 4.1 0

Paracetamol 4.6 5 3.6 0 3.6 0 4.8 5 4.3 0 4.1 3.3

Cotrimaxozole 0 0 0 0 0 0 2.4 0 0 0 2.7 0

ORS 1.5 0 0 0 1.2 0 0 0 0 0 1.4 3.3

Artemisinin 0 0 0 0 0 5 0 0 0 0 0 0

Don't know 21.5 0 34.5 5 8.4 0 26.2 0 11.4 0 13.5 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

158

Table 38: Percentage of CHV responses on danger signs of diarrhoea when to seek health care from a trained health worker immediately

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Persistent diarrhoea

23.1 60 38.2 55 51.8 65 57.1 45 51.4 65 37.8 67.7

Severe dehydration

50.8 15 16.4 25 37.3 25 54.8 30 32.9 10 32.4 23.3

Can't drink or breastfeed

9.2 10 20 10 3.6 20 42.9 20 15.7 10 25.7 13.3

Child cannot eat 7.7 10 10.9 10 18.1 25 7.1 10 8.6 10 24.3 10

Develop high fever

0 25 5.5 15 16.9 25 42.9 45 21.4 40 21.6 30

Diarrhoea vomiting

35.4 50 38.2 50 14.5 45 28.6 50 24.3 45 21.6 40

Blood in the stools

4.6 10 1.8 20 7.2 15 19 10 12.9 15 10.8 3.3

Fast or difficult breathing

4.6 10 1.8 15 9.6 10 2.4 20 7.1 25 6.8 16.7

Child doesn’t urinate

0 0 0 0 0 0 0 5 0 0 1.4 3.3

Sunken eyes . 5 . 15 . 0 . 10 . 20 . 10

Flaccid skin . 5 . 10 . 0 . 0 . 20 . 3.3

Not active . 15 . 10 . 10 . 15 . 5 . 3.3

Comas . 5 . 0 . 0 . 5 . 0 . 6.7

Don't know 9.2 5 10.9 0 10.8 0 4.8 0 1.4 0 6.8 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Table 39: Percentage of CHV who had heard of ORS for treating diarrhoea and correct knowledge of how to mix ORS

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Heard of Oralyte

96.9 100 85.5 100 98.8 100 95.2 100 98.6 100 98.6 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Volume of water to mix with ORS

1 Litre 88.9 95 72.3 90 82.9 95 95 95 97.1 100 89 100

Total number 63 20 47 20 82 20 40 20 69 20 73 30

2006 (n=374); 2011 (n=130)

159

Table 40: Percentage of CHV knowledge on the danger signs of other illnesses when to seek health care from a trained health worker immediately

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

High fever for two days

52.3 40 38.2 55 60.2 50 19 35 64.3 55 58.1 33.3

Fast and difficult breathing

15.4 25 3.6 5 36.1 15 7.1 30 25.7 30 21.6 30

Can't drink or breastfeed

10.8 5 18.2 15 16.9 15 50 15 12.9 5 18.9 3.3

Unconscious 4.6 10 32.7 25 19.3 5 66.7 30 8.6 5 17.6 33.3

Cough 10.8 25 9.1 20 20.5 45 0 10 27.1 40 16.2 33.3

Severely dehydrated

27.7 10 9.1 20 9.6 10 33.3 10 12.9 15 16.2 10

Child cannot eat 4.6 0 10.9 10 13.3 5 4.8 20 10 5 13.5 6.7

Has blood in stool

1.5 5 1.8 5 3.6 5 7.1 10 8.6 0 10.8 3.3

Chest drawn in 13.8 10 0 0 2.4 15 0 5 20 15 10.8 3.3

Drowsy 4.6 . 1.8 . 0 . 2.4 . 5.7 . 10.8 .

Noisy, wheezing breathing

0 20 1.8 15 8.4 25 7.1 10 4.3 10 8.1 6.7

Vomits continually

10.8 5 18.2 35 10.6 40 28.6 0 17.1 15 8.1 23.3

Worsening health despite home care

15.4 5 12.7 0 3.6 0 47.6 5 1.4 0 6.8 6.7

Severe diarrhoea/persistent diarrhoea

7.7 10 1.8 15 2.4 5 0 0 17.1 15 6.8 13.3

Convulsions 16.9 50 18.2 50 9.6 40 21.4 60 24.3 45 5.4 40

Has red dots on skin

4.6 0 1.8 0 0 5 0 0 10 5 4.1 0

Wasting/acute malnutrition

4.6 . 5.5 . 1.2 . 0 . 4.3 . 2.7 .

Not active/ exhausted

. 10 . 10 . 15 . 30 . 0 . 20

Headache . 0 . 5 . 0 . 10 . 0 . 0

Cadaverous . 0 . 10 . 0 . 5 . 0 . 3.3

Don't know 3.1 0 10.9 0 12 0 2.4 5 0 0 6.8 3.3

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

160

Table 41: Percentage of CHV knowledge on appropriate feeding recommendation for sick children under six months of age

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Give breast milk/more breast milk

84.6 90 61.8 95 84.3 95 90.5 85 90 100 98.7 90

Give more liquids than normal

1.5 0 7.3 5 8.4 0 7.1 0 2.9 0 9.5 0

Give less liquids than normal

0 0 3.6 0 9.6 0 0 0 12.9 0 5.4 0

Give less food than normal

0 0 10.9 0 3.6 5 0 0 15.7 0 5.4 0

Give normal amount of liquids

0 0 0 0 2.4 0 0 0 4.3 0 4.1 0

Give more foods than normal

0 5 5.5 5 4.8 5 7.1 20 0 10 2.7 16.7

Give normal amount of food

3.1 5 5.5 5 2.4 0 0 0 8.6 0 1.4 0

Don't know 13.8 5 14.5 0 8.4 0 0 0 2.9 0 6.8 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 42: Percentage of CHV knowledge on appropriate feeding recommendation for sick children older than six months of age

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Give breast milk/more breast milk

75.4 40 29.1 50 45.8 55 69 55 77.1 75 59.5 56.7

Give more liquids than normal

47.7 15 32.7 15 67.5 10 23.8 10 52.9 10 58.1 13.3

Give less liquids than normal

43.1 10 29.1 0 27.7 0 35.7 0 28.6 0 33.8 0

Give less food than normal

21.5 10 12.7 5 6 0 2.4 0 18.6 0 9.5 3.3

Give normal amount of liquids

0 10 1.8 0 2.4 5 0 0 20 0 9.5 10

Give more foods than normal

7.7 75 5.5 75 4.8 85 0 85 8.6 85 8.1 63.3

Give normal amount of food

0 20 1.8 30 7.2 10 2.4 10 4.3 5 1.4 23.3

Give breast milk less than food

. 0 . 0 . 0 . 0 . 5 . 0

Eat much more green vegetables

. 0 . 5 . 0 . 0 . 5 . 0

Don't know 9.2 0 5.5 0 6 0 0 0 4.3 0 4.1 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

161

8. Antenatal care

Table 43: Percentage of CHV knowledge on recommended number of ANC visits for pregnant women

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Less than 4 times

64.6 15 56.4 45 53 20 54.8 35 54.3 45 55.4 26.7

4 times 12.3 70 5.5 25 15.7 60 23.8 55 8.6 10 16.2 30

More than 4 times

18.5 15 16.5 30 26.5 20 19 10 22.9 45 21.6 43.3

Don't know 4.6 0 12.7 0 3.6 0 0 0 7.1 0 1.4 0

Missing 0 0 9.1 0 1.2 0 2.4 0 7.1 0 5.4 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

162

Table 44: Percentage of CHV knowledge of when pregnant women should go for first ANC visit and percentage response of what an ANC visit should include

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

When a woman should go for first ANC

As soon as she knows

1.5 45 0 40 1.2 50 2.4 15 1.4 50 12.2 40

1 to 3 months 64.6 55 63.6 60 79.5 50 69 85 82.9 50 55.4 60

4 to 6 months 27.7 0 27.3 0 15.7 0 26.2 0 14.3 0 25.7 0

More than 7 months

0 0 3.6 0 0 0 0 0 0 0 2.7 0

Do not know 6.2 0 5.5 0 3.6 0 2.4 0 1.4 0 4.1 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

What should ANC include (tests, information)

Measure abdomen

32.3 25 36.4 45 51.8 55 57.1 60 54.3 30 50 56.7

Get iron tablets 24.6 45 7.3 40 44.6 45 14.3 65 51.4 50 41.9 33.3

Weight measure 26.2 30 27.3 30 16.9 20 73.8 20 22.9 35 17.6 33.3

Tetanus vaccination

4.6 0 0 0 3.6 5 0 0 20 5 16.2 0

Blood pressure test

21.5 10 27.3 20 10.8 25 33.3 15 18.6 15 13.5 20

Height measure 10.8 5 16.4 15 8.4 10 61.9 10 7.1 20 10.8 23.3

Urine test 6.2 30 3.6 40 12 45 11.9 60 8.6 50 10.8 46.7

Pregnancy warning signs

6.2 20 1.8 20 6 25 11.9 15 5.7 20 9.5 16.7

Pregnancy complications

0 5 1.8 0 2.4 10 11.9 5 2.9 20 9.5 6.7

Blood test 0 20 7.3 35 10.8 30 9.5 35 4.3 45 5.4 33.3

Insecticide-treated net

0 15 0 10 0 0 0 25 0 0 0 10

Measure the heart beat

. 5 . 0 . 0 . 0 . 0 . 3.3

Recommend them to use vitamin A

. 0 . 5 . 0 . 0 . 5 . 0

Recommend them to eat as much as they can, and not work over

. 5 . 0 . 5 . 5 . 5 . 0

Check the pulse . 0 . 0 . 5 . 0 . 0 . 3.3

Antenatal check . 0 . 0 . 5 . 0 . 0 . 0

Measure the temperature

. 0 . 0 . 5 . 0 . 0 . 0

Don't know 43.1 20 34.5 15 21.7 10 16.7 10 12.9 0 10.8 10

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

163

Table 45: Percentage of CHV knowledge of iron supplementation during pregnancy and per cent distribution of recommended number of iron pills to take during pregnancy

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Pregnant women should take iron

95.4 100 60 90 95.2 100 95.2 100 91.4 100 87.8 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Number of iron pills recommended for the pregnant woman

Less than 90 pills

32.3 15 39.4 30 27.8 20 22.5 35 20.3 35 13.8 36.7

90 pills 15.4 35 12.1 45 27.8 80 57.5 50 25 50 32.3 20

More than 90 pills

0 30 0 5 0 0 0 0 3.1 5 0 13.3

Don't know 53.2 20 48.5 10 41.8 0 20 15 43.8 10 52.3 30

Missing 0 0 0 10 2.5 0 0 0 7.8 0 1.5 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 62 20 33 20 79 20 40 20 64 20 65 30

2006 (n=343); 2011 (n=130) Table 46: Per cent distribution of when pregnant women should be given vitamin A supplementation and timing for post-partum vitamin A supplementation

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

A pregnant woman should take vitamin A

Before giving birth

35.4 30 41.8 60 37.3 35 64.3 60 51.4 60 25.7 56.7

After giving birth 53.8 65 9.1 30 45.8 60 31 40 37.1 40 66.2 40

Do not know 10.8 5 49.1 10 15.7 5 4.8 0 10 0 8.1 3.3

Missing 0 0 0 0 1.2 0 0 0 1.4 0 0 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=343); 2011 (n=130)

Recommended timing for post-partum vitamin A supplementation

8 weeks or less 74.3 84.6 60 66.7 73.7 83.3 38.5 87.5 65.4 62.5 77.6 75

More than 8 weeks

0 0 0 0 0 0 7.7 12.5 3.8 0 0 16.7

Don't know 25.7 15.4 40 33.3 26.3 16.7 46.2 0 30.8 37.5 20.4 8.3

Missing 0 0 0 0 0 0 7.7 0 0 0 2 0

Total per cent 100 100 100 100 100 100 100 100 100 100 100 100

Total number 35 13 5 6 38 12 13 8 26 8 49 12

2006 (n=166); 2011 (n=59)

164

Table 47: Percentage of CHV response on healthy practices and eating/uncomplicated child birth

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Healthy women have healthy babies

90.8 30 96.4 5 98.8 20 100 0 100 15 100 6.7

Woman should eat less so her baby is smaller/easier to birth

13.8 80 25.5 90 28.9 95 4.8 90 30 80 23 93.3

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 48: Percentage of CHV knowledge on the warning signs of pregnancy when women should go to health centre immediately

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Vaginal bleeding 52.3 60 20 80 49.4 85 76.2 100 64.3 70 55.4 86.7

Swollen hands, feet or face

43.1 50 7.3 70 61.4 85 57.7 65 74.3 65 45.9 60

Severe anaemia 40 30 16.4 35 30.1 20 54.8 20 17.1 15 27 6.7

Severe headache and blurred vision

3.1 10 14.5 15 20.5 45 19 25 15.7 10 20.3 3.3

Fever 7.7 0 7.3 15 10.8 35 19 30 10 15 18.9 10

Stomachache 10.8 35 36.4 25 18.1 15 21.4 45 15.7 30 14.9 43.3

Premature rupture of membranes

3.1 10 1.8 10 9.6 45 9.5 25 2.9 15 14.9 10

Shock 0 0 3.6 0 7.2 0 4.8 0 18.6 5 5.8 3.3

Persistent vomiting

0 0 1.8 0 3.6 5 0 0 1.4 10 2.7 3.3

Yellow eyes 1.5 5 0 0 3.6 0 11.9 5 1.4 0 1.4 0

Black stools 0 0 0 0 0 0 16.7 0 0 0 0 0

Blood pressure . 5 . 0 . 0 . 0 . 0 . 16.7

Fetus has no action

. 0 . 0 . 0 . 5 . 5 . 3.3

Weak . 10 . 0 . 0 . 5 . 5 . 0

Don't know 18.5 0 29.1 0 8.4 0 4.8 0 2.9 0 14.9 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

165

9. Malaria

Table 49: Percentage of CHV reporting that malaria is common in their village and recommendations how to prevent the spread of malaria

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Malaria is common in village

63.1 0 100 70 28.9 5 97.6 25 21.4 10 25.7 13.3

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Recommended to prevent the spread of malaria

Sleep under bed net

89.2 90 92.7 90 80.7 95 95.2 100 82.9 100 85.1 96.7

Cut shrubs around the house

26.2 60 21.8 85 22.9 70 69 90 28.6 75 33.8 70

Protect water from mosquito larvae

13.8 65 3.6 75 27.7 50 40.5 60 5.7 80 14.9 80

Use mosquito repellent

0 0 7.3 0 2.4 0 7.1 5 8.6 0 10.8 0

Use mosquito coils

0 10 1.8 5 0 0 21.4 10 5.7 5 8.1 16.7

Use insecticide 4.6 20 1.8 0 6 5 14.3 5 12.9 5 8.1 23.3

Boil water/drink clean water

18.5 0 7.3 5 3.6 10 0 5 20 0 6.8 6.7

Use long pants/shirts

1.5 10 12.7 30 0 5 42.9 30 5.7 25 4.1 6.7

Light a smoky fire under house

0 0 5.5 10 2.4 5 28.6 15 1.4 0 2.7 6.7

Don't allow children to play in dark places

. 5 . 0 . 0 . 0 . 10 . 3.3

Using alt bet . 10 . 0 . 0 . 5 . 15 . 3.3

Ensure home has enough light

. 0 . 10 . 5 . 0 . 0 . 0

Don't know 7.7 0 1.8 0 13.3 0 0 0 5.7 0 9.5 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130) Table 50: Percentage of CHV responses on what can be done with a bed net to prevent malaria by domain

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Dip bed net with insecticide 81.5 90 98.2 100 54.2 95 100 100 58.6 95 77 100

Repair holes 24.6 15 9.1 10 16.9 10 19 5 41.4 5 23 6.7 Wash bed net regularly 1.5 15 1.8 10 14.5 5 4.8 0 22.9 5 8.1 3.3

Don't know 6.2 0 1.8 0 19.3 0 0 0 4.3 0 8.1 0

166

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

Table 51: Percentage of CHV responses on how to treat malaria

Kampong


Control Province

2006 2011 2006 2011 2006 2011 2006 2011 2006 2011 2006 2011

Go to health centre

64.6 100 61.8 95 80.7 100 52.4 100 88.6 100 64.9 96.7

Malarine 7.7 0 12.7 5 2.4 0 9.5 0 7.1 0 8.1 0

Quinine/ chloroquine

3.1 0 1.8 0 0 0 9.5 0 0 0 2.7 0

Antibiotics DK name

0 25 1.8 5 2.4 5 14.3 5 2.9 5 1.4 13.3

Mefloquine 0 0 5.5 0 0 0 0 0 1.4 0 1.4 0

Paracetamol 1.5 0 5.5 0 1.2 0 4.8 0 0 0 1.4 0

A+M (2,3,4) 0 0 10.9 5 0 0 14.3 0 0 0 0 0

Artemisinin 0 0 3.6 0 0 0 0 0 0 0 0 0

Intravenous drip 0 0 1.8 0 1.2 0 2.4 0 0 0 0 0

Herb . 0 . 0 . 5 . 0 . 0 . 0

Don't know 24.6 0 7.3 0 14.5 0 16.7 0 5.7 0 18.9 0

Total number 65 20 55 20 83 20 42 20 70 20 74 30

2006 (n=389); 2011 (n=130)

end of project evaluation of health behaviour … · at the end of the ec-funded health behaviour...

Documents