world relief’s cs health information system ccih m&e workshop may 28, 2005 melanie morrow...

World Relief’s CSHealth Information System

CCIH M&E Workshop May 28, 2005

Melanie MorrowWorld Relief

Presentation

• Overview of World Relief’s Vurhonga CSP in Mozambique

• Key components of the Vurhonga HIS– Regular surveys to monitor progress– Community HIS

• Tracking mortality

WR Vurhonga “Dawn” CSPs• Vurhonga 1 1995-1999

– Guija & Mabalane Districts– 107,000 population

• Vurhonga 2 1999-2003– Chokwe District– 2350 Volunteers trained in

173 Care Groups– 130,000 (140,000 EOP) pop– C-IMCI + HIV + BS

• Expanded Impact 2004-09– 5 New Districts in Gaza

Province– C-IMCI + HIV

Major Program Components

• Educating and mobilizing the community to prevent illness and seek appropriate treatment

• Creating and training VHCs to address health issues at village level

• Increasing access to care at village level

Vurhonga 2 Care Group Structure

5 Supervisors26 Animators

Care Groups of 10-15

Volunteers 10 HH per

volunteer

Reach: Every HH

One trainer can train & supervise 8 care groups, each with 8-10 volunteers.

Each volunteer is responsible for the 10-15 households on her block.

In Vurhonga 2: 26 trainers reached 24,500 HH via 2350

volunteers trained in 173 Care Groups

Care Group Meetings

• Volunteers verbally report and discuss statistics from the C-HIS

• Problem solve as a group

• Between meetings, Volunteers conduct home visits for the 10-15 HH in their “block”

Animator trains volunteers in interventions using pictures, stories, songs and drama

Home visit

• Volunteers greet family and inquire about their wellbeing

• Address current health concerns in HH

• Teach health lesson learned during most recent care group mtg.

• Make mental note of births, deaths or pregnancies

Other Care Groups

• Churches: Care Groups for pastors to teach BCC they share with their congregations.

• Grannies: Care Groups for grannies ensure support of elders.

Village Health Committees

Membership includes:– Chef de Saude – Village leader– Health Post Socorrista– Care Group leader– Neighborhood reps (max 5)– Church leader– Member of OMM (women’s organization)

Vurhonga HMIS Components

• Full count of beneficiaries at baseline and repeated as needed (can include retrospective birth and mortality questions)

• Baseline and Final KPC Survey• Monitoring surveys to track progress towards

project objectives (every 3-6 months)• Community-HIS (monthly Care Group statistics)

for monitoring vital events

Monitoring Surveys

• Abbreviated version of KPC instrument

• Similar to LQAS in that the sample is based on supervision areas; Random selection of 1 Care Group per animator

• Staff trade supervision areas and interview all HH with children <2 pertaining to that CG (sub sample OK)

• Track progress towards objectives

Monitoring Survey Form

Children 1-2 yrs old that have been fully vaccinated

88% 88% 86%89% 89% 91% 90% 91%

74%

71%59%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Sept'9

9

Dec'9

9

Mar

'00

Jun'0

0

Sept'0

0

Dec'0

0

Mar

'01

Jun'0

1

Sept'0

1

Jan'0

2

Mar

,02

Jun'0

2

Sept'0

2

Dec'0

2

Jun'0

3

Jul'0

3

Sept'0

3

Target

Actual

Objective: Increase from 0.3% to 70% use of mosquito nets year-round by all children under 5

0%

79%

68%

81% 81% 85%90%92%95%91% 93%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Target

Actual

Objective: Increase from 28% to 75% children treated within 24h for fever (suspected malaria) at health facility

28%

82%73%

80%86% 89% 90% 92% 96% 97%

90%

0%

20%

40%

60%

80%

100%

120%

Target

Actual

Objective: Increase to 50% children treated within 24h for rapid, difficult breathing at appropriate health facility

76%

63%

91%85%

60%

2%

100%100%100%99%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Target

Actual

Objective: Increase from 1.6% to 40% women who recognize at least 3 signs of STIs

1.60%6% 5%

44% 42% 41%

57% 59%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Sept'99

Dec'99

Mar

'00

Jun'0

0

Sept'00

Dec'00

Mar

'01

Jun'0

1

Sept'01

Jan'0

2

Mar

,02

Jun'0

2

Sept'02

Dec'02

Jun'0

3

Jul'0

3

Sept'03

%

Series1

Series2

Data use

• Revise strategies as appropriate (e.g. message re: mosquito nets; new pictures for reproductive health)

• Track performance by staff supervision areas, by individual villages & by district

• Share with care groups, VHCs and MOH to motivate and engage in problem solving

• Identify and respond to problems early

C-HIS (Monthly Care Group Stats)

Care groups form the basis for a sustainable community-HIS. Volunteers verbally report on vital events (births, deaths, pregnancies) that they discuss in their meetings. The information is shared with the community and MOH without dependence on project staff. Village Health Committees (VHCs) and the MOH make decisions using these data. Volunteers are motivated by the measurable impact they are having.

Care Group Statistics

• Volunteers verbally report vital events (births, deaths, pregnancies) for their block

• Mortalities are reported with the following information: Name, sex, age at death, date of illness onset, signs of illness, date of death.

• Literate volunteer records data

C-HIS (cont.)

• Discussion of illness signs during meeting used to determine most likely cause of death.

• “Questions of the month” can be added• Bi-directional learning with Animator• Summary data given to Vurhonga Animator and

to village Socorrista• Project staff together discuss monthly results and

implications during regular meetings, and take action.

Information Flow

.

CSP

Care Group

MOH

VHC

Animator

Socorrista

Sustainability of Info Flow

.

Care Group

MOH

VHCSocorrista

Vurhonga I volunteer retention 20 months post project

• Vols active at end of project: 1457• Vols who left post/moved: (92)• Vols who died: (44)• Replacement volunteers: 40

TOTAL No.VOLS STILL ACTIVE: 1361or 93%– Attrition 6.59%

50% of HH were visited by their volunteer in two weeks before survey

Time Cost of C-HIS Tabulation

• 30 minutes during CG meeting once/month

• 15 minutes for socorrista to compile village-wide data

• 30 minutes for district-wide tabulation at District Hospital

VHCs and CGs take action

• VHCs of 25 de Septembro and other villages noted increase in malnourished children in early 2002.

• Initiated Hearth community nutrition rehabilitation sessions using Care Group volunteers.

• Underweight children decreased from 13% in March 2002 to 7.2% in July despite food shortages.

Community goal setting

• Mapapa VHC noted that 19 HH lacked latrines; Set goal for all HH to have latrines within 3 months.

• 25 de Septembro VHC helped pass a local law requiring any HH that didn’t build a latrine to pay for the labor of others sent by VHC to do it for them.

Community-based accountability

• Muzumia village VHC noted pregnant women not using hygienic delivery huts assisted by TBA

• Data prompted community investigation

• Found TBA was demanding unauthorized payment

• Involved MOH to resolve issue

MOH preparedness

• Increases in diarrhea cases helped the MOH in Chokwe district to anticipate and stave off a cholera epidemic that other districts were unprepared for.

• Community has louder voice when backed by data

Benefits of community-specific data

• Local data used at local level prompted communities to:– Tackle malnutrition– Identify underlying problems– Involve district MOH in relevant matter– Set goals– Pass local laws– And appreciate impact of volunteers

Project benefits of regular HMIS feedback

• Keep project in touch with community

• Motivate staff—can see impact– Results graphed on office wall

• Identify and respond to problems – Personnel– Intervention/communication strategy

• Justify continued funding

Ingredients for Success

• Analysis and application of data by those involved in collecting it

• Only collect what actually use

• Link to lasting community structures (CGs and VHCs)

• Sustained volunteer participation (<2% drop out per year)

Volunteer Incentives

• Examples of tangible incentives – Year one: head scarf– Year two: kapulana traditional skirt– Year three: project T-Shirt

• Intangible incentives– Communication of respect and appreciation– Social support– Community recognition

End Result

As a result of CGs and VHCs using the C-HIS, the community has an effective system for monitoring and governing its own health—as well as interfacing with district MOH authorities.

200

3

FutureGenerations

Reasons to measure mortality rates and

changes in a CS project

Measurement of Mortality Rates and Causes of Mortality

• An Essential Tool for Maximizing Program Effectiveness?

• A CORE Function in Child Survival Programs?

The Initial Three-Tiered Approach to Monitoring and Evaluation

• Tier One: Counting the number of services provided

• Tier Two: Measuring coverage in the project population

• Tier Three: Measuring mortality impact

Arguments FOR Monitoring Mortality

• Is THE key indicator

• Can guide programming/increase program effectiveness

• Motivates staff

• Guides program policy formulation

Arguments AGAINST Monitoring Mortality

• Is too complicated, too time consuming, and takes high-level expertise, and must be carried out by outsiders

• Requires a “control” population• Is too expensive• Takes too many years to achieve impact• Takes a very large population in order to

document significant impact

“Gold Standard” for Demonstrating Mortality Impact

• Have an intervention and comparison area• Show that mortality rates in these two areas were similar

before the intervention• Show that the mortality decreased significantly more in the

intervention area than in the control area• Demonstrate that the mortality reduction should be

attributed to the intervention

Disseminating results

• Village chief and health committee are regularly informed of deaths and involved in discussion to learn from event.

• Trends are shared less often, at most every 6 months.

• Data are aggregated by project staff and (in Mozambique) by MOH, to promote sustainability.

C-HIS Information Flow from Care Groups

CSP

Care Group

MOH

VHC

Animator

Socorrista

Sustainability of data flow from Care Groups to Village

Health Committees & District MOH

Care Group

MOH

VHCSocorrista

Moz: U5 cause specific mortality Chokwe District (Mozambique) under 5 cause specific mortality March 2000-

July 2001

0

10

20

30

40

50

60

Malaria

Diarrhea

Pneumonia

Malnutrition

Cambodia: % of deaths attributable to EPI

EPI preventable deaths

19.921.3

15.3

11.78.6 9.5 8.5

1.8 2.14.2 3

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11

Quarters 2000-2002

Calculating mortality ratesInfant Mortality Rate:# deaths children 0-11 mo/1000 live births

Child Mortality Rate:# deaths in children age 12-59m/1000 live births

Under five mortality rate:• # deaths in children U5/1000 live births

Counts also useful if don’t know # births

Problems with Prospective Tracking

• Under-reporting of births—need to use a pregnancy register to catch all births

• Sensitivity needed to discern when culturally appropriate to visit family without waiting too long so that people forget important details.

• Hard to independently verify if all deaths have been captured.

Retrospective tracking of Mortalities in Mozambique

• Midterm count of all beneficiaries included inquiry about all births and deaths during preceding two years.

• Possible underreporting because– respondents inclined to leave out events that

occurred on the “border” of time asked about (though bounded by flood)

– Less likely to include more distant events

Mortality Data from Census/HIS

119

65

45

70

37

23.9

0

20

40

60

80

100

120

140

3/2000-2/2001 3/2001-2/2002 3/2002-2/2003

U5 Death Rate

Infant Death Rate

Retrospective tracking using pregnancy history

• At final eval, sample of 250 women interviewed about all pregnancies they have had during their lifetime and their outcomes.

• Intervals spanning 3 or more years without a birth were probed for possible miscarriages or unreported mortalities

Mortality Data from Pregnancy Histories

128.2

189.7

33.5

59.869

11.2

0

20

40

60

80

100

120

140

160

180

200

1998-1999 2000 2001-8/2003

Child Death Rate

Infant Death Rate

Comparison of Census vs. Pregnancy History

0

20

40

60

80

100

120

140

160

180

200

Year 2000** Year 2001 Year 2002

Census/HIS

Pregnancy History

**Pregnancy History data for 2000 includes flood deaths

Pregnancy History (cont)

• Pro: get complete history, has been validated in literature for accurate mortality estimates going back ~10 yrs.

Problems with Retrospective Studies

• Recall bias leading to under-reporting• If don’t use own staff, population reluctant to

talk about deaths; if use own staff scientific community reluctant to believe results

• Making lists is sometimes considered a suspect (politically destabilizing) activity

• Cultural definitions of child deaths (e.g. baby not considered a person until reaches a certain milestone or named)

Problems with Retrospective Studies (cont.)

• Determining adequate sample size for retrospective pregnancy history can be difficult

• Sample size can be quite large

• High maternal mortality rates could skew results

• Many confounders

Staff and Volunteers in Mapapa Village

Project Action

• Quarterly survey results showed children's ITN use changed as follows– Sept 1999 0%– April 2001 93%– Aug 2001 68%

• Refocused BCC to promote ITN use even in “winter” months– July 2002 81%

Key Ingredients for Success

• Analysis and application of data by those involved in collecting it

• Only collect what actually use

• Link to lasting community structures (VHCs)

• Sustained Volunteer Participation

Vurhonga I volunteer retention 20 months post project

• Vols active at end of project: 1457• Vols who left post/moved: (92)• Vols who died: (44)• Replacement volunteers: 40• TOTAL VOLS STILL ACTIVE: 1361

– Attrition 6.59%

• 50% of HH visited by volunteer in preceding two weeks

Volunteer Attrition in other WR CSPs

• 13.2% in Cambodia at end of year three (excluding deaths and relocation)– Lack of community identity

• 10% in Malawi at end of year one– Both men and women as volunteers– Association with established health institution

led to expectation of employment

Volunteer Motivation for Vurhonga

• Examples of tangible incentives – Year one: head scarf– Year two: skirt– Year three: project T-Shirt

• Intangible incentives– Communication of respect and appreciation– Social support– Community recognition

Benefits

• Accountability

• Contact with community

• Consensus-building

• Strengthening of partnerships

• Empowering communities to take responsibility for their health