achieving sustainable and equitable odf at scale - …...2018/11/04 · achieving sustainable and...

Achieving sustainable and equitable ODF at scale -evidence from sanitation and hygiene behaviour

change programmes

Hannah ChirgwinRadhika MenonWit Wichaidit

Matteus van der Velden

Water & Health Conference at UNC1 November 2018

Overview of the session

1. Introduction & background - WSSCC

2. WASH Sector Evidence gap map – 3IE

3. Outcome survey methodology for GSF programmes – UB

4. Systematic reviews & Impact study - 3IE

5. Discussion & closure

• Q&A after each presentation

2

Water, sanitation, and hygiene promotion for households, schools, and

health facilities – an evidence gap map update

Hannah Chirgwin

Research Associate

[email protected]

Water and Health 2018, UNC

Date of presentation: 01 November 2018

Hugh Waddington Hannah Chirgwin Duae Zehra John EyersSandy Cairncross

Authors and acknowledgements

• Thematic evidence collection on programmes, e.g. on a range of interventions

• Presents a matrix of policy relevant interventions, intermediate outcomes, and impacts

• Impact evaluations and systematic reviews

• Additional filters for region/country, study design, population, etc.

• A tool to navigate the evidence base

• A global public good

What is an evidence gap map?

Birte Snilstveit, Martina Vojtkova, Ami Bhavsar, Marie Gaarder 2013

Democratising evidence for accountability and learning

PRISMA flow chart

18, 037records identified through academic and trial registry database searching

430records identified through other

sources (grey literature searching, inclusion in 2014 map, etc.)

Title & abstract screening criteria:- Date- Country- Intervention/relevance- Study design- Relevant outcomes - Duplicate

13, 458records after duplicates

removed

1, 246 records manually excluded

11, 460 records excluded by machine

learning

Full-text screening criteria:- Date- Country- Intervention- Study design- Relevant outcomes - Excluded version (where

multiple publications)- Full-text unavailable

752records screened at full-text

Exclude on country: 7Exclude on intervention: 78

Exclude on study design: 140Exclude on relevant outcomes: 34

Exclude on version: 44Exclude as full-text unavailable:41

320 impact evaluations, 41 systematic reviews, and 47

protocols/trial registries included in the map

What vs. How

What vs. How

Source: Wager Lanoix1958, https://water1st.org/problem/f-diagram/

https://water1st.org/problem/f-diagram/

http://gapmaps.3ieimpact.org/evidence-maps/water-sanitation-and-hygiene-wash-evidence-gap-map-2018-update

http://gapmaps.3ieimpact.org/evidence-maps/water-sanitation-and-hygiene-wash-evidence-gap-map-2018-update

• Sector-wide evidence map covering interventions to improve WASH access in households, communities, schools and health facilities

• 320 completed impact evaluations

• 47 ongoing impact evaluations

• 42 completed systematic reviews

• 1 ongoing systematic review

• 39 IEs in schools

• 1 IE in health facilities

Key statistics

Evidence base in L&MICs

Interventions mechanisms

0.00%

10.00%

20.00%

30.00%

40.00%

Direct hardwareprovision

Health messaging Psychosocial'triggering'

Systems basedapproaches

Until 2008

Since 2009

Intervention technologies

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

40.00%

45.00%

50.00%

Water Sanitation Hygiene Combined

Until 2008

Since 2009

Frequency of outcomes

0 20 40 60 80 100 120 140 160 180 200

Hygiene behaviour

Construction, use, and maintenance of latrines

Open defecation

Sustainability and slippage

Diarrhoeal disease

Nutrition and anthropometry

Mortality

Drudgery, pain, and musculoskeletal disorders

Income, consumption, and poverty

Safety and vulnerability

Changes in what is being reported on?

0.00% 20.00% 40.00% 60.00% 80.00% 100.00%

Behavioural

Health

Socioeconomic

Post-2008

Pre-2008

Populations

0

50

100

150

200

250

300

350

Rural Urban Slum(informal

settlement)

Refugeecamp

People livingwith HIV

Humanitariancrisis

People withdisabilities

Outcomes

- Time use (22 studies)

- Psychosocial health (7 studies)

- Safety and vulnerability (4 studies)

BUT gender analysis rarely used to understand programme effects and most studies don’t even report sex-disaggregated outcomes!

- 20% of IEs and SRs report sex disaggregation

- Outcomes disaggregated include: psychosocial health (43%), education and cognitive development (40%), open defecation (33%), time use (26%)

Gender-sensitivity

Study designs

62%

33%

5%

Randomized ControlledTrial (RCT)

Non-Randomized Design

Natural Experiment

What are we learning from SRs?

Topic area High confidence

Disposal of excreta Clasen 2010De Buck 2015

Hand hygiene Aiello 2008Ejemot-Nwadjaro 2015De Buck 2015

Water quality Clasen 2015

Trachoma Rabiu 2012

MHM Hennegan 2015

WASH in humanitarian settings Ramesh 2015

Nutrition Dangour 2013

People living with HIV/AIDS Peletz 2013

Slums Turley 2013

Schools ?

Community-driven approach ?

• Sustainability and slippage

• Psychosocial health

• Menstrual care

• Vulnerable populations

• Health facilities

• Synthesis gaps

Gaps?

Outcome Surveys

WSSCC’s Global Sanitation Funds Outcome Surveys in

Malawi, India and Tanzania

Wit Wichaidit, MS and Pavani K. Ram, MD

University at Buffalo

11/16/2018 23

Global Sanitation Fund

• UN Human Development Report 2006 raised the issue of global sanitation crisis

• GSF established in 2008 by Water Supply & Sanitation Collaborative Council (WSSCC )

• The world's only fund dedicated solely to improving sanitation

• $117 million USD allocated to 13 countries

11/16/2018 24

Key Results & GSF Outcome Surveys

• To be conducted every two years

• Independent verification of programme results

• Measure sustainability of the results achieved

• Measure other information not part of the routine monitoring system 25

Theoretical Framework for the Outcome Survey

(Based on theory of change for collective behavior change programs, with individual- and household-level effects)

Learning beyond ODF: Objectives of core analyses of outcome surveys

To describe key sanitation and hygiene outcomes of Global Sanitation Fund in study households and public facilities

To describe emerging process and outcome indicators of the program, including exposure to program activities, social norms, and habits of latrine use

To describe the needs of marginal and vulnerable households/population, and assess equity and non-discrimination by disaggregation of the key indicators by equity variables (e.g., geographic area, demographics, wealth status)

To assess sustainability of open-defecation free (ODF) status among ODF-verified communities

Core Methods

• Cross-sectional survey

• Multi-stage cluster sampling

• Data collection sites: households, schools, health facilities, and communities

• Data collection methods• Structured interviews: 1) heads of households;

2) female household members; 3) elderly/persons with disabilities

• Visual inspection of WaSH facilities and surroundings

• Structured observation in subset of households

• Community inspection for ODF verification

11/16/2018 28

Survey Description

11/16/2018 29

Malawi India Tanzania

Research firm The Malawi Polytechnic, Blantyre

Centre for Media Studies, New Delhi

FXBT Health, Dar-Es-Salaam

Data collection dates

April - May 2017 April - May 2018 June - August 2018

Geographicdistribution

All 6 programmedistricts

3 states, 8 districts, 56 program blocks

All 3 programmedistricts

Sources of data 100 villages1594 households54 schools18 health facilities

65 villages1051 households 66 schools25 health facilities

73 villages 1185 households 46 schools31 health facilities

Findings from household data collection

11/16/2018 30

Access to sanitation facilities in survey populations

11/16/2018 32

*Data from Malawi did not allow for differentiation between Basic and Limited sanitation facilities

Reported information on latrine use (> 5 years old) and child feces disposal

11/16/2018 33

38

50

4

15

97

7

53

41

94

14

76

15

0 20 40 60 80 100

Person >5 years always uses the latrine

Person >5 always always or sometimesdefecate in the open

Feces of child <5 put in toilet

Child <5 uses the toilet

Tanzania Malawi India

Exposure to GSF program activities


(N=1065) (N=1835) (N=1485)

Participated in transect walk 53% 7% 6%

Participated in creating map of feces in the environment

39% 3% 2%

Reported that someone came to the home to talk about using a toilet

82% 34% 70%

Participated in at least one programme activity

88% 44% 85%

11/16/2018 35

Social norms of sanitation

Personal normative beliefs

Empirical expectations

Normative expectations

Bicchieri, Norms in the Wild, 2017

11/16/2018 36

Empirical Expectations: I see other peopleusing the latrines

11/16/2018 37


Males

(N=675)

Females

(N=1328)

Males

(N=1051)

Females

(N=784)

Males

(N=352)

Females

(N=1132)

Neighbors regularly defecate in the

open/field

Some/most/all of them 21% 23% 89% 88% 53% 29%

None of them 78% 74% 11% 12% 47% 71%

How many of your neighbors use a

toilet / latrine?

All of them 81% 82% 15% 14% 47% 54%

Most of them 19% 18% 85% 86% 53% 46%

Empirical expectations score (IQR)

(total possible score of 2)2 (2,2) 1 (0, 2) 1 (0,2)

11/16/2018 38

Empirical expectations of sanitation behavior

Normative Expectations: Other people care strongly about me using the latrine

11/16/2018 39


Male

responden

ts (N=675)

Female

respondents

(N=1328)

Male respondents

(N=1051)

Female

respondents

(N=764)

Male

respondents

(N=352)

Female

respondents

(N=1132)

Where do your neighbors think you should defecate? In the latrine / toilet

98% 97% 90% 88% 97% 97%

How many of your neighbors would agree with the statement: Everyone should use a toilet / latrine?

All of them

87% 84% 50% 52% 64% 70%

How many of your neighbors would agree with the statement: It is fine for everyone to defecate in the

open? None of them

86% 81% 63% 61% 77% 79%

Median normative expectations

score (IQR)

(total possible score of 5)

5 (5,5) 3 (2, 4) 5 (4,5)

11/16/2018 40

Normative expectations of sanitation behavior

Habit Definition and Measurement

• Questions developed based on work in social psychology (Verplanken & Orbell, 2003)

• A habit is a regular tendency or practice that a person:

1. Typically does (i.e, "repetition");

2. Does so without planning (i.e., "automaticity"), and;

3. Will feel uncomfortable not doing so (i.e., "identity").

11/16/2018 41

https://images.mentalfloss.com/sites/default/files/styles/mf_image_16x9/public/534067-istock-637130216.jpg?itok=WYcdFb71&resize=1100x1100

Verplanken, B., & Orbell, S. (2003). Reflections on Past Behavior: A Self-Report Index of Habit Strength. Journal of Applied Social Psychology, 33(6), 1313–1330. https://doi.org/10.1111/j.1559-1816.2003.tb01951.x

Latrine use habit


N=2003 N=1835 N=

Where do you usually go when you need to defecate? Always use

the latrine (repetition)

97% 42% 98%

Is using the latrine when you need to defecate something you

have been doing for many years? Yes (repetition)

98% 76% 96%

Is using the latrine when you need to defecate something that

you do without having to plan or think about? Yes

(automaticity)

60% 77% N/A

Is going to the bush when you need to defecate something that

you do without having to plan or think about? Yes

(automaticity)

25% 76% N/A

Is using the latrine when you need to defecate something you

would find hard not to do? Yes (identity)

89% 54% N/A

Median habit score (IQR) (total possible score of 6) 5 (5,6) 2 (1.3) N/A11/16/2018 42

Assessing equity and non-discrimination

• Gender

• Age

• Mobility and/or visual limitations

• Wealth

• Sub-national region

Latrine access to different categories of household members: Malawi

Region Wealth

Ntchisi District Rumphi District Lowest quintile Highest quintile

Girls 56% 75% 59% 70%

Boys 91% 88% 82% 95%

Men 96% 89% 87% 95%

Women 99% 97% 98% 97%

Latrine access to different categories of household members: India

Region Wealth

Assam Bihar JharkhandLowest

quintile

Highest

quintile

Girls 63% 85% 80% 69% 79%

Boys 70% 88% 91% 81% 81%

Men 84% 89% 96% 82% 88%

Women 85% 91% 98% 88% 89%

Equity and non-discrimination: persons with disability do not report difficulty in access in Malawi

• A household member with limited mobility or vision reported in 64 households (4%)

• 26 persons available and consented for interview• 23 reported being able to use the latrine every time necessary

• 24 reported privacy when using the toilet

• 15 reported being involved in toilet type decision of the household

Equity and non-discrimination: elderly and persons with disability do not report difficulty in access in India

ReportedPersons with visual or mobility

limitations (N=28)

Persons > 65 years old

(N=197)

Being able to use the latrine every

time necessary14 103 (92%)

Feeling very safe when using the

latrine during the night12 80 (72%)

Privacy when using the toilet 14 105 (94%)

Being involved in toilet type decision

of the household6 72 (59%)

Can use latrine without help from any

other person12 (75%) 102 (91%)

Sustained ODF status among ODF-verified communities: findings from Malawi

92 of 100 villages had been previously declared ODF, comprising 1467 households

Criteria for ODF:

• Households with access to a sanitation facility belonging to the respondent

• Households with a facility that ensures privacy

• No human feces observed in household area / compound

Adherence to ODF criteria, among households in previously ODF-declared villages, Malawi

Access to sanitationfacility

Facility ensuresprivacy*

No human fecesobserved

0

10

20

30

40

50

60

70

80

90

100

Overall Poorest Second Middle Fourth Wealthiest

Sustained ODF status among ODF-verified communities: findings from India

35 of 65 (54%) villages had been previously declared ODF, comprising 574 (55%) households

June 2015: Ministry of Drinking Water and Sanitation definition of ODF"ODF is the termination of fecal-oral transmission, defined by • a) no visible feces found in the environment/village; and • b) every household as well as public/community institutions using safe technology

option for disposal of feces”*

*”Safe technology option means no contamination of surface soil, ground water or surface water; excreta inaccessible to flies or animals; no handling of fresh excreta; and freedom from odour and unsightly condition)”

http://swachhbharatmission.gov.in/sbmcms/ writereaddata/images/pdf/Guidelines/Guidelines-ODF-Verification.pdf

http://swachhbharatmission.gov.in/sbmcms/ writereaddata/images/pdf/Guidelines/Guidelines-ODF-Verification.pdf

Adherence to ODF criteria, among households in previously ODF-declared villages, India

Access to latrine Fly proof latrine Latrine with safeseptage disposal

Soap and water in ornear the latrine

0

10

20

30

40

50

60

70

80

90

100

Overall Assam Bihar Jharkhand

A high bar for ODF status is difficult to sustain

Malawi

India

0

10

20

30

40

50

60

70

80

90

100

All households meetnational ODF criteria

<1 non-adherenthousehold

<2 non-adherenthouseholds

<3 non-adherenthouseholds

211

17

70

0

46

8694

% o

f co

mm

un

itie

s m

eeti

ng

OD

F ad

her

ence

cri

teri

a

Summary of the Findings

Item Malawi India Tanzania

Access to Basic Sanitation

15% of population*

48% of population 69% of population

ReportedLatrine Use

97% of household members

38% of household members

94%

Exposure** 88% 44% 85%

SocialNorms***

EE score = 2 (2,2)NE score = 5 (5,5)

EE score = 1 (0,2)NE score = 3 (2,4)

EE score = 1 (0,2)NE score = 5 (4,5)

Habits Scores Median (IQR) = 5 (5,6)

Median (IQR) = 2 (1,3)

N/A

11/16/2018 55

*Included both Limited and Basic sanitation **Participation in at least one programme activity among respondents*** Median (IQR); EE = Empirical Expectation; NE = Normative Expectation;

Summary of the Findings

Item Malawi India Tanzania

Latrine access equity and non-discrimination

Girls had lowest level of access;

Persons with disability do not

report difficulty in access

Girls had lowest level of access

Elderly and persons with disability do not

report difficulty in access

Analysis on-going

Sustained ODF status among ODF-verified communities

>= 98% of interviewed

households had no human feces in

the vicinity

Access to latrine in previously ODF-

declared villages ~ 55%

Analysis on-going

11/16/2018 56

Implications

•Developing and validating indicators is a process, not an endpoint•We’re still learning what are the best questions

to ask

•Construct validity of the data•How much do the answers actually tell? •Use of hypothetical scenarios and vignettes•Feasible methods for quantitative assessment

11/16/2018 57

Evaluation and Research Agenda

•Evaluation agenda•Multiple-countries comparisons•Comparison with previous outcome surveys•Collaboration with other actors in the WASH sector (e.g., UNICEF)

• Research agenda•Harvest the potential of multiple-countries harmonized surveys•Social norms in 13 countries•Structured observation in 13 countries

11/16/2018 58

CollaboratorsGlobal Sanitation Fund – Water Supply and Sanitation Collaborative CouncilRhiannon James, Carolien Van der Voorden, Matteus Van Der Velden, Rita Bonomally, Valerie Varela

Plan MalawiMike Khoza, Thoko Kaitane

The Malawi PolytechnicKondwani Chidziwisano, Limbani Kalumbi, Save Kumwenda, Khumbo Kalulu

Centre for Media StudiesAnisur Rahman, Vinod Mishra, Alok

Plan TanzaniaNyanzobe Malimi, Joyce Massile,

FXBT ResearchCharles Matiko, Donata Didas

University at BuffaloMoshood (Lanre) Omotayo, Kimmy Giacalone, Jia Hua, Megan Yoerg, Ryan Muldoon, Kimberly Giacalone,Pavani K. Ram 59

THANK YOU. ANY QUESTION?

11/16/2018 60

Achieving sustainable and equitable WASH outcomes at

scale

WSSCC-3ie supported evaluations and systematic reviews: what does the

evidence say?

Radhika Menon, 3ie

UNC Water and Health conference, Chapel Hill

1 November 2018

Assessing the impact of sanitation on psycho-social stress of women in Bihar, India

• Evaluation: Before and after ethnographic study of GSF-supported programme (phased out) in Gopalganj and PaschimChamparam districts, Bihar

• -1200 women surveyed at baseline, 41 interviews at baseline and 33 at endline

Context: Government led Swachh Bharat Mission (SBM) aims to end open defecation in India

Authors: Kathleen O’ Reilly et al., 2018

Conceptual framework

Findings: Bihar mixed-method study

Study showed that when intervention was successful there was impact on sanitation coverage and use, and women’s pyscho-social stress

In line with baseline findings which showed a 48% reduction in sanitation-related psycho-social stress scores among women with access to a sanitation facility

Findings

Implementation-related findings

• Within larger context of SBM:

Differences in contractor-built vs. self-financed latrine quality

Mukhiya-contractor nexus that may help build toilets but also encourages corruption, poor service delivery, focus on selected areas

Implications for policy: going to scale- Contractor-built latrines require standards

for usability that need to be monitored and enforced

- Improved information about self-financed latrines reimbursement requirements, role of SHGs

- More consistent community-based mobilisation and demand-generation activities for different groups

•

- Programme design and M&E should incorporate elements related to women’s satisfaction with latrine design and construction, and look at sanitation-related psycho social stress

What are systematic reviews?

Systematic reviews examine all the available existing evidence on the effectiveness of a particular intervention or programme.

WSSCC-3ie supported systematic reviews

Systematic review of handwashing and sanitation behavior change interventions

• Interventions to promote sanitation and handwashing behavior-change include community-based approaches, social marketing, messaging and theory-based approaches – De Buck et al. 2017

• Review synthesises evidence from 42 quantitative studies on effectiveness of behavior change approaches and 28 qualitative studies on implementation, focused on progs in 24 L&MICs

Systematic review findings

• Community-based approaches were effective in reducing open defecation and improving latrine use.

• Social marketing approaches in combined handwashing and sanitation programmes may improve latrine use but effect on handwashing unclear

• Implementation factors: facilitator, attitude of implementer, trust and cooperation

Systematic review findings• Sanitation and hygiene messaging may

improve handwashing in the short term, but no impact on open defecation behaviour or safe faeces disposal.

Further research is needed on theory-based approaches using elements of psychosocial theory.

Using interpersonal communication is effective in certain circumstances

Systematic review of the life-cycle approach in WASH

• Examines evidence from WASH policies, programmes and projects in 11 countries in South Asia and Sub-Saharan Africa during the MDGs – Annamalai et al. 2017

• The authors synthesised evidence from 59 policy documents and 131 programme and project documents

• Review uses a life-cycle lens to examine WASH portfolio, in comparison to geographic and socio-economic targettingstrategies

Life cycle approach to sanitation

Systematic review findings

• Shift towards using the life-cycle approach occurred during the MDG period in Africa and Asia

• Women were the focus of a relatively large number of policies, followed by children

However, life cycle segmentation is not always translated from policies to programmes and projects

Life cycle systematic review: findings• Robustness index higher for

geographic and socio-economic than life-cycle approach

• Life-cycle benefits were included more often in policies related to sanitation and hygiene than for water, despite impact on women

Programme and projects funded by multi-lateral agencies aided adoption of life-cycle segments during implementation, community participation also played an enabling role

What does the evidence say?

More WASH programmes and evaluations that draw on behavioural science

Evidence suggests we need a multi-pronged intervention strategy to improve outcomes

More evaluations looking at long-term use

More implementation research on programmes at scale

Systematic steps needed to incorporate life-cycle and gendered principles in targetting approach, programmedesign

achieving sustainable and equitable odf at scale - …...2018/11/04 · achieving sustainable and...

Documents