Evaluating Structural
Interventions
Kelly Hallman
“Pushing Forward with Structural Interventions for HIV Prevention – Where Are We Now
and How Do We Move Forward”
26 July 2012, IAS Conference
The reality
We are
hardwiredforthis
..and WHY do you want my sex?
• Intimacy, pleasure• Evolution, childbearing
• Challenge of HIV and STIs
• How do we tackle?• How to measure intervention impact?
Levels of “rigor”
Common HIV prevention outcomes
• HIV
• HSV
• Condom use
• Sexual partnering
• Abstinence, frequency
What’s common to these outcomes?
• Changes at individual-level
…and the “enabling” environment?
“Poverty isn’t created by poor people.
It’s created by wrong systems.
And we don’t notice it.”
- M. Yunus
Structural change
• Transformation that alters flows in• authority• rights• resources • opportunities• information• responsibilities
• Long-term process
Are these amenable to
study using an RCT?
R C T• Randomized• Denial of benefits a concern• Phased-in approach
• Control• All else held constant
• Learning, mid-course correction?
• Narrow intervention• or single bundled package • or expensive multi-arm
• Trial• Limited time to measure change
Perhaps a middle ground
Structural factors likely to impact HIV, health, well-being
• What is the theory of change?
• Clear conceptual framework
• Causal impact hypothesis
• Identify “proximate determinants”
• Outcomes along path to desired change
Theory Study outcomes, Study design
• Did it work?1. Structural level • Organizational theory• Systems analysis• Timeline
2. Individual level• Outcomes: Factors along pathway to
desired change• Design: e.g., case-control with careful
matching (PSM)• Timeline
Methods
• Why did it work (or not)?
• Monitoring
• Operations research
• Institutional change theory
• Qualitative approaches• Seasonality analysis• Time use patterns• etc.
Let us move creatively beyond….
Your study was not an
RCT!
Be gone with you!
“The first rule of social change:
Maximize your impact without being a jerk.
If you maximize impact while being a jerk, it is not worth it.”
- Jacob Harold, Hewlett
Foundation
The Population Council conducts biomedical, social science, and public health research. We deliver solutions that lead to more effective policies, programs, and technologies that improve lives around the world.
Few girls under age 15 in Burkina Faso reached by peer educators
7
3033
30
0
10
20
30
40
50
60
70
80
90
100
10-14 15-19 20-24 25+
Age of girls reached by peer educators
Yet younger girls much lower levels of HIV and SRH knowledge than older girls
15
40
2231
4759
0102030405060708090
100
Knows of STIsapart from HIV
Knows a sourcefor contraceptives
Knows ABC(abstinence, befaithful, use a
condom)
12-14 yrs
15-19 yrs
Few girls with little schooling reached by a peer educators
27
54
19
0102030405060708090
100
None Primary Secondary orhigher
Level of schooling of girls reached by peer educators
And …most girls in Burkina Faso have never been to school
63
11
26
0102030405060708090
100
None Primary Secondary orhigher
Level of schooling of 12-19-year-old girls
And their levels of preventive knowledge are much lower than those who ever attended school
16
46
2941
46
63
0102030405060708090
100
Knows of STIsapart from HIV
Knows a sourcefor contraceptives
Knows ABC(abstinence, befaithful, use a
condom)
Noschooling
Everattended
Siyakha Nentsha impact on self-reported sexual behaviors
Boys Girls
+ p<0.10; * p<0.05; ** p<0.01
Had sex past year
# of sexual partners in past
year
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan28a566028
SN combined (v. control)
(Yt2
- Y
t1)
- (Y
c2 -
Yc1
)
Had sex past year # of sexual partners in past
year
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan27a566027
0tan28a566028
SN combined (v. control)
*
+
Siyakha Nentsha impact on social and economic
Boys Girls
+ p<0.10; * p<0.05; ** p<0.01
# Social grants
knowledge
Attempted to open bank ac-count in
past year
Is saving
0tan27a566027
0tan28a566028
SN combined (v. control)
# Social grants
knowledge
Attempted to open bank ac-count in
past year
Is saving
0tan27a566027
0tan28a566028
SN combined (v. control)
(Yt2
- Y
t1)
- (Y
c2 -
Yc1
)
*
*
*