the days ahead monday-wednesday –training workshop on how to measure the actual reduction in hiv...
TRANSCRIPT
The days ahead• Monday-Wednesday
– Training workshop on how to measure the actual reduction in HIV incidence that is caused by implementation of MC programs
• Thursday & Friday– Share experiences about the problems countries are encountering in
initial scale-up of MC services – and the solutions they have found. Clarify what the unanswered questions are about how best to implement MC.
• Saturday– Explore how measurement of the impact on incidence could also
answer unresolved questions about how to best implement MC (e.g. optimal task sharing…)
• Monday-Wednesday next week– Third meeting on the use of the DMPPT tool to estimate the cost and
forecast the possible impact of MC programs to guide initial program design (and help decision-makers decide whether they want to implement MC)
How can we estimate effectiveness?
• E.g., What is the effect of:Male circumcision on HIV incidence
• In other words:“How much does an increase in male circumcision cause HIV incidence to decrease?”
How can we estimate effectiveness?
• If we implement MC in a community and incidence of HIV does not change does that mean that MC is not effective at reducing incidence of HIV?
• If we implement MC and incidence goes up, does that mean that MC increases HIV transmission?
• If we implement MC and incidence falls, do we know that the MC caused the fall?
• We are not asking whether incidence changes at the same time that MC is implemented – we are asking whether incidence changes because MC is implemented.
• Thus, we need to know how much would incidence have changed if MC was not implemented – and how different is that from how much it changes with MC
How can we estimate effectiveness?
• Unfortunately, we can’t rewind the clock and observe the same community over the same period of time with and without MC
• We need an estimate what would have happened to HIV incidence in the absence of an MC program
• We call this the counterfactual– i.e. the “factual” is what really happens where MC
is implemented and the “counterfactual” is what would have happened if MC had not been implemented
How can we estimate effectiveness?
Creating a Good Counterfactual
• Sometimes it is very simple:– If I dye my hair tonight and come in tomorrow
with black hair you will easily believe that the change in hair color was caused by the hair dye
– In other words, if we strongly believe that something is not changing on its own, then we can assume that the counterfactual = baseline
– Good example of that for HIV is treatment
Creating a Good Counterfactual
• With HIV prevention it is never simple
Creating a Good Counterfactual
Year 1 Year 2 Year 3 Year 40
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Baseline
MC Implemented
Creating a Good Counterfactual
Year 1 Year 2 Year 3 Year 40
0.5
1
1.5
2
2.5
3
3.5
4
4.5
No MC
MC
Baseline
MC Implemented
IMPACT
Creating a Good Counterfactual
Year 1 Year 2 Year 3 Year 40
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Baseline
MC Implemented
IMPACT ?MC doubles HIV incidence?
Creating a Good Counterfactual
Year 1 Year 2 Year 3 Year 40
0.5
1
1.5
2
2.5
3
3.5
4
4.5
No MC
MC
Baseline
MC Implemented
Real IMPACTMC Halves HIV incidence
Creating a Good Counterfactual
Year 1 Year 2 Year 3 Year 40
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Baseline
MC Implemented
IMPACT?MC reducesHIV incidence by 1/2
Creating a Good Counterfactual
Year 1 Year 2 Year 3 Year 40
0.5
1
1.5
2
2.5
3
3.5
4
4.5
No MC
MC
Baseline
MC Implemented
Real IMPACTMC reducesHIV incidenceby 1/3
Creating a Good Counterfactual
• Where in the World Are We?
Creating a Good Counterfactual
• Where in the World Are We?
• If the purpose of the experiment was to test the effectiveness of the hints, should we have given the hints to the “experts” or to the rest of us?
Creating a Good Counterfactual
Low INCIDENCE High
EFFECT
Ideally you want the communities that receive MC to be indistinguishable fromthose that don’t
Creating a Good CounterfactualNEGATIVE EFFECT?
If they are different, it can bias the estimation of the effect
Low INCIDENCE High
Creating a Good Counterfactual
If they are different, it can bias the estimation of the effect
HUGE EFFECT?
Low INCIDENCE High
Creating a Good Counterfactual
• The beneficiaries of the intervention and the counterfactual or control groups:– have identical characteristics, except for
benefiting from the intervention • With a good counterfactual, the only reason
for different outcomes between treatments and controls is the intervention (I)
Creating a Good Counterfactual
• The beneficiaries of the intervention and the counterfactual or control groups:– have identical characteristics, except for
benefiting from the intervention • With a good counterfactual, the only reason
for different outcomes between treatments and controls is the intervention (I)
Creating a Good Counterfactual for MC
• What if we compared the incidence among the men in a community who decide to get circumcised with the incidence among the men who decide not to get circumcised?
• What if we compared the incidence in communities who decide to initiate circumcision services in their health center with the incidence in the ones that don’t?
• What if we compare the incidence in towns without a clinic with the incidence in towns with a circumcision clinic?
Creating a Good Counterfactual for MC
• What if we opened up a circumcision clinic in Soweto and another in Khayelitsha and another in KwaMashu – but they could only serve a small number of the men who want to be circumcised. If we give lottery tickets to see who can go in what month, could we use the ones who will wait a year as a comparison (counterfactual) group?
• What if we focused circumcision services on the 15-24 year olds. Could we use the 25-30 year olds as a comparison group?