Evaluating HIV Prevention and Treatment Programs

Damien de WalqueDamien de Walque

Markus GoldsteinMarkus Goldstein

2

(Impact) Evaluating Health Programs is different

• Norm in medicine to use randomized control trials to figure out what works

• What we know less about (for example):

– Socioeconomic effects of health interventions

– How to get people to utilize treatment (e.g. vaccines)

– What is the most cost effective mode of treatment

3

This is an issue for HIV/AIDS

• Examples:

– ART works biologically, but what impact does it have on patients socio-economic status?

– Circumcision is effective in reducing HIV risk, how do we get people to volunteer?

– We know different prevention interventions have some effect – which is the most cost effective?

4

This presentation

• We will not focus on the medical findings

• We know something about non-medical outcomes through evaluation, but actually not that much rigorous (in a comparison group sense) evidence

• But, a lot in progress, and we will talk about some of that

5

Example 1• Duflo, Dupas, Kremer & Sinei: Education and HIV

AIDS Prevention

• The interventions:1. training teachers in the Kenyan government’s

HIV/AIDS-education curriculum2. organized debate and essay contest on the role of

condoms in protecting teens against HIV/AIDS3. reduced cost of education through the supply of school

uniforms4. information campaign for Kenyan teenagers to spread

the awareness of high HIV prevalence among adult men (uses earlier evaluation work)

6

Evaluation design– Schools randomly assigned to treatment groups

and control– Baseline and endline data collection to use a

difference in difference approach– Measured the effect of the program on:

• Teacher coverage of HIV/AIDS• Student HIV knowledge• Student attitudes• Self reported behavior• Child bearing (girls) and drop out rates

7

Main findings

– Teacher training led to no significant reduction in teen pregnancy, but increased likelihood that pregnancies occur within marriage

– Debate over condoms led to (some) increased report of condom use

– Reductions in the cost of schooling led to reduction in drop out rates and reduction in teen pregnancies

8

Prevention of HIV/AIDS

$575

$300

$91

$0

$100

$200

$300

$400

$500

$600

$700

Co

st

per

pre

gn

an

cy a

vert

ed

Teacher training Uniforms Info. Campaigns

Source: Duflo, Dupas, Kremer, and Sinei (2006)

9

Example 2• Thirumurthy, Graff Zivin, and Goldstein: The Economics of

AIDS Treatment: Labor Supply in Western Kenya

• The intervention

– Provide ART to patients, rule is for a CD4<200, but initial rationing

– Treatment for opportunistic infections

– Nutrition supplementation, but not much until later

• Data – two rounds of hh survey – random sample & patients

10

Evaluation DesignWe know what happens to counterfactual

groupMedical evidence: continued decline in health

and death

Allows estimation of upper bound of treatment impact

Zero labor supply in round 2

010

2030

Mea

n ho

urs

wor

ked

in p

ast w

eek

-50 0 50 100 150 200 250 300 350 400Days on ARVs

treated counterfactual

11

BMI Before and After ARV Therapy

Source: AMPATH Medical Records System – data as of March 2005.

.6.7

.8.9

Fra

ctio

n pa

rtic

ipat

ing

in la

bor

forc

e

-8 0 8 16 24 32 40 48 56Weeks on ARVs

LFP before and after ARV Therapy

Source (right): Household survey data.

19

20

21

22

Me

dia

n B

MI

-40 -30 -20 -10 0 10 20 30 40 50 60 70 80 90Weeks Before/After ARV Initiation

12

Main Findings

• Large and rapid labor supply response in patients

• Spillover benefits to other members of the household – young boys & women reduce their labor supply

• Patient earnings (relative to zero counterfactual) are close to the cost of treatment

Some work in progress

14

Question 1: Socio-economic impact of reducing premature adult mortality: the case of

ART

• In addition to labor supply:

• Schooling and welfare of children

• How household coping mechanisms change with ART

• Effects of ART on farming productivity

• Using evaluation results for macro models of the economic effects of AIDS & treatment

15

Question 2: Possible effects of ART on HIV transmission and prevention

Direction of effect

Beneficial(Slow transmission)

Adverse(Speed transmission)

Type of effect

Biological

Reduce infectiousness Select for resistance.

Longer duration of infectivity

Behavioral

Encourage prevention,especially testing

Off-setting behavior, “disinhibition”

16

Other questions (more on the supply- facility side)

• 3) Adherence to treatment

• 4) How to avoid the development and spread of resistance?

• 5) How are ART beneficiaries identified? How to encourage timely uptake?

• 6) How to assure the quality of HIV/AIDS service delivery?

• 7) How to encourage capacity building to reinforce the sustainability of ART delivery?

17

Methodology and data collection (Longitudinally)

• Biomedical follow-up including data on treatment regimen and treatment success (CD4 counts)

• Household surveys (HIV patients and general population) including health, schooling, labor force.

• 7 countries: Burkina Faso, Ghana, Kenya, India, Mozambique, Rwanda and, South Africa

• Surveys ongoing and scheduled. Data and preliminary analysis will be available by the end of 2008.

18

DeterminantsPatient

and ProviderBehavior

OutcomesImpact on

the entire country

Treatment Outcome, Resistance

Development

Socio-Economic benefits

for households

Prevention

PatientAdherence Information

socio-economicvariables

Community variables

Stigmatization

Training

Selection / Recruiting

Impact On health systemQuality of

service deliveryEquipment

Staffing and Incentives

Associations

EmployersSocio-

Economic benefits for firms

Framework for Learning Agenda

19

Methodological challenges: endogeneity

• Given issues with randomization, in some countries, we will evaluate some experiments on the conditions of ARV delivery.

• Rwanda: performance-based contracting for HIV/AIDS services in health facilities

• South Africa: food and counseling intervention as adherence support.

• Kenya: text messaging intervention as reminders for adherence

20

Some other common issues for HIV evaluations

• Measurement. Dupas & co. raise the issue of what should be the impact variable, particularly in prevention

– Issue of self reported behavior (Gersovitz)

– Think about using biomarkers – HIV or STI?

21

Comdom use at the last intercourse with the spouse: discordant reports

Source: DHS 2003 and 2004

N =

Burkina Faso

1630

Cameroon

1764

Ghana

1830

Kenya

1361

Tanzania

2497

Man no,

Woman no

88.9% 90.8% 91.9% 97.0% 90.9%

Man yes,

Woman yes

2.0% 2.2% 1.9% 1.0% 1.6%

Man yes, woman no

6.9% 4.2% 5.0% 1.8% 4.3%

Man no,

Woman yes

2.2% 2.8% 1.2% 0.3% 3.2%

22

Issues in HIV evaluation, cont.• Control Group: Thirumurthy et. al. point to the problem of a

medical intervention known to be effective and the problem of generating a counterfactual

– Sequel paper looks at children, here the authors use a range of control group techniques

• The shape & determinants of the epidemic vary across time and across countries, so results in one country may not apply to another…need to do multiple evaluations of the same intervention