Propensity Scores

October 2012

Alexander M. Walker MD, DrPHExtensive parts of this presentation incorporate the work of John D. Seeger, PharmD, DrPH

“In mathematics, you don't understand things. You just get used to them.”

- John von Neumann1903-1957

Research Goal Compare two treatments with respect to a health or

economic outcome “Counterfactual” ideal

If the same people had received B instead of A, how would their outcomes have differed?

What is achievable: “similar” not “same” Comparable treatment groups … insofar as you can tell!

4

Pictures for Confounding

Comparison of Heterogeneous Groups

5

E1 E2

6

E1 E2

Internal Composition May Differ

7

Affectedindividuals

50%

15%

50%

15%

E1 E2

Risks that Depend on Subgroup Status

8

These differences in risk are due to the covariate

structure of compared

populations, not to the differential effects of

E1 and E2

E1 E2

Internal Risk Factor Heterogeneity Creates an Differences in Group Risk

9

Propensity Scores to Create Populations with Similar Covariate Structure

10

E1 E2Covariate Heterogeneity

E1 has more Yellow

E2 has more Gray

11

E2

E1E1

E2Gray predicts

E2Yellow predicts

E1

Covariate Status as a Predictor of Treatment

12

Propensity Scores PS is the predicted probability of treatment, given

all the covariates Matching on the PS creates study populations that

have balance on the covariates Perfect for a single, dichotomous covariate Not perfect, but very good for multiple covariates

13

E1 E2

Propensity for Covariate PatternsThink of orange and green as two distinct covariate patterns that have the same predicted Pr(E1).

Pr(E1)=x

Pr(E1)=x

14

E1 E2

Pr(E1)=x

Pr(E1)=x

Gathering subjects with identical propensity puts all individuals with covariate patterns orange and green into the same stratum.

Conditioning on Propensity Permits Unconfounded Comparisons

At a given propensity level, there is no association between treatment and covariate patterns.

Formal Expression

Propensity(x) P(T=1|x) = E(T|x)The propensity associated with level x of the covariate X is the probability that treatment is present (equivalently, is “B” as opposed to “A”), given level x, and this is in turn equal to the expected value of treatment, given x.Note that the definition does not specify the parametric form of the Propensity(x) . The examples in this talk use a logistic function; others -- including nonparametric functions -- are also used.Notation. A single capital letter denotes a variable, a single lower case letter denotes a particular value for that variable.

Probability Calculus

Under propensity matching, how do x (covariate status) and t (treatment status) relate to one another?

1.Pr( x, t | p ) = Pr( x | p ) Pr( t | x, p ) Probability Theory

2.Pr( t | x, p ) = Pr( t | p ) p incorporates all information about t

that is in x Pr( x, t | p ) = Pr( x | p ) Pr( t | p )

Pr( x, t | p ) = Pr( x | p ) Pr( t | p )

Given a particular value of the propensity score variable, that is at P=p, the covariates X and T are uncorrelated.

At particular levels of P individually and therefore collectively as well (“conditionally on P), X cannot confound the association between T and any outcome.

18

Matching on Propensity Scores

Propensity Matching: Method

Identify candidate predictors of treatment B v A Perform a logistic regression of B v A Obtain from the regression a “predicted” probability of B v

A Sort all members of A and B according to this propensity Match A patients to B patients on the propensity

Duragesic and Long-Acting Opioids

Duragesic LA OpioidsN 504 2,20165+ years 29% 10%Male 35% 49%Periph Vasc Disease 4% 1%Sx of Abd or Pelvis 18% 10%> 2 hospitalztns 6 mo 9% 3%30 days NonRx Costs $1,136 $746

Straightforward Regressionproc logistic data = mother.propensity2 descending; model DuragesicUser = DischCostIndex EncCostIndex RxCostIndex OtherCostIndex RxCostPrior1 OtherCostPrior1 AnyRx OneDisch TwoDisch ThreePlusDisch AnyICD443 AnyICD719 AnyICD724 AnyICD787 AnyICD789 q3_95_new q4_95 q1_96 q2_96 q3_96 q4_96 q1_97 q2_97 q3_97 q4_97 q1_98 q2_98 q3_98 q4_98 hmo men young old /rl; where enrbaseflag = 1 and validindex = 1 and sameday = 0 and medicare = 0 and malignant = 0; output out = mother.propensity3 p = score ; run;

Propensity Output

Obs PATIENT score  1 2909 0.57475 2 3438 0.328993 3841 0.013244 5674 0.484115 5734 0.068926 8573 0.117807 10210 0.346928 13056 0.097379 13376 0.3735010 16026 0.1563511 16865 0.0610612 16949 0.10568

23

E1

Pr(E1)=x

Pr(E1)=x

E2(sample) E2 (residual)

Choose from E2 a sample that matches E1 in size.

Matching on Propensity

24

E1

Pr(E1) = 0.5

Pr(E1) = 0.5

E2At every level of propensity in the constructed cohorts, Pr(E1) = 0.5. Therefore, treatment is uncorrelated with propensity, and you can collapse all the propensity-matched groups together to form a cohort in which all covariate patterns are uncorrelated with treatment, and there will be no confounding bias.

Matching on Propensity

Stratum I II III IV V

Stratum I II III IV V

Duragesic and Long-Acting Opioids

Duragesic LA OpioidsN 504 2,20165+ years 29% 10%Male 35% 49%Periph Vasc Disease 4% 1%Sx of Abd or Pelvis 18% 10%> 2 hospitalztns 6 mo 9% 3%30 days NonRx Costs $1,136 $746

Propensity-Matched Cohorts

Duragesic LA OpioidsN 478 47865+ years 26% 25%Male 36% 33%Periph Vasc Disease 4% 3%Sx of Abd or Pelvis 17% 18%> 2 hospitalztns 6 mo 8% 8%30 days NonRx Costs $1,084 $1,043

Pharmacoepidemiol Drug Saf. 2005 Jul;14(7):465-76.

Do Statins Affect Risk of AMI? The purpose of the study was to assess whether

Statins affect the risk of risk of acute myocardial infarction (AMI)

Strong predictors for statin use that affect risk of AMI

How to design an observational study? Note: we would not ordinarily use observational

data for efficacy questions, but this serves as a suitable test case because there is a known gold standard

+Risk Factors: age (45M, 55F), diabetes, smoking, HTN, low HDL,family history of premature CHD-Risk Factor: high HDL

Risk Category LDL to initiate drug Tx

LDL Goal of drug Tx

No CHD and <2 Risk Factors

³190 <160

No CHD and ³2 Risk Factors

³ 160 <130

With CHD >130 £100

NCEP ATP II guidelines (1993)

Good Clinical Practice Creates Confounding

Gold Standard for the Effect of StatinsCA

RE Tr

ial R

esul

tsSa

cks F

M, e

t al

N E

ngl J

Med

. 199

6;33

5:10

01-9

Data Source

• Fallon Community Health Plan• Central Massachusetts HMO• ~200,000 members• Claims Data available on:

– Enrollment (age, sex, date)– Ambulatory care visits– Hospitalization– Pharmacy dispensings (drug & quantity)– Laboratory tests (tests & results)

Patient Entry, Analytic Sequence

1993 1994 1995 1996 1997 1998 1999

1 of 9 Blocks

1) Apply eligibility criteria• FCHP member for at least 1 year• At least one physician visit in last year• LDL, HDL, TG levels in last 6 months• At least one physician visit in cohort accrual block• No PAD diagnosis before index date• Not current statin user

2) Estimate propensity score (statin initiation)3) Match statin initiators with non-initiators4) Repeat for all blocks of time5) Follow matched groups for diagnosis of MI

2nd/94

~35,000Members

All Fallon members with any LDL > 130 mg/dl

Require 1 yearEnrollment

Current Statin Users (1501)

Statin Initiators, Eligible (77)

Statin Initiators, Not Eligible (34)

Non Statin Users, Not Eligible (24,799)

Non Statin Users, Eligible (9,639)

Month of1/1/94

PropensityScoreMatching

Total subjects in cohort (36,050)

“Typical” Statin Initiator and Non-Initiator Variable Init iators Non- Initiators P-value Lipid-related labs 26.04 13.58 <0.0001 Different Prescription Drugs 5.02 2.88 <0.0001 LDL level (mg/dl) 180.25 155.08 <0.0001 Triglyceride level (mg/dl) 202.66 166.91 <0.0001 Cardiovascular-related Prescription Drugs 0.59 0.26 <0.0001 Cardiovascular-related vis its 1.11 0.27 <0.0001 Age (years) 62.04 58.02 <0.0001 Physician Visits 7.69 6.31 <0.0001 Ischemic Heart Disease 20.27% 5.57% <0.0001 HDL level (mg/dl) 43.29 46.60 <0.0001 Cardiovascular-related diagnoses 0.28 0.12 <0.0001 Cardiovascular-related hospitalizations 0.56 0.13 <0.0001 MI 11.58% 2.92% <0.0001 Angina 11.92% 3.14% <0.0001 Unstable Angina 10.59% 2.22% <0.0001 Smoking 25.80% 18.10% <0.0001 Hypertension 19.96% 12.96% <0.0001 Labs 10.48 10.74 0.0074 Hospitalizations 0.22 0.08 <0.0001 Male 53.14% 47.61% <0.0001

111% (46%-204%)Risk Increase

Statin Non-Initiators

Statin Initiators

Months of Follow-Up

Cum

ulat

ive

Incid

ence

MI Outcome (Unmatched)

HR=2.11 (1.46-3.04)

Calculate Propensity Score Predict Treatment

Statin Initiation vs Not In Each 6-month Period of Cohort Accrual

Using Baseline Covariates Obtain Fitted Value From Regression Fitted Value is the Propensity Score

Construct Rich Model More than 8 events

per covariate leads to unbiased estimates

Many more persons exposed to drug of interest than study outcomes

In Drug Safety studies, usually the outcome is rare

Therefore can control for more covariates when exposure is dependent variable than when outcome is

Cepeda S, et al. Am J Epidemiol 2003;158:280-287.

*build model for 9501;proc logistic descending data=new1; model statin = male smok obes age9501 ang9501 usa9501 chf9501 isch9501 ath9501 cva9501 usa9501 mi9501 olmi9501 htn9501 tia9501 afib9501 ascv9501 hth9501 ost9501 cvs9501 htdx9501 circ9501 cond9501 rvsc9501 hhd9501 dysr9501 hrt9501 ns9501

ins9501 diab9501 skca9501 depr9501 adj9501 schz9501 deb9501 rheu9501 days9501 lres9501 tres9501 hres9501 hbac9501 cvhp9501 ekg9501 cvrx9501 cvvs9501 llab9501 lab9501 cvdg9501 hosp9501 rx9501 vist9501 diag9501; output out=psmodel pred=PROPSCORE; run;

Propensity Regression Parameter Estimates

Obs ID STATIN PROPSCORE  1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...

Output File – Propensity Scores

43

Obs ID STATIN PROPSCOR  1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...

Output File – Propensity Scores

Propensity Score Distribution (After Matching)

050

100150200250300350400450

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

Propensity Score

Num

ber o

f Per

sons

Statin Initiators Non-Initiators

Balance Achieved by Matching Matched at 0.01 Propensity Score Variable Initiators Non-Initiators N=2901 N=2901 P-value Lipid-related labs 24.90 24.64 0.4987 Different Prescription Drugs 4.57 4.54 0.7639 LDL level (mg/dl) 177.84 177.58 0.7837 Triglyceride level (mg/dl) 200.34 200.50 0.9626 Cardiovascular-related Prescription Drugs 0.51 0.51 0.9367 Cardiovascular-related visits 0.74 0.83 0.1249 Age (years) 61.47 61.68 0.5030 Physician Visits 7.25 7.27 0.8732 Ischemic Heart Disease 15.13% 15.48% 0.7428 HDL level (mg/dl) 43.51 43.55 0.9079 Cardiovascular-related diagnoses 0.21 0.23 0.2145 Cardiovascular-related hospitalizations 0.40 0.39 0.7929 MI 7.89% 8.69% 0.3169 Angina 8.51% 8.72% 0.8151 Unstable Angina 7.14% 7.31% 0.8393 Smoking 23.85% 24.27% 0.7355 Hypertension 16.58% 17.99% 0.1649 Labs 10.45 10.48 0.7874 Hospitalizations 0.16 0.16 0.6915 Male 52.33% 52.09% 0.8747

Only 1 of 52 variables sig. different at P<0.05

31% (7%-48%)Risk Reduction

Statin Non-Initiators

Statin Initiators

Months of Follow-Up

Cum

ulat

ive

Incid

ence

MI Outcome (After Matching)

HR=0.69 (0.52-0.93)

Interpreting Propensity Coefficients

49

Odds Ratio for Non-Statin Lipid Lowering Therapy as a Predictor of Statin Initiation

00.20.40.60.8

11.21.41.6

9402 9501 9502 9601 9602 9701 9702 9801 9802

Matching Block (Year and Half)

Odd

s R

atio

LDL as a Predictor of Statin Initiation

0.00

2.00

4.00

6.00

8.00

10.00

12.00

14.00

190+175-190160-175145-160130-145<130

LDL Category

Odd

s R

atio

for I

nitia

tion

Age as a Predictor of Statin Initiation

0.00

2.00

4.00

6.00

8.00

10.00

12.00

75+65-7555-6545-5534-45<35

Age Category

Odd

s R

atio

for I

nitia

tion

When Is the Model Sufficient?

53

Early Matching Results

New Variables Suggested post hoc for the Propensity Score

Cardiac Disease Cardiovascular

Diagnoses Hospitalizations Outpatient visits Medications

EKGs Number of labs Number of lipid labs

Other Causes of “Medicalization” Schizophrenia Adjustment Disorder Depression Non-Skin CA Skin CA Debility Rheumatic Disease

Imbalance on Non-Included Variables

NIVs are Predictors of Statin Initiation

New Ranking of Predictors# Variable c-stat OR P-value

Lipid lab tests 0.82 1.14 <0.011 Rxs 0.72 1.20 <0.012 LDL 0.71 1.02 <0.013 Triglycerides 0.61 1.00 <0.01

CVRxs 0.60 1.70 <0.01CVVisits 0.59 1.25 <0.01

4 Age 0.58 1.02 <0.015 Visits 0.57 1.04 <0.016 Ischemic Heart Ds. 0.57 4.20 <0.017 HDL 0.57 0.98 <0.01

CVDiagnoses 0.56 1.94 <0.01CVHospitalizations 0.55 1.23 <0.01

8 MI 0.54 4.63 <0.019 Angina 0.54 3.88 <0.0110 Unstable Angina 0.54 5.13 <0.0111 Smoking 0.54 1.55 <0.0112 Hypertension 0.54 1.77 <0.01

Balance on New Variables Original Matching Revised Matching Variable Statin Init.

(N=3579) Non-Init. (N=3579)

P-value Statin Init. (N=2901)

Non-Init. (N=2901)

P-value

Lipid lab tests 26.3 16.4 <0.01 24.9 24.6 0.50 CVRxs 0.51 0.53 0.09 0.51 0.51 0.94 CVVisits 0.96 0.54 <0.01 0.74 0.83 0.12 CVDiagnoses 0.25 0.17 <0.01 0.21 0.23 0.21 CVHospitalizations 0.46 0.31 <0.01 0.40 0.39 0.79 Lab tests 10.3 11.3 <0.01 10.5 10.5 0.79 EKGs 0.48 0.48 0.77 0.48 0.51 0.35 Schizophrenia 2.2% 3.6% 0.03 2.1% 2.2% 0.99 Depression 2.4% 2.9% 0.14 1.9% 2.7% 0.05 Non-skin cancer 5.3% 5.7% 0.47 5.2% 5.5% 0.60 Adjustment dis. 2.7% 3.6% 0.03 2.8% 3.1% 0.59 Skin cancer 2.5% 2.7% 0.71 2.2% 2.4% 0.73 Debility 1.7% 1.5% 0.78 1.6% 1.3% 0.44 Rheumatic dis. 1.3% 1.8% 0.07 1.2% 1.7% 0.19

Thank You!