nicholas jewell medicres world congress 2014

Statistical Methods for Observational Drug

Studies

Nicholas P. Jewell Departments of Statistics &

School of Public Health (Biostatistics) University of California, Berkeley

October 17, 2014

What is the Primary Scientific Question?

•  What is the one “number” you want to know (what is the measure of effect of interest)? •  What will be the “newspaper headline”?

•  What is the parameter of interest?

•  What would be a meaningful effect? (range of parameter values of interest)

© Nicholas P. Jewell, 2014

Types of Studies •  Administrative databases (FDA Adverse Event Reporting Ststem (FAERS), medical insurance/claims databases, registeries

•  Often no denominators, thereby compromising incidence estimates •  Exposures only have proxies available •  Temporality/causation

•  Observational studies •  Confounding/causation •  Selection bias •  Information bias (misclassification, detection bias etc)

•  Randomized Clinical Trials •  Meta-Analyses

•  apples & oranges


Observational Data Collection--Sampling

•  What is the target population? •  What is the Study Population?

•  How will individuals be sampled (case-control, cohort, longitudinal)?

•  How is exposure assigned to individuals?

•  What is now my parameter of interest? 4


Can I Draw a Prototype DAG to address Causation and

Confounding? •  Direct acyclic graphs (DAG) to related

variables, including exogeneous and selection variables if necessary

•  Is the parameter of interest identifiable from the design and under what assumptions?


5

Pre-specified Analysis Plan

•  Multiple Outcomes (Which one is primary)?

•  Confounders? •  Subgroups of Interest? •  Effect Modification of Interest? •  Mediation?


6

7

Confounding Variables

•  Classic Conditions for confounding –  C must cause D –  C must cause E

C

E D?


8

Directed Acyclic Graphs

•  A node A can be a collider on a specific pathway if the path entering and leaving A both have arrows pointing into A. A path is blocked if it contains a collider.

–  D is a collider on the pathway C-D-A-F-B; this path is blocked

BF

D

C

A


9

Using Causal Graphs to Detect Confounding

•  Delete all arrows from E that point to any other node

•  Is there now any unblocked backdoor

pathway from E to D? – Yes—confounding exists – No—no confounding


10

Using Causal Graphs to Detect Confounding

F C

DE

F C

DE

F C

DE

F C

DE© Nicholas P. Jewell, 2014

11

Vaccination & Autism Example

Medical Care Access

Vaccination Autism

Family History

SES


12

Checking for Residual Confounding

•  After stratification on one or more factors, has confounding been removed? – Cannot simply remove stratification factors

and relevant arrows and check residual DAG – Have to worry about colliders


13

Controlling for Colliders •  Stratification on a collider can induce an association

that did not exist previously

Rain

Sprinkler Wet Pavement

Diet sugar (B)

Fluoridation (A) Tooth Decay (D)


14

Checking for Residual Confounding

•  Delete all arrows from E that point to any other node

•  Add in new undirected edges for any pair of nodes that have a common descendant in the set of stratification factors S

•  Is there still any unblocked backdoor path from E to D that doesn’t pass through S ? If so there is still residual confounding, not accounted for by S .


15

Vaccination & Autism Example

Medical Care Access

Vaccination Autism

Family History

SES


16

Vaccination & Autism Example: Stratification on Medical Care Access

Vaccination Autism

Family History

SES

Still confounding: need to stratify additionally on SES or Family History, or both


17

Caution: Stratification Can Introduce Confounding!

C

E D

F

No Confounding Stratification on C introduces confounding!


Return to Randomized Studies

•  What is the Right Parameter to Estimate and How do We Interpret It?

•  Adjustment for Baseline Factors (Pre-randomization)?

•  Adjustment for Post-Randomization Factors?

18


What is the Right Parameter?

•  Two randomized treatment groups, continuous response

•  Interested in the difference in means

•  Use group sample averages and take differences

•  No model assumption underlying estimation/inference (we don’t need to assume Normal distributions)

•  Parameter being estimated (difference in group means) has a causal effect (both marginally and for specific subjects (i.e. conditionally))

19



•  Two randomized treatment groups, time to event data (usually right censored)

•  Interested in the difference in survival experience (hazards?)

•  Use Kaplan-Meier estimates and log rank test?

•  Summarize with estimated hazard ratio based on PH assumption?

–  Suggested by CONSORT guidelines and COCHRANE handbook

•  Parameter being estimated (difference in group means) has no causal interpretation if PH model is wrong (which it always is)

•  Parameter being estimated also depends on the censoring distribution!

20


Proportional Hazards

21

(courtesy of Hajime Uno) © Nicholas P. Jewell, 2014

Non-Proportional Hazards

22

(courtesy of Hajime Uno) © Nicholas P. Jewell, 2014

Alternatives with Survival Data

•  Ratio or difference in survival functions at time t •  Ratio or difference in survival percentiles •  Restricted (up to time t) mean of survival time

–  Difference in integrals of survival curves up to time t –  (Weighted) difference in Survival Curves –  Pre-specify t?

•  Differences in cumulative hazard –  Aalen’s additive hazard model

–  test statistic: time-weighted estimates of integrated hazard differences

23

h(t, x,�(t)) = �0(t) + �1(t)x

Z t

0�1(u)du


Non-PH Can Make a Difference! Ascot Clinical Trial (Statins vs. Placebo)

24



25



26


It Makes a Difference!

The assumption of statin benefit to women within ASCOT is based on a lack of heterogeneity test within a model that is incorrect. The results become clear and transparent when an alternative (less restrictive model is used)

27



•  Two randomized treatment groups, binary outcomes

•  Interested in the difference in “success” proportions

•  Use logistic regression to estimate treatment effect adjusting for important confounders/baseline predictors

•  Parameter being estimated (relative difference in group odds of success) in subgroups has a conditional interpretation but not as a marginal assessment of the treatment effect (this is typically smaller)

28


Should I adjust for baseline predictors in randomized studies

•  Two randomized treatment groups, binary outcomes

•  The answer depends on which parameter you are interested in: marginal or conditional

–  Marginally: can gain precision by adjsutment

–  Conditionally: always lose precision by adjustment (Robinson & Jewell) with simple logistic regression (is there a better estimator?)

–  Always can gain efficiency when testing the null hypothesis

•  It is important to construct estimates that leverage baseline predictors that do not depend on the model specification being right

29


Adjustment for Post-Randomization Factors

•  Beware! You are conditioning or selecting on a factor that is itself not randomized.

•  MIRA trial

•  Guarantee or Immortal Time Bias

–  Nobel prize winners have longer lives than the rest of us (as do Oscar winners etc).

•  Only considering those with complete data (eg ignoring early drop outs)

30


Mediation: The MIRA Trial •  Gates Foundation study to determine the

effectiveness of a latex diaphragm in the reduction of heterosexual acquisition of HIV among women

•  Two arm, randomized, controlled trial •  Primary intervention: diaphragm and gel provision

to diaphragm arm (nothing to control arm). •  Secondary Intervention: Intensive condom provision

and counseling given to both arms, plus treatment of STIs

•  Trial is not blinded •  5000 women seen for 18 months in three sites in

Zimbabwe and South Africa 31


MIRA Trial: Basic Intention to Treat Results

•  Basic Intent-to-Treat Analysis: – 158 new HIV infections in Diaphragm Arm – 151 new HIV infections in Control Arm

•  ITT estimate of Relative Risk is 1.05 with a 95% CI of (0.84, 1.30)

•  End of story . . . . .?

32 © Nicholas P. Jewell, 2014

Estimating Direct Effects: MIRA Trial

HIV Infection

Treatment Group (Diaphragm use)

Condom Use

Confounders

after stratification on condom use

HIV Infection

Treatment Group (Diaphragm use)

Confounders randomization hasn’t ruled out confounding of direct effect!

Now have to adjust for confounders (but we are still ITT) 33


Results of Direct Effects Analysis •  Relative Risk of HIV infection between Diaphragm

arm and Control arm by end of Trial, with Condom Use Fixed at “Never”: 0.59 (95% CI: 0.26, 4.56)

•  Relative Risk of HIV infection between Diaphragm arm and Control arm by end of Trial, with Condom Use Fixed at “Always”: 0.96 (95% CI: 0.59, 1.45)

Conclusion: No definitive evidence from direct effects analysis that diaphragms prevent (or don’t prevent) HIV.

34


Meta-Analyses

•  Random or Fixed Effects –  heterogeneity –  Measure of association (RR or ER)

•  Zero-event trials/use of all evidence –  Drug safety studies (Vioxx, Celebrex, Avandia, etc) –  Continuity corrections –  Study duration

35


Meta-Analyses and No Event Studies

•  One study: 30 events under Tx, 3 under

placebo (5000 individuals in both arms) – Risk difference of 0.0054 with exact 95%

confidence interval of (0.0032, 0.0076), p < 0.0001 •  Now add three (short duration) studies,all with

5000 in both arms, no events in either arms in all cases, use Tian et al. method – exact 95% confidence interval of (-0.0044, 0.0001),

p = 0.46

36


Overview

•  Opportunistic Data Torturing –  Multiple comparisons –  Multiple analyses –  Subgroup analyses

•  Procustean Data Torturing –  Subgroup analyses (or lack thereof) –  Meta-analyses

•  Sensitivity Analyses/Proxies/Assumptions/Honesty •  Blinding the Statistician (who funds the statistician?) •  Use of surrogate outcomes/exposures (similar to blank or spiked

samples) for comparisons of interest (HSV in MIRA trial) •  Pre-specified Data Driven Analyses •  Dissemination and Publication

37


References •  Jewell, N. P. Statistics for Epidemiology Chapman 7 Hall, 2003. •  Umo, H., Claggett, B., Tian, L., Inoue, E., Gallo, P., Miyata, T., Schrag, D., Takeuchi, M., Uyama,

Y., Zhao, L., Skali, H., Solomon, S., Jaconus, S., Hughes, M., Packer, M. & Wei, L.J. “Moving beyond the hazard ratio in quantifying the between-group difference in survival analysis,” J. Clinical Oncology, 32, 2014, 2380-2385.

•  Hosmer, D.W. & Royston, P. “Using Aalen’s linear hazards model to investigate time-varying effects in the proportional hazards regression model,” Stata Journal, 2, 2002, 331-350.

•  Sever, P.S., Dahlöf, B., Poulter, N.R., Wedel, H., Beevers, G., Caulfield, M., Collins, R. et al. “Prevention of coronary and stroke events with atorvastatin in hypertensive patients who have average or lower-than-average cholesterol concentrations, in the Anglo-Scandinavian Cardiac Outcomes Trial—Lipid Lowering Arm (ASCOT-LLA): a multicentre randomised controlled trial,” Lancet, 361, 2003, 1149-1158.

•  Rosenblum, M., Jewell, N.P., van der Laan, M., Shiboski, S., van der Straten, A. & Padian, N., “Analysing direct effects in randomized trials with secondary interventions: an application to human immunodeficiency virus prevention trials,” J. R. Statist. Soc. A, 172, 2009, 443–465.

•  Robinson, L.D. & Jewell, N.P.” Some surprising results about covariate adjustment in logistic regression models,” International Statistical Review 59, 1991, 227-240.

•  Tian, L., Cai, T., Pfeffer, M. A., Piankov, N., Cremieux, P.-Y. & Wei, L. “Exact and efficient inference procedure for meta-analysis and its application to the analysis of independent 2× 2 tables with all available data but without artificial continuity correction,” Biostatistics, 10, 2009, 275–281.

•  Bailar, J.C. III., ”Science. Statistics, and deception," Annals of Internal Medicine, 104, 1986, 259-260.

•  Mills. J.S., “Data torturing,” New England Journal of Medicine, 329, 1993, 1196-1199.

38


nicholas jewell medicres world congress 2014

Health & Medicine