David Sackett Symposium: A Substitute for Randomized Trials

Donald A. Redelmeier University of Toronto

Sunnybrook Hospital

General Internal Medicine

Institute for Clinical Evaluative Sciences

Conflicts of Interest Disclosure

•  Practicing Medicine in Ontario •  Canada Research Chairs Program •  Canadian Institutes of Health Research •  PSI Foundation of Ontario •  National Institutes of Health Research •  Heart and Stroke Foundation of Ontario •  Redelmeier’s own Ego Bias

Potential Conflicts of Interest

Background: Joyful Strengths • proof of causality • fabulous agricultural heritage • durable immortality • impressive quality control • simplistic writing • easily published • fabulous marketing • not often refuted • opportunities for more • lucrative grants

Each Strength has a Rebuttal! • fallible causal inference • forgotten agricultural heritage • transient clinical relevance • vexatious quality control • formulaic writing • slow to get published • simplistic marketing • impediments to replication • restricted access for analyses • exorbitant grants

Babies and Bathwater

• no methodology is perfect • different tools for the job • no denial of past success

A New Substitute for an RCT: Exposure Crossover Design

Dow Jones Industrial Average (6 months)

A Before and After Perspective

Intervention

Outcome }

Can Pregnancy Cause Venous Clots?

n = 507,262

0

50

100

150

200

250

300

350

400

450

500

Pregnancy Baseline Subsequent

Count

-4 -3 -2 -1 0 +1

Time (years)

Delivery

Does Pregnancy Prevent Bike Crashes?

n = 507,262

Pregnancy Baseline Subsequent

Count

-4 -3 -2 -1 0 +1

Time (years)

0

5

10

15

20

25

30

35

40

45

50

Delivery

Can Pregnancy Reduce Depression?

n = 507,262

Pregnancy Baseline Subsequent

Count

-4 -3 -2 -1 0 +1

Time (years)

0

50

100

150

200

250

300

350

400

450

500

Delivery

0

10

20

30

40

50

Does Kidney Transplantation Increase the Risk of Herpes Infections?

n = 4,905

Induction Baseline Subsequent

Count

-4 -3 -2 -1 0 +1

Time (years)

Transplantation

Do Physician Warnings Change the Risk of a Road Crash?

n = 100,075

Baseline Induction Subsequent

Count

-4 -3 -2 -1 0 +1

Time (years)

0

50

100

150

200

250

300

350

400

450

500

Warning

NEJM 2012

Collapsing a Display into a few Numbers

Baseline Rate*

4.76

Subsequent Rate*

2.73

* crashes per 1,000 annually

Relative Risk

0.55

Different Medical Diagnoses

DIAGNOSIS

Baseline Rate

Subsequent Rate

Relative Risk

95% Conf. Inter.

Alcoholism 7.24 4.91 0.64 0.46 – 0.91

Epilepsy 5.92 3.23 0.53 0.36 – 0.78

Dementia 2.92 0.86 0.31 0.18 – 0.58

Sleep Disorder 6.10 3.85 0.62 0.46 – 0.85

Syncope 5.91 2.95 0.49 0.38 – 0.64

Stroke 3.50 1.15 0.32 0.19 – 0.60

Diabetes 4.49 2.71 0.59 0.43 – 0.82

Depression 8.75 3.60 0.38 0.25 – 0.60

Different Physician Characteristics

RESPONSIBLE PHYSICIAN

Baseline Rate

Subsequent Rate

Relative Risk

95% Conf. Inter.

Age < 50 years 4.43 2.72 0.60 0.51 - 0.71 Age ≥ 50 years 5.51 2.76 0.48 0.38 - 0.61

Male gender 4.58 2.68 0.56 0.49 - 0.65 Female gender 5.92 3.05 0.51 0.37 - 0.73

General Practice 4.48 2.74 0.56 0.47 - 0.68 Specialty Practice 5.21 2.71 0.54 0.44 - 0.67

Years in Practice < 20 4.51 2.68 0.58 0.48 - 0.69 Years in Practice ≥ 20 5.13 2.80 0.53 0.43 - 0.65

Summary of Exposure Crossover Design

Allows self matching to control measured and unmeasured individual factors

Requires a durable intervention with a clear starting date

Focuses on outcomes that are recurrent, relapsing, or otherwise repeated

Limitations of Exposure Crossover Design

• unstable baseline interval • subtle forms of confounding • conglomerate interventions • inadequate sample size • insufficient time duration • potential survivor bias • selection bias of participants

Strengths of Exposure Crossover Design

• controls unmeasured stable confounders • straightforward basic statistics • intuitive interpretation of findings • simple graphical display of results • low marginal financial costs • no endless logistical hassles • zero need to assign placebos • “Real World” instead of “Disney World”

Other Self Matching Designs: Not Exposure Crossover Design

• Case Crossover Design (relative risks)

• N-of-1 Trials (brief effects)

• Self Controlled Case Series (single outcome)

• Time Series Analysis (aggregated perspective)

Name for Graph

Take Home Message The Exposure Crossover design is a new self-matching approach for testing clinical questions when a randomized trial is not possible.

Self-Matched Studies

Randomized Controlled Trials