quantifying the clinical measure of interest in the presence of … › public_access › autumn2016...
TRANSCRIPT
Quantifying the clinical measure of interest in the presence of missing data:
choosing primary and sensitivity analyses in neuroscience clinical trials
Sept 26, 2016
Elena Polverejan, Ph.D.Statistical Modeling & Methodology
Janssen R&D, Johnson & Johnson
His wif_ is not working today.
Impact of Missing Data
2
i or e?
Outline
• Background: Selection of the primary estimand and
statistical methods for a clinical trial
• Neuroscience trial simulation example:
• Assumptions
• Statistical methods
• Derived properties
• Role of simulations
3
Primary and Sensitivity Analyses
4
Estimand
Primary
Analysis
Sensitivity
Analysis 1
Sensitivity
Analysis k
Sensitivity
Estimate k
Sensitivity
Estimate 1
Primary
Estimate
…
…End of Trial
Challenges in Selection of Statistical Methods
Unique characteristics of each clinical trial:
Indication
Study population
Study design
Efficacy response
Likelihood of subjects remaining on treatment or in the trial etc.
Variety of statistical methods
Regulatory requirements that evolve over time
5
Simulation Example: Depression Efficacy Trial
Screening
Drug
Placebo
6
1:1 Randomization
N=66 Subjects/Group
4-week DB Efficacy Phase
Follow-Up
Phase
Primary time point
Primary efficacy measure:
depression scale
(MADRS) collected over
time in the DB phase
Primary efficacy endpoint:
change from baseline to
Week 4 in MADRS Total
Score
Simulation Evaluation Process
• Simulate full datasets (without any missing data)
• Create over time missing values based on various
assumptions (missing data cases)
• Apply considered statistical methods
• Determine operating characteristics of the considered
methods:
• Power (or Type I error rate)
• Estimated treatment difference and its variability
7
Simulate Full Datasets
Key Assumption: Over Time Efficacy Response
8
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
0 1 2 3 4 5 6 7 8 910111213141516171819202122232425262728
Day
Me
an
Re
sp
on
se
trt
Drug
Placebo
-6.5
Simulated Missing Data Cases
9
Averaged Distribution of Discontinuations for the Evaluated Cases
Case Group Mean Total %DC Mean %DC Other Mean %DC LOE
1 drug 30.6 25.1 5.4
1 control 26.0 14.7 11.2
2 drug 25.5 20.1 5.4
2 control 26.0 14.7 11.2
3 drug 18.8 15.1 3.7
3 control22.2 14.6 7.6
30.6
26.0
25.5
18.8
22.2
25.1
14.7
5.4
11.2
Statistical Methods?
26.0
DC = Discontinuation
LOE = Lack of Efficacy
Other discontinuation reasons include adverse events, lost to follow-up etc.
MAR vs MNAR
1
0
MAR = Missing at Random MNAR = Missing Not At Random
After withdrawal subjects would
tend to have similar efficacy to
subjects who remain in the
study after accounting for
observed characteristics
Assumptions need to be made on
the potential “trajectory” or
distribution of efficacy after
withdrawal, which will be different
from the one of subjects
remaining in the trial
Multiple Imputation Based Methods
Multiple imputation is a statistical technique for analyzing
incomplete datasets.
Application of this technique requires three steps:
c1_oc = observed change at Visit 1, c2_oc = observed change at Visit 2 etc.
11
DC c1_oc c2_oc c3_oc c4_oc
OTHER -1.06215 -5.17018 . .
_Imputation_ c1_mi c2_mi c3_mi c4_mi
1 -1.06215 -5.17018 -3.59436 -6.26150
2 -1.06215 -5.17018 -2.95910 -0.40440
3 -1.06215 -5.17018 -3.93861 -5.58729
4 -1.06215 -5.17018 -5.05197 -4.65008
5 -1.06215 -5.17018 -4.02463 -1.85226
6 -1.06215 -5.17018 -5.29770 -1.28843
7 -1.06215 -5.17018 -3.45573 -3.67338
8 -1.06215 -5.17018 -1.64735 -2.77245
9 -1.06215 -5.17018 -8.62956 -1.87126
10 -1.06215 -5.17018 -6.19145 -4.04486
_Imputation_ Diff vs Pbo StdErr
1 -0.6467 0.2717
2 -0.5798 0.2729
3 -0.4925 0.2800
4 -0.7350 0.2754
5 -0.4891 0.2722
6 -0.7668 0.2758
7 -0.7032 0.2699
8 -0.5977 0.2748
9 -0.6303 0.2709
10 -0.6180 0.2686
Pooled
Diff
Pooled
StdErr Pvalue
-0.625909 0.290099 0.0313
Imputation with multiple values
Analysis
Pooling
12September,
2014
Control-Based MI
Delta AdjustmentANCOVA_LOCF
ANCOVA_BOCF
Evaluated Statistical Methods
MMRM: Mixed Model for Repeated
Measures
MI_REG: MI with Regression Option
Missing at Random (MAR)
Based:
ANCOVA_FULL
MMRM_FULL
Single-Imputation (MNAR):
Multiple-Imputation (MI)
Under Missing Not at Random
(MNAR):
For Reference:
MI methods: MMRM for analysis
Joint Control-Based MI Methods
13
MIJOINT_J2R
MIJOINT_CR
Methodology and SAS macros developed by James Roger and shared through DIA missing data working group
site at http://www.missingdata.org.uk; Slide from O’Kelly & Davis short course at the 2015 ASA Biopharmaceutical
Workshop
MAR Methods
Delta Adjustment and Tipping Point Analysis
Analysis assuming that subjects who discontinue would,
on average, have their unobserved efficacy outcome
worse by the amount Delta compared to the observed
efficacy outcome of subjects who remain in the study.
Tipping Point Analysis: find the assumption at which
conclusions change from favorable to drug (statistically
significant) to unfavorable.
Recommended by the NRC 2010 report* and by FDA**
Delta adjustment may be applied:
only on the experimental groups (of regulatory interest)
on all treatment groups
for certain reasons of discontinuation.
14
* National Research Council report on the Prevention and Treatment of Missing Data in Clinical Trials
(2010)
**Thomas Permutt “Sensitivity analysis for missing data in regulatory submissions” Statist. Med. 2015
Delta Adjustment Types
Types of delta adjustment:
sequential (visit-by-visit)
marginal (multiple imputations first, then apply delta
adjustment)
In simulation exercise:
Marginal delta adjustment: MIDELTAMAR
first, MAR-based multiple imputation analysis (MI_REG)
apply delta mean worsening in active arm only
same delta adjustment across visits
sequence of delta of increasing severity: 0 to 10 by 1
15
Estimated Power
16
Case % DC
Control
% DC
Active
1 26% 31%
2 26% 26%
3 22% 19%
Assumption:
91% power for the “full dataset”,
before missing values applied
ancova_bocf
mijoint_j2r
mijoint_cr
ancova_locf
mi_reg
mmrm_oc
mmrm_full
ancova_full
60 70 80 90
Power
Me
tho
d
case
Case 1
Case 2
Case 3
Estimated Sample Size to Maintain 90% Power
17
Case % DC
Control
% DC
Active
1 26% 31%
2 26% 26%
3 22% 19%
Increasing the sample size
using expected amount of
drop-outs might not be
sufficient to maintain power:
Case 2: 26% adjustment to 66
subjects =~ 89 subjects ancova_bocf
mijoint_j2r
mijoint_cr
ancova_locf
mi_reg
mmrm_oc
mmrm_full
ancova_full
60 90 120 150 180
Number subjects per group to maintain 90% power
Me
tho
d
case
Case 1
Case 2
Case 3
mideltamar_0_10
mideltamar_0_9
mideltamar_0_8
mideltamar_0_7
ancova_bocf
mideltamar_0_6
mijoint_j2r
mideltamar_0_5
mideltamar_0_4
mideltamar_0_3
mijoint_cr
mideltamar_0_2
mideltamar_0_1
ancova_locf
mideltamar_0_0
mmrm_oc
mmrm_full
ancova_full
40 60 80
Power
Me
tho
d
case
Case 1
Case 2
Case 3
Estimated Power With Increasing Delta Adjustments
18
Case % DC
Control
% DC
Active
1 26% 31%
2 26% 26%
3 22% 19%
Probability(%) of Tipping From MAR MI Regression
19
Case % DC
Control
% DC
Active
1 26% 31%
2 26% 26%
3 22% 19%
0
5
10
15
20
25
30
35
40
45
50
55
1 2 3 4 5 6 7 8 9 10
Delta Adjustment
Pro
babili
ty o
f T
ippin
g f
rom
MA
R M
I_R
EG
Analy
sis
(%
)
case
Case 1
Case 2
Case 3 Tipping point analysis =
finding the delta worsening
that changes conclusions
from favorable to drug
(statistically significant) to
unfavorable
Role of Simulations
Understand at the study design stage:
the impact of amount and reason of missing data
the performance of the statistical analysis methods
considered for the primary and sensitivity analyses
potential sample size adjustments.
Incorporate clinical feedback
Communicate with regulatory agencies
Communicate the impact of missing data and the
importance to reduce the amount of missing
data through study design and conduct
20
21
Back-Up
ancova_bocf
mmrm_full
ancova_locf
ancova_full
mijoint_cr
mmrm_oc
mijoint_j2r
mi_reg
1.85 1.90 1.95 2.00 2.05 2.10
Mean SE
Me
tho
d
case
Case 1
Case 2
Case 3
ancova_full
mmrm_full
mi_reg
mmrm_oc
ancova_locf
mijoint_cr
mijoint_j2r
ancova_bocf
-6.5 -6.0 -5.5 -5.0 -4.5 -4.0
Mean Difference vs Control in LSMeans
Me
tho
d
case
Case 1
Case 2
Case 3
Estimated Mean Treatment Difference and
Mean Standard Error
23