designing an rct protocol (i) the...
TRANSCRIPT
1
Designing an RCT protocol(i) the context
1
Prof. Chris Maher
Affiliated with the University of Sydney
Overview
• Physiotherapy trials
• Common problems
• Elaborate on CONSORT statement
2
How many of you have performed a RCT?
How many of you have performed a SR?
How many have taken a course in critical appraisal, clin epi, trial methodology?
How many have done none of the above?
How many of you have a research question that they want to turn into a trial protocol?
3
4
Growth physiotherapy evidence
Year
1930 1940 1950 1960 1970 1980 1990 2000 2010
cu
mula
tive
nu
mb
er
of
reco
rds
0
2500
5000
7500
10000
12500
15000
17500
randomised controlled trials
systematic reviews
clinical practice guidelines
% of randomisesd controlled trials
0 20 40 60 80 100 120
inclusion and source
random allocation
concealed allocation
baseline comparability
blinded subjects
blinded therapists
blinded assessors
outcomes for > 85%
intention to treat analysis
between group comparisons
mean and variability data
5
Total PEDro score (/10)
0 1 2 3 4 5 6 7 8 9 10
% o
f ra
ndo
mis
ed
con
tro
lled
tria
ls
0
10
20
30
NATURE ISSUE Reporting RCT in title 44% Reporting All 4 elements of PICO in aim 61% Conduct Trial registered 0% Conduct Protocol published 6% Reporting Nominates primary outcome 33% Reporting Nominates primary outcome & time point 11%
Conduct Level of primary outcome Continuous Counts Time to event
88% 6% 6%
Conduct Design Parallel Cross-over
83% 17%
Conduct Placebo used? 6%
Analysis of 18 RCTs in the
period Jan 2007-2008
6
Analysis of 18 RCTs in the
period Jan 2007-2008
Reporting Describes method used to generate allocation sequence
33%
Reporting Describes methods used to implement the random allocation sequence
33%
Reporting Date of study specified 17% Reporting Basic power analysis reported 33% Analysis Power analysis includes alpha, beta, effect
size, SD 28%
Analysis Power analysis allows for loss to follow-up 17% Analysis Power analysis allows for treatment non-
compliance 0%
Analysis Statistical analysis appropriate 61% Conduct Measured side-effects/adverse events 17%
Conduct Rx duration (median & range) 8 (1; 48) Conduct Weeks of follow-up beyond Rx cessation
(median & range) 0 (0; 8)
Reporting Reported side-effects 11% Reporting Participant characteristic table includes
participant characteristics, important prognostic factors and primary and secondary outcomes
72%
Reporting Statistical comparison of groups for Table 1 83% Reporting Trial flow diagram 44% Conduct Minimum sample size per group (median &
range) 18 (9; 102)
Reporting Measure of variability for outcomes presented 94% Reporting Provides effect size and 95%CI 17% Analysis No of within group statistical comparisons
(median & range) ?* (0; 98)
Analysis No of between group statistical comparisons (median & range)
?* (1; 32)
Reporting Explicitly reports follow-up at each time point 56%
Analysis of 18 RCTs in the
period Jan 2007-2008
7
Analysis of 18 RCTs in the
period Jan 2007-2008
Reporting Explicitly reports follow-up at each time point 56% Conduct Best follow-up rate for primary outcome
(median & range) 89 (58; 100)@
Conduct Worst follow-up rate for primary outcome (median & range)
94 (75; 100)@
Analysis Method of dealing with missing data Last value carried forward Dropped case Not stated Not required as no loss to follow-up Worst case analysis
7% 27% 33% 17% 6%
Analysis Used mixed models for longitudinal data 0%
Rationale
• Is it worth doing?
• What has previous research shown?
• Systematic review/s?
• How would this trial alter the conclusions of previous research? Would it answer the question definitively?
8
Consolidated Standards of Reporting Trials (CONSORT) Statement
- *Extended for nonpharmacological trials
surgery, procedures, physical, alternate therapies
- Often more difficult to blind
- Often complex interventions
- Outcome more likely to be influenced by care provider/centre expertise and volume of care
Modified checklist of 22 items
Modified CONSORT Flow Diagram
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Extending the CONSORT statement to randomized trials
of nonpharmacologocial treatment: Explanation and elaboration. Ann Intern Med 2008;148:295-309.
Sample size
How sample size was determined and, when applicable,
explanation of any interim analyses and stopping rules
*Details of whether and how clustering by care providers or
centres was addressed: important for non-blinded studies
(Observations of participants treated by the same provider may be
correlated or ‘clustered’ which will reduce statistical power.This depends upon the intraclass correlation coefficient (defined as the
correlation between any 2 participants treated by the same health care
provider or centre) and the number of participants treated by each
provider or in each centre.)
Sample size
9
Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.
Randomisation (Sequence generation)
Method used to generate the random allocation sequence,
including details of any restriction (e.g. blocking,
stratification)
*Where applicable, how care providers were allocated to
each trial group
‘Expertise-based RCT’ (surgery vs physiotherapy –
different providers provide the different interventions)
Randomisation (sequence generation)
10
Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.
Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.
11
Randomisation (Sequence generation)
Method used to implement the random allocation
sequence (e.g.,numbered containers or central
telephone), clarifying whether the sequence was
concealed until interventions were assigned
Allocation concealment
Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.
12
Participants
Eligibility criteria for participants
Settings, locations
*Eligibility criteria for centres and those
performing the intervention
Eligibility criteria for centres
*Eligibility criteria for centres and those performing the
intervention should be justified as will inform applicability
of trial results
• expertise and training of surgeon or physical therapist
• years of training, professional qualifications
• years in practice
• number of interventions performed
• skill as assessed by complication rate for procedure
• specific training before trial initiation, expertise
• volume of care at centre
13
Blinding
� Blinding of participants controls for placebo effects and polite patient effects
� It can be achieved by providing a sham control
14
Strategies to blind participants
� Indistinguishable placebo
� Inert placebo
� Structural equivalence
� Naïve subjects
Blinding
� Blinding of assessors controls for assessor biases
� It can be achieved by using independent assessors
� When blinding assessors, evaluate the completeness of blinding
� When outcomes are self-reported, blinding of assessors implies blinding of participants
15
Blinding
� Blinding of therapists controls for implementation bias
� It can usually be achieved in drug trials, but rarely in trials of complex interventions eg stroke unit, surgery, physiotherapy
Blinding
� Blinding of statisticians prevents arbitrary decisions about analytical procedures being influenced by their effects on outcomes
� It can be achieved by presenting the statistician with coded data. The code is not broken until the primary analysis is complete
16
Imperfect placebos in LBP trials
• 126 RCTs described as placebo or sham
controlled
– Indistinguishable placebo?
– Inert placebo?
– Structural equivalence?
– Naïve subjects?
– Blinding assessed?
Machado et al J Clin Epi 2008
Machado et al J Clin Epi 2008
17
Examples of imperfect placebos in LBP trials
• Educational booklet
• Health education video
• Massage
• TENS
• Hot pack
• Abdominal wrap/advice to walk/video
• Diphenhydramine tablets (anti-histamine)
Machado et al J Clin Epi 2008
18
Acupuncturist:
skin penetration?
Patient:
skin penetration?
Patient:
de qi sensation
Yes No Yes No Yes No
real 35 25 48 12 48 12
sham 30 30 25 35 20 40
Acupuncturist: skin penetration χ2 = 0.833, p = 0.361
Patient: skin penetration χ2 = 18.502, p < 0.001; de qi, χ2 = 26.606, p < 0.001
19
Buchbinder R, Osborne RH, Ebeling P, et al. BMC Musculoskel Dis 2008;9:156.
20
21
Assessment of blinding
22
Treatment credibility
Exs &
advice
Sham exs &
advice
Exs & sham
advice
Sham exs &
sham advice
Confident that treatment
will relieve pain5 4 5 4
Confident that treatment
will help manage pain5 5 5 4
Confident to recommend
treatment to friend5 4 5 4
How logical is the
treatment6 5 6 5
Median treatment credibility scores at end of first treatment
0= not confident/logical; 6=absolutely confident/very logical
23
Assessment of pt blinding
• RCT of exs vs sham electrotherapy
• At 12 months, patients were asked
– “Which treatment did you receive? Real physical therapy treatment? Or a sham or pretend treatment?”
– Exs: 85% said real PT
– Sham: 84% said real PT
Costa et al Physical Therapy 2009 89: 1275-1286
CONSORT 2010
• The interventions for each group with
sufficient details to allow replication,
including how and when they were actually
administered
24
Extension for non-pharmacological trials
• Precise details of both the experimental treatment and comparator
• Description of the different components of the interventions and, when applicable, descriptions of the procedure for tailoring the interventions to individual participants
• Details of how the interventions were standardized
• Details of how adherence of care providers with the protocol was assessed or enhanced
Precise details of both the experimental
treatment and comparator
25
Costa et al Physical Therapy 2009 89: 1275-1286
Description of the different components of the interventions and, when applicable, descriptions of
the procedure for tailoring the interventions to individual participants
26
Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.
Details of how the interventions were
standardized
Costa et al Physical Therapy 2009 89: 1275-1286
27
Details of how adherence of care providers
with the protocol was assessed or enhanced
Pengel et al Ann Intern Med 2007 146: 787-796
28
Details of how adherence of care providers
with the protocol was assessed or enhanced
Pengel et al Ann Intern Med 2007 146: 787-796
29
30
Outcome measures for shoulder trials
Clearly defined primary and secondary outcome
measures and, when applicable, any methods
used to enhance the quality of measurements (e.g.
multiple observations, training of assessors)
Outcome measurement
Statistical Methods
Statistical methods used to compare groups for primary outcome(s). Methods for additional analyses, such as subgroup analyses and adjusted analyses
*When applicable, details of whether and how the clustering by care providers or centres was addressed
Standard methods of analysis that ignore clusters will result in incorrect estimation of treatment effect
Statistical Methods
31
• A lot of well-meaning but wasted effort– Clinically unimportant questions
– Poor methods so results not interpretable
• Determine the really important questions
• Consider multinational, multicentre mega-
trials
Conclusions
Trials should comply with CONSORT statement
– particular attention to non-pharmacologic interventions
– clearly define and describe their inclusion criteria
– report the training and expertise of the surgeon/physical
therapist/centre etc.
– blind participants/outcome assessment if feasible
Conclusions