designing an rct protocol (i) the...

1

Designing an RCT protocol(i) the context

1

Prof. Chris Maher

Affiliated with the University of Sydney

Overview

• Physiotherapy trials

• Common problems

• Elaborate on CONSORT statement

2

How many of you have performed a RCT?

How many of you have performed a SR?

How many have taken a course in critical appraisal, clin epi, trial methodology?

How many have done none of the above?

How many of you have a research question that they want to turn into a trial protocol?

3

4

Growth physiotherapy evidence

Year

1930 1940 1950 1960 1970 1980 1990 2000 2010

cu

mula

tive

nu

mb

er

of

reco

rds

0

2500

5000

7500

10000

12500

15000

17500

randomised controlled trials

systematic reviews

clinical practice guidelines

% of randomisesd controlled trials

0 20 40 60 80 100 120

inclusion and source

random allocation

concealed allocation

baseline comparability

blinded subjects

blinded therapists

blinded assessors

outcomes for > 85%

intention to treat analysis

between group comparisons

mean and variability data

5

Total PEDro score (/10)

0 1 2 3 4 5 6 7 8 9 10

% o

f ra

ndo

mis

ed

con

tro

lled

tria

ls

0

10

20

30

NATURE ISSUE Reporting RCT in title 44% Reporting All 4 elements of PICO in aim 61% Conduct Trial registered 0% Conduct Protocol published 6% Reporting Nominates primary outcome 33% Reporting Nominates primary outcome & time point 11%

Conduct Level of primary outcome Continuous Counts Time to event

88% 6% 6%

Conduct Design Parallel Cross-over

83% 17%

Conduct Placebo used? 6%

Analysis of 18 RCTs in the

period Jan 2007-2008

6

Analysis of 18 RCTs in the

period Jan 2007-2008

Reporting Describes method used to generate allocation sequence

33%

Reporting Describes methods used to implement the random allocation sequence

33%

Reporting Date of study specified 17% Reporting Basic power analysis reported 33% Analysis Power analysis includes alpha, beta, effect

size, SD 28%

Analysis Power analysis allows for loss to follow-up 17% Analysis Power analysis allows for treatment non-

compliance 0%

Analysis Statistical analysis appropriate 61% Conduct Measured side-effects/adverse events 17%

Conduct Rx duration (median & range) 8 (1; 48) Conduct Weeks of follow-up beyond Rx cessation

(median & range) 0 (0; 8)

Reporting Reported side-effects 11% Reporting Participant characteristic table includes

participant characteristics, important prognostic factors and primary and secondary outcomes

72%

Reporting Statistical comparison of groups for Table 1 83% Reporting Trial flow diagram 44% Conduct Minimum sample size per group (median &

range) 18 (9; 102)

Reporting Measure of variability for outcomes presented 94% Reporting Provides effect size and 95%CI 17% Analysis No of within group statistical comparisons

(median & range) ?* (0; 98)

Analysis No of between group statistical comparisons (median & range)

?* (1; 32)

Reporting Explicitly reports follow-up at each time point 56%

Analysis of 18 RCTs in the

period Jan 2007-2008

7

Analysis of 18 RCTs in the

period Jan 2007-2008

Reporting Explicitly reports follow-up at each time point 56% Conduct Best follow-up rate for primary outcome

(median & range) 89 (58; 100)@

Conduct Worst follow-up rate for primary outcome (median & range)

94 (75; 100)@

Analysis Method of dealing with missing data Last value carried forward Dropped case Not stated Not required as no loss to follow-up Worst case analysis

7% 27% 33% 17% 6%

Analysis Used mixed models for longitudinal data 0%

Rationale

• Is it worth doing?

• What has previous research shown?

• Systematic review/s?

• How would this trial alter the conclusions of previous research? Would it answer the question definitively?

8

Consolidated Standards of Reporting Trials (CONSORT) Statement

- *Extended for nonpharmacological trials

surgery, procedures, physical, alternate therapies

- Often more difficult to blind

- Often complex interventions

- Outcome more likely to be influenced by care provider/centre expertise and volume of care

Modified checklist of 22 items

Modified CONSORT Flow Diagram

Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Extending the CONSORT statement to randomized trials

of nonpharmacologocial treatment: Explanation and elaboration. Ann Intern Med 2008;148:295-309.

Sample size

How sample size was determined and, when applicable,

explanation of any interim analyses and stopping rules

*Details of whether and how clustering by care providers or

centres was addressed: important for non-blinded studies

(Observations of participants treated by the same provider may be

correlated or ‘clustered’ which will reduce statistical power.This depends upon the intraclass correlation coefficient (defined as the

correlation between any 2 participants treated by the same health care

provider or centre) and the number of participants treated by each

provider or in each centre.)

Sample size

9

Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.

Randomisation (Sequence generation)

Method used to generate the random allocation sequence,

including details of any restriction (e.g. blocking,

stratification)

*Where applicable, how care providers were allocated to

each trial group

‘Expertise-based RCT’ (surgery vs physiotherapy –

different providers provide the different interventions)

Randomisation (sequence generation)

10

Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.

Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.

11

Randomisation (Sequence generation)

Method used to implement the random allocation

sequence (e.g.,numbered containers or central

telephone), clarifying whether the sequence was

concealed until interventions were assigned

Allocation concealment

Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.

12

Participants

Eligibility criteria for participants

Settings, locations

*Eligibility criteria for centres and those

performing the intervention

Eligibility criteria for centres

*Eligibility criteria for centres and those performing the

intervention should be justified as will inform applicability

of trial results

• expertise and training of surgeon or physical therapist

• years of training, professional qualifications

• years in practice

• number of interventions performed

• skill as assessed by complication rate for procedure

• specific training before trial initiation, expertise

• volume of care at centre

13

Blinding

� Blinding of participants controls for placebo effects and polite patient effects

� It can be achieved by providing a sham control

14

Strategies to blind participants

� Indistinguishable placebo

� Inert placebo

� Structural equivalence

� Naïve subjects

Blinding

� Blinding of assessors controls for assessor biases

� It can be achieved by using independent assessors

� When blinding assessors, evaluate the completeness of blinding

� When outcomes are self-reported, blinding of assessors implies blinding of participants

15

Blinding

� Blinding of therapists controls for implementation bias

� It can usually be achieved in drug trials, but rarely in trials of complex interventions eg stroke unit, surgery, physiotherapy

Blinding

� Blinding of statisticians prevents arbitrary decisions about analytical procedures being influenced by their effects on outcomes

� It can be achieved by presenting the statistician with coded data. The code is not broken until the primary analysis is complete

16

Imperfect placebos in LBP trials

• 126 RCTs described as placebo or sham

controlled

– Indistinguishable placebo?

– Inert placebo?

– Structural equivalence?

– Naïve subjects?

– Blinding assessed?

Machado et al J Clin Epi 2008

Machado et al J Clin Epi 2008

17

Examples of imperfect placebos in LBP trials

• Educational booklet

• Health education video

• Massage

• TENS

• Hot pack

• Abdominal wrap/advice to walk/video

• Diphenhydramine tablets (anti-histamine)

Machado et al J Clin Epi 2008

18

Acupuncturist:

skin penetration?

Patient:

skin penetration?

Patient:

de qi sensation

Yes No Yes No Yes No

real 35 25 48 12 48 12

sham 30 30 25 35 20 40

Acupuncturist: skin penetration χ2 = 0.833, p = 0.361

Patient: skin penetration χ2 = 18.502, p < 0.001; de qi, χ2 = 26.606, p < 0.001

19

Buchbinder R, Osborne RH, Ebeling P, et al. BMC Musculoskel Dis 2008;9:156.

20

21

Assessment of blinding

22

Treatment credibility

Exs &

advice

Sham exs &

advice

Exs & sham

advice

Sham exs &

sham advice

Confident that treatment

will relieve pain5 4 5 4

Confident that treatment

will help manage pain5 5 5 4

Confident to recommend

treatment to friend5 4 5 4

How logical is the

treatment6 5 6 5

Median treatment credibility scores at end of first treatment

0= not confident/logical; 6=absolutely confident/very logical

23

Assessment of pt blinding

• RCT of exs vs sham electrotherapy

• At 12 months, patients were asked

– “Which treatment did you receive? Real physical therapy treatment? Or a sham or pretend treatment?”

– Exs: 85% said real PT

– Sham: 84% said real PT

Costa et al Physical Therapy 2009 89: 1275-1286

CONSORT 2010

• The interventions for each group with

sufficient details to allow replication,

including how and when they were actually

administered

24

Extension for non-pharmacological trials

• Precise details of both the experimental treatment and comparator

• Description of the different components of the interventions and, when applicable, descriptions of the procedure for tailoring the interventions to individual participants

• Details of how the interventions were standardized

• Details of how adherence of care providers with the protocol was assessed or enhanced

Precise details of both the experimental

treatment and comparator

25

Costa et al Physical Therapy 2009 89: 1275-1286

Description of the different components of the interventions and, when applicable, descriptions of

the procedure for tailoring the interventions to individual participants

26

Bennell K, Coburn S, Green S, Harris A, Forbes A, Buchbinder R. BMC Musculoskel Dis 2007;8:86.

Details of how the interventions were

standardized

Costa et al Physical Therapy 2009 89: 1275-1286

27

Details of how adherence of care providers

with the protocol was assessed or enhanced

Pengel et al Ann Intern Med 2007 146: 787-796

28

Details of how adherence of care providers

with the protocol was assessed or enhanced

Pengel et al Ann Intern Med 2007 146: 787-796

29

30

Outcome measures for shoulder trials

Clearly defined primary and secondary outcome

measures and, when applicable, any methods

used to enhance the quality of measurements (e.g.

multiple observations, training of assessors)

Outcome measurement

Statistical Methods

Statistical methods used to compare groups for primary outcome(s). Methods for additional analyses, such as subgroup analyses and adjusted analyses

*When applicable, details of whether and how the clustering by care providers or centres was addressed

Standard methods of analysis that ignore clusters will result in incorrect estimation of treatment effect

Statistical Methods

31

• A lot of well-meaning but wasted effort– Clinically unimportant questions

– Poor methods so results not interpretable

• Determine the really important questions

• Consider multinational, multicentre mega-

trials

Conclusions

Trials should comply with CONSORT statement

– particular attention to non-pharmacologic interventions

– clearly define and describe their inclusion criteria

– report the training and expertise of the surgeon/physical

therapist/centre etc.

– blind participants/outcome assessment if feasible

Conclusions