statistics for the terrified: trial design and sample size
DESCRIPTION
Statistics for the Terrified: Trial Design and Sample Size. Andrea Marshall Mark Williams. By the end of this session you will be aware of The different types of trial designs and why we use them Importance of sample size and requirements needed to be able calculate it - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/1.jpg)
Statistics for the Terrified:Trial Design and Sample Size
Andrea MarshallMark Williams
![Page 2: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/2.jpg)
Learning Objectives
By the end of this session you will be aware of
• The different types of trial designs and why we use them
• Importance of sample size and requirements needed to be able calculate it
• Types of statistical errors
• Statistical power
![Page 3: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/3.jpg)
Clinical Trial
• A prospective experiment • Designed to assess the effect of one or more
interventions in a group of patients
![Page 4: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/4.jpg)
Phases of Research
![Page 5: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/5.jpg)
Phase II Studies
• Generally small studies • 1 or more interventions • Provide initial assessment of efficacy
• Identify promising new interventions for further evaluation and screen out ineffective interventions
• Not able to compare interventions
![Page 6: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/6.jpg)
Choice of control
• Uncontrolled – no control – may be useful for small phase II studies looking to show a intervention is feasible or effective
• Historical control – trial compares patients treated with new intervention with earlier series of patients – but may not be truly comparable
• Controlled trial –intervention is compared to standard in patients over the same period of time
• Matched controls – patients allocated to intervention are matched to those on the standard intervention according to specific characteristics
![Page 7: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/7.jpg)
Types of Phase III trial design
• Parallel group• Factorial• Cross-over• Cluster-randomised• Adaptive designs
![Page 8: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/8.jpg)
Parallel group design
• Most simple design• Compares at least one intervention with the standard
in patients at the same time• Can consider two or more arms - i.e. have 2 or more
intervention options but have to be confident can get enough patients
• Patients on each arm are similar and only the allocated interventions differ
• Allocation to trial arms is made at random
![Page 9: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/9.jpg)
Example - Parallel group design
• SARAH trial: Assessing the effectiveness of exercise programme over and above usual care
Sample: People with Rheumatoid Arthritis
of the hand
Usual Care
Usual care + Exercise
![Page 10: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/10.jpg)
Factorial design
• Allows more than 1 question to be answered using the same patients and therefore less patients are needed
• If interested in two types of therapy• Alternative to running 2 parallel group trials instead
![Page 11: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/11.jpg)
Sunscreen anyone?
![Page 12: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/12.jpg)
Factorial design (2)
1. Looking at role of suncreen, i.e. Suncreen versus no sunscreen
2. Looking at role of betacarotene,
i.e. betacarotene versus no betacaroteneSample
(n=1620)
Suncreen(n=810)
Control(n=810)
Sample(n=1620)
Betacarotene(n=810)
Control(n=810)
![Page 13: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/13.jpg)
Factorial design (4) Example
1. Looking at role of sunscreen
2. Looking at role of Betacarotene
Sample(n=1621)
Placebo tabs only (n=393)
Betacarotene tabs only (n=416)
Sunscreen + placebo tabs
(n=408)
Sunscreen + betacarotene tabs (n=404)
No sunscreen (n=809) BC and Placebo tabs only
Sunscreen (n=812)Sunscreen+BC and Sunscreen+placebo
No BC (n=801) Sunscreen+placebo and
placebo tabs only
BC (n=820)Sunscreen+BC and BC tabs
only
![Page 14: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/14.jpg)
Crossover designs
• Patients are allocated to a sequence of two or more interventions
• Classified by number of interventions allocated to each patient – i.e. 2-period crossover = each person receives each of
the 2 interventions in a random order
Treatment A
Washout
period
Treatment
BSample Trea
tment B
Washout
Period
Treatment A
![Page 15: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/15.jpg)
Crossover designs (2)
• Limited to situations where:– Disease is chronic and stable– Different interventions can be administered to same
patient for a short period of time – Patients return to the same state afterward each
intervention s to avoid a carry over effect • Wash out period should be long enough for complete
reversibility of intervention effect• Generally fewer patients required as interventions
evaluated within the same patients
![Page 16: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/16.jpg)
Cross over design (3)- example
• Sleep apnoea victim?• Engleman et al (1999) Randomised placebo-
controlled cross-over trial of CPAP for mild sleep apnea/hypopnea syndrome
CPAP
Washout
period
Placebo tabs
Placebo tabs
Washout
Period
CPAP
Sample
![Page 17: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/17.jpg)
Cluster design
• Patients within any one cluster are often more likely to respond in a similar manner
• Intervention is delivered to or affects groups• E.g. Group exercise program
• Intervention is targeted at health professionals• E.g. Educational regarding disease management
• Contamination between individuals
![Page 18: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/18.jpg)
Cluster design (2) E.g. OPERA
Care Homes
Intervention – Physical
activation programme
Control – Depression Awareness
![Page 19: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/19.jpg)
Adaptive designs
• Ability to modify the study without undermining the validity and integrity of the trial
• Can save time and money • Need short-term evaluable• Possible modifications include:-
– Re-estimating sample size– Stopping trial early– Dropping interventions– Changing the eligibility criteria
![Page 20: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/20.jpg)
Size of trial
• Important to determine precisely at the design stage• Minimum number of subjects
– To reliably answer the question– Avoid patients unnecessarily receiving an inferior
treatment• Want trials to have
– High chance of detecting clinically important differences– Small chance of observing differences when they do not
exist
![Page 21: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/21.jpg)
Sample Size
• In order to calculate the required sample size we need to know:– Trial design
– Primary outcome and how it will be analysed
– Hypothesis being tested – superiority/non-inferiority/equivalence
– Significance level (generally set to 5%)
– Power (generally 80-90%.)
![Page 22: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/22.jpg)
Additional information
– What is expected on the standard arm (based on previous trials or experience)
– Size of a clinically relevant difference
– Expected dropout/loss to follow-up rate
![Page 23: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/23.jpg)
Primary outcome
• Ideally only one• Must be pre-defined• Validated• Most clinically relevant outcome • Able to provide convincing evidence to answer the
primary objective
![Page 24: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/24.jpg)
Types of primary outcomes
• Binary – yes/no, – E.g. toxicity suffered or response to intervention
• Continuous– E.g. Quality of life scores from the SF-12
• Time to event – E.g. Overall survival or time to hospital discharged
![Page 25: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/25.jpg)
Difference/Superiority trials
• To determine the effectiveness of a new intervention relative to a control
• Need to know what is a clinically relevant improvement/difference
• If no statistically significant difference, then cannot conclude interventions are equivalent only that NOT sufficient evidence to prove a difference
“Absence of evidence is not evidence of absence”
![Page 26: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/26.jpg)
Non-inferiority trials
• To determine if new intervention is no worse (by a certain amount) than control but maybe associated with e.g. less severe toxicity or better quality of life
• Need to know the largest difference to be judged clinically acceptable
• One sided test as only looking at whether no worse• Generally need more patients than superiority trials
![Page 27: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/27.jpg)
Equivalence trials
• Trying to show that a new intervention only differs by a clinically unimportant difference
• Need to know the size of equivalence margin, i.e. a difference which is clinically unimportant
• Two sided test• Generally need more patients than superiority trials
![Page 28: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/28.jpg)
Errors
• Type I error () – chance of detecting a difference when there is none
• Type II error ( ) – chance of failing to detect a difference when it DOES exists
• Power (1-) – chance of detecting the difference if it DOES exists
From data analysis
Truth
No difference There is a difference
Fail to detect a difference Correct decision Type II error (b)
Detected a difference Type I error (a) Correct decision1-b Power
![Page 29: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/29.jpg)
Sample size calculations
Important to • Allow for dropouts/loss to follow-up in calculations• Investigate sensitivity of sample size estimates to
deviations from assumptions used• Be able to answer secondary outcomes
![Page 30: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/30.jpg)
• Proportion of successes on control arm (P0)
• Difference you want to detect or the proportion (P1) expected with the new intervention
• E.g. With 5% two-sided significance level (=0.05 ),80% power for standard 2-arm parallel group trial with a 1:1 allocation and P0 = 0.25 and P1 = 0.40, i.e. 15% absolute differences in proportion of responsesSample size required is 152 patients in each arm giving a minimum total of 304 patients
Sample size for binary outcome
![Page 31: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/31.jpg)
Changes to the assumptions
• Numbers given are for the sample size in each arm
Difference P0 P1 Power
80% 90% 95%
15% 0.25 0.40 152 203 251
10% 0.25 0.35 329 440 543
5% 0.25 0.30 1251 1674 2070
15% 0.30 0.45 163 217 268
10% 0.30 0.40 356 477 589
5% 0.30 0.35 1377 1843 2278
![Page 32: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/32.jpg)
• Difference you want to detect in the means of the outcome and Standard deviation (SD)
• E.g. In MINT, – 1% two-sided significance level (=0.01 ),– 90% Power– To detect a difference of 3 points on the Neck
Disability Index with an estimated SD of 8 Sample size required is 211 patients in each arm
giving a minimum total of 422 patients
Sample size for continuous outcome
![Page 33: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/33.jpg)
Sample size for continuous outcome
• Or standardised difference/effect size – difference in means divided by the SD– small (0.2), median (0.3) or large (0.6)
Standardised difference
80% 90%
0.2 394 5270.3 176 2350.4 100 1330.5 64 860.6 45 60
![Page 34: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/34.jpg)
Sample size for time to event outcome• Survival rate at a particular time-point, e.g. 2 years,
or the median survival time (time at which 50% of patients have experienced the event) for those on the control arm
• Difference wanting to detect• Also can depend on
– Expected recruitment rate
– Duration of follow-up after recruitment closes
![Page 35: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/35.jpg)
Sample size for time to event outcome (2)E.g. In COUGAR-2, • To detect a 2 month improvement in median overall
survival from 4 months on the control arm to 6 months on the intervention arm
• 5% two-sided significance level (=0.05 ),90% Power
• 2 year recruitment period with analysis 6 months after completed recruitmentSample size required is 146 patients in each arm giving a minimum total of 292 patients
![Page 36: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/36.jpg)
Sample size for cluster trials
• Need to inflate sample size for the primary outcome to take into account clustering by the Design effect
– n = average number to be recruited per cluster– ρ = Intracluster correlation coefficient (ICC)
• Statistical measure for the level of between cluster variation
• Values between 0 and 1 (higher values represent greater between cluster variability)
• 0.05 is often used
![Page 37: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/37.jpg)
Sample size for cluster trials (2)
• Number of clusters– Better to have more clusters than a large number of
patients in fewer clusters– Even if the overall number of patients is large if the
number of clusters is inadequate the trial will be underpowered• E.g. Trials with 2 clusters equivalent to a trial with 2
patients– Absolute minimum of 4 clusters per arm
![Page 38: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/38.jpg)
• E.g. In OPERA – Clusters: Residential and nursing accommodation (RNH)– Control = depression awareness programme
versus Intervention = exercise programme– Primary Outcome = Proportion of residents depressed at end
of trial (Geriatric depression scale 15 score <5)– Clinical important benefit = 15% reduction from 40% for
controls to 25% for the intervention– 80% power and 5% significance level – Allocation of 1.5 controls to 1 intervention
A total sample size of 343 is needed for a patient randomised trial
Sample size for cluster trials (3) Example
![Page 39: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/39.jpg)
Sample size for cluster trials (4)
• To adjust for clustering – ICC=0.05 – Average cluster size =15Design effect = 1.7So Total sample size = 343 * 1.7 = 583
• A total sample size of 583 patients with assessments at end of the trial is needed (233 in intervention arm and 350 in control arm)
• At least 39 clusters (RNH) are required (=583/15)
![Page 40: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/40.jpg)
• Effect of varying ICC and number of patients per cluster
Sample size for cluster trials (5)
Total number without clustering
ICC Number per cluster
Design effect
Total with clustering
Number of clusters
343 0.05 15 1.7 583 39
343 0.05 50 2.45 840 17
343 0.15 15 2.1 720 48
343 0.15 50 7.35 2521 50
![Page 41: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/41.jpg)
Summary
• Phase of clinical trial depends on evidence already acquired
• Phase III trials = RCT• Trial design depends on
– Outcome and questions to be answered– Number and type of Interventions being compared– Unit of allocation and controlling of bias
• Bigger sample sizes gives us increased confidence in our results (but there is a limit!)
• Always consult a statistician!!!
![Page 42: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/42.jpg)
Quiz
![Page 43: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/43.jpg)
How many arms can a parallel group trial have?
0%
43%
0%
57%1. 12. 23. 34. 2 or more
![Page 44: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/44.jpg)
Which type of trial would you need a washout period incorporated into the design?
0%7%
93%
0%
1. Parallel design2. Factorial design3. Cross-over design4. Cluster design
![Page 45: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/45.jpg)
Which type of trial is likely to require the largest sample size?
Phas
e I t
rial
Phas
e II
trial
Phas
e III
(par
...
Phas
e III
(clu
...
4%
70%
26%
0%
1. Phase I trial2. Phase II trial3. Phase III (parallel) trial4. Phase III (cluster) trial
![Page 46: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/46.jpg)
What is the generally acceptedminimum level of power for a trial?
60%
70%
80%
90%
4% 7%
89%
0%
1. 60%2. 70%3. 80%4. 90%
![Page 47: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/47.jpg)
Which of the following information do you NOT need to calculate the sample size for a binary outcome?
33%
17%
8%
42%
1. Clinically relevant difference2. Standard deviation3. Power4. Significance level
![Page 48: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/48.jpg)
If sample size to detect 10% differences in a binary outcome with 5% significance level and 80% power is not obtainable, how could you decrease the sample size required?
Det
ect d
iffer
e...
Use
85%
pow
er
Use
1%
sig
nifi...
All
of the
abo...
None
of above
46%
0%
19%
31%
4%
1. Detect differences of 15%2. Use 85% power3. Use 1% significance level4. All of the above5. None of above
![Page 49: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/49.jpg)
If wanting to detect with a continuous outcome: Trial A: difference in means of 5, SD = 10 Trial B: difference in means of 4, SD = 8 Trial C: Standardised difference of 0.5would you need ... ?
More
pat
ients
...
More
pat
ients
...
More
pat
ients
...
Sam
e fo
r all
t...
9%
50%
18%23%
1. More patients with Trial A2. More patients with Trial B3. More patients with Trial C4. Same for all trials
![Page 50: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/50.jpg)
References
• The handbook of clinical trials and other research Ed. Alan Earl-Slater
• Sample size tables for clinical studies. Machin, Campbell, Fayer and Pinol.
![Page 51: Statistics for the Terrified: Trial Design and Sample Size](https://reader033.vdocuments.site/reader033/viewer/2022061602/568139d1550346895da18324/html5/thumbnails/51.jpg)
Questions?