réalisation d’une méta-analyse - ucl mont-godinne...2016/03/04 · example: assessment of...
TRANSCRIPT
Haguet Hélène
Meta-analysis: Methodology
Example: Assessment of cardiovascular safety profile of new generation BCR-ABL TKIs in patients with CML
04/03/2016
Meta-analysis
= statistical combination of results from two or more
separate studies to answer a common question
• Compute effect size + variance for each study
• Assign weights based on study variance
• Compute the weighted mean
Assign weight depending of the study precision
2
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Design the protocol
PRISMA protocols
• Rationale – potential interest
• Objectives: define the question: PICO
• Participants (diseases, conditions)
• Interventions (treated arm)
• Comparators (control arm)
• Outcomes of interest
• Eligibility criteria: PICO + study design
• Search strategy
• Data collection
• Statistical analysis (model)
3
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Shamseer L, et al. Bmj. 2015;349:g7647.
Random or fixed-effect model?
• Fixed-effect model
• Estimates a single effect that is assumed to be common to
every study
• Random-effects model
• Allow that the true effect size may vary from study to study
• Observed variance = within-studies + between-studies
variance
both used in weights assignment
4
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Random or fixed-effect model?
5
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Borenstein M, et al. 2010;1(2):97-111.
Fixed-effect model Random-effects model
Search strategy
• Databases
• Bibliographic databases
E.g. PubMed, Scopus, Cochrane Library, EMBASE
• Abstracts from international congresses
• Clinical trial registers
E.g. www.clinicaltrials.gov, WHO international clinical trial register, EudraCT
• Keywords and Boolean operators
6
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Search strategy
• Example
#1: ponatinib [Title] (69)
#2: AP24534 [Title] (8)
#3: ((#1) OR #2) (71)
#4: (#3) and "randomized controlled trial" (1)
#5: (#3) and "randomized trial" (0)
#6: (#3) and "randomized clinical trial" (0)
#7: (#3) and "randomised controlled trial" (0)
#8: (#3) and "randomised trial" (0)
#9: (#3) and "randomised clinical trial" (0)
#10: (((((#4) OR #5) OR #6) OR #7) OR #8) OR #9 (1)
7
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Study selection
• Removal of duplicates
• In 2 stages
• Screening of abstracts and titles independently by 2
reviewers (+ discussion with 3rd reviewers)
• Selection of included studies based on the entirety of
the paper
• Process described in flow diagram
PRISMA statement
8
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
PRISMA flow chart
9
Moher D, et al. Bmj. 2009;339:b2535.
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Quality assessment
• Use quality scoring system
• By 2 reviewers independently
• Exclude low quality studies
E.g JADAD score, Chalmers scale
10
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Data collection
• Extracted by 2 independent reviewers
• Standardized data extraction form
E.g. study characteristics, study design, population, outcomes
11
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Data collection
• Selection of data by the 2 reviewers
12
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Statistical analysis
• Direction and size of the effect?
Effect size measure + 95%CI
• Is the effect consistent across studies?
Heterogeneity assessment
• What is the strength of evidence for the effect?
Quality assessment
13
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Effect size measures
= summarises the observed intervention effect
• Depend of the type of data
• Means (e.g. improvement of blood pressure)
• Binary data (e.g. survival)
• Correlational data
• Binary data
• Risk ratio
• Odds ratio (often the best choice)
• Risk difference
14
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Statistical analysis
15
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
• Comprehensive Meta-Analysis software version 2.2.046
Heterogeneity assessment
• Cochran’s Q statistics
• Reported with p value
• Low power to detect heterogeneity
• Common use of 0.10 as cut-off value for significance
• I2 statistic
• I2: % of observed total variation across studies that is due to
real heterogeneity rather than chance
• <25%: low heterogeneity
• 25-50%: moderate heterogeneity
• 50-75%: high heterogeneity
• Less dependent of the number of studies
16
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Heterogeneity assessment
• No heterogeneity: similar to fixed-effect model
• In case of heterogeneity:
• Explore the causes:
• Subgroup analysis (!! False negative and positive)
E.g. Treatments, population characteristics …
• Meta-regression
• Check data and effect size measure
17
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Assessing risk of bias
• Publication bias
• Funnel plots
• Assessment of the funnel plot asymmetry
• Egger’s linear regression test
• Begg and Mazumbar rank correlation test
• Fail-safe number
= how many new studies averaging a null result are required to bring
the overall treatment effect to nonsignificant
18
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Study name Treatment Statistics with study removed Peto odds ratio (95%
CI) with study removedLower Upper
Point limit limit p-Value
NCT01650805 EPIC Ponatinib 3,410 2,317 5,020 0,000NCT00574873 BELA Bosutinib 3,443 2,382 4,977 0,000NCT00471497 ENESTnd Nilotinib 3,518 2,140 5,785 0,000NCT00760877 ENESTcmr Nilotinib 3,335 2,293 4,852 0,000NCT01275196 ENESTchina Nilotinib 3,395 2,360 4,885 0,000NCT00802841 LASOR Nilotinib 3,303 2,281 4,782 0,000NCT00852566 NordCML006 Dasatinib 3,392 2,358 4,881 0,000NCT00070499 Dasatinib 3,395 2,360 4,884 0,000NCT00481247 DASISION Dasatinib 3,321 2,278 4,840 0,000NCT00103844 START-R Dasatinib 3,404 2,363 4,903 0,000NCT00320190 Dasatinib 3,524 2,449 5,070 0,000NCT01460693 SPIRIT2 Dasatinib 3,645 2,465 5,390 0,000
3,418 2,379 4,909 0,000
0,01 0,1 1 10 100Imatinib New generation TKI
Vascular occlusive events
One-way sensitivity analysis
Aim: explore the robustness of the findings
• Remove one single study at a time
19
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Douxfils J, et al. JAMA Oncol. 2016.
Forest plot
20
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Douxfils J, et al. Journal of the American Heart Association. 2014;3(3):e000515.
Criticisms
• 1 number cannot summarize a research field
Assessment of heterogeneity and dispersion between studies
• Publication bias overestimation of the true effect size
Assessed by funnel plot + asymmetry tests
• “Mixing apples and oranges”
If heterogeneity between studies, it should be investigated
23
Be clear and transparent
Resources
• Borenstein M, et al. Introduction to Meta-Analysis 2009.
• Cooper H, et al. The Handbook of Research Synthesis and Meta-Analysis,
2nd Edition 2009.
• Higgins J, et al, Cochrane Handbook for Systematic Reviews of
Interventions
24
1
Vascular occlusive events
•Ponatinib: arterial and venous occlusive events identified1
• Temporal suspension of ponatinib marketing (FDA
decision)
• Early abortion of the phase III clinical trial (EPIC trial)
• Risk minimization measures2
1Giles, F. J. et al. Leukemia 27(6): 1310-1315. 2EMA. European Medicines Agency recommends changes in use of leukaemia medicine Iclusig (ponatinib) in order to minimise risk of blood clots
Vascular occlusive events
•Dasatinib: no vascular occlusive events3
•Bosutinib: no vascular occlusive events4
•Nilotinib: serious cases of PAOD5,6
• Cardiac and arterial occlusive events included in
labeling information
2
3Food and Drug Administration. “Label information – SPRYCEL.” 4Food and Drug Administration. “Label information BOSULIF.”
5Food and Drug Administration. "Label information - TASIGNA." 6Quintas-Cardama A., et al. Clin Lymphoma Myeloma Leuk 12(5): 337-340.
Rationales
3
7Giles, F. J. et al. (2013). Leukemia 27(6): 1310-1315.
• TKIs may alter other tyrosine kinases
class effect?7
• 2nd generation TKIs have demonstrated
higher efficacy on surrogate outcomes in the
treatment of CML
Question
Risk of CV occlusive events
associated to new generation
BCR-ABL TKIs in CML
compared with imatinib?
4
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Systematic review and Meta-analysis
5
Protocol: online registration: PROSPERO 2014: CDR42014014147
Literature search:
• Scientific articles: Pubmed, scopus and Cochrane library
• Congress abstracts: ASH, ASCO, ESMO
• Clinical trial register: www.clinicaltrials.gov
Study selection:
• Randomized clinical trials
• Comparing new generation TKIs vs imatinib
• Patients with CML
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Systematic review and Meta-analysis
6
Data collection:
• Vascular occlusive events (+ OS and MMR)
• Arterial occlusive events, venous occlusive events
• Study characteristics, population characteristics, JADAD scale
Statistical analysis:
• Random-effects model
Exception: fixed-effect model for venous occlusive events
• Effect size measure: Odds ratio using Peto method
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Peto odds ratio
Odds Ratio
𝑂𝑅 =𝐴𝐷
𝐵𝐶
Peto Odds Ratio
7
Brockhaus AC, et al. Statistics in medicine. 2014;33(28):4861-74.
Systematic review and Meta-analysis
8
Statistical analysis:
• Stratification
• By treatment
• Heterogeneity
• Q statistic
• I2 value
• Publication bias: funnel plots
• Robustness: 1-way sensitivity analysis
Problem formulation
Literature search
Study selection
Data collection
Statistical analysis
Reporting the results
Vascular occlusive events8
10
Treatment Group by
Comparison
Study name Statistics for each study Peto odds ratio and 95% CI
Peto Lower Upper Relative
odds ratio limit limit p-Value weight
Bosutinib Bosutinib NCT00574873 BELA 2,768 0,388 19,769 0,310 100,00
Bosutinib 2,768 0,388 19,769 0,310
Dasatinib Dasatinib NCT00852566 NordCML006 8,092 0,160 409,341 0,296 3,30
Dasatinib Dasatinib NCT00070499 7,389 0,147 372,385 0,317 3,31
Dasatinib Dasatinib NCT00481247 DASISION 4,855 1,301 18,120 0,019 29,30
Dasatinib Dasatinib NCT00103844 START-R 4,460 0,230 86,505 0,323 5,78
Dasatinib Dasatinib NCT00320190 0,085 0,002 4,614 0,227 3,19
Dasatinib Dasatinib NCT01460693 SPIRIT2 2,317 0,887 6,052 0,086 55,12
Dasatinib 2,913 1,428 5,942 0,003
Nilotinib Nilotinib NCT00471497 ENESTnd 3,307 1,949 5,611 0,000 80,15
Nilotinib Nilotinib NCT00760877 ENESTcmr 4,826 1,178 19,777 0,029 11,26
Nilotinib Nilotinib NCT01275196 ENESTchina 7,334 0,146 369,606 0,319 1,46
Nilotinib Nilotinib NCT00802841 LASOR 7,475 1,270 43,982 0,026 7,13
Nilotinib 3,700 2,305 5,940 0,000
Ponatinib Ponatinib NCT01650805 EPIC 3,470 1,231 9,779 0,019 100,00
Ponatinib 3,470 1,231 9,779 0,019
Overall 3,418 2,379 4,909 0,000
0,01 0,1 1 10 100
Imatinib New generation TKI
Vascular occlusive events
8Douxfils J, et al. JAMA Oncol. 2016.
Vascular occlusive events8
10
Treatment Group by
Comparison
Study name Statistics for each study Peto odds ratio and 95% CI
Peto Lower Upper Relative
odds ratio limit limit p-Value weight
Bosutinib Bosutinib NCT00574873 BELA 2,768 0,388 19,769 0,310 100,00
Bosutinib 2,768 0,388 19,769 0,310
Dasatinib Dasatinib NCT00852566 NordCML006 8,092 0,160 409,341 0,296 3,30
Dasatinib Dasatinib NCT00070499 7,389 0,147 372,385 0,317 3,31
Dasatinib Dasatinib NCT00481247 DASISION 4,855 1,301 18,120 0,019 29,30
Dasatinib Dasatinib NCT00103844 START-R 4,460 0,230 86,505 0,323 5,78
Dasatinib Dasatinib NCT00320190 0,085 0,002 4,614 0,227 3,19
Dasatinib Dasatinib NCT01460693 SPIRIT2 2,317 0,887 6,052 0,086 55,12
Dasatinib 2,913 1,428 5,942 0,003
Nilotinib Nilotinib NCT00471497 ENESTnd 3,307 1,949 5,611 0,000 80,15
Nilotinib Nilotinib NCT00760877 ENESTcmr 4,826 1,178 19,777 0,029 11,26
Nilotinib Nilotinib NCT01275196 ENESTchina 7,334 0,146 369,606 0,319 1,46
Nilotinib Nilotinib NCT00802841 LASOR 7,475 1,270 43,982 0,026 7,13
Nilotinib 3,700 2,305 5,940 0,000
Ponatinib Ponatinib NCT01650805 EPIC 3,470 1,231 9,779 0,019 100,00
Ponatinib 3,470 1,231 9,779 0,019
Overall 3,418 2,379 4,909 0,000
0,01 0,1 1 10 100
Imatinib New generation TKI
Vascular occlusive events
Bosutinib: 3 events/248 pts Imatinib: 1 event/251 pts
8Douxfils J, et al. JAMA Oncol. 2016.
Vascular occlusive events8
10
Treatment Group by
Comparison
Study name Statistics for each study Peto odds ratio and 95% CI
Peto Lower Upper Relative
odds ratio limit limit p-Value weight
Bosutinib Bosutinib NCT00574873 BELA 2,768 0,388 19,769 0,310 100,00
Bosutinib 2,768 0,388 19,769 0,310
Dasatinib Dasatinib NCT00852566 NordCML006 8,092 0,160 409,341 0,296 3,30
Dasatinib Dasatinib NCT00070499 7,389 0,147 372,385 0,317 3,31
Dasatinib Dasatinib NCT00481247 DASISION 4,855 1,301 18,120 0,019 29,30
Dasatinib Dasatinib NCT00103844 START-R 4,460 0,230 86,505 0,323 5,78
Dasatinib Dasatinib NCT00320190 0,085 0,002 4,614 0,227 3,19
Dasatinib Dasatinib NCT01460693 SPIRIT2 2,317 0,887 6,052 0,086 55,12
Dasatinib 2,913 1,428 5,942 0,003
Nilotinib Nilotinib NCT00471497 ENESTnd 3,307 1,949 5,611 0,000 80,15
Nilotinib Nilotinib NCT00760877 ENESTcmr 4,826 1,178 19,777 0,029 11,26
Nilotinib Nilotinib NCT01275196 ENESTchina 7,334 0,146 369,606 0,319 1,46
Nilotinib Nilotinib NCT00802841 LASOR 7,475 1,270 43,982 0,026 7,13
Nilotinib 3,700 2,305 5,940 0,000
Ponatinib Ponatinib NCT01650805 EPIC 3,470 1,231 9,779 0,019 100,00
Ponatinib 3,470 1,231 9,779 0,019
Overall 3,418 2,379 4,909 0,000
0,01 0,1 1 10 100
Imatinib New generation TKI
Vascular occlusive events
(overall)
(overall)
(overall)
8Douxfils J, et al. JAMA Oncol. 2016.
Limitations
• Lack of individual data
• Time to event
• Number of deaths due to VOE
• Lack of homogeneity in evaluation criteria (VOE)
between studies8,9
13
8Yang EH, et al. Future Oncol. 11(14):1995-8. 9Groarke JD, et al. The New England journal of medicine. 369(19):1779-81.
Strengths
• Consistent with the signal
• Robustness confirmed by the sensitivity analysis
•No heterogeneity between studies
•No evidence of publication bias
• Published & unpublished studies included
14
Funnel plot
-3 -2 -1 0 1 2 3
0
1
2
3
Sta
nd
ard
Erro
r
Log Peto odds ratio
Funnel Plot of Standard Error by Log Peto odds ratio
15
Acknowledgments
•Douxfils Jonathan
•Mullier François
• Chatelain Christian
•Graux Carlos
•Dogné Jean-Michel
42
FEM vs REM
• Both: weights depend of the precision of the estimation
(= of the overall study error variance)
• Difference REM vs FEM: definition of this variance
Fixed-effect model
Only 1 source of variation: the
estimation error εi
= difference between common true mean
and observed mean
Within-study variance depends of:
- The variance of individual observation
- The size of the sample
Random-effects model
2 sources of variation
- The estimation error within study εi
- The estimation error between study ξi
Overall study error variance: combination
of the variance of these 2 parameters
Borenstein M, et al. 2010;1(2):97-111.
FEM vs REM
Fixed-effect model Random-effect model
Circle: true mean Square: observed mean (differs from the true mean because of estimation error)
Adapted from Borenstein M, et al. 2010;1(2):97-111 Adapted from Borenstein M, et al. 2010;1(2):97-111
ε1 ε1
ξ1
Effect size: means
• Raw mean difference
• Only if studies used the same scale
• Standardized mean difference
• In case of different evaluation of the outcome (the scale of
measurement differed)
• Division of the mean difference in each study by study’s standard
deviation
• Response ratios
• When the measure is unlikely to be 0, but has a natural 0 point
(e.g. length)
• …
Odds ratio vs risk ratio
Holcomb WL, Jr., et al. Obstetrics and gynecology. 2001;98(4):685-8.
• Divergence large when the outcome is common
Avoid quantitative statements about OR
Meta-analysis of observational studies
• Controversial: potential biases
• Diversity of study designs and populations
• Publication bias (could have particular impact)
• Recommendations:
• Use broad inclusion criteria
• Perform analysis relating suspected source of bias and
variability
• Investigate heterogeneity
50
Stroup DF, et al. JAMA. 2000;283(15):2008-12.
Meta-analysis of single-arm clinical trials
• Controversial: lack of control: effect of site-specific
variables
• Interrupted-time series (ITS) study
For studies with multiple time-points before and after an
intervention (at least 3 data points before and 3 after the
intervention)
• Repeated measures study: if the measures are repeated in
the same individuals
51
Effective Practice and Organisation of Care (EPOC). 2013. Available at: http://epoc.cochrane.org/epoc-specific-resources-review-authors