réalisation d’une méta-analyse · 2016. 4. 15. · •low power to detect heterogeneity...

Haguet Hélène

Meta-analysis: Methodology

Example: Assessment of cardiovascular safety profile of new generation BCR-ABL TKIs in patients with CML

04/03/2016

Meta-analysis

= statistical combination of results from two or more

separate studies to answer a common question

• Compute effect size + variance for each study

• Assign weights based on study variance

• Compute the weighted mean

Assign weight depending of the study precision

2

Problem formulation

Literature search

Study selection

Data collection

Statistical analysis

Reporting the results

Design the protocol

PRISMA protocols

• Rationale – potential interest

• Objectives: define the question: PICO

• Participants (diseases, conditions)

• Interventions (treated arm)

• Comparators (control arm)

• Outcomes of interest

• Eligibility criteria: PICO + study design

• Search strategy

• Data collection

• Statistical analysis (model)

3

Problem formulation

Literature search

Study selection

Data collection



Shamseer L, et al. Bmj. 2015;349:g7647.

Random or fixed-effect model?

• Fixed-effect model

• Estimates a single effect that is assumed to be common to

every study

• Random-effects model

• Allow that the true effect size may vary from study to study

• Observed variance = within-studies + between-studies

variance

both used in weights assignment

4

Problem formulation

Literature search

Study selection

Data collection



Random or fixed-effect model?

5

Problem formulation

Literature search

Study selection

Data collection



Borenstein M, et al. 2010;1(2):97-111.

Fixed-effect model Random-effects model

Search strategy

• Databases

• Bibliographic databases

E.g. PubMed, Scopus, Cochrane Library, EMBASE

• Abstracts from international congresses

• Clinical trial registers

E.g. www.clinicaltrials.gov, WHO international clinical trial register, EudraCT

• Keywords and Boolean operators

6

Problem formulation

Literature search

Study selection

Data collection



http://www.clinicaltrials.gov/

Search strategy

• Example

#1: ponatinib [Title] (69)

#2: AP24534 [Title] (8)

#3: ((#1) OR #2) (71)

#4: (#3) and "randomized controlled trial" (1)

#5: (#3) and "randomized trial" (0)

#6: (#3) and "randomized clinical trial" (0)

#7: (#3) and "randomised controlled trial" (0)

#8: (#3) and "randomised trial" (0)

#9: (#3) and "randomised clinical trial" (0)

#10: (((((#4) OR #5) OR #6) OR #7) OR #8) OR #9 (1)

7

Problem formulation

Literature search

Study selection

Data collection



Study selection

• Removal of duplicates

• In 2 stages

• Screening of abstracts and titles independently by 2

reviewers (+ discussion with 3rd reviewers)

• Selection of included studies based on the entirety of

the paper

• Process described in flow diagram

PRISMA statement

8

Problem formulation

Literature search

Study selection

Data collection



PRISMA flow chart

9

Moher D, et al. Bmj. 2009;339:b2535.

Problem formulation

Literature search

Study selection

Data collection



Quality assessment

• Use quality scoring system

• By 2 reviewers independently

• Exclude low quality studies

E.g JADAD score, Chalmers scale

10

Problem formulation

Literature search

Study selection

Data collection



Data collection

• Extracted by 2 independent reviewers

• Standardized data extraction form

E.g. study characteristics, study design, population, outcomes

11

Problem formulation

Literature search

Study selection

Data collection



Data collection

• Selection of data by the 2 reviewers

12

Problem formulation

Literature search

Study selection

Data collection




• Direction and size of the effect?

Effect size measure + 95%CI

• Is the effect consistent across studies?

Heterogeneity assessment

• What is the strength of evidence for the effect?

Quality assessment

13

Problem formulation

Literature search

Study selection

Data collection



Effect size measures

= summarises the observed intervention effect

• Depend of the type of data

• Means (e.g. improvement of blood pressure)

• Binary data (e.g. survival)

• Correlational data

• Binary data

• Risk ratio

• Odds ratio (often the best choice)

• Risk difference

14

Problem formulation

Literature search

Study selection

Data collection




15

Problem formulation

Literature search

Study selection

Data collection



• Comprehensive Meta-Analysis software version 2.2.046


• Cochran’s Q statistics

• Reported with p value

• Low power to detect heterogeneity

• Common use of 0.10 as cut-off value for significance

• I2 statistic

• I2: % of observed total variation across studies that is due to

real heterogeneity rather than chance

• <25%: low heterogeneity

• 25-50%: moderate heterogeneity

• 50-75%: high heterogeneity

• Less dependent of the number of studies

16

Problem formulation

Literature search

Study selection

Data collection




• No heterogeneity: similar to fixed-effect model

• In case of heterogeneity:

• Explore the causes:

• Subgroup analysis (!! False negative and positive)

E.g. Treatments, population characteristics …

• Meta-regression

• Check data and effect size measure

17

Problem formulation

Literature search

Study selection

Data collection



Assessing risk of bias

• Publication bias

• Funnel plots

• Assessment of the funnel plot asymmetry

• Egger’s linear regression test

• Begg and Mazumbar rank correlation test

• Fail-safe number

= how many new studies averaging a null result are required to bring

the overall treatment effect to nonsignificant

18

Problem formulation

Literature search

Study selection

Data collection



Study name Treatment Statistics with study removed Peto odds ratio (95%

CI) with study removedLower Upper

Point limit limit p-Value

NCT01650805 EPIC Ponatinib 3,410 2,317 5,020 0,000NCT00574873 BELA Bosutinib 3,443 2,382 4,977 0,000NCT00471497 ENESTnd Nilotinib 3,518 2,140 5,785 0,000NCT00760877 ENESTcmr Nilotinib 3,335 2,293 4,852 0,000NCT01275196 ENESTchina Nilotinib 3,395 2,360 4,885 0,000NCT00802841 LASOR Nilotinib 3,303 2,281 4,782 0,000NCT00852566 NordCML006 Dasatinib 3,392 2,358 4,881 0,000NCT00070499 Dasatinib 3,395 2,360 4,884 0,000NCT00481247 DASISION Dasatinib 3,321 2,278 4,840 0,000NCT00103844 START-R Dasatinib 3,404 2,363 4,903 0,000NCT00320190 Dasatinib 3,524 2,449 5,070 0,000NCT01460693 SPIRIT2 Dasatinib 3,645 2,465 5,390 0,000

3,418 2,379 4,909 0,000

0,01 0,1 1 10 100Imatinib New generation TKI

Vascular occlusive events

One-way sensitivity analysis

Aim: explore the robustness of the findings

• Remove one single study at a time

19

Problem formulation

Literature search

Study selection

Data collection



Douxfils J, et al. JAMA Oncol. 2016.

Forest plot

20

Problem formulation

Literature search

Study selection

Data collection



Douxfils J, et al. Journal of the American Heart Association. 2014;3(3):e000515.

Guidelines: PRISMA checklist

21

Moher D, et al. Bmj. 2009;339:b2535.

Guidelines: PRISMA checklist

22

Moher D, et al. Bmj. 2009;339:b2535.

Criticisms

• 1 number cannot summarize a research field

Assessment of heterogeneity and dispersion between studies

• Publication bias overestimation of the true effect size

Assessed by funnel plot + asymmetry tests

• “Mixing apples and oranges”

If heterogeneity between studies, it should be investigated

23

Be clear and transparent

Resources

• Borenstein M, et al. Introduction to Meta-Analysis 2009.

• Cooper H, et al. The Handbook of Research Synthesis and Meta-Analysis,

2nd Edition 2009.

• Higgins J, et al, Cochrane Handbook for Systematic Reviews of

Interventions

24

1


•Ponatinib: arterial and venous occlusive events identified1

• Temporal suspension of ponatinib marketing (FDA

decision)

• Early abortion of the phase III clinical trial (EPIC trial)

• Risk minimization measures2

1Giles, F. J. et al. Leukemia 27(6): 1310-1315. 2EMA. European Medicines Agency recommends changes in use of leukaemia medicine Iclusig (ponatinib) in order to minimise risk of blood clots


•Dasatinib: no vascular occlusive events3

•Bosutinib: no vascular occlusive events4

•Nilotinib: serious cases of PAOD5,6

• Cardiac and arterial occlusive events included in

labeling information

2

3Food and Drug Administration. “Label information – SPRYCEL.” 4Food and Drug Administration. “Label information BOSULIF.”

5Food and Drug Administration. "Label information - TASIGNA." 6Quintas-Cardama A., et al. Clin Lymphoma Myeloma Leuk 12(5): 337-340.

Rationales

3

7Giles, F. J. et al. (2013). Leukemia 27(6): 1310-1315.

• TKIs may alter other tyrosine kinases

class effect?7

• 2nd generation TKIs have demonstrated

higher efficacy on surrogate outcomes in the

treatment of CML

Question

Risk of CV occlusive events

associated to new generation

BCR-ABL TKIs in CML

compared with imatinib?

4

Problem formulation

Literature search

Study selection

Data collection



Systematic review and Meta-analysis

5

Protocol: online registration: PROSPERO 2014: CDR42014014147

Literature search:

• Scientific articles: Pubmed, scopus and Cochrane library

• Congress abstracts: ASH, ASCO, ESMO

• Clinical trial register: www.clinicaltrials.gov

Study selection:

• Randomized clinical trials

• Comparing new generation TKIs vs imatinib

• Patients with CML

Problem formulation

Literature search

Study selection

Data collection



http://www.clinicaltrials.gov/


6

Data collection:

• Vascular occlusive events (+ OS and MMR)

• Arterial occlusive events, venous occlusive events

• Study characteristics, population characteristics, JADAD scale

Statistical analysis:

• Random-effects model

Exception: fixed-effect model for venous occlusive events

• Effect size measure: Odds ratio using Peto method

Problem formulation

Literature search

Study selection

Data collection



Peto odds ratio

Odds Ratio

𝑂𝑅 =𝐴𝐷

𝐵𝐶

Peto Odds Ratio

7

Brockhaus AC, et al. Statistics in medicine. 2014;33(28):4861-74.


8

Statistical analysis:

• Stratification

• By treatment

• Heterogeneity

• Q statistic

• I2 value

• Publication bias: funnel plots

• Robustness: 1-way sensitivity analysis

Problem formulation

Literature search

Study selection

Data collection



Vascular occlusive events8

10

Treatment Group by

Comparison

Study name Statistics for each study Peto odds ratio and 95% CI

Peto Lower Upper Relative

odds ratio limit limit p-Value weight

Bosutinib Bosutinib NCT00574873 BELA 2,768 0,388 19,769 0,310 100,00

Bosutinib 2,768 0,388 19,769 0,310

Dasatinib Dasatinib NCT00852566 NordCML006 8,092 0,160 409,341 0,296 3,30

Dasatinib Dasatinib NCT00070499 7,389 0,147 372,385 0,317 3,31

Dasatinib Dasatinib NCT00481247 DASISION 4,855 1,301 18,120 0,019 29,30

Dasatinib Dasatinib NCT00103844 START-R 4,460 0,230 86,505 0,323 5,78


Dasatinib Dasatinib NCT01460693 SPIRIT2 2,317 0,887 6,052 0,086 55,12

Dasatinib 2,913 1,428 5,942 0,003

Nilotinib Nilotinib NCT00471497 ENESTnd 3,307 1,949 5,611 0,000 80,15

Nilotinib Nilotinib NCT00760877 ENESTcmr 4,826 1,178 19,777 0,029 11,26

Nilotinib Nilotinib NCT01275196 ENESTchina 7,334 0,146 369,606 0,319 1,46

Nilotinib Nilotinib NCT00802841 LASOR 7,475 1,270 43,982 0,026 7,13

Nilotinib 3,700 2,305 5,940 0,000

Ponatinib Ponatinib NCT01650805 EPIC 3,470 1,231 9,779 0,019 100,00

Ponatinib 3,470 1,231 9,779 0,019

Overall 3,418 2,379 4,909 0,000

0,01 0,1 1 10 100

Imatinib New generation TKI


8Douxfils J, et al. JAMA Oncol. 2016.


10

Treatment Group by

Comparison





Bosutinib 2,768 0,388 19,769 0,310







Dasatinib 2,913 1,428 5,942 0,003





Nilotinib 3,700 2,305 5,940 0,000


Ponatinib 3,470 1,231 9,779 0,019

Overall 3,418 2,379 4,909 0,000

0,01 0,1 1 10 100



Bosutinib: 3 events/248 pts Imatinib: 1 event/251 pts



10

Treatment Group by

Comparison





Bosutinib 2,768 0,388 19,769 0,310







Dasatinib 2,913 1,428 5,942 0,003





Nilotinib 3,700 2,305 5,940 0,000


Ponatinib 3,470 1,231 9,779 0,019

Overall 3,418 2,379 4,909 0,000

0,01 0,1 1 10 100



(overall)

(overall)

(overall)


Limitations

• Lack of individual data

• Time to event

• Number of deaths due to VOE

• Lack of homogeneity in evaluation criteria (VOE)

between studies8,9

13

8Yang EH, et al. Future Oncol. 11(14):1995-8. 9Groarke JD, et al. The New England journal of medicine. 369(19):1779-81.

Strengths

• Consistent with the signal

• Robustness confirmed by the sensitivity analysis

•No heterogeneity between studies

•No evidence of publication bias

• Published & unpublished studies included

14

Funnel plot

-3 -2 -1 0 1 2 3

0

1

2

3

Sta

nd

ard

Erro

r

Log Peto odds ratio

Funnel Plot of Standard Error by Log Peto odds ratio

15

6th NTHC symposium

18

Acknowledgments

•Douxfils Jonathan

•Mullier François

• Chatelain Christian

•Graux Carlos

•Dogné Jean-Michel

42

Thank you for your attention!

PRISMA protocol checklist

Shamseer L, et al. Bmj. 2015;349:g7647.

FEM vs REM

• Both: weights depend of the precision of the estimation

(= of the overall study error variance)

• Difference REM vs FEM: definition of this variance

Fixed-effect model

Only 1 source of variation: the

estimation error εi

= difference between common true mean

and observed mean

Within-study variance depends of:

- The variance of individual observation

- The size of the sample

Random-effects model

2 sources of variation

- The estimation error within study εi

- The estimation error between study ξi

Overall study error variance: combination

of the variance of these 2 parameters

Borenstein M, et al. 2010;1(2):97-111.

FEM vs REM

Fixed-effect model Random-effect model

Circle: true mean Square: observed mean (differs from the true mean because of estimation error)

Adapted from Borenstein M, et al. 2010;1(2):97-111 Adapted from Borenstein M, et al. 2010;1(2):97-111

ε1 ε1

ξ1

Effect size: means

• Raw mean difference

• Only if studies used the same scale

• Standardized mean difference

• In case of different evaluation of the outcome (the scale of

measurement differed)

• Division of the mean difference in each study by study’s standard

deviation

• Response ratios

• When the measure is unlikely to be 0, but has a natural 0 point

(e.g. length)

• …

Odds ratio vs risk ratio

Holcomb WL, Jr., et al. Obstetrics and gynecology. 2001;98(4):685-8.

• Divergence large when the outcome is common

Avoid quantitative statements about OR

Meta-analysis of observational studies

• Controversial: potential biases

• Diversity of study designs and populations

• Publication bias (could have particular impact)

• Recommendations:

• Use broad inclusion criteria

• Perform analysis relating suspected source of bias and

variability

• Investigate heterogeneity

50

Stroup DF, et al. JAMA. 2000;283(15):2008-12.

Meta-analysis of single-arm clinical trials

• Controversial: lack of control: effect of site-specific

variables

• Interrupted-time series (ITS) study

For studies with multiple time-points before and after an

intervention (at least 3 data points before and 3 after the

intervention)

• Repeated measures study: if the measures are repeated in

the same individuals

51

Effective Practice and Organisation of Care (EPOC). 2013. Available at: http://epoc.cochrane.org/epoc-specific-resources-review-authors

52

Effective Practice and Organisation of Care (EPOC). 2013. Available at: http://epoc.cochrane.org/epoc-specific-resources-review-authors

réalisation d’une méta-analyse · 2016. 4. 15. · •low power to detect heterogeneity...

Documents