meta-analysis and systematic review gianluigi savarese, md, fesc, acc fit department of advanced...

META-ANALYSIS AND SYSTEMATIC REVIEW

Gianluigi Savarese, MD, FESC, ACC FIT

Department of Advanced Biomedical Sciences, Federico II University, Naples, ItalyDepartment of Medicine, Solna, Karolinska Institutet, Stockholm, Sweden

What is a Systematic Review?

“A review that is conducted according to clearly stated, scientific research methods, and is designed to minimize biases and errors inherent to traditional, narrative reviews.”

Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD, “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal of Hand Surgery. Vol. 31A No.10 December 2006. p.1671

What is the significance of Systematic Reviews?

• The large amount of medical literature requires clinicians and researchers alike to rely on systematic reviews in order to make an informed decision.

• Systematic Reviews minimize bias. “A systematic review is a more scientific method of summarizing literature because specific protocols are used to determine which studies will be included in the review.”

Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839

Why are Systematic Reviews Necessary?

“The volume of published material makes it impractical for an individual clinician to remain up to date on a variety of common conditions. This is further complicated when individual studies report conflicting conclusions, a problem that is prevalent when small patient samples and retrospective designs are used.”


Characteristics of Systematic Reviews

• Two possible approaches:

– or qualitative synthesis

– statistical synthesis of data (meta-analysis) if appropriate and possible

Hypothesis

A systematic review should be based on principles of hypothesis testing, and the hypotheses must be conceived a priori.

• Identify your studies• Determine eligibility of studies apriori to

avoid bias– Inclusion: which ones to keep– Exclusion: which ones to throw out

• Abstract Data from the studies• Analyze data in the studies statistically

Four steps

Literature Search

A comprehensive and reproducible literature search is the foundation of a systematic review.

• Be methodical: plan first• List of popular databases to search

– Pubmed/Medline– Embase– Cochrane Review– ISI Web of Science– SCOPUS

• Other strategies you may adopt– Trial registries (clinicaltrials.gov)– Abstracts from meetings– Hand search (go to the library...)– Personal references– References from published reviews/meta-analysis/trials– Contact experts – Web, eg. Google (http://scholar.google.com)

Literature Search

Database bias!!!

Grey litterature

Literature Search – Risk of Bias

• English-language bias - occurs when reviewers exclude papers published in languages other than English

• Citation bias - occurs when studies with significant or positive results are referenced in other publications, compared with studies with inconclusive or negative findings

• Publication Bias - selective publication of articles that show positive treatment of effects and statistical significance.

Data Collection• The list of data to be extracted should be decided a priori.

• A data extraction form should be used so that the same data are extracted from each study and missing data are clearly apparent.

• To be sure that data extraction is accurate and reproducible, it should be performed by at least two independent readers.

• Disagreement between readers could be solved by agreements or by a third reviewer


Data Collection

Collected data includes:– Study characteristics (year and journal of publication,

number of patients in each arm, treatments performed, duration of follow-up)

– Sample demographics (age, % males or females)

– Sample characteristics (traditional CV risk factors - % hypertensive pts, % diabetic pts, % dyslipidemic pts, % smokers – concomitant treatments, comorbidities, etc)

– Outcome data (all-cause death, CV death, MI, stroke, etc)

Quality Assessment

The validity of a systematic review ultimately depends on the scientific method of the retrieved studies and the reporting of data.”


GRADE

Grading of Recommendations Assessment, Development and Evaluation

Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008 May 3;336(7651):995-8.

Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008 May 3;336(7651):995-8.

Data Synthesis

Data could be summarized quantitatively if study designs are not too different in: •outcome definition (composite outcome?); •population sizes•population characteristics•interventions

HETEROGENEITY

Data Synthesis

Once the data have been extracted and their quality and validity assessed, the outcomes of individual studies within a systematic review may be pooled and presented as summary outcome or effect

META-ANALYSIS

What is meta analysis?

Quantitative approach for systematically combining results of previous research to

arrive at conclusions about the body of research.

What does it mean?

• Quantitative : numbers• Systematic : methodical• combining: putting together• previous research: what's already done• conclusions: new knowledge

Meta-analysis: Statistical Models

• There are 2 statistical models used in a meta-analysis: – Fixed effects:1.Effect of treatment is the same for every study;2.Low heterogeneity

– Random effects:1.True effect estimate for each study varies;2.High heterogeneity3.Provide larger CI

Heterogeneity• Clinical heterogeneity: variability in the participants,

interventions and outcomes studied+

• Methodological heterogeneity: variability in study design

• Statistical heterogeneity: Variability in the intervention effects being evaluated in the different studies. It is a consequence of clinical or methodological diversity, or both, among the studies.

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

Heterogeneity assessment• If confidence intervals for the results of individual studies (generally

depicted graphically using horizontal lines) have poor overlap, this generally indicates the presence of statistical heterogeneity.

• Cochrane Q statistic: It is calculated as the weighted sum of squared differences between individual study effects and the pooled effect across studies, with the weights being those used in the pooling method. A p value decided apriori defines the presence of significant heterogeneity.

• I2 statistic: It describes the percentage of variation across studies that is due to heterogeneity rather than chance.

• 0% to 40%: heterogeneity might not be important;• 30% to 60%: may represent moderate heterogeneity;• 50% to 90%: may represent substantial heterogeneity;• 75% to 100%: considerable heterogeneity.


Strategies for addressing heterogeneity

• Check again that the data are correct• Do not do a meta-analysis• Explore heterogeneity (subgroup analysis, meta-

regression)• Ignore heterogeneity (there is no an intervention

effect but a distribution of intervention effects)• Perform a random-effects meta-analysis (when

heterogeneity cannot be explained)• Change the effect measure (different scales in different

studies)• Exclude studies (outlying studies)


Sensitivity analysis

• One study removed meta-analysis• Meta-regression analysis

Publication Bias

Publication bias arises when trials with statistically significant results are more likely to be published and cited, and are preferentially published in English language journals and those indexed in Medline

Jüni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. International Journal of Epidemiology 2001;31:115-123.

Publication Bias

• A funnel plot is a simple scatter plot of the intervention effect estimates (OR, logOR) from individual studies against some measure of each study’s size or precision (standard error, 1/standard error, sample size, 1/sample size, log(sample size), log(1/sample size), Mantel-Haenszel weight).

• The best choice of x axis for detecting the small sample effect is the log odds ratio. This is because the scale is not constrained and because the plot will be the same shape whether the outcome is defined as occurrence or non-occurrence of event.

Sterne JAC, Egger M. Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology 2001;54:1046-1055.

Symmetrical plot in the absence of bias (opencircles indicate smaller studies showing no beneficial effects)

Asymmetrical plot in the presence of publication bias (smaller studies showing no beneficial effects are missing)

Asymmetrical plot in the presence of bias due to low methodological quality of smaller studies (open circles indicate small studies of inadequate quality whose results are biased towards larger beneficial effects)

Jonathan A et al. The Stata Journal 2004; 4:127

Publication Bias

Ntot is the total sample size, NE and NC are the sizes of the experimental and control intervention groups, S is the total number of events across both groups and F = Ntot – S. Note that only the first three of these tests (Begg 1994, Egger 1997a, Tang 2000) can be used for continuous outcomes.

Protocols

The purpose of PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines is to provide proper procedures for conducting a meta-analysis and to standardize the methods of reporting a meta-analysis.

Savarese G et al J Am Coll Cardiol 2013;61:131

• Angiotensin-converting enzyme inhibitors (ACE-Is) are recommended for reduction of cardiovascular (CV) events in patients at high CV risk without heart failure (HF).

• In contrast, CV effects of angiotensin receptor blockers (ARBs) on major clinical outcomes in patients without HF are less certain as major clinical trials comparing ARBs vs placebo reported conflicting results.

Background


Methods – Inclusion Criteria

• Report of at least one clinical outcome (all-cause death, CV death, myocardial infarction, stroke, new onset heart failure, new onset diabetes mellitus).

• Randomized, placebo-controlled trials using ACE-Is or ARBs as treatments.


• Meta-analysis was performed to assess the influence of treatments on outcomes.

• Statistical homogeneity was assessed using Q statistic and further quantified with the I2 statistic.

• Meta-regression was performed to test the influence of potential effect modifiers on results.

• Publication bias was assessed using linear regression test by Egger and Macaskill’s modified test.

Methods – Statistical methods


Results – Search Strategy


Results – Population characteristics

ACE-Is significantly reduced the risk of the composite outcome by 14.9% compared to placebo (p=0.001).

ARBs significantly reduced the risk of the composite outcome by 7.0% compared to placebo (p=0.012).

Results – Composite Outcome


10% reduction of CV death did not achieve statistical significance in ACE-Is trials (p=0.087).

ARBs did not reduce the risk of CV death (p=0.768)

Results – CV Death


ACE-Is significantly reduced the risk of MI by 17.7% (p<0.001)

9.5% reduction of MI risk did not achieve statistical significance in ARBs trials (p=0.086).

Results – Myocardial infarction


ARBs significantly reduced the risk of stroke by 9.1% (p=0.011).

ACE-Is significantly reduced the risk of stroke by 19.6% (p=0.004)

Results – Stroke


ACE-Is reduced the risk of all-cause death by 8.3% (p=0.008).

No significant effect was found on the risk of all-cause death in ARBs trials (p=0.866).

Results – All Cause Death


No significant effect was found on the risk of new onset HF in ARBs trials (p=0.866).

ACE-Is reduced the risk of new onset HF by 20.5% (p=0.001).

Results – New onset HF


ACE-Is reduced the risk of new onset diabetes by 13.7% (p=0.012)

ARBs significantly reduced the risk of new onset DM by 10.6% (p<0.01).

Results – New onset DM


Results – Subgroup analysis


Results – Meta-regression analysis

• In comparison to placebo, ACE-Is substantially reduce the composite of CV death/MI/stroke as well as all-cause death, new onset HF and new onset DM in high-risk patients without HF, mostly with coronary or other vascular diseases.

• ARBs, in high-risk patients mostly with DM or impaired glucose tolerance, without HF, reduce the composite outcome and new onset DM, but do not appear to reduce rates of all-cause death or new onset HF.

Conclusions


Types of Meta-analysis/Terminology

Systematic Review

Meta-analysis(Overview)

Extract data from published reports(aggregated data meta-analysis)

Collect individual patient data (IPD)

Frequentist Approach

Bayesian Approach

Network

Frequentist approachClassical methods are, usually based on algorithms using explicit

formulas.

main assumptions

of the model

results of studies (usually

RCTs)

Transformations of input data

Results of Meta-analysis

Bayesian Meta-Analysis Including extra (prior) information

Extra data e.g. results of non-randomized

trials, historical observations, etc.

establishing prior

distributions basing on:

main assumptions

of the model

results of randomized

studies


MCMC simulations

Setting the level of conviction to this data

!

Bayesian Meta-AnalysisAssessing clinical significance


MCMC simulations

non-informative prior

distributionsresults of studies

Answering the question: How probable is that the result is clinically significant?

Possible to obtain due to knowledge of

whole distribution

establishing the level of clinical significant result

(e.g. RR > 1.2)

main assumptions

of the model

!

Frequentist vs Bayesian approach

Bayesian approach Frequentialist methods

philosophy First: assumptions and constructionthen: inputing results of studies

Construction based on the results of studies

flexibility YES NO

computation Makov Chain Monte Carlo simulations formulas

software specialistic, e.g. WinBUGS no special requirements

55

Network Meta-Analysis(Multiple Treatments Meta-Analysis, Mixed Treatment Comparisons)

• Combine direct + indirect estimates of multiple treatment effects

• Internally consistent set of estimates that respects randomization

• Estimate effect of each intervention relative to every other whether or not there is direct comparison in studies

• Calculate probability that each treatment is most effective

• Compared to conventional pair-wise meta-analysis:

• Greater precision in summary estimates

• Ranking of treatments according to effectiveness

Indirect Comparisons of Multiple Treatments – Network Meta-Analysis

Trial

1 A B

2 A B

3 B C

4 B C

5 A C

6 A C

7 A B C

• Want to compare A vs. BDirect evidence from trials 1, 2 and 7Indirect evidence from trials 3, 4, 5, 6 and 7

• Combining all “A” arms and comparing with all “B” arms destroys randomization

• Use indirect evidence of A vs. C and B vs. C comparisons as additional evidence to preserve randomization and within-study comparison

paroxetine

sertraline

citalopram

fluoxetine

fluvoxamine

milnacipran

venlafaxine

reboxetine

bupropion

mirtazapineduloxetine

escitalopram

sertraline

milnacipran

bupropion

paroxetine

milnacipran

duloxetine

escitalopram

fluvoxamine

?

19 meta-analyses of pairwise comparisons published

Network Meta-Analysis(Multiple Treatments Meta-Analysis, Mixed Treatment Comparisons)

What is an individual patients data Meta-analysis?

• Involves the central collection, checking and analysis of updated individual patient data

• Include all properly randomised trials, published and unpublished

• Include all patients in an intention-to-treat analysis

Benefits of IPD

• Carry out time-to-event analyses• Only practical way to do subgroup analyses• More flexible analysis of outcomes• Carry out detailed data checking• Ensure quality of randomisation and follow up• Ensure appropriateness of analysis• Update follow up information

Other Benefits

• More complete identification of trials• Better compliance in providing missing data• More balanced interpretation of results• Wider endorsement and dissemination of results• Better clarification of further research• Collaboration on further research

THANKS FOR THE ATTENTION!!!

Sorrento, Naples, Italy.

meta-analysis and systematic review gianluigi savarese, md, fesc, acc fit department of advanced...

Documents

systematic reviews necessary

studiesanalyze data

individual studies

narrative reviews

outabstract data

language bias

englishcitation bias

data extraction form