introduction to meta-analysis - aureliotobias.com · introduction to meta-analysis aurelio tobías...

Introduction to Meta-analysis

Aurelio Tobí[email protected]

Aims of the session

• At the end of this session you should be able to understand1) The basic principles for meta-analysis of interventions

2) How to do meta-analysis in practice using OpenMetaAnalyst

Introduction

• Meta-analysis is “… the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings” (Glass 1976)

• A meta-analysis is basically a way of calculating an average effect

Where to use meta-analysis?

• Systematic reviews– Identification and evaluation of all the available scientific

evidence– Meta-analysis may be appropriate to combine the

estimates of studies included in the systematic review

Systematic review

• PICO question• Literature search• Critical appraisal • Meta-analysis• Reporting results and

conclusions (PRISMA)

Systematic review

Systematicreview

Meta-analysis

• Statistical methods (meta-analysis) can be used or not to summarize the results of the included studies included in a systematic review

Where to use meta-analysis?

• Systematic reviews– Identification and evaluation of all the available scientific

evidence– Meta-analysis may be appropriate to combine the

estimates of studies included in the systematic review• Multi-centre studies – Each participating centre analyses its own individual data

under the same (or not) protocol– Meta-analysis may be appropriate to combine the

estimates reported from each study-centre

Why do a meta-analysis?

• Objectives– To increase statistical power and precision– To assess consistency of results– To answer questions not comprised by the individual

studies • Views– Synthetic view summarizes study results– Analytic view identifies differences between study results

Usual meta-analysis ‘slang’

Some tips

• All studies included in the meta-analysis must answer the same research question

• Only include studies with the same design• Same effect size and measure of variability for all

studies included in the meta-analysis• Meta-analysis techniques allow to combine effect

sizes in additive (symmetric) scale• Other measures of frequency of the disease

(incidence, prevalence) or effect sizes (correlation) can also be used

Data to be collected

• Effect sizei) Binary outcomes

– Risk Difference (RD)– Risk Ratio (RR)– Odds Ratio (OR)

ii) Continuous outcomes– Mean difference (D)

• Variabilitya) From confidence interval

– 95% CI: (lcli to ucli)– SE: si = (ucli – lcli)/2�1.96– Variance: si

2 = si�si

b) to variance– Variance: si

2

– SE: si = √si2

– 95% CI: yi� 1.96�si

Row data to be collected

• Binary outcome– 2�2 table

– Effect size in additive (symmetric) scale• RD = (a/(a+b)) − (c/(c+d))

– Effect sizes in multiplicative (asymmetric) scale• RR = (a/(a+b)) / (c/(c+d))• OR= (a/b) / (c/d)

Outcome (y)Treat. (x) Yes NoYes a bNo c d


• Binary outcome– 2�2 table

– Effect size in additive scale and its variance• DR with s2

DR = (ab/(a+b)3)-(cd/(c+d)3)– Effect sizes to additive scale and their variances

• log(RR) with s2log(RR) = (1/a)-(1/(a+b))+(1/c)-(1/(c+d))

• log(OR) with s2log(OR) = (1/a)+(1/b)+(1/c)+(1/d)

Outcome (y)Treat. (x) Yes NoYes a bNo c d


• Continuous outcome– Table of means

– Mean difference and its variance• D = m1 – m2 with s2

D = ((n1-1)s12+(n2-1)s2

2)/(n1+n2-2) – Standardised mean difference and its variance

• d = (m1 – m2)/s with s2d = (n1+n2)/(n1n2)+d2/2(n1+n2-2)

with s = √((n1-1)s12+(n2-1)s2

2)/(n1+n2-2)

Outcome (y)Treat. (x) n mean (sd)Yes n1 m1 (s1)No n2 m2 (s2)

Fixed effects model

Favours to treatment Favours to control

Common fixed effect (q)

yi yi = q

Fixed effects model

Favours to treatment Favours to control

(ei)

Common fixed effect (q)

yi yi = q + ei

assumes ei ~ N(0, si2)

Fixed effects model• Homogeneity of effects must be assumed

H0: y1 = y2 = … = yk

• Meta-analysis pooled estimate

– Weights defined as, wi = 1/si2

– Inverse variance weight method (Cochran 1937)

!!!!

€

⌢ θ = wiyi∑ / wi∑ !!

€

with!s⌢ θ 2 =1/ wi∑

º

∑

�

1/si2

Borenstein et al. "Introduction to Meta-analysis” (Capítulo 19, página 151)

Heterogeneity• Testing for heterogeneity

– H0: y1 = y2 = … = yk

– The test has low power when there are few studies, but high when there are many, so the p-value is difficult to interpret

!!!!

€

Q = wi yi −⌢ θ ( )

2

∑ ~ χk−12

• Quantifying heterogeneity– Based on Q’s

– Proportion of total variability explained by heterogeneity

– Cut-off values of 25%, 50% and 75% might be considered as low, moderate, high and very high heterogeneity, respectively (Higgins and Thompson 2003)

!!

€

I2 =Q −(k −1)

Q×100%



Random effects models

Favours to treatment A favor del control

Study-specific effect (qi)

yi yi = µ + di + ei

assumes di~N(0, t2) ei~N(0, si

2)

t2Random effects distribution

(ei)

1/(si2+t2)

∑

X


Random effects model• Assumes that the true effect in each study is

randomly distributed across studies with a fixed mean and variance (t2)

• Meta-analysis pooled estimate

– Weights defined as, wi = 1/(si2+t2)

– DerSimonian & Laird’s method (1986)

!!!!

€

⌢ µ = wi

*yi∑ / wi*∑ !!

€

with!s⌢ µ 2 =1/ wi

*∑

Fixed vs. random effects model

Fixed effects• Interpretation

– All studies share an identical true treatment effectxxxxxxxxxxxxxxxx

– The meta-analysis estimates this effect

• Strength– Easy to understand and simple

to carry out• Limitation

– Homogeneity of effects must be assumed

Random effects• Interpretation

– The true treatment effect in each study is randomly distributed across studies

– The meta-analysis estimates an average of these effects

• Strength– It is not limited by a restrictive

assumption • Limitation

– Does not answer why results between studies are different?

What is heterogeneity?

• Heterogeneity is the variation in the true effect which may be shown in more observed variation than expected by chance

• Individual studies are conducted in different places, times, populations and conditions leading to different between-study estimates

• Heterogeneity should not be ignored, must be explained

Causes of heterogeneity

1) Study characteristics– Variations in the study

designs– Development of studies– Attrition

• Methodological (or statistical) heterogeneity due to bias

2) Population characteristics− Type of participants− Temporal/geographical

settings− Treatment (or exposures) and

outcome measures

• Clinical heterogeneity due to biological diversity

Other data to be collected

• Study design characteristics– Method of randomization, blinding, type of comparison

and analysis, loss to follow up, etc. • Population characteristics– Conditions under investigation, eligibility criteria,

geographical, information, etc. • Details of treatment (or exposure)– Dose, duration, type of treatment (or exposure), type of

control, etc.

Subgroup meta-analysis

• Synthetic vs. analytic views - Stratified meta-analysis by study/population characteristics

- Clear definitions of subgroups is essential

- Identify homogeneity between studies within each subgroup, but heterogeneity between subgroups

Analytic

Effect modifier (xi)

Synthetic

Effe

ct si

ze (y

i)

1.83/1.49=1.23


Limitations

• Small number of studies (low statistical power)• Observational relationship between studies

(confounding bias)• Definition of sub-groups and use of aggregated

covariates (information bias)• Too many sources of methodological and clinical

heterogeneity (interpretation bias)

Summary

• Simple statistical basis for meta-analysis– Homogeneity of effects assumption– Weighted mean

• In case of heterogeneity between studies– The fixed effects model is clearly inappropriate … but the

random effects models is inappropriate too – Heterogeneity should not be ignored, must be explained.

But avoid over-interpretation of findings

Further readings

• Borestein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-analysis (mainly Chapters 4, 5, 11, 12, 13, 16, 19 and 20). Wiley: Chichester, 2009

• Egger M, Davey-Smith G, Altman D (Eds.). Systematic Reviews in Health Care: Meta-Analysis in Context 2nd Edition. BMJ Books, 2001

• Higgins JPT, Green S (Eds.). Cochrane Handbook for Systematic Reviews of Interventions. The Cochrane Collaboration, 2011 [http://handbook-5-1.cochrane.org]

• http://www.cebm.brown.edu/openmeta/

OpenMetaAnalyst

OpenMetaAnalyst

OpenMetaAnalyst

Meta-analysis Sub-groupMeta-analysis

Import dataOpen dataset

Hands on!

• gastric – Meta-analysis of 13 randomized clinical trials to evaluate the effectiveness of the use of adjuvant chemotherapy in gastric cancer remission.

• biomtx – Meta-analysis of 20 randomized clinical trials to evaluate the effectiveness of the use of biological agents for patients with rheumatoid arthritis.

• depression – Meta-analysis of 10 clinical randomized clinical trials to evaluate the effectiveness of exercise as an intervention for the treatment of depression.

introduction to meta-analysis - aureliotobias.com · introduction to meta-analysis aurelio tobías...

Documents