generalised evidence synthesis
Post on 06-Jan-2016
38 Views
Preview:
DESCRIPTION
TRANSCRIPT
Generalised Evidence Synthesis
Keith Abrams, Cosetta Minelli, Nicola Cooper &
Alex SuttonMedical Statistics Group
Department of Health Sciences,
University of Leicester, UK
CHEBS Seminar
‘Focusing on the Key Challenges’
Nov 7, 2003
Outline
• Why Generalised Evidence Synthesis?
• Bias in observational evidence
• Example: Hormone Replacement Therapy (HRT) & Breast Cancer
• Discussion
Why Generalised Evidence Synthesis?• RCT evidence ‘gold standard’ for assessing efficacy (internal
validity)
• Generalisability of RCT evidence may be difficult (external validity), e.g. CHD & women
• Paucity of RCT evidence, e.g. adverse events
• Difficult to conduct RCTs in some situations, e.g. policy changes
• RCTs have yet to be conducted, but health policy decisions have to be made
• Consider totality of evidence-base – (G)ES beyond MA of RCTs
Assessment of Bias in Observational Studies - 1
• Empirical evidence relating to potential extent of bias in observational evidence (Deeks et al. 2003)
• Primary studies: – Sacks et al. (1982) & Benson et al. (2000)
• Primary & Secondary studies (meta-analyses): – Britton et al. (1998) & MacLehose et al. (2000)
• Secondary studies (meta-analyses):– Kunz et al. (1998,2000), Concato et al. (2000) &
Ioannidis et al. (2001)
Assessment of Bias in Observational Studies - 2
• Using a random effects meta-epidemiology model (Sterne et al. 2002)
– Sacks et al. (1982) & Schultz et al. (1995) ~ 30%– Ioannidis et al. (2001) ~ 50%– MacLehose et al. (2000) ~ 100%
• Deeks et al. (2003) simulation study: comparison of RCTs and historical/concurrent observational studies – Empirical assessment of bias – results similar to
previous meta-epidemiological studies– Methods of case-mix adjustment, regression &
propensity scores fail to properly account for bias
Approaches to Evidence Synthesis
• Treat sources separately, possibly ignoring/downweighting some implicitly
• Bayesian approach & treat observational evidence as prior for RCTs & explicit consideration of bias:– Power Transform Prior– Bias Allowance Model
• Generalised Evidence Synthesis
Example – HRT
• HRT used for relief of menopausal symptoms
• Prevention of fractures, especially in women with osteoporosis & low bone mineral density
• BUT concerns have been raised over possible increased risk of Breast Cancer
Odds Ratio (log scale)
<
<
<
<
<
<
<
>
>
>
Viscoli 2001 5 / 337 5 / 327
<
<
<
<
<
<
<
>
>
>
Mosekilde 2001 2 / 502 5 / 504
<
<
<
<
<
<
<
>
>
>
Gallagher 2001 0 / 243 4 / 246 <
<
<
<
<
<
<
>
>
>
Herrington 2000 1 / 204 0 / 105
<
<
<
<
<
<
<
>
>
>
Angerer 2000 0 / 215 1 / 106
<
<
<
<
<
<
<
>
>
>
Alexandersen 2000 1 / 150 1 / 50
<
<
<
<
<
<
<
>
>
>
Komulainen 1999 2 / 232 1 / 232
<
<
<
<
<
<
<
>
>
>
Hulley (HERS) 1998 32 / 1380 25 / 1383
<
<
<
<
<
<
<
>
>
>
Steele 1997 1 / 37 0 / 37
<
<
<
<
<
<
<
>
>
>
Speroff 1996 6 / 1128 0 / 137
<
<
<
<
<
<
<
>
>
>
PEPI 1995 7 / 701 1 / 174
<
<
<
<
<
<
<
>
>
>
Munk-Jensen 1994 2 / 100 0 / 51
<
<
<
<
<
<
<
>
>
>
Maheux 1994 0 / 30 1 / 30
<
<
<
<
<
<
<
>
>
>
Aloia 1994 0 / 31 1 / 70
<
<
<
<
<
<
<
>
>
>
Lufkin 1992 1 / 39 1 / 36
<
<
<
<
<
<
<
>
>
>
Gallagher 1991 0 / 62 1 / 20
<
<
<
<
<
<
<
>
>
>
Genant 1990 1 / 116 1 / 40
<
<
<
<
<
<
<
>
>
>Christiansen 1980 2 / 56 1 / 259
<
<
<
<
<
<
<
>
>
>
Nachtigall 1979 0 / 84 4 / 84
RCT Year HRT Control
Pooled
0.05 0.1 1 5 10 20
HRT & Breast Cancer – RCT Evidence before July 2002
Source: Torgerson et al. (2002)
OR 0.97 95% CI 0.67 to 1.39
Odds Ratio (log scale)
Iowa Womens Health 1991 355 / 1338 178 / 702Netherlands Cohort 1988 30 / 125 306 / 1076Nurses Health 1986 618 / 2442 714 / 3084Schairer 1985 341 / 1418 370 / 1422Canadian NBSS 1985 205 / 954 243 / 976
Cohort
Pooled
Stanford 1989 117 / 134 149 / 161Yang/Gallagher 1989 132 / 148 269 / 2774 State Study 1988 604 / 720 1892 / 2297Long Island 1984 157 / 122 519 / 547Ewertz 1983 136 / 109 400 / 414Bain 1983 39 / 86 226 / 458Hislop 1981 86 / 84 275 / 282CASH 1981 437 / 542 335 / 420Brinton 1976 808 / 932 714 / 869
Case-Control (Population)
Pooled
Franceschi 1992 151 / 132 1265 / 1379Katsouyanni 1990 42 / 70 404 / 770La Vecchia 1987 119 / 64 1496 / 1386Vessey 1982 47 / 51 369 / 411Morabia 1974 80 / 144 104 / 178
Case-Control (Hospital)
Pooled
HRT No HRT
0.5 1 2
ALL Observational
RCTs
HRT & Breast Cancer – Observational Evidence*
Source: Lancet (1997)
* Adjusted for possible confounders
All Observational OR 1.18 95% CI 1.10 to 1.26
RCTs OR 0.97 95% CI 0.67 to 1.39
Use of Observational Evidence in Prior Distribution
Case-ControlCase-ControlQuasi RCTsQuasi RCTs
RCTsRCTs
CohortCohort
Prior
Synthesis
Empirical EvidenceBias
Empirical EvidenceBias
Following Ibrahim & Chen (2000)
• 0 1 is degree of downweighting = 0 total discounting = 1 accept at ‘face value’• Evaluate for a range of values of
Power Transform Prior
)()|()|()|( PObsLRCTsLDataP
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0.50
0.75
1.00
1.25
1.50
Alpha
Odd
s R
atio
(lo
g sc
ale)
Power Transform Prior – Results 1
* is unbiased true effect in observational studies is bias associated with observational evidence 2 represents a priori beliefs regarding the possible
extent of the bias
),0(~ &
for prior form obs. i.e. |~
),(~ & )(~ : ObsMA
),(~ & ),(~ :RCTsMA
2*
22,
22
N
z
NvNz
NsNy
jjjj
iiii
Bias Allowance ModelFollowing Spiegelhalter et al. 2003
Bias Allowance Model - Results
Belief/Source Bias 2 OR 95% CrI P(OR>1)
‘Face Value’ 0% 0 1.14 1.07 to 1.20 1.00
Total Discounting % 0.87 0.30 to 1.60 0.31
Sacks & Schultz 30% 0.02 1.08 0.85 to 1.37 0.72
Ioannidis 50% 0.08 1.00 0.68 to 1.45 0.50
MacLehose 100% 0.24 0.94 0.56 to 1.49 0.40
HRT & Breast Cancer: Evidence – July 2002
HERS II (JAMA July 3) [Follow-up of HERS]– n = 2321 & 29 Breast Cancers – OR 1.08 (95% CI: 0.52 to 2.25)
• WHI (JAMA July 17) [Stopped early]– n= 16,608 & 290 Breast Cancers– OR 1.28 (95% CI: 1.01 to 1.62)
• HERS II & WHI– OR 1.26 (95% CI: 1.01 to 1.58)
• Revised Meta-Analysis of RCTs– WHI 68% weight – OR 1.20 (95% CI: 0.99 to 1.45)
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0.50
0.75
1.00
1.25
1.50
Alpha
Odd
s R
atio
(log
sca
le)
HERS II & WHI
Power Prior Posterior
30%
Power Transform Prior – Results
Generalised Evidence Synthesis
• Modelling RCT & observational (3 types) evidence directly;
– Hierarchical Models (Prevost et al, 2000;Sutton & Abrams, 2001)
– Confidence Profiling (Eddy et al, 1990)
• Overcomes whether RCTs should form likelihood & observational studies prior
Generalised Evidence Synthesis
Quasi RCTsQuasi RCTs Case-ControlCase-Control RoutineRoutineRCTsRCTs CohortCohort
Beliefs
Synthesis
Decision Model
Utilities Costs
Hierarchical Model
],[~ & ],[~ ],,[~
],[~
],[~
,,1 & ,,1 ],[~
22
2
2
2
j
j
jjij
jijijij
N
N
JjnisNy
HRT: Hierarchical Model - Results
Independent Hierarchical
OR 95% CrI OR 95% CrI
RCT 0.89 0.39 to 1.52 1.02 0.76 to 1.27
Cohort 0.98 0.84 to 1.12 1.01 0.89 to 1.13
CC-P 1.06 0.96 to 1.06 1.05 0.97 to 1.15
CC-H 1.23 0.94 to 1.55 1.12 0.93 to 1.36
Overall 1.05* 0.98 to 1.13 1.05 0.87 to 1.24
* Ignores study-type
Hierarchical Model - Extensions
• Inclusion of empirical assessment of (differential) bias with uncertainty, i.e. distribution
• Bias Constraint, e.g. HRT
CCHCCPCohRCT
Discussion – 1 • Direct vs Indirect use of non-RCT evidence
– Direct: intervention effect, e.g. RR– Indirect: other model parameters, e.g. correlation
between time points
• Allowing for bias/adjusting at study-level– IPD if aggregate patient-level covariates are
important, e.g. age, prognostic score– Quality – better instruments for non-RCTs &
sensitivity of results to instruments
Discussion – 2
• Subjective prior beliefs regarding relative credibility (bias or relevance) of sources of evidence– Elicitation
• Bayesian methods provide …– A flexible framework to consider inclusion of all
evidence, & …– which is explicit & transparent, BUT …– Require careful & critical application
ReferencesDeeks JJ et al. Evaluating non-randomised intervention studies. HTA 2003;7(27).
Eddy DM et al. A Bayesian method for synthesizing evidence. The Confidence Profile Method. IJTAHC 1990;6(1):31-55.
Ibrahim JG & Chen MH. Power prior distributions for regression models. Stat. Sci. 2000 15(1):46-60.
Prevost TC et al. Hierarchical models in generalised synthesis of evidence: an example based on studies of breast cancer. Stat Med 2000;19:3359-76.
Sterne JAC et al. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat. Med. 2002;21:1513-1524.
Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian Approaches to Clinical Trials & Health-care Evaluation. London: Wiley, 2003.
Sutton AJ & Abrams KR. Bayesian methods in meta-analysis and evidence synthesis. SMMR 2001;10(4):277-303.
top related