Introduction SMAA-2 Model generation Data model ADDIS Conclusions

ADDIS: Aggregate Data Drug Information Systemfor drug benefit-risk analysis and automated

evidence synthesis

Tommi Tervonen

Faculty of Economics and Business, University of Groningen, The Netherlands

Presentation @ Universidade Federal de Santa Catarina11 June 2010

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Introduction: evidence-based medicine (EBM)

Evidence-based medicine aims to apply the best availableevidence gained from scientific research to medical decisionmaking

A large share of decisions made by health care professionalsare informed by evidence-based medicine, e.g. prescription,regulatory- and reimbursement policy decisions

Although the scientific evidence is transparent and achievedwith methodological rigour, the actual decisions are oftenunstructured, ad hoc and lack transparency as the treatmentbenefit-risk valuation is not explicit

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Introduction: application of EBM in drug benefit-riskanalysis

For a drug to be granted marketing authorization, it must beproven efficant, safe, and have a sufficient benefit-risk (BR)profile compared to other drugs already in the market

Eichler & al., Nature Drug Disc, 2008

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Project Escher

Escher is a national research project of the Dutch TopInstitute Pharma that aims to improve drug regulationthrough science

16 PhD students and 4 PostDocs working in 5 universities(RUG/UMCG, UU/UMCU, Erasmus MC) in collaborationwith the industry (MSD, GSK, Amgen, WINap)

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Project Escher

Escher is a national research project of the Dutch TopInstitute Pharma that aims to improve drug regulationthrough science

16 PhD students and 4 PostDocs working in 5 universities(RUG/UMCG, UU/UMCU, Erasmus MC) in collaborationwith the industry (MSD, GSK, Amgen, WINap)

Work package 3.2 (RUG/UMCG withSchering-Plough/Merck) aims to bridge the gap betweenaggregate clinical data and evidence-based drug regulation byhaving useful methods for benefit-risk analysis implemented inusable software (which would then be used in real-life decisionmaking)

Useful/Usable/Used: Keen & Sol, IOS Press, 2008

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Drug benefit-risk analysis

BR analysis should includeall relevant evidence, andtherefore apply (network)meta-analysis

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Drug benefit-risk analysis

BR analysis should includeall relevant evidence, andtherefore apply (network)meta-analysis

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Drug benefit-risk analysis

Problems

1 Inclusion of all relevantevidence in themeta-analysis is notguaranteed

2 The BR analysis isunstructured andnon-transparent

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Drug benefit-risk analysis

Problems

1 Inclusion of all relevantevidence in themeta-analysis is notguaranteed

2 The BR analysis isunstructured andnon-transparent

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Case study of BR analysis

Hansen & al. (Ann Intern Med, 2005) assessed safety andefficacy of four second generation antidepressants andconcluded that there are “no significant differences among thedrugs”

In general, the assessment of antidepressants is hard; placeboeffect is always present causing high uncertainty on the results

Q’s:1 How can the benefit-risk assessment of second-generation

antidepressant be structured based on evidence from theclinical trials?

2 Can we come up with something better than “no significantdifferences”?

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Case study: data from meta-analysis

Hansen & al., Ann Intern Med, 2005

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Case study: data from meta-analysis

Hansen & al., Ann Intern Med, 2005

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Case study: data from meta-analysis

Hansen & al., Ann Intern Med, 2005

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Case study: data from meta-analysis

Hansen & al., Ann Intern Med, 2005

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Approach

Separate clinical data (measurements) from the valuejudgements (MCDA)

Include all data present in the original analysis (imprecisemeasurements)

Provide metrics for decision uncertainty

Enable model generation for re-applicability

We chose to apply Stochastic Multicriteria AcceptabilityAnalysis (SMAA)

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Approach

Separate clinical data (measurements) from the valuejudgements (MCDA)

Include all data present in the original analysis (imprecisemeasurements)

Provide metrics for decision uncertainty

Enable model generation for re-applicability

We chose to apply Stochastic Multicriteria AcceptabilityAnalysis (SMAA)

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

SMAA(-2)

SMAA(-2) is a multi-criteria decision aiding (MCDA) methodfor ranking a set of alternatives evaluated on basis of a set ofcriteria

Preference information expressed with a weight vector and avalue function of a commonly accepted shape (in practiceoften linear one)

Uncertain or imprecise criteria values are represented bystochastic variables that map the deterministic value functionsto value distributions

Lahdelma & Salminen, EJOR, 1998 / Tervonen & Figueira, JMCDA, 2008

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Traditional MCDA

Lahdelma & Salminen, Springer, 2010

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Inverse approach

Lahdelma & Salminen, Springer, 2010

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Weight space

Total lack of preferenceinformation is represented bya uniform joint probabilitydistribution of the weightspace

If some preferenceinformation is available, theweight space can berestricted with linearconstraints

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Rank acceptability index

Describes the share of different weights and criteria measurementsranking an alternative on a certain rank

bri =

∫ξ∈χ

fχ(ξ)

∫w∈W r

i (ξ)fW (w) dw dξ

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Central weight vector & confidence factor

Central weight vector describes the preferences of a typicalDM supporting this alternative with the assumed preferencemodel

CW’s are used for inverse approach: instead of askingpreferences and giving results, answers the question “whichpreferences support an alternative to be the most preferredone?”

Confidence factor is the probability for an alternative to bethe preferred one with the preferences expressed by its centralweight vector

CF measures whether the criteria measurements are accurateenough to discern the efficient alternatives

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Computation

Analytical techniques based on discretizing the integrals withrespect to each dimension are infeasible, so the integrals areestimated through Monte Carlo simulation

10000 simulations provide sufficient accuracy for the indices

Algorithm has less-than squared mean complexity and is veryfast in practice

Tervonen & Lahdelma, EJOR, 2007

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

MCDA Model Generation

Tervonen, URPDM’2010

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

When cannot the MCDA-BR-model be generated?

Paroxetine

Bupropion

(1)

Duloxetine

(1)

Mirtazapine

(2)

Venlafaxine

(2)

Sertraline

(3)

(1)

Escitalopram

(2)

Fluoxetine

(8)

(2)

(1)

(1)

(7)

Fluvoxamine

(2)

(6)

Citalopram

(1)

(3) (1) (2)

(1) (2)

Figure: Evidence network of studies comparing efficacy of 2nd genantidepressants

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Meta-analysis limitations

Fluoxetine

Paroxetine

(8)

Sertraline

(5)

Venlafaxine

(6)

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Meta-analysis limitations

Paroxetine

Bupropion

(1)

Duloxetine

(1)

Mirtazapine

(2)

Venlafaxine

(2)

Sertraline

(3)

(1)

Escitalopram

(2)

Fluoxetine

(8)

(2)

(1)

(1)

(7)

Fluvoxamine

(2)

(6)

Citalopram

(1)

(3) (1) (2)

(1) (2)

Figure: Evidence network of studies comparing efficacy of 2nd genantidepressants

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Meta-analysis limitations

Paroxetine

Bupropion

(1)

Duloxetine

(1)

Mirtazapine

(2)

Venlafaxine

(2)

Sertraline

(3)

(1)

Escitalopram

(2)

Fluoxetine

(8)

(2)

(1)

(1)

(7)

Fluvoxamine

(2)

(6)

Citalopram

(1)

(3) (1) (2)

(1) (2)

Uncertainty about fluoxetine not represented explicitly

What happens if we choose another baseline?

Other studies included → possibly different results

Not all drugs can be included (escitalopram)

We’re “double counting” multi-arm trials

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Solution: apply network meta-analysis

Paroxetine

Bupropion

(1)

Duloxetine

(1)

Mirtazapine

(2)

Venlafaxine

(2)

Sertraline

(3)

(1)

Escitalopram

(2)

Fluoxetine

(8)

(2)

(1)

(1)

(7)

Fluvoxamine

(2)

(6)

Citalopram

(1)

(3) (1) (2)

(1) (2)

Include all evidence in one mixed-treatment comparison(MTC) analysis

Van Valkenhoef & al., manuscript, 2010

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Network meta-analysis problems

Paroxetine

Bupropion

(1)

Duloxetine

(1)

Mirtazapine

(2)

Venlafaxine

(2)

Sertraline

(3)

(1)

Escitalopram

(2)

Fluoxetine

(8)

(2)

(1)

(1)

(7)

Fluvoxamine

(2)

(6)

Citalopram

(1)

(3) (1) (2)

(1) (2)

Model considerably more complex (Bayesian instead ofregression)

Treatment network inconsistency must be evaluated

No algorithms for generating MTC models exist(ed)

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

MTC/SMAA application

Zhao, MSc thesis @ KTH.se, 2010 (to appear)

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Clinical trial data models

Previous advances in clinical trial informatics have aimed for acomprehensive data model

Comprehensiveness is conflicting with usefulness

We provide a minimal data model to enable useful analyses

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Minimal data model

*

*

1

*

name: Stringdirection: Enum

OutcomeMeasure *1 sampleSize:

Integer

Measurement

name:StringStudy

name: Stringcode: Long

Indication

*

1

1 2 .. *size: Integer

Arm

name: StringatcCode: String

Drug

*

1

Dosequantity: Doubleunit: Enum

1 1

1 .. *

1

{One measurement for every arm-

OutcomeMeasure pair}

rate: IntegerRateMeasurement

mean: DoublestdDev: Double

ContinuousMeasurement

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Aggregate Data Drug Information System

Import & store trial design & results

Generation & computation of1 meta-analyses2 network meta-analyses3 SMAA benefit-risk models

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Time for live demo!

ADDIS v0.9-SNAPSHOT from www.drugis.org

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Conclusions

Drug benefit-risk analysis can be structured with multi-criteriadecision analysis (MCDA)

The MCDA models can take into account all relevant clinicalevidence in their original format by applying SMAA+MTC

Minimal data model is required for generation of usefulanalysis models

The ADDIS software is open source, and implements alreadyall this!

Obrigado pela sua atencao!

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Conclusions

Drug benefit-risk analysis can be structured with multi-criteriadecision analysis (MCDA)

The MCDA models can take into account all relevant clinicalevidence in their original format by applying SMAA+MTC

Minimal data model is required for generation of usefulanalysis models

The ADDIS software is open source, and implements alreadyall this!

Obrigado pela sua atencao!

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Conclusions

Drug benefit-risk analysis can be structured with multi-criteriadecision analysis (MCDA)

The MCDA models can take into account all relevant clinicalevidence in their original format by applying SMAA+MTC

Minimal data model is required for generation of usefulanalysis models

The ADDIS software is open source, and implements alreadyall this!

Obrigado pela sua atencao!

Introduction SMAA-2 Model generation Data model ADDIS Conclusions

Conclusions

Drug benefit-risk analysis can be structured with multi-criteriadecision analysis (MCDA)

The MCDA models can take into account all relevant clinicalevidence in their original format by applying SMAA+MTC

Minimal data model is required for generation of usefulanalysis models

The ADDIS software is open source, and implements alreadyall this!

Obrigado pela sua atencao!