lessons about risk-benefit assessment from omeract

Jasvinder Singh, MBBS, MPHAssociate Professor of Medicine, UAB School of Medicine & VA

Medical Center, Birmingham, AlabamaMayo Clinic School of Medicine, Rochester, MN

IMMPACT-XIV, Arlington, VA June 16-17, 2011

Entire OMERACT Executive: Developed OMERACT 3 X 3

Slides: Dr. Maarten Boers

Comments: Drs. Maarten Boers & Lee Simon

Definitions Summary and Discussions at 2008

OMERACT Drug Safety Summit OMERACT approach Benefit Risk Action Team (BRAT) approach

◦ Pharmaceutical Research and Manufacturers of America

Questions and potential solutions The Future

Risk: ◦ Oxford: “a situation involving exposure to danger”◦ Merriam-Webster: “possibility of loss or injury”◦ Medical context: an effect that is harmful to the

patient’s or public’s health and which can relate to safety, efficacy or quality of a product.

Benefit: ◦ Oxford: “an advantage or profit gained from

something”◦ Merriam-Webster: “something that promotes well-

being “

Assigning causality:◦ Bradford-Hill criteria + biological plausibility +

Bayesian methods◦ No agreement on magnitude of frequency, i.e.,

when does something become a “signal” Utility of large trials to define risk

◦ Large, simple trials such as prospective randomized open blinded endpoint (PROBE)

◦ Randomization to two effective treatments◦ Alternative: observational studies with

confounding by indication and channeling bias

Utility of metanalyses of RCTs◦Multiple small RCTs ?statistically

equivalent to a large single trials with same denominator

◦200-300 events needed for credible estimates

◦Role for indirect comparisons- unclear◦Observational studies: may be included; Selection and confounding bias issues Outcome definition and measurement

and follow-up

Postmarketing Surveillance◦ Essential for drug safety evaluation◦ All drugs in a class should be analyzed identically

and concurrently◦ Desired components:

More than one defined population Full protocol with outcomes assessed at regular

intervals Concurrent control subjects Outcomes- patient-recoded + documented in electronic

records Pre-defined hypothesis Oversight by FDA and data management council

Utilities of Drug registries◦ Advantages: real world experience, long-term FU◦ Disadvantages: lack of comparator group, data

quality, patients not obligated to follow protocol, loss to follow-up

Pharmacoepidemiologic studies◦ Cohort studies generally better than case-control

studies in providing risk estimates◦ Issues need to addressed

Misclassification bias Validation of outcomes Guidelines for confounding bias and methods for

adjustment

Simple versus complex metric◦Simple: OMERACT 3 x 3 Benefit and risk categorized into 3 levels

each- none, substantial, (near) remission or death

◦Complex: multicriteria decision analysis• Complex Frameworks: Quantitative methods◦Decision analysis method◦Conjoint analysis◦Incremental net-benefit◦BRAT approach

GRADE approach◦ Classifies evidence into 4 levels: high, moderate, low

and very low based on Study design, weaknesses, special strengths (large

effect, dose response)◦ 2 recommendations: strong and weak◦ RCTs always starting at high and non-RCTs starting

at low? Other models of Risk: Nontreatment

◦ Risk and value of available treatment versus nontreatment options

◦ Type, intensity and severity of adverse event◦ Acute, subacute or chronic; manageable, treatable

or not

Create three categories of harm and benefit

Benefit: none/minimal, major, (near) remission

Harm: none/minimal, major, (near) death

Key components of this approach:◦Harms versus benefits

measurement of benefit is measurement of benefit is specific and evolved, but specific and evolved, but measurement of risk (or harm) is measurement of risk (or harm) is generic and more primitivegeneric and more primitive

Benefit and harm not placed on Benefit and harm not placed on a single scale a single scale

single scale benefit-harm 14

placing value judgments on scientific factsplacing value judgments on scientific facts◦ values will vary depending on

the perspective of the assessor comparing benefit and risk involves: comparing benefit and risk involves:

◦ comparing apples and pears◦ trading off (and discounting)

long-term against short-term effects◦ assessing multiple benefits and risks

assessment mostly driven assessment mostly driven by the need to make decisions, by the need to make decisions, whereas most research is ‘truth-driven’whereas most research is ‘truth-driven’


...placing value judgments on scientific facts

Requires qualitative research

...multiple comparisons and trade-offsDifficult scienceRequires decision analysis

...mostly driven by decision-makersLow speed of developmentsLiterature difficult to access


COBRA trialCOBRA trial VIGOR trialVIGOR trial


RCT comparing combination therapy RCT comparing combination therapy (COBRA) (COBRA) including step-down high-dose prednisolone including step-down high-dose prednisolone with single drug therapy (sulfasalazine, with single drug therapy (sulfasalazine, SSZ) SSZ) in early rheumatoid arthritisin early rheumatoid arthritis

Main result showed COBRA to be more Main result showed COBRA to be more effective and result in less side effects than effective and result in less side effects than SSZSSZ


20

Major benefit = Disease activity score <=3.7(near) Remission = Disease activity score <=2.4Severe Harm = treatment discontinuation due to serious harm, loss of efficacy or both(near) death = death or severe inacapacitation


ARR: Unqualified success: 80-54% = 26%; NNT = 4ARR: Unmitigated failure: 5-14% = –9%; NNH = ?


Unqualified success

Unmitigated failure

large industry-sponsored trial to compare high-dose rofecoxib with naproxen in RAobject was GI safety, not efficacyrofecoxib proved safer for GI, but less safe for cardiovascular eventsMerck withdrew the drug voluntarily amidst much controversy

no access to individual patient datasafety and efficacy assumed independent



Data from original report and from Strand V. Lancet 2007;370:2138-51.Major harm is defined as cardiovascular event; gastrointestinal event; death.


Unqualified success

Unmitigated failure

Specifying the therapeutic context◦ Explicit definition of product, indication, target

population, formulation, dosage and contraindications

Specifying the comparator◦ Standard of care, best in class, watchful waiting,

placebo and nonpharmacological intervention Defining the time horizon

◦ Duration of exposure and time period over which benefit-risk events are measured

Specifying the stakeholder perspective◦ Sponsor, regulators, patients, physicians

1Coplan PM Clin Pharma Therap 2011; 89:2: 312-15

Identifying pool of candidate outcomes for the assessment

Deciding which outcomes to include or exclude from the framework

Documenting all critical assumptions for these inclusion and exclusion decisions

Value tree is a visual hierarchical presentation of key ideas, values or concepts


Information may come from multiple data sources

A repository of all data, called data source table is kept


Customization done based on data quality and limitations

Study end-points are organized in value tree at 2 levels◦ Body system category of the benefit or risk◦ The end-point measured in studies

Approaches to capturing level of severity of outcomes identified and the value tree is enhanced


Assess relative weights◦ Stated choice methods◦ QALYs◦ Utilities◦ Value functions

Application of weighted approach in analysis◦ Net clinical benefit◦ Number needed to harm (NNH)◦ Multicriteria decision analysis


Key risk benefit table◦ Flexible table summarizing the key information

needed to quantify outcomes in the value trees

Additional items to enhance key table◦ Heat map color coding◦ Embedded graphs


1Levitan Clin Pharma Therap 2011; 89:2: 217-234

How to come up with method (s) for risk-benefit assessment that is/are◦ Easily understood by regulators, patients and physicians◦ Take into account non-treatment or alternative

treatments◦ Allow sensitivity analyses?

How to mandate and conduct efficient postmarketing surveillance studies?◦ Requirement by FDA and EMEA etc.◦ Funding◦ Preventing bias◦ Design and follow-up methodology

Can efficacy trials be better designed to allow better short-term risk-benefit analysis?

What steps must be taken to use large databases to conduct postmarketing studies?◦ Standardization of methods for reduction of bias◦ Data integrity and privacy issues

How to achieve an agreement on methods by major regulators (FDA, EMEA)?

Collaborative networks of academia, regulatory agencies, patients and pharma

Use of preferences, values, cost and delay in drug approval (risks of alternatives) in risk-benefit analysis

Universally applicable methods Ability to vary values, weights and perform

sensitivity analyses Utilize non-randomized studies Design and conduct useful large simple

comparative effectiveness trials

lessons about risk-benefit assessment from omeract

Documents

developed omeract

risk estimatesissues

entire omeract executive

casecontrol studies

large single trials

signalutility of large

simple trials

data quality