Selecting Study Designs to Discover New Therapies during Infectious Disease Outbreaks

Andre Kalil, MD, MPH, FACP, FIDSA, FCCM

Professor Infectious Diseases DivisionDirector, Transplant ID ProgramUniversity of Nebraska Medical

Center

Hong Kong Global Health Risk Framework – August 20,2015

Conflicts of Interest

Financial:

• None

Intellectual:

• I am a practicing Clinician and Clinical Researcher with focus on critically ill patients and transplant recipients

• I believe that both randomized and observational study designs are relevant for Medicine, but their role and purpose are distinct

Which study design is the most optimal for evaluating experimental drugs with low availability in diseases affecting low-resource areas?

The most optimal and ethical study design is the one that offers the highest probability of detecting true success or true failure with the experimental drug.

The study design with the highest probability of detecting the effect of an experimental drug and distinguishing its true signal from true noise (patient selection bias, confounding, chance) is the Randomized Controlled Design.

The rationale for choosing randomized trials is based on several factors:

SCIENTIFIC: No other study design offers more robust methodology with the lowest probability of selection bias.

BIOLOGICAL: No other study design offers a direct comparison of cases to concurrent/contemporaneous controls with the lowest probability of spectrum bias.

ETHICAL:

No other study design offers a higher benefit/risk ratio with the most treatment equipoise to ALL patients

No other study design better utilizes the limited availability of experimental drugs

No other study design evaluates the efficacy of experimental drugs in a timely fashion for the benefit of humanity TODAY

No other study design accrues as valid and reliable results for the benefit of ALL patients who will be affected by future outbreaks

The rationale for choosing randomized trials is based on several factors:

Arguments recently used against randomization (in favor of observational studies):

Argument:Patients and health care professionals in Africa would not accept randomization because the EVD mortality is too high.

Counterargument:Randomized trials for acute illnesses with very high mortality are being done all over the world every day in order to evaluate new therapies for many diseases: septic shock, hospital-acquired infections, ARDS, encephalitis, necrotizing fasciitis, and many others!

Argument:Patients and health care professionals in Africa would not consent if they were told that they have a random chance to receive either the experimental drug or control.

Counterargument:• Randomized trials for acute and chronic illnesses have

been performed with the appropriate consent process in tens of thousands of patients in Africa for a multitude of diseases such as severe sepsis, tuberculosis, malaria, HIV, meningitis and many others.

• If a consent would not be deemed ethical for a randomized trial, why would a consent for an observational trial (which provides more uncertainty and less scientific information) be considered ethical?

Arguments recently used against randomization (in favor of observational studies):

Argument:Observational trials can be done faster than randomized studies.

Counterargument:Randomized trials can be done quickly, especially with the adaptive Bayesian and sequential group analysis design. Several experimental therapies can be tested in a timely fashion, and less patients are unnecessarily exposed to an ineffective or harmful drug while more patients are exposed to effective or beneficial drugs during the trial.

Arguments recently used against randomization (in favor of observational studies):

Argument:Randomized trials are more expensive and require more resources.

Counterargument:For a new experimental drug, randomized designs are actually less costly and require less resources than other designs because the information gathered will bring at once (instead of multiple observational studies) a more definitive (less biased) result regarding the drug efficacy.

Arguments recently used against randomization (in favor of observational studies):

Argument:Randomized trials are more expensive and require more resources.

Counterargument (continued):The addition of an adaptive Bayesian design makes randomized trials even more cost effective.

Only half of the randomized trial sample size will receive experimental drug; this further facilitates the evaluation of drugs with limited availability, and provides robust equipoise with a more just distribution of treatment and resources among all patients.

Arguments recently used against randomization (in favor of observational studies):

Argument:Certain experimental drugs have demonstrated very high survival success in small animals and non-human primate (NHP) models, so why bother with randomized studies?

Counterargument:Our medical history is full of drugs that showed impressive benefits in small animals and NHP models but failed terribly in randomized human studies, examples being monoclonal and polyclonal antibodies against LPS, TNF, IL-1, PAF in septic shock.

Arguments recently used against randomization (in favor of observational studies):

How much have we learned from the investigational treatment of all patients treated in Africa, EU and US?

Based on the small uncontrolled single-arm studies done in Africa and on the approximate 30 patients treated with experimental drugs in the EU and US – over one year after the beginning of the outbreak - which is your treatment drug of choice on August 20, 2015?

Uncontrolled anecdotal administration of convalescent plasma, ZMaap, MIL77, TKM-Ebola, brincidofovir, favipiravir (approximate 30 patients): based on the clinical, virological, and laboratory assessments, no conclusions can be made in favor of or against ANY of these drugs!

How much have we learned?

Uncontrolled single-arm study designed by the University of Oxford/WT to evaluate brincidofovir (planned sample size=?), which has been withdrawn (Jan 30, 2015). Results?

Uncontrolled single-arm study designed by the University of Oxford/WT to evaluate TKM-Ebola (planned sample size=?), which has been withdrawn (Jun 19, 2015). Results?

Uncontrolled single-arm study designed by the Gates Foundation (planned sample size=?), terminated (?date). Results?

How much have we learned?

Uncontrolled single-arm study designed by the INSERM, France, to evaluate favipiravir-JIKI (planned sample size=225), with results from first 69 patients available.

Overall mortality results:All Ct values: – JIKI 33/69 (48%) vs. Hist. Control 272/478 (57%):P=0.16Ct values <20:– JIKI 26/28 (93%) vs. Hist. Control 197/232 (85%):P=0.39Ct values ≥20:– JIKI 6/39 (15%) vs. Hist. Control 75/246 (30%):P=0.052

How much have we learned?

How much have we learned?

Study name Statistics for each study Odds ratio and 95% CI

Odds Lower Upper ratio limit limit Z-Value p-Value Favi Pre-Trial Total

CT<20 2.31 0.52 10.17 1.11 0.27 26 / 28 197 / 232 223 / 260

CT>20 0.41 0.17 1.03 -1.89 0.06 6 / 39 75 / 246 81 / 285

0.88 0.17 4.69 -0.15 0.88 32 / 67 272 / 478 304 / 545

0.1 0.2 0.5 1 2 5 10

Favors Favipiravir Favors Control

JIKI Study: Mortality by Baseline CT Value

Heterogeneity: Q=3.74; P=0.05; I-Squared=73%

How to interpret the JIKI findings?Based on the statistically significant heterogeneity and interaction tests, there are two possibilities:

If you believe that favipiravir saved lives in the subgroup with CT≥20, then you must also be prepared to believe that favipiravir took lives in the subgroup with CT<20. What would be the biological rationale for that?

If you believe that the detection of a CT<20 was the reason for the very high mortality, then you must be prepared to believe that the detection of a CT≥20 (not favipiravir) was the reason for the lower mortality.

Increasing the sample size cannot solve this critical interpretation dilemma because selection bias and confounding cannot be overcome by ↑ trial size.

How much have we learned?

The statistically significant heterogeneity (Q) and the very high I-squared result indicate that 73% of the differences in mortality outcome may be explained by the differences in CT value!

The mortality analysis based on different CT values was not planned prospectively, so this post-hoc subgroup analysis may be further affected by patient selection bias and false-positive findings due to multiplicity.

When comparing the outcome effects between randomized and historical control studies, the outcome effect magnitude differs by >50% (falsely inflated) in 70% of historical control studies (JAMA 2001;286:821).

How much have we learned?

What about the reliability of the JIKI historical control mortality?

The JIKI 3-month historical control mortality (Overall=57%; CT≥20=30%) may already be outdated and higher than the current actual mortality.

How much have we learned?

Changing Mortality for Ebola Virus Infection Over Time

Date N Site Mortality Change from Prior

Interval

Change from May-

JuneMay-June,

2014*106 Kenema,

Sierra Leone74% - -

Sept.-Oct.,2014**

151 Freetown, Sierra Leone

48% -35% -35%

Oct.-Nov., 2014**

126 Freetown, Sierra Leone

32% -33% -55%

Nov.-Dec., 2014**

304 Freetown, Sierra Leone

23% -28% -69%

*Schieffelin JS, et al. New Eng J Med 371:2092-100, 2014**Ansumana R, et al. New Eng J Med 372:587-88, 2014

JUST IMAGINE…

If all the patients who suffered from EVD in Africa, EU, and US were offered the opportunity to receive experimental therapies in the context of a Randomized Controlled Trial for the last 18 months…

…instead of numerous small uncontrolled single-arm studies in Africa and anecdotal use of multiple/unproven drugs in EU and US…

Today we would unquestionably be much closer to know if any of these drugs work or not!

In order to provide all patients with the very same probability of receiving the investigational drug or control, three American institutions (NIH, University of Nebraska, and Emory University) have agreed to design a RCT with a Bayesian adaptive design, and in conjunction with Liberia and Sierra Leone, to evaluate several new potential therapies against EVD.

A Multicenter Randomized Safety and Efficacy Study of Putative Investigational Therapeutics in the Treatment of Patients with Known Ebola Infection 

Short Title: MCM RCT Protocol 

Sponsored by: Office of Clinical Research Policy and Regulatory Operations (OCRPRO)National Institute of Allergy and Infectious Diseases 5601 Fishers LaneBethesda, MD 20892

 

NIH Protocol Number: 15-I-0083

Protocol IND #: 125530

ClincalTrials.gov Number: NCT02363322

Participating Sites: MCM RCT PROTOCOL (First drug to be tested: ZMapp)

U.S.• NIH Clinical Center• University of Nebraska Medical

Center• Emory University Hospital• WRNMMC• PI: Richard T. Davey, Jr.

Sierra Leone• PTS1• PTS2 (Closed – merged with

PTS1)• Emergency• Chinese Friendship Hospital at

Jui• PI: Dr. Foday Sahr

Liberia• MMU• ELWA2 (moving to ELWA3)• PI: Dr. Moses Massaquoi

MCM RCT PROTOCOL

Primary Endpoint: Mortality at Day 28

Secondary Endpoints:

Adverse events Clinical and virology effects associated with EVD Comparative effects of different investigational

agents on plasma viral load kinetics Time to viral load clearance Duration of hospital stay

The very first patient was randomized at the NIH, in Bethesda, Maryland, USA; and 60 more patients have been enrolled to date in Liberia and Sierra Leone, and Guinea.

Observational StudiesSingle-armHistorical ControlsCompassionate Use“Emotional” Protection

Randomized StudiesReal Protection

If new experimental therapies for most diseases - both acute and chronic - are approved based on randomized trials in the EU and US, why are African patients being exposed to lower quality study designs in the fight against a disease like EVD?

Conclusions

No patients and no countries affected by EVD should be offered or subjected to sub-optimal study designs with low probability to produce interpretable, valid, and replicable results.

ALL newly diagnosed patients with EVD in ALL countries should have the equal opportunity to be offered an experimental drug within the context of a randomized trial.

Proposal Unless the infectious disease outbreak is associated with 100%

mortality, no single-arm trials should be allowed to be initiated in any country due to the following reasons:

High likelihood to produce uninterpretable and uninformative results

High likelihood to produce biased and not replicable results

Results are too uncertain too be applied to the ongoing outbreak, so it would not benefit patients currently suffering from the disease

Would further deplete the availability of experimental drugs for randomized trials

Proposal Based on the reasons above the following are the possible

consequences of a single-arm trial during an outbreak:

Patient Loss: Patients suffering from the current outbreak would not have the opportunity to benefit from a new therapy

Societal Loss: Patients form other countries and from future outbreaks would not have the opportunity to benefit from a new therapy

Proposal Based on the reasons above the following are the possible

consequences of a single-arm trial during an outbreak:

Scientific Loss: Results from a single-arm trial cannot bring a robust and informative evidence in order to change the current outbreak course

Business Loss: A drug may be found falsely ineffective and/or unsafe due to the natural course of the infectious disease itself (unrelated to the drug), or falsely effective and/or safe due to the same reason.

Proposal During a major infectious disease outbreak, the initiation of clinical

research is as vital as the provision of human resources and immediate relief.

If clinical research is not started at the onset of an outbreak, no amount of human resource will suffice to bring substantial impact on mortality reduction.

The only way to achieve significant mortality reduction during highly lethal infectious disease outbreaks is by providing BOTH supportive care and clinical research AT THE SAME TIME.

Wolfe ND, Dunavan CP, Diamond J. Nature 2007;447:279

Proposal Bayesian adaptive randomized clinical trials

should be the method of choice Platform trial design; response-adaptive

randomization RCTs should be launched as early as possible

during all outbreaks of infectious diseases Multiple therapies should be evaluated in the

same trial; order to be defined by ad hoc prioritization

The moment one drug is deemed to be effective, it should be offered to all patients (treatment and controls), as well as to all patients anywhere as part of the new standard of care

Proposal Key players that must be engaged for the success of the clinical

trial implementation:

Local population and community leaders Local health care workers Local academic centers Local government Country minister of health and other leaders Global support and participation from other

countries, academia, industry, and not for profit organizations

Proposal Based on the reasons above the following are the possible

consequences of a randomized clinical trial during an outbreak:

Patient Gain: Patients suffering from the current outbreak will have the opportunity to benefit from the new therapy

Societal Gain: Patients from other countries and from future outbreaks will have the opportunity to benefit from the new therapy

Proposal Based on the reasons above the following are the possible

consequences of a randomized clinical trial during an outbreak:

Scientific Gain: Results from a randomized trial will bring a robust and informative evidence in order to change the current outbreak course.

Business Gain: A drug will be found ineffective and/or unsafe due to the drug itself (unrelated to the natural course of the infectious disease), or effective and/or safe due to the same reason.

“It is a capital mistake to theorize before you have all the evidence. It biases the judgment”.

A Study in Scarlet (1887), Dr Sir Arthur Conan Doyle