thoughts on the use of decision analysis in the review of new drug applications october 3, 2007 todd...
TRANSCRIPT
![Page 1: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/1.jpg)
Thoughts on the Use of Decision Analysis in the Review of New Drug Applications
October 3, 2007
Todd Durham
![Page 2: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/2.jpg)
Outline
NDAs and the nature of the decisionPotential benefits and challenges of
decision analysis An illustrationLearning and opportunities
![Page 3: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/3.jpg)
Mental Exercise
Imagine that tomorrow you are diagnosed with a disease from which you will die in exactly 7 days.
If you could take a pill: That would definitely cure you from this disease,
how much would you pay for it? That would give you a 25% chance of a cure, how
much would you pay for it? That would give you a 25% chance of a cure, how
much risk (s%) of a debilitating stroke would you accept?
![Page 4: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/4.jpg)
New Drug Applications
Marketing applications for new drugs FDA reviewed between 20-30 NDAs (for
NMEs) per year between 2001-2003 (FDA, Critical Path, 2004)
Data submitted with a NDA Human evidence of benefit Human evidence of risk Manufacturing controls Animal data on toxicology and carcinogenicity
![Page 5: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/5.jpg)
Objective in Reviewing a NDA
Decide if a drug’s benefits outweigh its risks
Evolved historically with various changes in the law to:Avoid misleading doctors or consumersKeep dangerous drugs out of the system
What does the law really say?
![Page 6: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/6.jpg)
“Substantial Evidence” from FDC Act of 1962
Substantial evidence was defined in section 505(d) of the Act as “evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts qualified by scientific training and experience to evaluate the effectiveness of the drug involved, on the basis of which it could fairly and responsibly be concluded by such experts that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling or proposed labeling thereof.”
(FDA, Clinical Evidence of Effectiveness, 1998)
![Page 7: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/7.jpg)
Sufficient Criteria for Demonstration of Efficacy
Choice of Primary Endpoint Reliably measures a clinically relevant characteristic Statistically sensitive to treatment Identified a priori (with corresponding analysis methods)
Results for Primary Endpoint Treatment effect is “statistically significant” in at least two studies Magnitude of treatment effect (Δ) is clinically relevant
Results for Secondary Endpoints Results from secondary endpoints further describe the relevance
of Δ (primary endpoint) if results from primary endpoint in the same study are statistically significant
![Page 8: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/8.jpg)
The Case of Carvedilol
“… the usual two-study FDA paradigm does not make sense under all situations. This much is clear. But I would also suggest, as stated above, that experience has shown the paradigm to be a very useful guideline; exceptions should therefore be relatively unusual, and, when in doubt; one should err on the side of conservatism. Nevertheless, it strikes me as absurd in extreme cases to insist that if one does not meet the original primary end point in a study, that conclusions can never be definitive but only hypothesis generating.” (Fisher, 1999)
![Page 9: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/9.jpg)
Criteria Used in Reviewing a NDA
Benefit Quantity of evidence Quality of evidence Typically restricted to one or a few “endpoints” Leads to a labeled claim consistent with results
Safety From any number of reported adverse events Cardiac safety studies (e.g., QTc) Potentially animal studies (e.g., risk to fetus)
Manufacturing
![Page 10: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/10.jpg)
Decision to be Made
Approve the new drugReject the new drugAsk the sponsor for more information
(“approvable”)
![Page 11: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/11.jpg)
Influences on the Decision
Statistical robustness of the apparent benefit, with appropriate statistical control of the false positive rate
Clinical relevance of the benefit Excess safety risks, with no control of the false
positive rate Severity of the disease Availability of other treatments
![Page 12: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/12.jpg)
When a Drug is Approved
Can be legally marketed in the U.S. Doctors have a prescribing option Patients have a treatment option Pharmaceutical companies make revenue Need for education all around
Safety will continue to be monitored Surveillance has less rigor than RCTs
May be studied further Expand the label Clarify the role of the new drug or its effects
![Page 13: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/13.jpg)
When a Drug is “Approvable”
Can not be legally marketed in the U.S.Doctors can not prescribe itPatients can not take it
May be studied furtherPharmaceutical companies spend more
money on researchTime for further research and submission
![Page 14: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/14.jpg)
When a Drug is Rejected
Sponsor may withdraw applicationCan not be legally marketed in the U.S.
Doctors can not prescribe itPatients can not take it
![Page 15: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/15.jpg)
“Easy” Approval Decisions
A lot of evidence of clear benefit Clinically relevant Statistically robust (very unlikely due to chance)
At least moderately sized safety database reflects reasonable risks
No evidence of toxic or carcinogenic effects No other available treatments or just a few
treatments with some toxicities
![Page 16: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/16.jpg)
“Easy” Rejection Decisions
Obvious hazards with little benefitPoor manufacturing controls
![Page 17: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/17.jpg)
Decisions are Much Harder When
Mixed results for benefitDrug which has been shown to have a
benefit in some populations but not others.A lot of studies, only a few of which were
successful.Statistical criteria for “success” are not met.
Some significant trade-offs must be reckoned with.
![Page 18: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/18.jpg)
Made Even More Difficult
Changing landscape Regulatory standards (e.g., emerging concerns) Medical advances Changing standard of care Ex-US medical care
External pressures Congress Patient advocates Pharmaceutical industry
![Page 19: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/19.jpg)
Benefits of a Decision Analysis
Transparency of the decision Many objectives possible (identified, weighting) Influences for all stakeholders
Role of uncertainties Which ones make the most difference?
Model that can be applied to many products in the same therapeutic area, but evolve over time.
Dissection of the problem greater understanding
![Page 20: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/20.jpg)
Transparency
PatientsTo pharmaceutical companyWithin the FDACongress
![Page 21: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/21.jpg)
Role of Uncertainties
How much do the following uncertainties bear on the consequences? Quality or quantity of evidence of benefit Medical need, population affected Available therapies How many patients will actually use the treatment?
Don’t need to be accurate but having a grasp on the range of uncertainties can still be instructive (through tornado diagrams)
![Page 22: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/22.jpg)
An Evolving Model
Changes in medicineChanges in how medical expenses are
reimbursedChanges in societal priorities or norms
![Page 23: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/23.jpg)
Dissection of the Problem
Factors which most influence the best decision can lead to new prioritiesRole of available therapy compare the
new treatment to available therapyQuantity of evidence additional
informationThe safety/benefit tradeoff patient
involvementInsensitivity of the model to various
uncertainties can make decisions easier
![Page 24: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/24.jpg)
Challenges of DA for this Application
How to define the safety risks?All of them?Control of false positive rate?
How to assess the consequencesBy whom?Using what measure?
![Page 25: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/25.jpg)
Consequences
TimeMoneyHuman livesUnwanted eventsQuality-adjusted life yearsCredibility / trust (how to value?)Quality of information (what is its value?)
![Page 26: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/26.jpg)
Basic Decision Tree
NDA Decision
Approved
Approvable
Rejected
Consequences
C1 C2 C3
![Page 27: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/27.jpg)
Waiting for More Information
NDA Decision
Approved
Approvable
Rejected
New Study?
OutcomeYes
No
Success
Failure
Presumably, success would lead to a greater chance of regulatory approval, but what are the consequences of having made this decision to wait for more information?
![Page 28: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/28.jpg)
Illustration: Serious Diagnosis
Advanced cancer that affects 50,000 individuals per year
Current expected life-span (median) is 20 months from diagnosis.
The one available treatment is not tolerated well such that most patients choose not to take it.
Loosely adapted from story in New York Times, 2007.
![Page 29: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/29.jpg)
Results from Clinical Trials
New treatment compared to placebo Efficacy:
Treatment effect is ~4 months of survival (benefit) in two studies.
In one study survival had a nominal p-value <=0.050, but it was a secondary endpoint.
Primary endpoint was stopping progression of cancer (failed in both studies).
Safety: Most common side effect is flu-like symptoms 1-2% chance of a stroke from new treatment
![Page 30: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/30.jpg)
Considerations
DA could address the consequences of a world with (now or later) and without the new treatment Lives lost in a period of time New strokes in a period of time Bouts of flu-like symptoms
Was survival a false positive finding? Zero survival benefit What to do with the conventional hypothesis testing
interpretation? Won’t the benefit depend on how many patients might
use the new treatment?
![Page 31: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/31.jpg)
Could this Ever Be Applied?
Modest proposals: FDA could conduct an exercise by writing out an influence
diagram for approval decisions in one therapeutic area. Carry out research on how to best communicate risk to
patients (both benefit and safety). Increased emphasis on risk communication to patients.
Steiner, 1999 has tremendous insight on the topic. More difficult proposal:
Conduct focus groups with patients to examine ability to elicit their trade-offs. Howard has written on ways to value life and other outcomes.
Fantasy-land proposal: Make all drugs available for marketing and change the
regulatory paradigm such that regulators verify accuracy of labeling and educate doctors and the public.
![Page 32: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/32.jpg)
Learning from Experience
Unexpected clarity, almost profound new understanding of the decision to be made.
Ability to proceed without regret knowing the problem had been understood as best as humanly possible.
Training is important. Even highly intelligent people do a poor job of estimating uncertain quantities.
![Page 33: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/33.jpg)
Illustration: What if…
The benefit was only 0-4 months of survival, with a great deal of skepticism that 4 months from the trials was “real”?
Some patients might trade the chance of a stroke for a chance at an extra month or two of life.But they can’t make this choice unless the
drug is made available to them.We won’t know unless we ask.
![Page 34: Thoughts on the Use of Decision Analysis in the Review of New Drug Applications October 3, 2007 Todd Durham](https://reader031.vdocuments.site/reader031/viewer/2022013011/55151d08550346c77d8b502b/html5/thumbnails/34.jpg)
References
US Department of Health and Human Services, Food and Drug Administration, 2004. Challenge and opportunity on the critical path to new medical products. Available from www.fda.gov.
US Department of Health and Human Services, Food and Drug Administration, 1998, Providing clinical evidence of effectiveness for human drug and biological products. Available from www.fda.gov.
Fisher L. Carvedilol and the Food and Drug Administration (FDA) Approval Process: The FDA Paradigm and Reflections on Hypothesis Testing. Controlled Clinical Trials 1999;20:16–39.
Steiner J. Talking About Treatment: The Language of Populations and the Language of Individuals. Annals of Internal Medicine 1999; 130,7: 618-622.
Howard RA. On Making Life and Death Decisions. Readings on the Principles and Applications of Decision Analysis. Howard RA and Matheson JA, editors. 1989. Strategic Decisions Group.
Howard RA. On Fates Comparable to Death. Readings on the Principles and Applications of Decision Analysis. Howard RA and Matheson JA, editors. 1989. Strategic Decisions Group.
Andrew Pollack, “Panel Endorses New Anti-Tumor Treatment,” The New York Times (March 30, 2007).