managing tough decisions with decision analysis informs southern california chapter meeting...
TRANSCRIPT
Managing Tough DecisionsWith Decision Analysis
INFORMS Southern California Chapter Meeting
California State University, Northridge
Phil Beccue
Decision Sciences INFORMS CSUN 2
What makes resource allocation decisions difficult?
Many needs,limited resources
A
B
CYr
Timing/Staging complexities
?
Risk/Uncertainty
Competing Objectives
Wall Street
Patient Benefit
Organizationimage/publicperception
Novelproducts
Long-termFinancial
value
Dispersed Information/Multiple Stakeholders/
Differing Opinions Projects that won’t die
Decision Sciences INFORMS CSUN 3
Many companies use informal approaches when allocating resources which may not be effective
R I S K
Trust Us Approach
Ostrich Approach
DO NOT
DISTURB
Locked Door Approach
Squeeeak
Squeaky Wheel Approach
Dominant Personality Approach
FOR SALE
Washington Monument Approach
Fair Share Approach
Decision Sciences INFORMS CSUN 4
Decision Analysis (DA) addresses the challenges of real-world decisions
• Provides a method to decompose complex problems
– Broad set of alternatives
– Specific agreement (up front) on criteria
– Explicit accounting of uncertainty
• Offers a set of tools and processes to bring clarity to the best course of action
• Based on foundational axioms of utility theory
• Is a prescriptive approach to decision-making applied to important, real-world problems
Decision Sciences INFORMS CSUN 5
The Decision Analysis process provides a guide to think systematically about R&D decisions
• Decision analysis is a rigorous, transparent, quantitative approach for balancing the difficult tradeoffs inherent in R&D decisions.
• A formal process provides a common language for thinking and communicating about decisions within a multidisciplinary team.
StructuringModeling and Data Collection
EvaluationCommunication
and Integration
ActionActionDecision Problem
Decision Sciences INFORMS CSUN 6
A case example illustrates the application of DA to a tough decision for a drug company
• Phase 2 trials for Leapogen to address an unmet medical need (jumping ability) are nearly complete
• High efficacy shown in a few of the 17 major potential clinical settings
• Management had differing views on the best way to proceed to develop and commercialize Leapogen
• Senior management review meeting in 2 months
• We started by carefully crafting a decision statement to keep the team focused:
How should we develop and commercialize Leapogen for the athletic jumping indication?
Decision Sciences INFORMS CSUN 7
We took a comprehensive look at 8 strategies and included all significant issues• 8 well-defined Strategies• Approved Label
– Narrow– Medium– Broad
• Probability of Tech Success• Probability of Reg Approval• Launch Timing• Price• R&D Costs• S&M Costs• COGS
• Patient Population by Indication– Treated patients– Patient growth rate– Disease Severity– Therapeutic Penetration– Competition– Market Share
• Marketing Focus
Decision Sciences INFORMS CSUN 8
Ph 2
Ph 3
None
Ph 2
Ph 3
INV IND
None
Ph 2
Ph 3
INV IND
None
Ph 2
Ph 3
INV IND
None
Ph 2
Ph 2/3
Ph 3
INV IND
None
Ph 2
Ph 3
INV IND
None
Ph 2
Ph 3
INV IND
None
Strategy 1 $128M
Strategy 2 $140M
Strategy 3 $ 82M
Strategy 4 $ 40M
Strategy 5 $120M
Strategy 6 $ 103M
Strategy 7 $ 25M
Strategy 8 $180M
A strategy table narrowed the feasible alternatives to 8 clearly defined development strategies
Strategy Name Dev Cost
ClinicalSetting A
(Basketball)
Clinical Clinical Clinical Clinical Clinical ClinicalSetting B(Soccer)
Setting C(Volleyball)
Setting D(Long Jump)
Setting E(High Jump)
Setting F(Hurdles)
Setting G(Rugby)
Decision Sciences INFORMS CSUN 9
We carefully defined 3 label outcomes for each strategy by specifying the clinical setting included
Narrow Medium BroadS1 adefmn cdefmnopq bcdefghijklmnopqS2 adefmn defmnopq bcdefghijklmnopqS3 adefmn defmnopq bcdefghijklmnopqS4 adef adefmnoq adefmnopqS5 def cdef cdefmnS6 defmn cdefmnopq abcdefghijklmnS7 mn mno mnoqS8 adefmn adefmnopq abcdefghijklmnopq
Label
Str
ate
gy
KEYa – basketballb – soccerc – volleyballd – long jumpe – high jumpf – hurdlesg – rugbyh – balleti – gymnasticsj – figure skatingk – …
Decision Sciences INFORMS CSUN 10
The decision tree specified the scenarios to be analyzed for each strategy
The model calculated NPV for over 4 million scenarios!
Low
Total NPV Nom
Total NPV High
Total NPV
Low
Nom
High
DiseaseSeverity
Low
Nom
High
TherapeuticPenetration
Region 1
Region 2
Region 3
Region 4
MarketShare
Early
Late
MarketingFocus
Narrow
Medium
Broad
CompetitionTiming
Yes
Label
No
Pay R&D Costs
Yes
RegulatoryApproval
No
Pay R&D Costs
Yes
TechnicalSuccess P3
No
Pay R&D Costs
S1
S2
S3
S4
S5
S6
S7
S8
TechnicalSuccess P2
LeapogenStrategy
Decision Sciences INFORMS CSUN 11
An influence diagram defined the key input requirements to compute NPV
DecisionUncertaintyValue
COGSDemand
NPV
Cost
Revenue
S&MCost
DevCost
Price
PatientsTreated
Dose perpatient per
cycle
Marketshare
Competition
Today'sPts receiving
treatment
Cyclesper patient
COGSper gm
GrowthRate
Futuretrends for treatment
Patientpopulation
Pts treatedwith anytherapy
TechnicalSuccess
Label
LeapogenStrategy
MarketingFocus
Decision Sciences INFORMS CSUN 12
We specified uncertain inputs using probability distributions
10% of area10% of areaPro
bab
ility
500 1000 2000
Clinical Disease Severity Patients TreatedSetting Lo Nom High Lo Nom High
A 3% 18% 30% 3,200 4,400 7,500 B 3% 18% 30% 10,500 20,600 28,000 C 10% 15% 30% 1,500 4,800 7,000 D 5% 8% 10% 6,000 10,000 12,000 B 60% 80% 90% 6,000 9,000 12,000 E 20% 40% 80% 4,000 5,600 7,800 F 1% 5% 8% 13,000 19,000 25,000 G 10% 20% 35% 15,000 17,100 34,000 H 6% 9% 12% 45,000 66,000 90,000
Decision Sciences INFORMS CSUN 13
Technical success probabilities were assessed through structured conversations and compared to industry benchmark data
Approved
Not Approved
Success
Approval
Failure
Success
P3
Failure
Success
P2
Failure
P1
.60
.91
.73
.46
New, Active Substances (NAS)CMR Int’l Data, 1997
Launch
Decision Sciences INFORMS CSUN 14
The probabilities of technical and regulatory success varied by strategy
Narrow Medium Broad 1 30% 100% 90% 5% 2 30% 100% 95% 5% 3 35% 100% 95% 5% 4 25% 0% 100% 95% 5 30% 100% 30% 0% 6 15% 100% 90% 10% 7 40% 100% 90% 0% 8 20% 100% 95% 0%
Strategy
ProbabilityTechnicalSuccess
Probability ofRegulatory Success
Given Label
Decision Sciences INFORMS CSUN 15
0%
10%
20%
30%
40%
50%
-100 -50 0 50 100
Expected NPV ($M)
Pro
bab
ility
of
Tec
hn
ical
Su
cces
s
The top strategy (#4) provides over $150M additional value than the status quo (#8) strategy
4
8
3
6
7
2
5
1
Size of bubble is proportional to expected peak sales.
Decision Sciences INFORMS CSUN 16
NPV ($M)
Cu
mu
lative
Pro
ba
bility
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
-100 -50 0 50 100 150 200 250 300 350
The expected NPV for Strategy 4 is $10M, including all technical and commercial risks
There is a 30% chance of launch
There is only a 10% chance of getting an NPV greater than $100M
Probability-weighted average = $10 million
Decision Sciences INFORMS CSUN 17
Assuming technical success, peak year revenue could vary from $100 to $250 M, with expected sales of $150M
Peak Sales ($M)
Pro
ba
bili
ty D
en
sity
75 100 125 150 175 200 225 250 275 300 325
ExpectedValue
Decision Sciences INFORMS CSUN 18
Competition
Peak Share Basketball
Label
Severity of Disease Long Jump
Peak Share High Jump
Peak Share Long Jump
Severity of Disease Long Jump
Peak Share Soccer
Severity of Disease High Jump
Peak Share Volleyball
Peak Share Hurdles
Severity of Disease Volleyball
Peak Share Gymnastics
Peak Share Ballet
0 50 100 150 200 250 300 350 400
Peak Sales ($M)
The key drivers of risk for Strategy 4 are Competition, Peak Share for Basketball, and Label
Assumes technical and regulatory success
The base value ($270 MM) is calculated by setting all uncertain inputs to their base case.
Each bar in the tornado diagram shows the impact on commercial value of moving one uncertain input across its range of uncertainty while holding all other inputs to the base case.
Decision Sciences INFORMS CSUN 19
Revenues for Strategy 4 are uncertain, and key contributors of value are clinical settings A, D, and E
0
50
100
150
200
250
2000 2002 2004 2006 2008 2010 2012 2014 2016
Su
m o
f R
even
ues
($M
)
-
50
100
150
200
1
Strategy 4
Pea
k S
ales
($M
)
A C D E F M N O P Q
Decision Sciences INFORMS CSUN 20
Risk/return tradeoffs for each strategy were made explicit
Peak Sales ($M)
Pro
bab
ility
Strategy 8
Strategy 4
Strategy 7
0 50 100 150 200 250 300 350
Decision Sciences INFORMS CSUN 21
Strategy 3 will provide positive value if the chance of success exceeds 50%
-80
-60
-40
-20
0
20
40
60
80
100
0% 20% 40% 60% 80% 100%
Probability of Technical Success for Strategy 3
Ex
p N
PV
($
M)
NominalProbability = 25%
Decision Sciences INFORMS CSUN 22
The optimal marketing focus depends on the outcome of future uncertainties
Region 1
Region 2
Region 3
Region 4
Early
MarketingFocus
Region 1
Region 2
Region 3
Region 4
Late
Narrow
CompetitionTiming
Region 1
Region 2
Region 3
Region 4
Early
Region 1
Region 2
Region 3
Region 4
Late
Medium
CompetitionTiming
Broad
S1
Label
S2
S3
S4
S5
S6
LeapogenStrategy
Bold line indicateshighest NPV path
Decision Sciences INFORMS CSUN 23
Key insights from the strategic analysis
• Strategy 4 (a focused strategy) has best overall value
• Peak sales is $150 million
• Optimal marketing focus should be determined closer to launch
• High value indications (% of total value):
– Clinical setting A (62%)
– Clinical setting E (26%)
– Clinical setting D (12%)
• Chance of success = 30%
• Clinical setting G is not as critical as once thought
Decision Sciences INFORMS CSUN 24
The initial DA had a significant impact at Amgen
• Jumping ability is a viable indication for Leapogen• Structuring the complex set of development options provided
direction and a clear development plan• Senior management agreed to follow the recommended
strategy of a focused program• The recommended strategy gave $150M additional value over
the status quo strategy• Key drivers were identified as needing further investigation;
senior management asked to update the decision analysis in the future
• A few months later, the team asked our group to translate the decision model into a forecasting tool for ongoing use
• We have performed additional strategic projects for Leapogen based on the initial work
Decision Sciences INFORMS CSUN 25
It is critical to keep management informed at each step of the process
Action !Action !
Decision BoardDecision Board
Decision Analysis TeamDecision Analysis Team
Information ExpertsInformation Experts
Time
ProblemStructuring
Modeling/Data
Collection
Evaluation Communicationand
Integration
Decision Sciences INFORMS CSUN 26
The benefits of using decision analysis should be weighed against the costs
• Documents the decision process
• Information collection is focused and efficient
• Helps resolve conflicts and debates
• Fewer “surprises” because uncertainty is explicitly considered
• Avoids common pitfalls in analyzing complex situations
– solving the wrong problem
– analyzing what is known rather than what is important
– getting lost in the process
• More time required of decision-makers
• More time for analysis team
• Possible discomfort with new process
• Reveals logic of decision
– confidential information
– lack of knowledge
– embarrassing motivations
Benefits Costs
Decision Sciences INFORMS CSUN 27
The significant problems we face cannot
be solved at the same level of thinking
we were at when we created them.
- Albert Einstein