RATIONAL OPTIMISM

IN

RESEARCH & DEVELOPMENT

Dennis Lendrem

RATIONAL OPTIMISM

IN

RESEARCH & DEVELOPMENT

Dennis Lendrem

Lendrem D.W., Senn S.J., Lendrem B.C., Isaacs J.D. (2015),

R&D productivity rides again?, Pharmaceut. Statist., 14, 1–3, doi: 10.1002/pst.1653

Lendrem D., Senn S.J., Lendrem B.C., and Isaacs J.D. (2015), R&D productivity rides

again?, Pharmaceut. Statist., 14, 1–3, doi: 10.1002/pst.1653

Years

Cycle Time

= 4 yrs

1

2

3

4

5

6

Cycle Time

= 6 yrs

1

2

3

4

5

6

Before After

Years

Cycle Time

= 4 yrs

1

2

3

4

5

6

Cycle Time

= 6 yrs

1

2

3

4

5

6

Before After

Rational Optimism

• By placing tasks in parallel to reduce cycle time we

inadvertently:

• Increased R&D costs

• Increased late-stage attrition rates

• Increased expected time to market

Rational Optimism

• By placing tasks in parallel to reduce cycle time we

inadvertently:

• Increased R&D costs

• Increased late-stage attrition rates

• Increased expected time to market

Lendrem DW, Lendrem BC Torching the Haystack: Modelling fast-fail strategies in drug

development. Drug Discov Today. 2013 Apr;18(7-8):331-6.

http://dx.doi.org/10.1016/j.drudis.2012.11.011

Lendrem DW, Lendrem BC The Development Speed Paradox: Can increasing development

speed reduce R&D productivity? Drug Discov Today. 2014 Mar;9(3):209-214

http://dx.doi.org/10.1016/j.drudis.2013.09.002

Rational Optimism

Looking for a needle in a haystack?

Torch the haystack.

Rational Optimism

• It is more important to do the right science than to do

the wrong science quickly.

• Build opportunities to kill projects earlier in the

development cycle.

• Focus on tests that prevent projects from entering the

development process.

• Focus on tests that allow elimination of projects early

in the development process.

Rational Optimism

Rational Optimism

• Progression Bias

• terminations are viewed as ‘losses’ and project teams are loss-

averse

• sunk-costs fallacy – terminating a project risks losing investment

that has already been lost

• marginal cost associated with a decision to continue development

may seem trivial compared to the losses associated with

terminating a potential NTD

• organizational and market incentives to load the pipeline with

‘promising’ candidates

• Progression bias leads to continued development even in

the light of strong evidence supporting termination.

Rational Optimism

• Rational Optimism

• Continued pursuit of a line of research in the knowledge that it is

likely to be successful.

Rational Optimism

• Rational Optimism

• Continued pursuit of a line of research in the knowledge that it is

likely to be successful.

• Mindless Optimism

• Continued pursuit of a line of research in the hope that it will be

successful.

Rational Optimism

NTD

p1-β A

β B

1-pα C

1-α D

Success

Miss

False Discovery

Success

Rational Optimism

NTD

p1-β A

β B

1-pα C

1-α D

Success

Miss

False Discovery

Success

Rational Optimism

NTD

p1-β A

β B

1-pα C

1-α D

Success

Miss

False Discovery

Success

Rational Optimism

• Opportunity Costs:

Loss due to false negatives = B

NTD

p1-β A

β B

1-pα C

1-α D

Success

Miss

False Discovery

Success

Rational Optimism

• Opportunity Costs:

Loss due to false negatives = B

Loss due to false positives = C

NTD

p1-β A

β B

1-pα C

1-α D

Success

Miss

False Discovery

Success

Rational Optimism

• Opportunity Costs:

Loss due to false negatives = B

Loss due to false positives = CxA

NTD

p1-β A

β B

1-pα C

1-α D

Success

Miss

False Discovery

Success

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100I I I I I I

Cost of False Positives

when α = 67%

β =10%

mindless

optimism

rational

optimismvs

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100I I I I I I

β =15%Cost of False Positives

when α = 67%

mindless

optimism

rational

optimismvs

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100I I I I I I

mindless

optimism

rational

optimism β =20%

Cost of False Positives

when α = 67%

vs

Rational Optimism

• So?

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100I I I I I I

Cost of False Positives

when α = 67%

Cost of False Negatives

when β =10%

mindless

optimism

rational

optimismvs

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100I I I I I I

mindless

optimism

rational

optimism Cost of False Negatives

when β =20%

Cost of False Positives

when α = 67%

vs

The false negative rate has to be rather high before it outweighs

the cost of false positives!

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100I I I I I I

β =20%

Cost of False Positives

when α = 67%

mindless

optimism

rational

optimismvs

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100I I I I I I

β =20%

mindless

optimism

rational

optimismvs

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100I I I I I I

β =20%

mindless

optimism

rational

optimismvs

Rational Optimism

• So?

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100

I I I I I I

Cost of False Negatives

when β =20%

Cost of False Positives

when α = 67%

mindless

optimism

rational

optimismvs

Opportunity

Costs

(%)

Probability (%)

20

15

10

5

0

25 -

-

-

-

-

-0 20 40 60 80 100

I I I I I I

Cost of False Negatives

when β =20%

mindless

optimism

rational

optimismvs

Early Development Late Development

Rational Optimism

• So?

• We fear false negatives, but the opportunity costs of false positives

are huge.

• Terminating projects frees up R&D resources to pursue other

prospects.

• Thinking about opportunity costs allows project teams to reframe

termination decisions as prospective future gains.

Rational Optimism

Lendrem D.W., Lendrem B.C., Peck R.W., Senn S.J., Day, S., Isaacs J.D. (2015),

Progression Bias and Rational Optimism in Research & Development,

Nat. Rev. Drug Discov., In press

NTD

p1-β1

1-β2

1-β3

1-β4

1-β5

1-β6

1-β7

1-β8

β8β7

β6β5

β4β3

β2β1

1-pα1

α2

α3

α4

α5

α6

α7

α8

1-α81-α7

1-α61-α5

1-α41-α3

1-α21-α1

Rational Optimism

• Richard Peck (Roche)

• Stephen Senn (CRP Santé)

• Simon Day (MHRA)

• John Isaacs (MRG)

• John Loughlin (MRG)

NIHR Newcastle BRC

Rational Optimism

• Richard Peck (Roche)

• Stephen Senn (CRP Santé)

• Simon Day (MHRA)

• John Isaacs (MRG)

• John Loughlin (MRG)

• Clare Lendrem (AHG)

NIHR Newcastle BRC

fin