“Breaking Down the Appraisal Process to Empower Manufacturers to Improve Internal Decision-Making and Successfully Achieve Making and Successfully Achieve Reimbursement”

Matt Stevenson

Quick Intro

• NICE Appraisal Committee Member

• Working Group Member for the revision of the • Working Group Member for the revision of the NICE Methods Guide

• Technical Director for ScHARR-TAG (the largest academic group undertaking work for NICE)

• Professor of Health Technology Assessment

Outline• The concept of opportunity cost

• Errors seen in submissions

• Mixed Treatment Comparisons

• Crossover in RCTs• Crossover in RCTs

• Innovation

• Patient Access Schemes / Rapid Reviews

• Value Based Pricing

Opportunity Cost

• Should be a familiar concept to all, as all decisions on food, refreshment, housing, schooling, leisure, holidays, savings ..... are taken in the context of a fixed budget

• Any increased expenditure on one item will result in a • Any increased expenditure on one item will result in a reduction in expenditure of another item.

• Given a fixed healthcare budget, the uptake of drug X will mean a reduction in the use of drug Y.

Theory behind the threshold approach

• In principle, the adoption of any new technology, with a cost burden, would result in the least cost-effective option currently used being displaced (regardless of disease area)disease area)

• Assuming zero inflation this is a downward force on the threshold.

• This process aims to increase societal QALYs . The converse view to ‘drug rationing’ is ‘health maximisation’

Limitations

• The process fails if PCTs reduce expenditure in cost-effective procedures rather than those that are not cost-effective

• Is the current threshold correct?• Is the current threshold correct?

Alternative approaches?

“I'm afraid we've got a budget problem, Marge.

Your boy picked a bad time to fall down a well. time to fall down a well.

If he had done it at the beginning of the fiscal year, no problemo.”

Errors seen in submissionsSome errors are inexcusable:

• Incorporating the unconfirmed price of the drug, or the discount rate in the probabilistic analyses

• Assuming different utilities for patients prior to • Assuming different utilities for patients prior to receiving the intervention and comparator

• ‘Broken’ models

• PAS altering results for comparator drugs

Sampling from a simple linear regression

10

15

20

10

15

20

X

X

X

XX X

X

X

X

XX X

Omission of correlation

0

5

10

-1 1 2 3 4 5x0

5

10

-1 1 2 3 4 5x

XX

X

X

XX

XX

X

X

X

XX

X

X

XX

XX

X

X

X

Slope and intercept sampled independently

Slope and intercept sampled dependently

Ignoring extended dominance

3000

4000

5000

6000

7000

8000

9000

10000

Co

st

(£)

A

B

C

As the gradient from A to B is greater than the gradient from A to C and C produces more QALYs than B then B suffers extended dominance by A and C. By using a combination of A and C the same QALYs can be produced as

0

1000

2000

0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9

QALYs Gained

the same QALYs can be produced as in B for a lower cost, or alternatively for the same expenditure as in B, more QALYs can be accrued.

If ICERs do not monotonically increase then interventions

which are extendedly dominated have not be removed.

Misinterpretation of CEACs

The CEAC shows the probability of being most cost-effective, but does not incorporate the ramifications of being non-optimal, nor indicate non-optimal, nor indicate that similarity in the results of strategies may have similar.

The mean cost per QALY must be presented.

Misinterpretation of CEACs

To reinforce the point. Being paid £1 to play Russian Roulette (6 chambers, 1 bullet), would produce the following CEAC

Choosing when to extrapolate

In this example the manufacturer defined an arbitrary rule as to when a parametric curve should replace £80,000

£90,000

£100,000

Base case ICER ICER - Based on alternative time points

ICER

curve should replace the Kaplan Meier data. In this example the assumed switch appeared favourable to the intervention.

£50,000

£60,000

£70,000

£80,000

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39

Cycles

Refusal to look at sequences

• Often treatments should be aimed at a second-line position, due to a cheap, safe and effective incumbent treatment.

• In one appraisal a company refused to consider • In one appraisal a company refused to consider sequencing claiming it was unethical given their model indicated the intervention was more cost-effective than the comparator.

• This was a misunderstanding of the need for a full incremental analysis

Inappropriate assumption of certainty

• In the appeal of sorafenib for HCC a key issue centred on whether the survival distribution was better represented by: a lognormal distribution better represented by: a lognormal distribution (≈£51,000 per QALY); a weibull distribution (CIC but considerable higher); or whether a prudent view would be to assume it lay somewhere inbetween.

Continually selecting assumptions that favour the intervention.

• This is akin to ‘resurrection with a thousand bandages’.

• The committee and the • The committee and the Assessment Groups are experienced and are very likely to see that the central ICER is biased.

Mixed Treatment Comparisons

• MTCs (also known as Network Meta Analyses) are becoming more common place in assessing the efficacy of treatments. There has been a rapid rise in the number of MTCs being presented to NICE appraisal committees.

• Although NICE maintains a preference for Head to Head studies, the evidence from MTCs are fully considered. When multiple treatments are considered, the distinction between ‘direct’ and ‘indirect’ analyses is difficult to determine.

A Taxonomy of Comparisons

AB

AC

A B Direct Comparison (Head to head)

‘Naïve’ Indirect Comparison:Absolute effect estimates from individual trial arms

‘Adjusted’ Indirect Comparison:AC

B C

AC

B C

A B

‘Adjusted’ Indirect Comparison:Relative effect estimates between treatments

Mixed Treatment Comparison/’Network’ Meta-Analysis:‘Adjusted’ indirect comparison extended to more complex networks of trial evidence (i.e. head to head and indirect evidence)

Are ‘Adjusted’ Indirect Comparisons Systematically Biased?

Song et al (2003). BMJ, 326:472-475

• Significant discrepancy in 3 out of 44 comparisons (95% CI of difference between methods did not cross zero)

• Equally likely to over or underestimate difference

• Indirect comparisons may be useful when direct evidence • Indirect comparisons may be useful when direct evidence is absent or insufficient. Indirect evidence could strengthen conclusions based on direct evidence

• Direct evidence generally regarded as best, but at times may be flawed

One compulsion for performing MTCs

“. . . to ignore indirect evidence either makes the unwarranted claim that it is irrelevant, or breaks the established precept of or breaks the established precept of systematic review that synthesis should embrace all available evidence”

Lu & Ades (2004). Combination of direct and indirect evidence in

mixed treatment comparisons. Stat Med, 23: 3105-3124

Recommendations – Song (2009)

• Literature search needs to be systematic in order to identify all relevant studies

• Naïve indirect approaches should be avoided

• Methods for investigating heterogeneity in standard meta-• Methods for investigating heterogeneity in standard meta-analysis can be applied to assess trial similarity in indirect comparisons (e.g. subgroups, meta-regression etc)

Recommendations – Song (2009)

• Evidence from head to head RCTs should not be excluded in analyses that use indirect comparisons

• Direct and indirect evidence should be separately presented and comparedpresented and compared

• Consistency should be explicitly assessed before direct and indirect evidence is combined

Crossover within RCTs

• Where an intervention has been shown to be efficacious it may be unethical to leave patients on the comparator drug. This confounds the standard intention to treat analysis.

• Two methods have been used to adjust for the crossover in submissions to NICE: the Rank Preserving Structural Failure Time model (RPSFT); and the Inverse Probability of Censoring Weights (IPCW)

RPSFT

• The RPSFT uses the randomisation assumption to estimate treatment effect such that counterfactual survival (a function of observed survival time, observed treatment and the survival time, observed treatment and the treatment effect) would have been equal in randomised groups, had no experimental treatment been given to any patients.

IPCW• The IPCW approach treats crossover patients as informatively censored, and applies time-varying weights to uncensored survival probabilities based on the probability of patients crossing over given their covariate history. The IPCW over given their covariate history. The IPCW approach is reliant on the assumption of no unmeasured confounders, and upon the existence of a reasonable number of uncensored observations.

Crossover within RCTs

• Research is currently ongoing to determine which method is most applicable in health technology evaluations

Innovation• NICE is expected to take into account the potential for long-term benefits to the NHS of innovation

• Above a most plausible ICER of £20,000 the Committee will consider the innovative nature of Committee will consider the innovative nature of the technology, specifically if the innovation adds demonstrable and distinctive benefits of a substantial nature which may not have been adequately captured in the QALY measure.

Innovation

• However, should patient health be sacrificed for the pursuit of innovation, resulting in a higher threshold for innovative products?

• Industry may have a conflicting view to those tasked with maximising societal health from the fixed budget.

£120,000,000

£140,000,000

£160,000,000

£180,000,000

£200,000,000

Pre

sent valu

e o

f in

novation a

t T

Patent expires and generic entry at year 15

Generic price are 25% of the brand

Discount rate of 3.5%

Private sector share

Value of the innovation

£120,000,000

£140,000,000

£160,000,000

£180,000,000

£200,000,000

Pre

sent valu

e o

f in

novation a

t T

Patent expires and generic entry at year 15

Generic price are 25% of the brand

Discount rate of 3.5%

£120,000,000

£140,000,000

£160,000,000

£180,000,000

£200,000,000

Pre

sent valu

e o

f in

novation a

t T

Patent expires and generic entry at year 15

Generic price are 25% of the brand

Discount rate of 3.5%

Private sector share

Value of the innovation

Where cost/QALY at threshold

£0

£20,000,000

£40,000,000

£60,000,000

£80,000,000

£100,000,000

0 5 10 15 20 25 30

Years from launch (T)

Pre

sent valu

e o

f in

novation a

t T

NHS share

£0

£20,000,000

£40,000,000

£60,000,000

£80,000,000

£100,000,000

0 5 10 15 20 25 30

Years from launch (T)

Pre

sent valu

e o

f in

novation a

t T

£0

£20,000,000

£40,000,000

£60,000,000

£80,000,000

£100,000,000

0 5 10 15 20 25 30

Years from launch (T)

Pre

sent valu

e o

f in

novation a

t T

NHS share

£100,000,000

£150,000,000

£200,000,000

£250,000,000

£300,000,000

Pre

sent valu

e o

f in

novation a

t T

Private sector share

Value of the innovation

Accept price > P* during patent

because price < P* when generics enter

£100,000,000

£150,000,000

£200,000,000

£250,000,000

£300,000,000

Pre

sent valu

e o

f in

novation a

t T

Private sector share

Value of the innovation

Accept price > P* during patent

because price < P* when generics enter Private sector share

Value of the innovation

Accept price > P* during patent

because price < P* when generics enter

Enhanced price for ‘Innovation’

-£150,000,000

-£100,000,000

-£50,000,000

£0

£50,000,000

£100,000,000

0 5 10 15 20 25 30 35 40

Years from Launch (T)

Pre

sent valu

e o

f in

novation a

t T

NHS share

-£150,000,000

-£100,000,000

-£50,000,000

£0

£50,000,000

£100,000,000

0 5 10 15 20 25 30 35 40

Years from Launch (T)

Pre

sent valu

e o

f in

novation a

t T

NHS shareNHS share

Patient Access Schemes

• It was perceived that the manufacturer would incorporate a PAS with the initial submission, or following the release of guidance (using a Rapid Review)following the release of guidance (using a Rapid Review)

• In exceptional circumstances(!) a PAS would be accepted following the ACD.

Historic use of PAS

• Up to June 2012, 29 products have submitted PAS.

• The initial PAS had been criticised for imposing a burden both in terms of administrative and operational costs both in terms of administrative and operational costs (Williamson, Lancet 2010). As time has progressed there has been a move towards commercial in confidence discounts within PAS, which are viewed as less burdensome

Recommendations following a PAS

• The trend in the acceptance of products with PAS is interesting.

1

-1

0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Comments on PAS

• The ability for the manufacturer to submit multiple PAS (albeit it requiring the agreement with DH) allows the potential for some gaming

• Recommending an intervention at the threshold will not produce an increase in societal QALYs.

• Can be viewed as a fore-runner to value based pricing, with NICE moving towards a position of price setter.

Value Based Pricing

However..........

Some key principles

• Any additional weight provided to the young will result in a lesser weight being provided to the elderly and vice versa. Ditto for male/female severe condition / mild condition .......severe condition / mild condition .......

• Any additional benefits provided to evaluated interventions must also be taken into consideration for those items that will be displaced.

Some key principles

• If some interventions are ‘recommended’ following VBP that wouldn’t have been under the current system, it follows that there will be interventions that would have there will be interventions that would have been ‘recommended’ that are now not.

• This is, in essence, a zero-sum game

Additional details

• It is likely that NICE will no longer have the powers to recommend (or not) interventions. It will instead be the body that decides on the value offered by an intervention.

• Setting a value based price is not compulsory. The only sanction is that “it would be the company’s responsibility to explain to the public why it was not prepared to offer that drug at an appropriate price”

Any Questions?