Markov Model in Health Care

Septiara PutriCenter for Health Economics and Policy Studies

4/10/2015 1

Objectives

• Introduce and familiarize participants with basic Markov modeling for economic evaluation

• Familiarize participants with ‘step by step’ Markov model construction

• Prepare participants to conducting health economic models (Microsoft Excel application)

4/10/2015 2

Introduction “a health-care evaluation model as an analytic methodology that

accounts for events over time and across populations, that is based on data drawn from primary and/or secondary sources, and whose

purpose is to estimate the effects of an intervention on valued health consequences and costs” [1]

• Decision analysis: systematic-mathematical approach, informing clinical decisions, uncertainty in decision making

• Decision analytic model have been increasingly applied in health economic evaluation

• Markov modeling for health economic evaluation

4/10/2015 3

[1] Weinstein, Milton C., et al. "Principles of Good Practice for Decision Analytic Modeling in Health-Care Evaluation: Report of the ISPOR Task Force on Good Research Practices—Modeling Studies." Value in health 6.1 (2003): 9-17.

Decision Tree: An overview

4/10/2015 4

Which treatment should be chosen?

Disease A

Drug A

Drug B

Die

Survive

Die

Survive

Severe

Moderate

Mild

Severe

Nodes:decision point

between treatment options

possible events that patients experienced

terminal nodes

*Mutually exclusive for pathway*probability should be 1.0 in the end

Decision tree

4/10/2015 5

0.60

0.50

0.40

0.50

0.40

0.45

0.15

0.40

0.50

0.10

Hint: Expected Cost= probability x costExpected utility=probability x utility

Limitations of Decision Tree

4/10/2015 6

Limited for predicting continuous values Expected cost and effect/utility are less visible Limited to one output Difficulties for regression application and

representing long term prognoses Difficulties to apply when patients experiencing

recurrence events, particularly chronic disease Applying discounting and outcome over time

become issues. Inability to determine when events occur (start and

later?) Possibly excessive branches

Markov

Markov Process• Form of stochastic

process• Future event could

not be accurately predicted by past event

• Able to look at long sequence

• Memory less assumption

4/10/2015 7

Markov Chains

• Discrete time• Homogenous transition

probabilities • Statistical model

application real world

• Probability of particular transitionstransition matrix

Steps in ConstructingMarkov Model

Determine relevant states

• Mutually exclusive• Representative (clinically and

economically)• Available and consistent

evidence of clinical pathway-not contradicted, widely accepted

• Determine cycle length• Time horizon-long enough

4/10/2015 8

Asymptomatic

Progressive

Death

Steps in ConstructingMarkov Model

Determine transitions

• Determines relevant transition from each states

• Identify transition probabilities (most representative data)

• Cycle length should be reflective

4/10/2015 9

Asymptomatic

Progressive Death

Asymptomatic

Steps in ConstructingMarkov Model

Model parameters

• Outcomes? (Life years, QALYs)

• Determines transition probabilities, utilities, effectiveness and costs related to states

• To understand: transition matrix

4/10/2015 10

Progressive Death

A to P

P to D

A to D

Analyzing Markov Model • Model ParametersTransition Probabilities:A to P: 0.105P to D: 0.411A to D: 0.202

Costs (progressive state) : Rp 12,000,000 (drug A) Rp9,500,000 (drug B)

Utility (progressive state): 0.70 (drug A) 0.82 (drug B)Effectiveness: RR: 0.60Discounting: 3% (both cost and effect)

4/10/2015 11

Analyzing Markov Model • Transitions Matrix

4/10/2015 12

Asymptomatic Progressive Death

Asymptomatic A to A A to P A to D

Progressive 0 P to P P to D

Death 0 0 D to D

Asymptomatic Progressive Death

Asymptomatic 1-0.105-0.202 0.105 0.202

Progressive 0 1-0.411 0.411

Death 0 0 1

Cohort Simulation

4/10/2015 13

1000 00

105693 2022

480 135 385

Asymptomatic DeathProgressive Cycles

1

3

2

0.693=693

0.693=480

1.00=202

0.589=62

Analyzing Markov Model • Cohort Analysis

4/10/2015 14

Life Years

Asmptomatic Progressive Death Check Uncorrected Corrected

0 1 No Discounting

1 0,693 0,105 0,202 1,000 0,798 0,899

2 0,480 0,135 0,385 1,000 0,615 0,706

3 0,333 0,130 0,537 1,000 0,463 0,539

4 0,231 0,111 0,658 1,000 0,342 0,402

5 0,160 0,090 0,750 1,000 0,250 0,296

6 0,111 0,070 0,820 1,000 0,180 0,215

7 0,077 0,053 0,871 1,000 0,129 0,155

8 0,053 0,039 0,908 1,000 0,092 0,111

9 0,037 0,029 0,935 1,000 0,065 0,079

10 0,026 0,021 0,954 1,000 0,046 0,056

Incremental Cost Effectiveness Ratio

4/10/2015 15

Sensitivity Analysis [2]

4/10/2015 16

“There remains several types of uncertainty relatedmethods in analysis”

“Sensitivity analysis can help the reviewer to determinewhich parameters are the key drivers of a model’sresults”

“By reporting extensive outputs from sensitivityanalysis, modellers are able to consider a wide range ofscenarios and, as such, can increase the level ofconfidence that a reviewer will have in the model”[2] Taylor Matthew, 2009. Retrieved fromhttp://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/What_is_sens_analy.pdf

One way sensitivity analysis• The simplest form of sensitivity

analysis• vary one value in the model by a

given amount, and examine theimpact that the change has on themodel’s results.

• This is known as one-way sensitivityanalysis, since only one parameter ischanged at one time.

[2] Taylor Matthew, 2009. Retrieved fromhttp://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/What_is_sens_analy.pdf[3] Tornado diagram showing results of the one-way sensitivity analysesfor the difference (vaccination versus no vaccination) in total cost pervaccinated child. The vertical line indicates where the incremental costsfor the vaccine strategy were more than €0.Knoll et al. Health Economics Review 2013 3:27 doi:10.1186/2191-1991-3-27

4/10/2015 17

Tornado diagram

Two way sensitivity analysis

4/10/2015 18

Smith, T., Jordaens, L., Theuns, D. A., van Dessel, P. F., Wilde, A. A., & Hunink, M. M.(2013). The cost-effectiveness of primary prophylactic implantable defibrillatortherapy in patients with ischaemic or non-ischaemic heart disease: a Europeananalysis. European heart journal, 34(3), 211-219.

• Assesing the impact of pairs of variables in a CEA by varying them across a plausible range of values and combinations with all other variables constant at their baseline value.

• If varying more than two, it is called multi-way sensitivity analysis'.

Probabilistic sensitivity analysis (PSA)

4/10/2015 19

“A form of sensitivity analysis inwhich probability distributionsare applied to the ranges for amodel’s input parameters, andsamples from these distributionsare drawn at random togenerate an empiricaldistribution of the relevantmeasure of cost-effectiveness.”

Andronis, L., Barton, P., & Bryan, S. (2009). Sensitivity analysis in economicevaluation: an audit of NICE current practice and a review of its use andvalue in decision-making.

Pharoah, P. D., Sewell, B., Fitzsimmons, D., Bennett, H. S., & Pashayan, N. (2013). Cost effectiveness of the NHS breast screening programme: life table model. BMJ: British Medical Journal, 346.

Probabilistic sensitivity analysis (PSA)

4/10/2015 20

Useful Reading Lists• Briggs, Andrew, Mark Sculpher, and Karl Claxton. Decision modelling

for health economic evaluation. Oxford university press, 2006.• Philips, Zoe, et al. "Good practice guidelines for decision-analytic

modelling in health technology assessment”• Weinstein, Milton C., et al. "Principles of Good Practice for Decision

Analytic Modeling in Health-Care Evaluation: Report of the ISPORTask Force on Good Research Practices—Modeling Studies." Valuein health 6.1 (2003): 9-17.

• Weinstein, Milton C. "Recent developments in decision-analyticmodelling for economic evaluation." Pharmacoeconomics 24.11(2006): 1043-1053.

• Briggs, Andrew, and Mark Sculpher. "An introduction to Markovmodelling for economic evaluation." Pharmacoeconomics 13.4(1998): 397-409.sment." Pharmacoeconomics 24.4 (2006): 355-371.

4/10/2015 21

Practical

• All the parameters information has already filled

• Move to sheet 3• Follow the available directions• Remember that all parameters are named,

just put the formula directly and running your model

• If your sum probability is not 1.0 yet…just relax!, try once more time

4/10/2015 22