Approaching Predictive ModelingApproaching Predictive Modeling

Steven S. Eisenberg, MDSteven S. Eisenberg, MDChief Science OfficerChief Science OfficerUnited HealthCareUnited HealthCare

4th Annual DM Colloquium June 23, 2005

OverviewOverview

•• Types of predictive modelingTypes of predictive modeling•• What predictive modeling can doWhat predictive modeling can do•• The usual The usual •• The less usual (focus)The less usual (focus)•• SummarySummary•• Q&AQ&A

The future The future ain't what ain't what it used to it used to

bebe

My Favorite Philosopher on Predictive My Favorite Philosopher on Predictive ModelsModels

What Are We Hoping to Learn?What Are We Hoping to Learn?•• From an From an actuarialactuarial perspectiveperspective

–– more accurately predict utilization and cost of more accurately predict utilization and cost of populations populations

–– adjunct to better and more accurate pricing decisions adjunct to better and more accurate pricing decisions –– Perhaps flattening the actuarial cyclePerhaps flattening the actuarial cycle

•• From a From a medical managementmedical management perspectiveperspective–– identify individuals at very high risk for high utilizationidentify individuals at very high risk for high utilization–– open the door to managing those individuals open the door to managing those individuals

•• case management/disease management case management/disease management •• prior to the high utilization prior to the high utilization

–– mitigating some of the impactmitigating some of the impact•• healthier populationhealthier population•• lower costslower costs

Leading toLeading toImprovements in:

Improvements in:Disease Disease Management

ManagementPatient CarePatient Care

QualityQuality

Cost Cost ManagementManagement

The ToolsThe Tools

•• Model is a mathematical representation of realityModel is a mathematical representation of reality–– Relevant, consistent input data are neededRelevant, consistent input data are needed–– The outcome The outcome mustmust be measurablebe measurable–– A way to relate the two mathematically A way to relate the two mathematically mustmust existexist

•• Currently well over 100 different modelsCurrently well over 100 different models•• For our (healthcare) purposes these really break For our (healthcare) purposes these really break

down into three groups:down into three groups:–– Artificial IntelligenceArtificial Intelligence–– Statistical ModelsStatistical Models–– Rules Based AlgorithmsRules Based Algorithms

ACG's

DCG's

ETG's

ERG's

CCG's

CRG's, Others

CART

One Size Fits All ?One Size Fits All ?

What Predictive Modeling Can DoWhat Predictive Modeling Can Do

•• Stratify membersStratify members–– Primary or secondaryPrimary or secondary–– Enhance impact of interventionsEnhance impact of interventions

•• Identification of high utilizersIdentification of high utilizers–– Assign risk scoresAssign risk scores

•• Describe comparative severity of illnessDescribe comparative severity of illness

•• Identify members not receiving proper care/requiring Identify members not receiving proper care/requiring special carespecial care–– Case Management/Disease ManagementCase Management/Disease Management

•• Highlight inconsistency/inefficiency of careHighlight inconsistency/inefficiency of care•• Prospectively identify adverse eventsProspectively identify adverse events•• Allow focused interventionsAllow focused interventions

–– Maximize benefits of disease managementMaximize benefits of disease management–– Allow intervention earlier in disease cycleAllow intervention earlier in disease cycle

•• Financial forecasting Financial forecasting –– Actuarial riskActuarial risk

Application of Predictive ModelsApplication of Predictive Models

•• Identifying/managing complexly ill members Identifying/managing complexly ill members (hospitalization avoidance)(hospitalization avoidance)

•• Refining disease management strategiesRefining disease management strategies•• Managing pharmacy services (integrated with Managing pharmacy services (integrated with

medical management)medical management)•• Underwriting more preciselyUnderwriting more precisely•• Reimbursement based on illness burdenReimbursement based on illness burden•• Assessing physician management strategiesAssessing physician management strategies

Additional Uses of ModelingAdditional Uses of Modeling•• Influence adoption of best practicesInfluence adoption of best practices•• Track effectiveness of interventionsTrack effectiveness of interventions•• Establish pay for performanceEstablish pay for performance•• Set more accurate premiumsSet more accurate premiums•• Develop contracts with providersDevelop contracts with providers

–– ActuarialActuarial•• Help plan network compositionHelp plan network composition

–– Based on member needsBased on member needs•• Develop specific, targeted interventionsDevelop specific, targeted interventions

–– Probabilities for certain outcomesProbabilities for certain outcomes–– Practice guidelinesPractice guidelines–– Practice standardizationPractice standardization

•• Decrease variationDecrease variation

Choosing the Right ModelChoosing the Right Model

•• There is no one model that does There is no one model that does everything the besteverything the best

•• What are you trying to do?What are you trying to do?–– Is there a model that fits the problem (or Is there a model that fits the problem (or

the data) better?the data) better?

•• What is available to you?What is available to you?–– Can you use whatever model/data Can you use whatever model/data

you have available?you have available?

•• What can you afford?What can you afford?•• What are you willing to compromise What are you willing to compromise

on?on?

The “Usual Suspects”The “Usual Suspects”

•• Most DM programs / Most DM programs / HealthplansHealthplans use grouper use grouper rules based algorithms prospectivelyrules based algorithms prospectively–– ERG’s/ETG’sERG’s/ETG’s–– DCG’sDCG’s–– ACG’sACG’s

•• Most Fraud & Abuse programs useMost Fraud & Abuse programs use–– Decision treesDecision trees–– Rules based algorithmsRules based algorithms–– Neural netsNeural nets–– Pattern analysisPattern analysis

Some Less Usual SuspectsSome Less Usual Suspects

•• There are some “inexpensive” and less There are some “inexpensive” and less cumbersome ways to do cumbersome ways to do somesome predictive predictive modelingmodeling–– Trend linesTrend lines–– Time seriesTime series–– Markov ModelsMarkov Models–– Pharmacy only modelsPharmacy only models

Statistical Statistical ModelsModels

Trend Lines = RegressionTrend Lines = Regression

•• Definition: the technique of fitting a simple equation to Definition: the technique of fitting a simple equation to real data pointsreal data points

•• Linear regression is the most common typeLinear regression is the most common type–– e.g. y=e.g. y=a+bx+ea+bx+e

•• Other TypesOther Types–– Multilinear regressionMultilinear regression–– Logistic RegressionLogistic Regression

•• It is a mathematical way of It is a mathematical way of assessing the impact and contribution of diverse/disparate assessing the impact and contribution of diverse/disparate variables on a process or outcomevariables on a process or outcome

•• Linear regressionLinear regression is used for is used for continuous variablescontinuous variables•• Logistic regressionLogistic regression is used for is used for binomial variablesbinomial variables

Change in $PMPM Cost Over TimeChange in $PMPM Cost Over TimeChange in $PMPM Cost Over Time

Trend LinesTrend Lines

•• “Poor man’s” Predictive model“Poor man’s” Predictive model•• Built into ExcelBuilt into Excel

ExampleExampleChange in PMPM Cost over TimeChange in PMPM Cost over TimeChange in PMPM Cost over Time

Doing a PredictionDoing a Prediction

Change in PMPM Cost over TimeChange in PMPM Cost over TimeChange in PMPM Cost over Time

Double click on the trendline

Double click on Double click on the the trendlinetrendline

Project the Project the TrendlineTrendline 6 Months Forward6 Months Forward

By doing so you are making By doing so you are making the assumption thatthe assumption that

all the variables are and will remainall the variables are and will remainconstantconstant

Change in PMPM Cost over TimeChange in PMPM Cost over TimeChange in PMPM Cost over Time

The PredictionThe PredictionThe Prediction

Time Series Time Series •• Time series analysis accounts for the fact that data Time series analysis accounts for the fact that data

points taken over time may have an internal structure points taken over time may have an internal structure reflecting a pattern or more than one pattern reflecting a pattern or more than one pattern –– Trend Trend –– Seasonal variation (seasonality)Seasonal variation (seasonality)

•• General aspectsGeneral aspects–– TrendTrend

•• systematic linear or (most often) nonlinear component that changsystematic linear or (most often) nonlinear component that changes es over time and does not repeat or at least does not repeat withinover time and does not repeat or at least does not repeat within the the time range captured by our data time range captured by our data

–– SeasonalitySeasonality•• May have a relationship similar to trend May have a relationship similar to trend

but tends to repeat itself in but tends to repeat itself in sytematicsytematicintervals over time intervals over time

Common Uses of Time SeriesCommon Uses of Time Series•• Economic Forecasting Economic Forecasting •• Sales Forecasting Sales Forecasting •• Budgetary Analysis Budgetary Analysis •• Stock Market Analysis Stock Market Analysis •• Yield Projections Yield Projections •• Process and Quality Process and Quality

Control Control •• Inventory Studies Inventory Studies •• Workload Projections Workload Projections •• Utility Studies Utility Studies

•• Census AnalysisCensus Analysis

Example:Example:Pharmacy UtilizationPharmacy Utilization

ExampleExample

•• Pharmacy Utilization over time Pharmacy Utilization over time –– Excel w Trend Excel w Trend LineLine

Total Pharmacy $ Paid by Month

$0.00

$2,000,000.00

$4,000,000.00

$6,000,000.00

$8,000,000.00

$10,000,000.00

$12,000,000.00

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun

TrendLine

TrendTrendLineLine

Example When Analyzed via Example When Analyzed via Time SeriesTime Series

JanJan JunJunFebFeb MarMar AprApr MayMay JulJul AugAug SepSep OctOct NovNov DecDec

Trend ComponentTrend Trend ComponentComponent

Seasonality ComponentSeasonality Seasonality ComponentComponent

95%CI

95%95%CICI

ExampleExample

The Prediction Using Time SeriesThe Prediction Using Time Series

Markov ModelsMarkov Models

–– A probabilistic process over a finite set of A probabilistic process over a finite set of possibilities, {S1, ..., Sk}, usually called its possibilities, {S1, ..., Sk}, usually called its statesstates

•• The model is capable of The model is capable of showing the probability of any given showing the probability of any given state coming up nextstate coming up next, pr(xt=Si), and this may depend on the prior , pr(xt=Si), and this may depend on the prior history (to thistory (to t--1). 1).

–– originally introduced in the late 1960’s and early originally introduced in the late 1960’s and early 1970’s 1970’s

•• used for a variety of applications in science and technologyused for a variety of applications in science and technology–– Markov disease state simulations Markov disease state simulations portray the progression of portray the progression of

disease over timedisease over time•• It does this by dividing the disease into discrete “It does this by dividing the disease into discrete “statesstates,”,”•• specifying the risks of progression per unit time between those specifying the risks of progression per unit time between those states,states,•• assigning utilities and costs to each state, assigning utilities and costs to each state, •• and conducting a simulation with a defined endand conducting a simulation with a defined end--point.point.

Markov Models Are Often Represented Markov Models Are Often Represented GraphicallyGraphically

•• State Transition FigureState Transition Figure•• Transition Probability Transition Probability

MatrixMatrix

The Midwest Healthplan ProjectThe Midwest Healthplan Project

A Real World Example of UsingA Real World Example of UsingMarkov ModelsMarkov Models

Project OverviewProject Overview

•• A large Midwest Healthplan wants to understand A large Midwest Healthplan wants to understand the movement of members by segments over the movement of members by segments over time and be able to identify future high cost time and be able to identify future high cost utilizers utilizers –– Leverage their investment in medical management Leverage their investment in medical management

programs/techniquesprograms/techniques–– Help control runaway medical inflationHelp control runaway medical inflation–– Make points with their large employers by showing Make points with their large employers by showing

proactive managementproactive management•• Wanted to do it without having to buy and support Wanted to do it without having to buy and support

another technologyanother technology

Project Overview, cont.Project Overview, cont.

•• Focusing on certain higher risk parts of their book of business Focusing on certain higher risk parts of their book of business we developed the Markov Model for them to better understand we developed the Markov Model for them to better understand their historical movement of members from one disease state their historical movement of members from one disease state level (severity) to another over time level (severity) to another over time –– at a population levelat a population level

•• The model is then being run to predict what is predicted to The model is then being run to predict what is predicted to occur over the ensuing 6occur over the ensuing 6--12 months12 months

•• The Healthplan can then deThe Healthplan can then de--encrypt and identify these encrypt and identify these members and reach out to them with case/disease members and reach out to them with case/disease managementmanagement–– at an individual levelat an individual level

•• Outcomes and costs can be monitored over timeOutcomes and costs can be monitored over time–– PrePre-- Post AnalysisPost Analysis–– Matched Cohort AnalysisMatched Cohort Analysis

Diabetes Example Diabetes Example –– Predicting Member Predicting Member Counts Counts (By Age Band and Gender)(By Age Band and Gender)

(all diabetes)

The Distribution of Disease StatesThe Distribution of Disease States

Baseline Population LevelBaseline Population Level

The Markov Model PredictionThe Markov Model Prediction

Member # A012556

Individual Member Transition PredictionIndividual Member Transition Prediction

1

3

4

273.17%20.50%

00.03%

6.30% 0%

Final Analysis*Final Analysis*

* with thanks to Ken Kubisty, Bearing Point Solutionswith thanks to Ken Kubisty, Bearing Point Solutions

•• Rules based, member centric Rules based, member centric •• Uses only pharmacy, demographic, and eligibility data Uses only pharmacy, demographic, and eligibility data

as the inputsas the inputs•• Developed by Developed by Symmetry Health Data SystemsSymmetry Health Data Systems•• Assigns weighted risk score individuals based onAssigns weighted risk score individuals based on

–– distribution of drugs a member is taking, age, and sexdistribution of drugs a member is taking, age, and sex–– weights differ by:weights differ by:

•• Threshold assumption Threshold assumption ---- $250K, $100K, $50K, $25K$250K, $100K, $50K, $25K•• StopStop--loss amount is typically used as the cutloss amount is typically used as the cut--off pointoff point

•• Combines PRG profile and weightsCombines PRG profile and weights–– represents relative health risk for a member for future represents relative health risk for a member for future

periodperiod

Pharmacy Risk GroupsPharmacy Risk Groups

AdvantagesAdvantages

•• DataData–– AvailabilityAvailability–– Cleanliness and accuracyCleanliness and accuracy–– TimelinessTimeliness

•• Cost effective Cost effective –– IT and administrationIT and administration•• Supports more frequent risk assessmentSupports more frequent risk assessment•• Predictive accuracyPredictive accuracy

–– R squared and other predictive measures close R squared and other predictive measures close to those of claims based systemsto those of claims based systems

DisadvantagesDisadvantages

•• Pharmacy Pharmacy plusplus medical claims can improve medical claims can improve accuracy accuracy –– e.g.e.g.–– Members w/ medical use, w/o pharmacy useMembers w/ medical use, w/o pharmacy use–– Conditions where drugs not integral component Conditions where drugs not integral component

of treatmentof treatment–– Further stratification within a diseaseFurther stratification within a disease

•• Incentives Incentives –– linking risk to specific drug treatments may not linking risk to specific drug treatments may not

provide best incentives for efficient and quality careprovide best incentives for efficient and quality care•• Linking risk to disease prevalence Linking risk to disease prevalence

–– harder to do without disease categorizationharder to do without disease categorization

ExampleExample

0.5340.534Antidepressants, Antianxiety Antidepressants, Antianxiety AgentsAgents

32000 32000 –– Fluoxetine HCLFluoxetine HCL

10.32710.327Agents to treat ALSAgents to treat ALS34604 34604 –– RiluzoleRiluzole

11.89211.892Prospective Risk ScoreProspective Risk Score

1.0311.031Males, 55 to 64Males, 55 to 64Male, 58Male, 58

AgeAge--Sex GroupSex GroupAgeAge--SexSex

WeightWeightPRG ProfilePRG ProfileDCCsDCCs

The DHS Pilot ProjectThe DHS Pilot Project

A Real World Example of UsingA Real World Example of UsingPRG’sPRG’s

State of MN, Department of Human State of MN, Department of Human ServicesServices

•• Desire to extend disease management to FFS Desire to extend disease management to FFS Medicaid populationMedicaid population–– ~100,000~100,000–– High risk populationHigh risk population

•• High morbidity/Chronic IllnessHigh morbidity/Chronic Illness•• Very low incomeVery low income•• Distrust of managed care Distrust of managed care

•• Need to demonstrate to legislature that concepts Need to demonstrate to legislature that concepts work for this populationwork for this population–– Establish the Establish the opportunityopportunity for a formalized DM approach to for a formalized DM approach to

this populationthis population–– Collect a series of success storiesCollect a series of success stories–– Provide the data and the stories to the legislatureProvide the data and the stories to the legislature

The ApproachThe Approach

•• Use the tool as the first pass to provide the Use the tool as the first pass to provide the basic output file basic output file –– Rank order of patients by prospective riskRank order of patients by prospective risk

•• Analyze the medical history of the highest risk Analyze the medical history of the highest risk members members –– Create a clinical vignette of their medical Create a clinical vignette of their medical

historyhistory–– ? Focus on those conditions and diseases that ? Focus on those conditions and diseases that

have a track record of success in disease or have a track record of success in disease or case management case management

•• Focus only on the top few percent of highest Focus only on the top few percent of highest risk membersrisk members–– About 250 for the pilot projectAbout 250 for the pilot project

The PilotThe Pilot•• Members includedMembers included

–– Medicaid FFS onlyMedicaid FFS only–– Continuously enrolled for at least 18 monthsContinuously enrolled for at least 18 months

•• Members excludedMembers excluded–– Primary serious mental health diagnosesPrimary serious mental health diagnoses–– Members in skilled or unskilled nursing homesMembers in skilled or unskilled nursing homes

•• Primary concern is cognitionPrimary concern is cognition

•• Need for a short time frameNeed for a short time frame–– Program began mid January ‘04Program began mid January ‘04–– Program ended mid June ’04Program ended mid June ’04

•• Final Dataset Final Dataset –– 14,443 members 14,443 members –– members with highest prospective risk score had a complete members with highest prospective risk score had a complete

claims dump for the prior 18 monthsclaims dump for the prior 18 months–– Highest 2% underwent detailed claims analysisHighest 2% underwent detailed claims analysis

Results Results –– Top 24 Patients Top 24 Patients •• 11 females, 13 males11 females, 13 males

–– Range 20Range 20--63 y.o. 63 y.o. •• average 45.4 y.o.average 45.4 y.o.

–– Costs for 18 months Costs for 18 months •• $5,432 $5,432 -- $491,331$491,331•• Average $117,945Average $117,945

–– Total # of Claims/Pt Total # of Claims/Pt •• 195 195 –– 2,5312,531•• Average 1,129Average 1,129

•• DiagnosesDiagnoses–– Diabetes Diabetes -- 1111–– Chronic Renal Chronic Renal

Failure/ESRD Failure/ESRD -- 99–– Post kidney transplant Post kidney transplant

-- 55–– HIV positive HIV positive –– 55–– AIDS AIDS -- 22–– Cystic Fibrosis Cystic Fibrosis -- 33–– Active malignancy Active malignancy -- 22–– Smokers Smokers -- 55

Clinical VignettesClinical Vignettes

•• Member #1 Member #1 is a 30 year old woman with long standing is a 30 year old woman with long standing Cystic Fibrosis. She has problems with malaise, fatigue, skin Cystic Fibrosis. She has problems with malaise, fatigue, skin disease, and hair loss as well as multiple dislocated vertebrae disease, and hair loss as well as multiple dislocated vertebrae in her neck. She had a very rocky 18 month course with in her neck. She had a very rocky 18 month course with multiple recurring episodes of pneumonia requiring multiple recurring episodes of pneumonia requiring hospitalization as well as multiple episodes of dehydration and hospitalization as well as multiple episodes of dehydration and bouts of painful Herpes Simplex. bouts of painful Herpes Simplex.

•• Her prospective risk score was over Her prospective risk score was over 2727 and she had the and she had the highest total expenditure in the dataset of highest total expenditure in the dataset of $491,000$491,000 for the 18 for the 18 month time period.month time period.

Summary and ConclusionsSummary and Conclusions

•• Predictive Modeling is a toolPredictive Modeling is a tool–– It is a method, not an answer in itselfIt is a method, not an answer in itself–– Modeling is only an arrow to add to the quiverModeling is only an arrow to add to the quiver——it is not it is not

the whole quiverthe whole quiver

•• Consider the use of multiple modelsConsider the use of multiple models–– just as multiple forms of assessment are done for just as multiple forms of assessment are done for

diagnosis diagnosis –– May increase reliability and accuracyMay increase reliability and accuracy

•• Predictive modeling is also a way to better Predictive modeling is also a way to better understand your data accuracyunderstand your data accuracy–– and conversely where you have problems with and conversely where you have problems with

your datayour data

Challenges of Predictive ModelingChallenges of Predictive Modeling•• All of the models are more accurate at the All of the models are more accurate at the

aggregate (population) level than at the aggregate (population) level than at the individual levelindividual level–– Most results published are at the population levelMost results published are at the population level–– Population level may work well for actuarial Population level may work well for actuarial –– Medical Mgmt is typically focused on the individualMedical Mgmt is typically focused on the individual

•• You can adjust (improve) the results by You can adjust (improve) the results by changing the threshold, the specificity, changing the threshold, the specificity, sensitivity, etc.sensitivity, etc.–– Models demonstrate better R squared values when outliers Models demonstrate better R squared values when outliers

are excludedare excluded•• e.g. Stope.g. Stop--loss amountsloss amounts

–– But the But the outliers may be exactly the members that you are outliers may be exactly the members that you are trying to findtrying to find to have the impact you are looking forto have the impact you are looking for

Summary & ConclusionsSummary & Conclusions

•• There is no one clearly superior There is no one clearly superior predictive modelpredictive model–– Certain approaches may be more valuable for Certain approaches may be more valuable for

underwritingunderwriting–– Other approaches may be more valuable for Other approaches may be more valuable for

managing caremanaging care

•• The The actionabilityactionability quotient must also be consideredquotient must also be considered–– If you cannot act on the results, the study is merely If you cannot act on the results, the study is merely

interesting interesting

•• Linking models with interventions can help you Linking models with interventions can help you improve quality and efficiency of careimprove quality and efficiency of care

Summary and ConclusionsSummary and Conclusions

•• All predictive models tend to All predictive models tend to overpredictoverpredict low utilizers and low utilizers and under predict very high utilizersunder predict very high utilizers–– Some of this may be mitigated by using a threshold and excludingSome of this may be mitigated by using a threshold and excluding costs costs

beyond a certain point (typically at a stopbeyond a certain point (typically at a stop--loss amount)loss amount)–– But this can exclude exactly those folks you may want to identifBut this can exclude exactly those folks you may want to identifyy

•• None of the models can predict “random” eventsNone of the models can predict “random” events–– TraumaTrauma–– PregnancyPregnancy–– Catastrophic ClaimsCatastrophic Claims

•• Measurement of “success” is very difficultMeasurement of “success” is very difficult–– How do you “How do you “unmanageunmanage” a case to determine savings?” a case to determine savings?

•• But the tools are very valuable, getting better, But the tools are very valuable, getting better, and and cancan be made to workbe made to work–– You will see increasing success over the next You will see increasing success over the next

several yearsseveral years

You can either take action You can either take action or you can hang back and or you can hang back and

hope for a miracle.hope for a miracle.

Miracles are great, but Miracles are great, but they are so unpredictablethey are so unpredictable..

Peter Peter DruckerDrucker