Download - Simulation Modeling Review
ABSTRACT
An examination of models used to inform health reform in the United States. For this
paper I examined the technical appendix or methodology paper associated with six
major microsimulations used to inform the passage of the Patient Protection and
Affordable Care Act (PPACA) and to predict the effects of implementing the act’s
provisions. See Table 1 for a list of specific models. I focus on models used to
address access with an eye toward possible uses of improved models and
simulations to address quality.
2 [ ]
Introduction
“It is hard to ignore that in 2006, the United States was number one in terms of
health care spending per capita but ranked 39th for infant mortality, 43rd for adult
female mortality, 42nd for adult male mortality, and 36th for life expectancy. Health
expenditures in the United States neared $2.6 trillion in 2010, over ten times the $256
billion spent in 1980. i The rate of growth in recent years has slowed relative to the late
1990s and early 2000s, but is still expected to grow faster than national income over the
foreseeable future.ii Since 2002, employer-sponsored health coverage for family
premiums have increased by 97%, placing increasing cost burdens on employers and
workers.iii In total, health spending accounted for 17.9% of the nation’s Gross Domestic
Product (GDP) in 2010.iv “Despite having the most expensive health care system, the
United States ranks last overall compared to six other industrialized countries—
Australia, Canada, Germany, the Netherlands, New Zealand, and the United Kingdom—
on measures of health system performance in five areas: quality, efficiency, access to
care, equity and the ability to lead long, healthy, productive lives.”v These facts have
fueled a question now being discussed in academic circles, as well as by government and
the public: Why do we spend so much to get so little?”vi
Solutions to the health care crisis; succinctly described above, focus on
improving the triad of Cost, Quality, and Access. Major reform efforts have culminated
in the passage of The Patient Protection and Affordable Care Act of 2010 (PPACA), which
[ ] 3
primarily addresses the issue of access. Economic models and simulations that
predicted policy outcomes supported reform efforts. Quality is presently under
investigation and modeling may have a role to play in this segment of reform.
Simulation modeling is becoming increasingly popular as a format for
understanding problems as well as designing and predicting the effects of health care
policy. The ability to model and simulate policy changes to predict their effects with
accuracy could help effectively prioritize legislation. They say you don’t know what you
don’t know but perhaps understanding how models and simulations are limited could
help to focus research efforts by addressing those limitations. Simulation models have
been shown to provide “reasonably accurate estimates, with confidence bounds of
~30%”.vii I believe simulations could become an even more reliable tool in policy
analysis and design over time with improved economic paradigms and standardized
methods as collaboration among disciplines is enabled by technology. The economic
models that were used were limited to the current neoclassical paradigm. As health
economics and heterodox views emerge and are full of possibilities, modeling and
simulation remains tied to the neoclassical economic paradigms.
Model & Simulation Basic Structure
Models provide the framework for simulations that predict outcomes. Microsimulations
predict behaviors over time. The models I examined all used the same basic
methodology of defining a baseline model, running a simulation with and without
specific policies applied, then comparing outcomes.
4 [ ]
View
In order to achieve the desired view, the modeler will make certain decisions about the
parameters, data sources, behavioral assumptions and outputs. View is not just a
reflection of micro vs. macro, it is also defined by what is considered endogenous and
exogenous to the model based on relevancy to desired outputs. Parameters are the
measurable factors used to define the system – microsimulations use the most discrete
parameters. In most cases they are represented as decision-making units or “agents”
subject to behavioral assumptions – providers, payers (public and private), individuals or
households, and employers. Data source selection is based on available data and best fit
for the desired view as well. The data may not match the view being sought precisely;
therefore, some manipulation takes place to achieve the most accurate approximation.
Closed or Open
If a model is closed, the population at the start remains constant throughout the
simulation. If it’s open then the population changes as agents enter and depart the
system.
Static or dynamic
Dynamic means behaviors change over time in response to system changes. In a static
model, actors maintain the same behaviors over time.
[ ] 5
Outputs
Coverage, cost and economic impact are the main outputs in the models I examined.
Health, however, is not an output. Models used to support quality efforts would need
to include health status as an output.
Models Examined
Purposes
I examined the methodology or technical documents for six major microsimulations
used to support the access improvement aspects of health reform. I looked at one
developed by the CBO to analyze an array of public policy options involving health
insurance coverage in 2007.viii Also, COMPARE, developed by RAND in 2009 as a way of
projecting how households and firms would respond to health care policy changes
based on economic theory and existing evidence from smaller changes.ix Jon Gruber of
MIT developed GMSim in 2009 to predict the effect of health market interventions on
the movement of people and dollars within the US healthcare system.x Lewin Group also
developed HBSM as a platform for analyzing the impact of health reform proposals -
created to provide comparisons of the impact of alternative health reform models on
coverage and expenditures for employers, governments and households.xi CMS
developed OHRM in 2011, a model that simulates the impact of health reform legislative
provisions on both household and employer decision-making in regard to health
insurance coverage and health spending. Finally, the Urban Institute developed HIPSM
6 [ ]
in 2011, which estimates the cost and coverage effects of proposed health care policy
options.xii
Parameters, Data, & Variables
Populations
Each model represents the population using some combination of three available
sources: Current Population Survey Annual Social and Economic Supplement (CPS-
ASEC), survey of Income and Program Participation (SIPP), and Medical Expenditure
Panel Survey Household Component (MEPS-HC).
Employers
Employers are represented and defined using MEPS-HC, MEPS-IC (the Insurance
Component), or the Kaiser Family Foundation/HRET Employer Health Benefits Survey.
Cost & Prices
Premium Prices
Medical Spending and Premium Prices are not treated as a behavioral characteristic of
agents but are instead estimated using three data sources MEPS-HC, MEPS-IC and
KFF/HRET. The microsimulations use one of two methods to estimate baseline
premiums for ESI.xiii Many modelers simulate insurance premiums using information on
health care expenditures and applying an estimate of insurer loading costs.xiv Loading
cost calculations change over time and the details of those changes are not studied or
understood, although they are incorporated into some of the models to determine the
[ ] 7
price of insurance offered using averages. This is somewhat helped by the 80/20 or
85/15 provision of ACA provided no major loopholes emerge. Baseline premium
constructions for the microsimulation models are represented in Table 2 (borrowed
from Jean M. Abraham, Ph.D., University of Minnesota). National Health Expenditure
Accounts (NHEA) and MEPS are the most relied upon sources of data used to
understand expenditures. While this may be fine in the higher views, microsimulations
would be better served with information about behaviors at a more discrete level.
Agents and behaviors
The patient-consumer, a key decision maker in these models, is represented as an
individual or a household. A central behavior modeled is the take-up of insurance.
Utility-based microsimulations build on an economic framework of utility maximization -
each agent (decision making entity – individual, household, firm etc.) compares all
available health insurance options and selects the option that best serves them.
Elasticity based microsimulations build on data collected from household surveys and
describes individual behaviors in terms of the responsiveness of insurance participation
to changes in the price of coverage. Utility tends to be more discrete and less
dependent on past data but data are often adjusted in order to more closely match
currently accepted elasticity data. Which is better depends on the view and desired
outputs of the model.
8 [ ]
Coverage
The effect of the coverage mandate is often predicted using data from Massachusetts
(97% compliance).
Limitations
Accuracy
A review of history suggests, unsurprisingly, that modelers are most accurate when
estimating the effects of incremental reforms that affect relatively small proportions of
the population.xv Accuracy is essentially determined by the modelers’ sense of validity
or, likely, what is perceived to be the audience’s validity tolerance. In my conclusion, I
will point to possibilities for improving accuracy in the aggregate by using standard
methods of development and loosely coupled component models.
Behavioral Assumptions
Behavioral assumptions may be the most limiting characteristic of these models. The
economic models that have been used to inform solutions are limited in their ability to
understand and express human behaviors and evaluate social impacts.
Behaviors
In modeling, behaviors are represented as formulas or logic derived from historical data
combined with theories that assume “rational behavior”xvi - “A major challenge facing
effective—mathematical—modeling… is to develop models that can take into account…
[ ] 9
[agents'] capacity for self-modification according to internally constructed and defined
goals. Basically, the assumptions required for tractable mathematics steer models away
from the most important aspects of human behavior.”xvii A deeper understanding of
human behavior could improve the use of the utilityxviii concept but it would need to
solve the problem of measuring value independent of price.
Value
Measuring the values inherent in transactions is a particularly difficult problem for
economists. Alfred Marshall (the father of neoclassical economics) thought that "We
might as reasonably dispute whether it is the upper or the under blade of a pair of
scissors that cuts a piece of paper, as whether value is governed by utility or cost of
production".xix I think the problem in this statement is that value is not the outcome of
tension between cost of production and utility – that is simply the optimal price a
producer can charge a consumer. It may seem semantic, but the difference becomes
clearer when we attempt to improve quality in health care. What a person is willing and
able to pay only represents the dollar value to the producer or seller although the price
does reflect some aspect of value to the consumer. I think including the degree to
which the transaction improves an individual’s wellbeing provides a more complete
picture of the value to the consumer. Because people have different resources, how
much they are able to pay says little about how valuable a good or service is to that
person in absolute terms. There is lost value in things that the consumer needs but
cannot afford. The value to society is the net contribution to productivity and the
improved wellbeing of others as a result of the transaction. In other words, a single
10 [ ]
transaction has different values to different agents throughout the system. For
example, the picture is incomplete if a model accounts for uncompensated care
(unemployed and uninsured individuals who seek medical care for which they cannot
pay) as a cost to the provider but, not for the fact that health status and employment
are bi-directionally related and by not receiving care, the probability of employment and
ability to pay for future care declines. So, by providing effective care, a provider still
adds value to the individual, the system and potentially saves itself the cost of future
uncompensated care. If the goal of economics is simply to improve the financial returns
of producers then neoclassical economic approach to defining value makes sense. But,
especially when the product is health status, the real value has other dimensions to the
individual, household, and society writ large.
Utilization
Utilization is a key component to understanding cost growth. The relationship
between coverage and utilization is a point of contention among experts. Studies that
provide evidence of this relationship are limited primarily to the RAND Health Insurance
Experiment from 1971 and a few statistical analyses related to major changes in
Medicare and Medicaid.
One major limitation of the RAND HIE for the purposes of assessing PPACA is that it
did not include an uninsured group. Additionally, since the completion of the RAND HIE,
many changes have occurred that could affect behaviors, for instance:
Direct to consumer advertising of pharmaceuticals were made legal in 1985
[ ]11
Reimbursement models have changed more than once
Managed care has peaked and faded
Tax treatment of private payers has changed
The Mental Health Parity Act of ’96 made psychiatric benefits more prevalent
The Balanced Budget Act of 97 further changed Medicare reimbursement
HIPPA, COBRA and CHIP
The rise of PPOs
Major hospital mergers
The Internet has become a more accessible source of information (affecting the
ability to maximize utility)
Applying policy changes effectively to agents in the 1971 regulatory environment
doesn’t include several changes that may affect the way those agents behave today.
Many statistical methods of updating consumer behavior have been used however,
qualitative analysis of individual perceptions of health insurance could lend a lot to the
effort to improve the effect of insurance plans on the quality of care.
Utilization in microsimulations appears to be corrolated mainly to coverage in a one-
way positive direction. (As coverage rises, utilization rises.) because the view is limited
to insurance coverage and the direct effects of PPACA related to coverage. Changing
one person’s health coverage will change an individual’s behavior; changing everyone’s
will also likely alter the demand facing providers and perhaps affect choices throughout
the health care system.xx
12 [ ]
Provider behavior is not detailed into these models even though reimbursement
structure and patient volume are thought to influence behavior and therefore
utilization. The CBO estimates that a 10% reduction in fees paid to physicians will
induce a 2.8% increase in service utilization in a 2007 background paper entitled Factors
Underlying the Growth in Medicare’s Spending for Physicians’ Services. Physician
behavior is not limited to response to reimbursement rates.
Defensive medicine is another behavior not included but, commonly thought to
influence utilization and therefore costs. Details of the relationship between public and
private reimbursement fees are unclear and may have an impact on provider behavior.
Provider decisions to participate in various networks and their market power to
negotiate dollar conversion factors are not well understood or represented. Finally,
physicians may vary in their commitment to patient outcomes based on other factors
that would be important to understand if we want to improve quality.
Employer behaviors are modeled in terms of the decision of employers to offer
health insurance. This decision too can be looked at in terms of elasticity of price or
utility. There is very little research about how employers make decisions regarding
offers of health insurance. There is some discussion about crowd out which is
considered by modelers in various ways. Some employers may be driven by a belief that
healthy employees are productive employees. Proving or disproving this and
quantifying it may be influential in improving quality.
Private payer behavior in the new (post PPACA) environment is almost a
complete unknown and may limit the capabilities of these models – particularly at the
[ ]13
state level. Concerns about crowd-out cannot be reliably modeled, simulated or
predicted without more certainty about employer, provider, and patient-consumer
behaviors.
Unknowns
There are non-price factors that remain unknown without some comprehensive
qualitative studies of all agents involved. Environment is exogenous to these models.
Education is not considered to be variable but may play a prominent role in health
status as indicated by many studies.
Another potential future consideration lies in upcoming immigration reform’s
impact on uncompensated care. Forecasts of the Medicare program substantially
underestimated the cost per insured person because the enormous change in coverage
induced systemic effects on health spending such as new investments in hospitals. This
effect may not apply – the characteristics of the uninsured are unknown - including and
especially health status.
The effect of mental health and wellbeing on somatic health can play a role in
improving how we define quality.
The Future
Much expert attention is now focused on attempting to improve quality of care;
that is to say, outcomes and cost-effectiveness. The predominant belief is that
insurance reimbursement models have unintentionally caused over-utilization of
14 [ ]
unnecessary care, redundancies, and other inefficiencies. But, how to pay for outcomes
is difficult for the payer to conceptualize. In my opinion, the value of a dollar spent is
dependent on whose perspective you take and the circumstances surrounding the
transaction. Utility does not accurately capture the depth of this reality at the individual
level. Further, the measure of economic success of any entity should not be limited
soley to the accumulation of wealth but should include quality of life and social impact
elements as well. Modeling to include a variety perspectives is a future possibility.
“Complexity economics is the application of complexity science to the problems of
economics. It studies computer simulations to gain insight into economic dynamics, and
avoids the assumption that the economy is a system in equilibrium.”xxi There has been
"little work on empirical techniques for testing dispersed agent complexity models".xxii
The modeling process could be more transparent and collaborative if given a
common format and design principles. There are software design concepts and
principles that could be applied to model design and development that might facilitate
their maturation. There may be ways for simulations to be linked to live data.
The accuracy of aggregating and projecting functions may improve if modelers
were to consider using major principles of open source programming, and service
oriented architecture using a common modeling language for collaboration. Then they
could work on smaller, more discrete – (read: more accurate) interoperable (loosely
coupled) pieces. Those more accurate pieces could then be used in a larger, shared
environment or copied to a separate environment creating a flexible, sharable
environment. Rapid, even real-time, turn around may be possible. Imagine a library of
[ ]15
agents and behaviors and data sets and a visual interactive environment where models
can be built and simulations can be run using compounds of other models.
I think any economic model that uses aggregate consumer data could be
redesigned to take the input of actual people. It would be quite interesting to compare
projected outcomes to outcomes produced by the participants. I would think it might
be particularly valuable in the limitations described of the Rand Health Insurance
Experiment. Consider conducting studies with people providing data directly to the
catalog. With more people working on a problem, more creative solutions can be
attained. Google, Mozilla, and Apple have successfully used these methods. To improve
data, the Maryland Exchange could include opt-in surveys or other such information
gathering techniques. There are many opportunities that might be worth exploring in
collaboration with computer scientists and qualitative researchers. The possibilities are
only limited by imagination.
Conclusions
These models have been useful despite the breadth of limitations identified. A strong
case can be made for further improving their usefulness and reliability with focused
research efforts, unified methods and collaborative development. The opportunities to
reconsider economic paradigms in the process of addressing the quality aspect of health
reform are huge. Prometheus gave fire & optimism to humans – with fire, they could be
optimistic; with optimism, they could use fire constructively, to improve the human
condition. “'Philanthropía'—loving what it is to be human—was thought to be the key
16 [ ]
to and essence of civilization.” xxiii I am optimistic that we have the tools we need to
achieve our potential and improve the human condition – models and simulations are
among those tools. New methods of evaluating the value of transactions are required to
effectively determine appropriate policy and market conditions that optimize the health
and well being of our citizens and modeling has a role to play in that process. All agents
have valuable insights and information and all perspectives are relevant. Modeling to
reveal the interrelationship between the micro and the macro; the various agents; and
the economic and social impacts are possibilities worth pursuing as we attempt to
improve the health of our citizens.
[ ]17
Table 1
18 [ ]
Table 2
Table 2. Model Comparison: Baseline Premium Construction
ESI Premium Construction Non-Group Premium Construction
CBO Expected aggregate spending of a firm’s workers Actuarial values assigned based on firm size, income, health status 9% to 27% loading fee applied depending on firm size Incorporates state-specific information
Factor-based approach using information on age, sex, health, experience, and state 29% loading fee applied Includes state-specific information/adjustments
GMSIM (Gruber) _Individual-level cost index (age/sex/health rating) averaged over synthetic firm; index aligned to employer premium distribution to assign premium to firm _Actuarial value assigned based on income and firm size _Loading fee implicit in premium _Adjusted for state variation in premiums
_Age-health status spending distribution from MEPS applied to CPS _Loading fee applied with fixed (15%) and variable components; varying load component equal to 30% of average unloaded non-group cost, based on age interval _Includes state-specific information/adjustments
COMPARE (RAND) _Firm-specific premiums based on experience-rated and
community-rated estimates (former use predicted spending of workers and dependents, while the latter use 12 pools based on 4 census regions by 3 firm sizes) _Actuarial values assigned based on firm size _8.3% to 20% loading fee applied depending on firm size _No detailed information available about incorporation of state-specific information
_Age-health status risk pools to estimate spending from MEPS-HC for those reporting individual coverage _Loading fee applied (details not available) _Includes state-specific information/ adjustments (approximated)
HBSM (Lewin) _Use expenditures of workers and apply rating practices (e.g., small group market) to estimate premiums; premiums also estimated for self-funded plans _Actuarial values assigned based on comparison of employer plan to standard benefits _5.5% to 40% loading fee applied depending on firm size _State code imputed to MEPS and state small-group rating rules applied
_Predicted spending and rating practices (age, sex, health status) _40% loading fee applied _Includes state-specific information/adjustments
HIPSM (Urban Institute) _Built from risk pools from underlying health care costs;
blend of actual and expected costs _Actuarial value assigned based on firm size _Loading fee applied depending on industry and firm size _Accounts for state variation in spending
_Predicted spending among those in non-group market; model based on age sex, health status, and “typical” rating rules _Loading fee applied (details not available) _Includes state-specific
[ ]19
information/adjustments
i Centers for Medicare and Medicaid Services, Office of the Actuary, National Health Statistics Group, National Health Care Expenditures Data, January 2012. ii Robert Wood Johnson Foundation, High and rising health care costs: Demystifying U.S. health care spending, October 2008. iii Kaiser Family Foundation and Health Research and Educational Trust. Employer Health Benefits 2012 Annual Survey. September 2012.iv Martin, A.B. et al. January 2012. Growth in US health spending remained slow in 2010; Health share of gross domestic product was unchanged from 2009. Health Affairs 31(1): 208-219.v http://www.commonwealthfund.org/News/News-Releases/2010/Jun/US-Ranks-Last-Among-Seven-Countries.aspx vi This article (10.1056/NEJMp0910064) was published on January 6, 2010, at NEJM.org. http://www.nejm.org/doi/full/10.1056/NEJMp0910064 vii Glied, Sherry and Tilipman, Nicholas 2012. Simulation Modeling of Health Care Policy Department of Health Policy and Management, Mailman School of Public Health, Columbia University, New York, New York 10032.viii CBO October 2007 Health Insurance Simulation Model: A Technical Descriptionix RAND Corporation 2006. The Health Insurance Experiment A Classic RAND Study Speaks to the Current Health Care Reform Debatex Gruber, Jonathan Microsimulation Estimates of the Effects of Tax Subsidies for Health Insurance Massachusetts Institute of Technology, National Tax JournalVol. Llll, No. 3, Part 1xi The Lewin Group 2009. The Health Benefits Simulation Model (HBSM): Methodology and Assumptionsxii Blumberg, Linda J. Shen, Yu-Chu Nichols, Len M. Buettgens, Matthew Dubay, Lisa C. McMorrow, Stacey 2003. The Health Insurance Reform Simulation Model (HIRSM): Methodological Detail and Prototypical Simulation Resultsxiii Abraham, Jean M. March 2012. Predicting the Effects of the Affordable Care Act: A Comparative Analysis of Health Policy Microsimulation Models from the State Health Reform Assistance Network. xiv Glied, Sherry and Tilipman, Nicholas 2012. Simulation Modeling of Health Care Policy Department of Health Policy and Management, Mailman School of Public Health, Columbia University, New York, New York 10032.xv Zabinski D, Selden T, Moeller J, Banthin J. 1999. Medical savings accounts: microsimulation results from a model with adverse selection. J. Health Econxvi Rational choice theory, also known as choice theory or rational action theory, is a framework for understanding and often formally modeling social and economic behavior. Rationality, interpreted as "wanting more rather than less of a good", is widely used as an assumption of the behavior of individuals in microeconomic models and analysis and appears in almost all economics textbook treatments of human decision-making. It is also central to some of modern political science, sociology, and philosophy. It attaches "wanting more" to instrumental rationality, which involves seeking the most cost-effective means to achieve a specific goal without reflecting on the worthiness of that goal. - http://en.wikipedia.org/wiki/Rational_choice_theory xvii http://www.pnas.org/content/99/suppl.3/7288.short
xviii In economics, utility is a representation of preferences over some set of goods and services. Preferences have a utility representation so long as they are transitive, complete, and continuous. Utility is usually applied by economists in such constructs as the indifference curve, which plot the combination of commodities that an individual or a society would accept to maintain a given level of satisfaction. Individual utility and social utility can be construed as the value of a utility function and a social welfare function respectively. When coupled with production or commodity constraints, under some assumptions, these functions can be used to analyze Pareto efficiency, such as illustrated by Edgeworth boxes in contract curves. Such efficiency is a central concept in welfare economics. - http://en.wikipedia.org/wiki/Utility#Utility_as_probability_of_success
xix Marshall, Alfred Principles of Economics (1890)xx Remler DK, Zivin JG, Glied SA. 2004. Modeling health insurance expansions: effects of alternate approaches. J. Policy Anal. Manag.xxi http://en.wikipedia.org/wiki/Complexity_economics xxii Rosser, J. Barkley, Jr. "On the Complexities of Complex Economic Dynamics" Journal of Economic Perspectives, V. 13, N. 4 (Fall 1999): 169-192.xxiii http://nvs.sagepub.com/content/39/3/385