data for outcomes research andy bindman md department of medicine, epidemiology and biostatistics
TRANSCRIPT
Data for Outcomes Research
Andy Bindman MD
Department of Medicine, Epidemiology and Biostatistics
What is Outcomes Research
Studies of the quality of care as judged by patients’ outcomes
IOM domains of quality– Effectiveness– Safety– Timeliness– Equity– Efficiency– Patient-Centered
Donabedian Model of Quality
Structure Process Outcome
Donabedian Model of Quality
Structure Process OutcomeNumber of nurses per hospital bed
Physicians per capita
Donabedian Model of Quality
Structure Process OutcomeBeta blocker following MI
Immunizations
Donabedian Model of Quality
Structure Process OutcomeSurvival
Functional status
Satisfaction
Which is Best to Monitor Quality?
Structure - necessary but not sufficient
Process - many things we do/recommend don’t have proven health benefit
Outcomes - our ultimate responsibility but related to more than just the care we
provide
Predictors of Outcomes
Outcomes = intrinsic patient risk factors
treatment effectiveness
quality of care
random chance
Goals of Risk-Adjustment
Account for intrinsic patient risk factors before making inferences about effectiveness, efficiency, or quality of care
Minimize confounding bias due to nonrandom assignment of patients to different providers or systems of care
How is Risk Adjustment Done
On large datasets Uses measured differences in compared groups Model impact of measured differences between
groups on variables shown, known, or thought to be predictive of outcome so as to isolate effect of predictor variable of interest
When Risk-Adjustment May Be Inappropriate
Processes of care which virtually every patient should receive (e.g., immunizations, discharge instructions)
Adverse outcomes which virtually no patient should experience (e.g., incorrect amputation)
Nearly certain outcomes (e.g., death in a patient with prolonged CPR in the field)
Too few adverse outcomes per provider
When Risk-Adjustment May Be Unnecessary
If inclusion and exclusion criteria can adequately adjust for differences
If assignment of patients is random or quasi-random
When Risk-Adjustment May Be Impossible
If selection bias is an overwhelming problem If outcomes are missing or unknown for a large
proportion of the sample If risk factor data (predictors) are extremely
unreliable, invalid, or incomplete
Data Sources for Risk-Adjustment
Administrative data are collected primarily for a different purpose (billing), but are commonly used for risk-adjustment
Disease registries
Sources of Administrative Data
Federal Government– Medicare– VA
State Government– Medicaid (Medi-Cal)– Hospital Discharge Data
Private Insurance
Dataset Resources
http://www.epibiostat.ucsf.edu/courses/RoadmapK12/PublicDataSetResources/
http://base.google.com/base/search?a_n0=clinical+trials&a_y0=9&hl=en&gl=US
Advantages of Administrative Data
Computerized, inexpensive to obtain and use Uniform definitions Ongoing data monitoring and evaluation Diagnostic coding (ICD-9-CM) guidelines Opportunities for linkage (vital stat, cancer)
Administrative Hospital Discharge Data Admission Date • Race Discharge Date • Sex Type of Admission • Date of Birth Source of Admission • Zip Code Principal Diagnosis • Patient SSN Other Diagnoses • Total Charges Principal Procedure and Date • Expected Source of Payment Other Procedures and Dates Disposition of Patient External Cause of Injury Pre-hospital Care and Resuscitation (DNR)
Disadvantages of Administrative Data
No control over data collection process Missing key information about physiologic and
functional status Quality of diagnostic coding can vary across sites Non capture of out of plan/out of hospital/out of state
events
Linking Administrative Data
Strategy for enhancing number of predictor or outcomes variables
Linkage dependent on reliable shared identifiers such as social security numbers in both datasets
Probabilistic matching of less specific variables (age, sex, race, date of birth, etc)
Some Routinely Available Data Linkages
California hospital discharge data and vital statistics– Example: 30 day mortality following AMI
SEER -Medicare– Example: utilization patterns for those with breast cancer
National Health Interview Survey-Medical Expenditure Panel Survey– Example: health care costs for those with self-reported chronic
conditions
California Hospital Discharge Data and Medicaid Eligibility Files
Creates a continuous monthly record of an individual’s pattern of Medicaid enrollment
Discharge data captures all hospitalizations regardless of whether in or out of Medicaid
Have found a 3 fold increase in hospitalizations for ambulatory care sensitive conditions for those with interrupted Medicaid coverage
Health Plans/Delivery Systems
Health insurance claims– Inpatient, outpatient, pharmacy, diagnostics, etc
Electronic Medical Records– VA– Kaiser– SF Dept of Public Health (THREDS)
THREDS
~120,000 patients per year seen in DPH clinics/SFGH
Data begin in 1996 and updated daily Includes demographics, insurance status,visit hx,
diagnostic codes, tests ordered and results, pharmacy, link to death registry
http://gcrcsfgh.ucsf.edu/?page=threds
Disease Registries Attempt to capture all or large sample of the cases of a
specified condition Often include more clinical information than
administrative datasets Many of these can support assessments of survival
beyond acute period May require permission/approved protocol to access all
or some of the data
Example Registries UNOS:national registry of patients with end stage renal
disease
SEER Cancer Registry
Coronary Artery Bypass Graft Surgery: California Office of Statewide Health Planning and Development
Doing Your Own Risk-Adjustment vs. Using an Existing Product
Is an existing product available or affordable? Would an existing product meet my needs?
- Developed on similar patient population
- Applied previously to the same condition or procedure
- Data requirements match availability
- Conceptual framework is plausible and appropriate
- Known validity
Conditions Favoring Use of an Existing Product
Need to study multiple diverse conditions or procedures
Limited analytic resources Need to benchmark performance using an external
norm Need to compare performance with other providers
using the same product Focus on resource utilization, possibly mortality
A Quick Survey of Existing ProductsHospital/General Inpatient
APR-DRGs (3M) Disease Staging (SysteMetrics/MEDSTAT) Patient Management Categories (PRI) RAMI/RACI/RARI (HCIA) Atlas/MedisGroups (MediQual) Cleveland Health Quality Choice Public domain (MMPS, CHOP, CSRS, etc.)
A Quick Survey of Existing ProductsIntensive Care
APACHE MPM SAPS PRISM
A Quick Survey of Existing ProductsOutpatient Care
Resource-Based Relative Value Scale (RBRVS) Ambulatory Patient Groups (APGs) Physician Care Groups (PCGs) Ambulatory Care Groups (ACGs)
How Do Commercial Risk-Adjustment Tools Perform
Better than age/sex to predict health care use/death Better retrospectively (~30-50% of variation) than
prospectively (~10-20% of variation) Lack of agreement among measures More than 20% of in-patients assigned very different
severity scores depending on which tool was used (Iezzoni, Ann Intern Med, 1995)
Co-Morbidity or Severity?
Are patients at risk for an outcome because they have multiple conditions (co-morbidities), a more severe version of a disease (disease stage) or both?
Before adjusting for co-morbidity and or severity consider whether either is a complication of treatment (or non treatment) rather than an independent health characteristic of the patient
Summary Risk adjustment is a multivariate modeling technique
designed to control for patient characteristics so that judgments can be made about the quality of care
Risk adjustment requires large datasets such as administrative datasets or disease registries
Commercial risk adjustment products exist for patients in different health care settings
There are many reasons why one might choose to develop a risk adjustment model - we will talk about how to do this next week!