Understanding the Outcomes Research Toolbox

Justin B. Dimick, MD, MPH Henry King Ransom Professor of Surgery Director, Center for Healthcare Outcomes & Policy University of Michigan ACS Outcomes Research Course 2014

Agenda:

1. Conceptual overview – three major areas of research

2. For each area, explore major

problems and tools for fixing them

Comparative effectiveness

Randomized clinical trials

Quality of life

? Qualitative research

Patient centered outcomes

Decision analysis

Policy evaluation

Disparities

How can we make sense of these different research areas?

Groups of 3 to 4

5 minutes

Evaluating Disease Management (Patient-Level)

Healthcare Delivery System (Macro-systems)

Understanding Local Provision of Care (Micro-systems)

Patient-Centered Outcomes Comparative Effectiveness

Data Synthesis

Quality Improvement Patient Safety

Policy Evaluation Payment Reform

Work Force

Coverage Decisions Technology Adoption

Cost Effectiveness

Implementation Science

Quality Measurement

Problems & Solutions

Evaluating Disease Management(Patient-Level)

Patient-Centered OutcomesComparative Effectiveness

Data Synthesis

Major Problems?

VS.

VS.

Problem: Patients getting treatment A and B are different

Treatment (Endo vs. open

aneurysm)

Outcome (1-year death)

Confounding variables

(Illness levels)

Confounding

Selection bias & confounding

Toolbox: • Randomized clinical trials • Multivariate adjustment using

regression • Propensity scores • Instrumental variable analysis

Problem: External Validity

Randomized Clinical Trials – External Validity?

Toolbox: Pragmatic trials Cluster randomized trials Observational studies

CER Methodologies

Randomized trials

Observational Studies

Pragmatic Trials Cluster Randomized Trials Natural Experiments Cohort

*Lohr, Medical Care (2007) 45 (10); S5

Problems with external validity

Problems with confounding

What about the patient perspective?

Toolbox: Patient centered outcomes; Quality of life measures; Survey research

Dr: “Congratulations, your hernia if fixed” Patient: “Great, but why is my groin pain worse?”

What about the patient perspective?

Understanding Local Provision of Care (Micro-systems)

Quality ImprovementPatterns of Care Analyses

Patient Safety

Major problems?

“The mere existence of evidence-based practices, however, does not always result in the use of such practices because of the complexities inherent in translating evidence into practice.” How do we better understand these factors?

Toolbox: Qualitative research – Observation, key informant interviews, Focus groups

In-depth phone and in-person interviews were conducted between October 2006 and September 2007 with 86 participants (31 physicians). Active resistance to evidence-based practice change was pervasive. Successful efforts to overcome active resisters included benchmarking infection rates, identifying effective champions, and participating in collaborative efforts.

Complexity in frontline systems

• Complex assembly of people, information, resources, equipment and procedures working toward a common goal

• System vulnerability reflects exposure to

events and factors that can make the system less safe or more prone to adverse events

Toolbox: Human factors engineering, systems analysis

Human Factors Engineering: Design systems and environments for safer, more effective and more efficient use by humans

“There is a general consensus by intensivists and nonsurgical providers that surgeons hesitate to withdraw life-sustaining therapy on their operative patients despite a patient's or surrogate's request to do so.” How do you better understand this?

Healthcare Delivery System(Macro-system)

Policy EvaluationPayment Reform

Work Force

Major problems?

VS.

Problem: Patients getting treatment A and B are different

Evidence supporting policy change

What we know: Outcomes in Medicare patients improved after the national coverage decision

What we don’t know:

Improvements due to secular trends or the policy itself?

Change over time

Toolbox: Health economics, econometrics, natural experiments, health policy

Differences-in-differences technique

Isolates the impact of a policy change on outcomes above and beyond any changes to a control group

Policy implementation

After Before

Out

com

es

Time

Expected Improvement

A B

Impact of policy change

Control

Intervention

Toolbox: Health economics, econometrics, natural experiments, health policy

Difference-in-difference methods – Econometric techniques for “subtracting out” effect of time Instrumental variables analysis – Econometric technique for isolating causal effects of interventions

Evaluating Disease Management (Patient-Level)

Healthcare Delivery System (Macro-systems)

Understanding Local Provision of Care (Micro-systems)

Patient-Centered Outcomes Comparative Effectiveness

Data Synthesis

Quality Improvement Patterns of Care Analyses

Patient Safety Policy Evaluation Payment Reform

Work Force

Coverage Decisions Technology Adoption

Cost Effectiveness

Implementation Science

Quality Measurement

Major theme Research areas Research tools Resources

Evaluation of disease management (patient level)

• Comparative effectiveness

• Patient centered outcomes

• Data synthesis

• Randomized trials • Pragmatic trials • Advanced

observational study designs

Understanding local health care systems (micro-systems)

• Quality measurement and improvement

• Patterns of Care Analyses

• Patient Safety

• Qualitative research methods

• Human factors theory

• Process engineering

Understanding influence of healthcare delivery system (macro-systems)

• Policy Evaluation • Payment Reform • Surgical work force

• Large database analyses

• Econometric methods

• Health care policy