tball acsm lecture 2016 - healthcare pa evaluation methods_linked video

1

Methods for Evaluating Effectiveness of Promoting Physical Activity in Healthcare

Trever J. Ball, PhD, MPH Senior Outcomes AnalystHealth Promotion & Wellness Institute for Healthcare Delivery Research Intermountain [email protected]

World Congress on Exercise is MedicineMay 31, 2016Boston, MA

2

Background

Intermountain Healthcare• Not-for-profit health system• Based in Salt Lake City, UT• 22 hospitals• 185 clinics

4

QI: Role in Driving Evidence Base

Outcomes Research

Program Evaluation

Clinical Effectiveness

Implementation Science

Quasi-Experimental

Intervention Studies

Quality Improvement

Courtesy of L. SavitzInstitute of Medicine (IOM). “Health System Leaders Working Toward High-Value Care through Integration of Care and Research,” The National Academies, Workshop in Brief, June, 2014.

Driving the science of change/innovation…

5

Role of Research at Intermountain

Priorities:

1. High operational utility2. Of interest to clinical investigators3. Of interest to close partners4. Of interest to external investigators/entities

6

Quality Improvement Foundational Leaders

Homer Warner, MD, PhD– Medical informatics founder– 1950s – computer assisted CV

decision support– 1970s – HELP system

developed

Dr. Brent James, MD, MStat– CQI - Standardization of clinical

care through data analysis

7

Institute for Healthcare Delivery Research

Clinical Programs

Data Analysts

Delivery System Science

Data Analysts

Health Promotion & WellnessCommunity Health

ImprovementOutcomes Analyst

8

VP Community Benefit

Community Health

Improvement

Health Promotion &

Wellness

Behavioral Health Reporting

Outcomes Analyst

9

Health Promotion & Wellness

Insurance Members

Patients & Providers

Intermountain Employees Community Comm. Business

Development

10

Physical Activity “Vital Sign” (PAVS)

1. Ball TJ, Joy EA, Gren LH, Shaw JM. Concurrent Validity of a Self-Reported Physical Activity "Vital Sign" Questionnaire With Adult Primary Care Patients. Prev Chronic Dis. 2016;13:E16.

2. Ball TJ, Joy EA, Gren LH, Cunningham R, Shaw JM. Predictive Validity of an Adult Physical Activity "Vital Sign" Recorded in Electronic Health Records. J Phys Act Health. 2015.

3. Ball T, Joy E, Goh T, Hannon J, Gren L, Shaw J. Validity of two brief primary care physical activity questionnaires with accelerometry in clinic staff. Prim Health Care Res Dev. 2014;28:1-9.

4. Greenwood JLJ, Joy EA, Stanford JB. The Physical Activity Vital Sign: A Primary Care Tool to Guide Counseling for Obesity. Journal of Physical Activity & Health. 2010;7(5):571-576.

11

Physical Activity “Vital Sign”

14

Reasons to Evaluate

• Ensure efforts produce desired results

• Justifies need• Monitor goals• Facilitates accountability

15

Steps for PA Evaluation & Quality Improvement

1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research

16

Team Structure

Guidance Team

Project Team

Front Line Workers

Process

Health Promotion & WellnessPatient & Provider Development Team

Lead: Trever Ball, PhD, MPH

Clinic Receptionists MAs Physicians

Scott Hansen, MD Locke Ettinger, PT, PhD Tamara Sheffield, MD Roger Peterson, PT

Laurel Newman, RN Liz Joy MD, MPH Clinic Manager

Direction, control

Participation, ownership

Reports monthly

Fundamental knowledge

17



18



19

Engage Stakeholders

Describe projects

Focus evaluation

plans

Gather credible evidence

Justify conclusions

Ensure use and

disseminate

RE-AIM Framework: www.re-aim.org CDC Evaluation Framework. http://www.cdc.gov/eval/framework/index.htm

StandardsUtility

FeasibilityProprietyAccuracy

RE-AIM

http://www.re-aim.org/

http://www.cdc.gov/eval/framework/index.htm

20



21



22

Mission

Increase proportion of primary care visits for which physical activity levels of patients are asked

advised

23



24



25

Logic Modeling (“Outcomes Roadmap”)

26

Purposes of Logic Models

• Graphic depiction of relationship between activities and intended outcomes.

• Helps focus evaluation on most central and important questions.

CDC Evaluation Framework. http://www.cdc.gov/eval/framework/index.htm

http://www.cdc.gov/eval/framework/index.htm

27

Activities Outcomes

Early Later

28

Activities Outcomes

Early Later

Contact with Clinical Program Leads

Identify Champions Identify areas that health promotion fits into their workflow

Engage Intermountain Clinicians with tools Engage Clinician in their own health:

Nursing Study (Liz) LiVe Well for Employees (see Employee) Design engagement opportunities for physicians, nurses and other clinicians to improve their personal wellness habits.

Execute care processes for healthy behaviors

PAVS DPP Hypertension CHF – With CV Clinical Program Weigh to Health Falls – Stepping On (this may be in CB community) Potential Partners that are being considered

DATU Omada Takada

Determine a Behavior Change Framework Carry out research and development on “best practices” for achieving healthy lifestyles. Study and expand application of integrative wellness services.

Identify CP LiVe Well champion(s) Identify Health Pathway(s) to include tools from the Lifestyle CPM Build healthy lifestyles into High Blood Pressure CPM Nursing intern student study results (Liz) Health pathways include assessment and management of prevention and wellness Meet with each CP to prioritize efforts aimed at prevention and wellness Clinical wellness champions identified and trained on LWM CPM in relevant CPs Dissemination & implementation of DPP to relevant CPs Priorities identified within each CP for implementation in 2015-2016 # of CPs with identified and trained clinical champion # of CP that have implemented

Orient and engage the champion to LiVe Well programs and tools Education event, or online resource for clinicians Measure patient participation in exercise a d wt loss. BMI CPM include assessment and management of prevention and wellness strategies Number of patients not converting to Diabetes Number of patients not progressing to CHF Number of patients increasing or maintaining PA. Number of patients with controlled BP Obesity Rates in adults and children PA rates in Children and adults Other population health measures

29

Activities Outputs Outcomes

Early Later

Early Later

Contact with Clinical Program Leads

Identify Champions

Identify areas that health promotion fits into their workflow Engage Intermountain Clinicians with tools

Identify clinical wellness champions and train on LWM CPM in relevant CPs Meet with each CP to prioritize efforts aimed at prevention and wellness

Identify Health Pathway(s) to include tools from the Lifestyle CPM Build healthy lifestyles into High Blood Pressure CPM Dissemination & implementation of DPP to relevant CPs Identify priorities within each CP for implementation in 2015-2016 Orient and engage the champion to LiVe Well programs and tools

Education event, or online resource for clinicians

Execute care processes for healthy behaviors

PAVS

DPP

Hypertension

CPM include assessment and management of prevention and wellness strategies Patient participation in exercise and wt loss # of CPs with identified and trained clinical champion # of CP that have implemented operational workflow to refer patients into DPP # of new CPMs with measureable and evidence based wellness and prevention strategies

Culture of Wellness Health pathways include assessment and management of prevention and wellness # of patients adhering to recommendation for CHF lifestyle Clinical Values Lifestyle measures Rate of physicians recording and counseling on PAVS Rate of physicians counseling on BP compliance that includes lifestyle

BMI Patient rate not converting to Diabetes Patient rate not progressing to CHF Patient rate increasing or maintaining PA. Patients rate with controlled BP Obesity rates in adults and children PA rates in adults and children Other population health measures Lifestyle measures

30

ReachAdoptionImplementationMaintenance Effectiveness

Glasgow, Vogt et al. 1999; Gaglio, Shoup et al. 2013; Yeh, Chang et al. 2014; Thomas, Krevers et al. 2015

RE-AIMFramework

31

Example Central Evaluation Question:

Do efforts to adopt and implement care

processes related to the Physical Activity Vital Sign improve patient

physical activity?

32




physical activity?

34

Detailed health promotion

strategies for patients and

providers

35

Physical Activity “Value Chain”

Activities Outcomes

Adapted from: Program Measurement & Evaluation Guide: Core Metrics for Employee Health Management, 2015. HERO & PHA

36


Activities Outcomes

Ask AssessAdvise

AssistArrange

PA Behavior

Clinical Outcomes

Financial Outcomes


37



38



39

Aim

Increase the proportion of patient visits at the (choose clinic) that have mins/week of physical activity recorded from __% to __% by (date).

40



41



42

Quality Improvement Tools

• RE-AIM framework• Cause & effect diagrams• Driver diagrams• Pareto charts• Flow charts/logic modeling• Run charts• SPC charts

43

Why is physical activity not asked and recorded?

Staff Factors

Health ITOperations

Patient Factors

Healthy staff

Care Manager

MAs

Clinicians

Research Findings

Age disparity

BMI disparity

Data Architecture

Decision Support

New EHR integration

Activating team

Educating teams benefits of PA

44

Odds of Treatment for Physical Activity by Patient Demographic

Female Male

18-29

30-39

40-49

50-64 >6

4<1

8.5

18.5-

24.9

25.0-

29.9

30.0-

34.9

35.0-

39.9

≥40.0

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

Standard PAVS Only PA Advice Only PAVS + PA Advice

n=257,869

45


Female Male

18-29

30-39

40-49

50-64 >6

4<1

8.5

18.5-

24.9

25.0-

29.9

30.0-

34.9

35.0-

39.9

≥40.0

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00


n=257,869

46


Female Male

18-29

30-39

40-49

50-64 >6

4<1

8.5

18.5-

24.9

25.0-

29.9

30.0-

34.9

35.0-

39.9

≥40.0

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

1.33

3.30

5.23

7.24

1.84 1.831.32

2.58


n=257,869

47

Why is physical activity not asked and recorded?

Staff Factors

Health ITOperations

Patient Factors

Self-complete formsHealthy staff

Care Manager

MAs

Clinicians

Research Findings

Age disparity

BMI disparity

Tools

Clinic support for process

Assessment & operations of clinic education needs

Support for feedback/reporting

Data Architecture

Decision Support

New EHR integration

Explanation of terms

Activating team

Educating teams benefits of PA

48



49



50

Aim Key Drivers Strategies

Increase proportion of patient adult visits at the _____ clinic that have PA recorded from __ % to __ % by [date]

Clinical Champion

Clinical excellence goal part of value recognition program

Health IT

Academic detailing our LWM CPM; presentations

Maintain work with PCCP

Work with new EHR team to ensure PAVS is there and easy to complete

51

Framework Aim Key Drivers Strategies

Ask

Increase the proportion of patient visits at the _____ (clinic/region) that have physical activity recorded (minutes/week) from __% to __% by (date).

Clinical championAcademic detailing our LWM CPMs to cliniciansPresentations

Clinical excellence goal Maintain work with PCCPWork with IMG

Health ITWork with iCentra team to ensure PAVS is there, is easy to complete, and is visible to clinicians

Assess & Advise

Increase the proportion of patient visits at the ______ (clinic/region) that have physical activity advice recorded from __% to __% by (date).

iCentraWork with iCentra development team to make sure product is placed well for iCentra workflow for clinician

Clinical excellence goal Maintain work with PCCP Work with IMG

iCentra +Manipulate patient activity graph to serve as a prompt for clinician, including if patient meets or does not meet PA guidelines

Assist & Arrange

Increase the proportion of SH MA members that access the fitness benefit from __% to __% by (date).

Referral networkSee Greenville, SC model (EIM slides)Member educationDevelop referral network

Increase the reimbursement/member for the fitness benefit (mean & median) from $__ to $__ by (date).

Payer factors for referral network

Member education

Develop referral network

52



53



54

DATA ANALYTICS 101

56

Intermountain Healthcare EDW

57

Query & Track Outcomes

TrackAnalyze

Disseminate

Interpret

Graphical presentation

Stratifications

VisualizeWrite & QA

query

QueryAgree on

evaluation question

Identify data elements

Consult

58

Query & Track OutcomesPatient_id

Patient_sex

Appt_st_dt

Ob_NCID

Encounter_num

001002003004005

Facility_ncid

Encounter_facility_id

Encounter_num

Primary_provider

Unit_num

Unit_num

Encounter_facility_id

Encounter_num

Primary_provider

Unit_num

59


Patient_id

APPOINTMENT_START_DTTM

APPOINTMENT_STATUS_NCID

APPOINTMENT_STOP_DTTM

APPOINTMENT_TYPE_NCID

ENCOUNTER_FACILITY_NCID

ENCOUNTER_NUMBER

LAST_UPDATE_DT

MED_RECORD_FACILITY_NCID

MEDICAL_RECORD_NUMBER NOTE

PATIENT_BIRTHDATE

PATIENT_HOME_PHONE

PATIENT_NAME

PATIENT_SEX

PATIENT_WORK_PHONE

PROVIDER_NAME

PROVIDER_NCID

REASON

REFERRING_PROVIDER_NAME

REFERRING_PROVIDER_NCID

RESPONSIBLE_PARTY

SCHEDULED_DTTM

SCHEDULED_FACILITY_NCID

SCHEDULED_LOCATION_NCID

SCHEDULER_NCID

UNIT_NUMBER

00001

00002

00003

00004

00005

00006

00007

00008

00009

00010

60


• 25,422 tables• 2,197,993,362 Queries/month• 157,797,141,557 Rows• 467,789 Columns

Courtesy of Dallin Rogers, Intermountain EDW Team

62


64


Activities Outcomes

Ask AssessAdvise

AssistArrange

PA Behavior

Clinical Outcomes

Financial Outcomes


65




physical activity?

66



67



68


Outcomes Research

Program Evaluation



Quasi-Experimental


Quality Improvement



69


Outcomes Research

Program Evaluation



Quasi-Experimental


Quality Improvement



70


Outcomes Research

Program Evaluation



Quasi-Experimental


Quality Improvement



71

Designs & Methods

• Knowing what works requires that we design a study and use those methods that allow us to answer key questions.

• Design Approaches:– Intervention– Quasi-experimental– Observational– Time Series

• Methods:– Quantitative– Qualitative– Mixed

Courtesy of L. Savitz

72

Design: observational

Are there differences in level of treatment for physical inactivity by patient demographics?

• Logistic regression

𝑦 (𝑃𝐴𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑙𝑒𝑣𝑒𝑙 )=𝛽0+𝛾 𝐹 ( h𝑑𝑒𝑚𝑜𝑔𝑟𝑎𝑝 𝑖𝑐𝑠 )+… 𝛽𝑘 ( covariates )+ μ

73

Design: observational

How much impact on patients do levels of treatment for physical inactivity have?

• Mixed effects regression

𝑦 (𝑃𝐴 ; h h𝑒𝑎𝑙𝑡 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 )=𝛽0𝑖+𝛾 𝐹 (𝑃𝐴𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 )+𝛽𝑖𝑗 (𝑟𝑎𝑛𝑑𝑜𝑚)… 𝛽𝑖𝑗 ( 𝑓𝑖𝑥𝑒𝑑)+𝜇𝑖𝑗

74

Method: mixed

What are behavioral and other differences in providers using and not using the PAVS?

• Descriptive statistics• Survey constructs of Theory of Planned Behavior

Adapted from: https://commons.wikimedia.org/w/index.php?curid=42261999

Attitude

Subjective Norm

Perceived Behavioral Control

Intention Behavior

75

Design: quasi-experimental

Nonrandomized observational comparisons

Example:Does a program designed to increase ascertaining and recording patients’ self-reported physical activity as part of the initial visit intake improve glycemic control among patients with diabetes?

Grant RW. (2014). J Gen Intern Med, 2014;29(2):341-348

76

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4EVS Clinic HbA1c Change Non-EVS Clinic HbA1c Change

Per

cent

Cha

nge

Changes in HbA1c Over Time Among Patients with Diabetes Attending EVS Versus Non-EVS

Medical Centers

Adapted from: Grant RW. (2014). J Gen Intern Med, 2014;29(2):341-348 EVS: Exercise Vital Sign

Al patients w/diabetes (n=70,083)

Patients w/HbA1c>7 (n=30,487)


77

Series1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4EVS Clinic HbA1c Change Non-EVS Clinic HbA1c Change

Per

cent

Cha

nge

Changes in HbA1c Over Time Among Patients with Diabetes Attending EVS Versus Non-EVS

Medical Centers

Adapted from: Grant RW. (2014). J Gen Intern Med, 2014;29(2):341-348 EVS: Exercise Vital Sign

Al patients w/diabetes (n=70,083)



78


1. Organize Team2. Identify evaluation

framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research

79




80




Thank youQuestions?

tball acsm lecture 2016 - healthcare pa evaluation methods_linked video

Documents