1

HIT Return on Investment:Evaluating Progress in a Sea of Change

John Hsu, MD, MBA, MSCE

AHRQ Conference

27 September 2007

2

HIT Background

Great potential for transforming clinical care, especially for patients with chronic diseases

Adoption of HIT across the U.S. is limited but growing

Actual benefits of HIT unclear: Initial benefits of HIT depend on how routinely and systematically

clinicians use the HIT tools and resulting information Little information on HIT effects in the ambulatory setting with

commercially-available systems

Actual benefits and costs of HIT are difficult to quantify Comprehensive identification Methodological challenges

3

Preliminary Results - IMPACT Study

Impact of Information Technology on Clinical Care: An Evaluation of the Technology on Quality, Safety and Efficiency of Chronic

Disease Care

John Hsu, MD, MBA, MSCE (KP DOR)Ilana Graetz (KP DOR)Huihui Wang (KP DOR)

Jie Huang, PhD (KP DOR)Mary Reed, DrPh (KP DOR)

Bruce Fireman, MA (KP DOR)Joseph Selby, MD, MPH (KP DOR)

Yvonne Zhou, PhD (KP)Jim Bellows, PhD (KP CMI)Naomi Bardach, MD (UCSF)

Julian Wimbush (UCB)Tom Rundall, PhD (UCB)

Robert Miller, PhD (UCSF)Richard Brand, PhD (UCSF)

Funding: AHRQ R01HS015280

4

• Design: – Longitudinal study with quasi-experimental changes in

exposure to HIT, and using a pre-post analytic design with concurrent controls

• Study Period: 2004-2008

• Population: IDS Members with any of five chronic diseases in January 2004 (Asthma, CAD, DM, HF, Htn)

• Data: - Automated databases- Annual surveys

Overview

5

Basic HIT Tools

  CIPS eChart eRx/eRefill eConsult

First Available: 1995 March 2004

Functions:

Data-Review  

Documentation      

Order-Entry      

Communication  

Paper-alternative: No Yes Yes Yes

Integrated: Not integrated with other applications (i.e., need log onto each application separately)

Description:  

Viewing lab results

Viewing medication list

Writing free-text visit notes

Using standard note templates

Viewing medication list

Viewing medication allergies

Entering orders for new prescription or refills

Requesting referrals or consultations

Sending messages to other providers

6

First Available: Staggered implementation (2005-2008)Functions:

Data-Review

Documentation

Order-Entry

Communication  

Paper-alternative: No

Integrated: Fully Integrated

Description:  

•Viewing medication list, allergies, lab results•Using standard note templates & writing free-text visit notes•Order new prescription or refills with decision support•Ordering Disease-specific sets (drugs and labs)•Sending messages to other providers & requesting referrals or consultations•Sending and receiving messages from patients•Point-of-care access to decision-support tools– including:

–Online references and resources for current treatment guidelines –Care Management Institute protocols, and standard tests/screens

KP HealthConnect Ambulatory Suite

7

Potential Benefits of HIT

• Improved information availability (value of information)

• Clinical benefits• Financial benefits: e.g., greater efficiency, lower

administrative costs, better coding

Benefits predicated on clinician use of HIT tools

HIT Use

9

HIT Implementation and Use

0

20

40

60

80

100

4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6

2004 2005 2006

Percen

t o

f V

isit

s U

sin

g H

IT

KPHCe-Chart

* Among office visits in department of Medicine or Family Practice

10

CPOE Implementation and Use

0%

10%

20%

30%

40%

50%

60%

70%

4 5 6 7 8 9 11 12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12

2004 2005 2006

Perc

ent o

f New

Pre

scrip

tions

Usi

ng H

IT

* New prescriptions are defined as new prescriptions doctor wrote, can be refills for existing drugs or completely new drugs

11

HIT Use

• Implementation ≠ use

• Use of one type of HIT ≠ use of all HIT tools

Information Quality

13

Data Availability: Diagnoses Completed on Visit Date

* Among office visits in department of Medicine or Family Practice

0

10

20

30

40

50

60

70

80

90

1004 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12

2004 2005 2006

Per

cen

t o

f V

isit

s w

ith

Sam

e-d

ay D

iag

no

sis

En

try

Clinical Benefits

15

Methodological Challenges for Assessing Clinical Benefits

• Measures of use

• Temporal trends - concurrent control groups

• Patient- and physician-level differences

• Reliable pre-implementation clinical data - differentiating

documentation vs. care

• Multi-level effects

• Adequate power

16

Methods

• Study Period: 04/2004-12/2006

• Study Population– Active KPNC members who continuously enrolled during the study

period– 18 years and older as of 04/01/2004– In diabetes registry as of 1st quarter of 2004 – Members in 5 medical centers where KPHC implemented before

07/2006 during the study period – In teams which existed all the time during the study period – With at least one LDL measurement in pre-HIT period and one in post-

HIT period

• Predictor Measures: Presence of HIT (HealthConnect)

• Model: Mixed model with random effects at PCP and Patient level, adjusted for patient age, gender, race/ethnicity, neighborhood SES, time of measurement and Medical centers

17

Definitions of Presence of HIT

Definition 1: Medical center level KPHC rollout schedule– HIT=0: before KPHC was implemented at the first team in the medical

center– HIT=1: within six months after KPHC was implemented at the first team

in the medical center – HIT=2: six months after KPHC was implemented at the first team in the

medical center

Definition 2: Primary care team level actual use– HIT = 0: low use (<80% at team level) of eChart or KPHC– HIT = 1: starting from the first month when eChart used >=80%– HIT = 2: starting from the first month when KPHC used >=80%

18

Mean LDL in Each Month in KPNC

90

92

94

96

98

100

4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12

2004 2005 2006

19

Association between HIT and LDL

    Estimate 95% CI

1. Implementation at Medical Center (roll-out schedule)  

Before KPHC 1.00 ref. group  

First 6 months of KPHC -0.50 -1.15 0.15

6+ months of KPHC -0.64 -1.58 0.30

2. Actual use by Primary Care Team (% of total visits)  

Low HIT use (<80% of visits) 1.00 ref. group  

EChart used in >80% of visits -0.89 -1.55 -0.23

KPHC used in >80% of visits -1.72 -2.68 -0.76

Costs

21

Investment

• Investment costs– Equipment– Personnel/productivity– Training

• Maintenance costs– IT support staff– Future upgrades– Continued training

Other Relevant Features

23

Dynamic Environment

• Changes in HIT– Decision support– Information use

• Changes in Care Delivery– Clinical coordination– Delivery system

• Changes in Medical Therapy– Information on effectiveness– Dissemination of new knowledge

• Changes in the Market– Payment features, e.g., risk adjustment, reporting, performance incentives– Payment mix

24

Conclusions

• Benefits– Some potential clinical benefits related to better information at the

point-of-care– Unclear benefits associated with improvements in clinical

information at the system level– Transaction benefits perhaps easiest to quantify– Financial benefits depend market and reimbursement mix

• Costs– Investment costs beyond equipment costs can be difficult to

quantify– Maintenance costs also important

• Dynamic/changing systems and markets....

25

Summary: Need for Better Empirical Studies

26

HIT as Basic Infrastructure