deriving value from patient flow analytics · deriving value from patient flow analytics session...

1

Deriving Value from Patient Flow AnalyticsSession 200, February 14, 2019

Justin Boyle & Sankalp Khanna

CSIRO Australian e-Health Research Centrewww.csiro.us

2

Justin Boyle, BEng(Hons1) PhD, Principal Research Scientist

Has no real or apparent conflicts of interest to report.

Sankalp Khanna, BEng,MInfTech, PhD, Senior Research Scientist

Has no real or apparent conflicts of interest to report.

Conflict of Interest

3

• Learning Objectives

• Introduction – Where we are from

• The Australian Health System – Why we do what we do

• Patient Flow Analytics @ CSIRO – Our approach

• Case Studies

Agenda

4

• Identify strategies to ensure effective impact from developed

health analytics solutions

• Form effective partnerships between problem solvers and problem

owners to ensure success

• Apply appropriate statistical rigor in planning and executing data

science projects

• Identify strategies to decongest the health system

Learning Objectives

5

Australia’s National Science Agency

Over 5000 Research Scientists

58 Sites globally, Research activities in 80 Countries

$1 Billion Annual budget

Top 1% Of Global research agencies

Hosts Boeing’s largest R&D facility outside of the US

Run NASA’s spacecraft tracking facilities in Australia

Invented WiFi, used in five billion devices globally.

CSIRO: Commonwealth Scientific and Industrial Research Organisation

6

CSIRO’s Digital Health Research Program

BIOMEDICALINFORMATICS

Biostatistics, imaging and

genomics based -clinical

workflows

How: Leveraging operational &

clinical data through analytics,

modelling, decision support &

automation

HEALTHINFORMATICS

Improving health system

performance & productivity from

electronic health data

How: Meaningful data

interoperability and analysis for

decision support, analytics,

modelling and reporting

HEALTHSERVICES

Improving access to services &

management of chronic

diseases

How: Service delivery models

utilising telehealth, mobile

health & remote monitoring

7

The Australian Health System

Over the Past DecadeOver the Past Decade

Health

Spending

Population

Growth50%50% 17%17%

Reference: Australia's health 2018, https://www.aihw.gov.au/reports/australias-health/australias-health-2018/

9

Overcrowding in hospitals: an international crisis

- Increased wait times. - Increased walkouts.

- Increased medical errors. - Ambulance diversion.

- Increased length of stay. - Patient safety at risk.

- Increased medical negligence claims - Unnecessary deaths.

ED admissions

Elective surgery

Reference: American College of Emergency Physicians, Emergency Department Crowding: High-Impact Solutions, 2008

10

Do we have a problem with Patient Flow ?

OVERCROWDED HOSPITALS

AGEING POPULATIONS

EXPLODING HEALTH BUDGETS

AMBULANCE RAMPING

LONG WAITING TIMES

STAFFING SHORTAGES

Image Source: Fairfax Media

11

Fixing Patient Flow - Why does it matter?

Reference: Boyle J, Zeitz K, Hoffman R, Khanna S, Beltrame J. Probability of severe adverse events as a function of hospital occupancy. IEEE J Biomed Health Inform. 2014 Jan;18(1):15-20

0.7 0.75 0.8 0.85 0.9 0.95 1 1.05 1.1 1.150

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Occupancy

Pro

ba

bilit

y o

f S

AC

1/S

AC

2 I

ncid

en

t

No event/day

1 event/day

2 events/day

3 events/day

12

Patient Flow Analytics @ CSIRO

Improving public hospital performance through efficiency improvements

Improving public hospital performance through efficiency improvements

Creating an evidence base to support policy and decision making

Creating an evidence base to support policy and decision making

13

• Better Capacity Management

• 85% occupancy delivers optimum patient flow

• Higher levels result in increased patient risk and regular overcrowding

• Is 85% a one size fits all ?

• How do I manage my hospital capacity ?

• How unsafe is my crowded hospital ?

• Early Discharge

• Early discharge should help ease crowding.

• Where is the evidence ?

• How much benefit could it potentially deliver ?

Hospital Crowding: The Magic Fix

14

Q > How full can I run my Hospital

at before Capacity Crisis ?

Case Study 1 : Capacity Management

ED, Admission & Discharge vs occupancy

3 key choke points - performance declines:

• A - Admission/discharge surge

• B - ED overwhelmed

• C - Admissions overwhelmed

Overcrowding affects :

Access Block

ED Length of Stay (Inpatients)

Inpatient Length of Stay

Inpatient Admissions from ED

ED Length of Stay (not admitted)

15

Q > Do we see this trend across hospitals of all sizes ?

Case Study 1 : Capacity Management

051015202530354045500102030405060708090

71

%

80

%

89

%

98

%

10

7%

11

6%

OCCUPANCY

Inpatient Admissions (patients/hr) (Y1 axis) Inpatient Discharges (patients/hr) (Y1 axis) ED Presentations (patients/hr) (Y1 axis)

ED Discharges (patients/hr) (Y1 axis) Inpatient Admissions from ED (patients/hr) (Y1 axis) Inpatient Length of Stay (days) (Y2 axis)

ED Length of Stay (inpatients) (hours) (Y2 axis) ED Length of Stay (others) (hours) (Y2 axis) ED Access Block Cases (inpatients) (patients/hr) (Y2 axis)

0

5

10

15

20

25

30

35

40

45

0

10

20

30

40

50

60

70

80

90

75

%

80

%

85

%

90

%

95

%

10

0%

10

5%

11

0%

11

5%

OCCUPANCY

GROUP 3

A

B

C

300 >= Beds

0

5

10

15

20

25

30

35

40

45

0

10

20

30

40

50

60

70

80

90

75

%

80

%

85

%

90

%

95

%

10

0%

10

5%

11

0%

OCCUPANCY

GROUP 2

A

B

C900 >= Beds > 300

0

5

10

15

20

25

30

35

40

45

0

10

20

30

40

50

60

70

80

90

70

%

75

%

80

%

85

%

90

%

95

%

10

0%

OCCUPANCY

GROUP 1

A

B

C

Beds > 900YES … but at different levels

Group 1 (Large hospitals) :

• Choke Point A : 86%

• Choke Point B : 90%

• Choke Point C : 94%

Group 2 (Mid-size hospitals) :




Group 3 (Small hospitals) :




16

Case Study 2 : Discharge Timing

Q > Does discharge peak timing affect ED LOS and Access Block

5 hours

Admissions

Discharges‘d1’

5 hours

Discharges‘d2’

Category 1 Category 2 Category 4 Category 5

Category 3

Hour of Day

Num

ber

of P

atie

nts

Define

discharge

peak timing

Reference: Khanna S, Boyle J, Good N, Lind J, Impact of Admission and Discharge Peak Times on Hospital Overcrowding, Proc. 19th Australian National Health Informatics Conference (HIC 2011), 2011, 82-88

17


Q > Does discharge peak timing affect ED LOS and Access Block

Reference: Khanna S, Boyle J, Good N, Lind J, Impact of Admission and Discharge Peak Times on Hospital Overcrowding, Proc. 19th Australian National Health Informatics Conference (HIC 2011), 2011, 82-88

0

50

100

150

200

250

75%

80%

85%

90%

95%

100%

105%

110%

115%

1 2 3 4 5

Acc

ess

Blo

ck C

ase

s p

er d

ay

Occ

up

an

cy (

%)

Category

23 HospitalsMean Occupancy (Y1 Axis)Mean PeakOccupancy (Y1 Axis)Mean AB Cases (Y2 Axis)

6.0

6.5

7.0

7.5

8.0

8.5

9.0

9.5

10.0

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

1 2 3 4 5

Len

gth

of

Sta

y (h

ou

rs)

Len

gth

of

Sta

y (d

ays

)

Category

23 HospitalsMean LOS (days)

Mean EDLOS (hours)(Y2) Cat 5 vs Cat 1

• 13% Higher Peak

Occupancy

• 60 cases/day higher

Access Block

• 0.7 hours higher

Mean ED LOS

18


0

50

100

150

200

250

300

350

400

450

55

60

65

70

75

80

85

90

95

100

105

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Dis

char

ges/

ho

ur

Occ

up

ancy

(%

)

Time of Day (hour)

2 Hours Early

1 Hour Early

Actual

1 Hour Late

2 Hours Late

2 Hour Early Discharge (all 23 Hospitals) :

• Average Occupancy reduced from 93.7% to

91.6%.

• Maximum Occupancy reduced from 110.8%

to 106.1%.

• Time spent above 95% occupancy reduced

from 34.7% to 21.5%.

Q > Can we quantify the impact of Early Discharge ?

Q > What happens if overcrowding delays Discharge ?

Reference: Khanna et al. Unravelling relationships: Hospital occupancy levels, discharge timing and emergency department access block. Emerg Med Australas. 2012 Oct;24(5):510-7

19


Q > How does this affect my KPIs ?

Q > How can I operationalise this ?

Scenario Description

150% of patients to be discharged by 10am, 80% by 12pm and 100% by 2pm.

2 35% of patients to be discharged by 11am, 70% by 2pm and 100% by 5pm.

3 50% of patients to be discharged by 11am, 70% by 2pm and 100% by 5pm.

4 80% of patients to be discharged by 11am.

540% of patients to be discharged by 10am, 70% by 2pm, 90% by 5pm and 100% by 10pm.

6Select the same patients as for Scenario 5 but change only the emergency discharge times, leaving elective patient discharge times unchanged.

7Select the same patients as for Scenario 3 but change only the emergency discharge times, leaving elective patient discharge times unchanged.

Reference: Khanna S, Sier D, Boyle J, Zeitz K. Discharge timeliness and its impact on hospital crowding and emergency department flow performance. Emerg Med Australas. 2016 Apr;28(2):164-70

20


Q > How does this affect my KPIs ?

Q > How can I operationalise this ?

Reference: Khanna S, Sier D, Boyle J, Zeitz K. Discharge timeliness and its impact on hospital crowding and emergency department flow performance. Emerg Med Australas. 2016 Apr;28(2):164-70

% change as compared to baseline

Scenario Scenario DescriptionNEAT

Performance

Ave Bed

Occupancy

Ave Inpatient

LOS

Ave wait for

Inpatient Bed - ED

Ave wait for

Inpatient Bed - All

150% by 10am, 80% by 12pm,

100% by 2pm.+16.1% -1.5% -1.7% -25.5% -24.2%

235% by 11am, 70% by 2pm,

100% by 5pm.+5.7% -0.2% -0.3% -6% -5.7%

350% by 11am, 70% by 2pm ,

100% by 5pm.+9.4% -0.5% -0.5% -11.8% -10.5%

4 80% by 11am. +16.2% -1.5% -1.6% -24.9% -23.5%

540% by 10am, 70% by 2pm,

90% by 5pm, 100% by 10pm.+7.3% -0.3% -0.4% -8.6% -7.7%

6Same as Scenario 5

but ED only+6.9% -0.2% -0.3% -7.3% -6.4%

7Same as Scenario 1

but ED only+15.7% -1.2% -1.3% -22.7% -20.5%

21

How Many of What Beds?Access Target:

ED LOS<4hrs

22

Case Study 3 : Predicting Demand

• Forms a regular component

of daily bed management

across major QLD public

hospitals

• Licensed in Australia and

overseas

• Estimated to deliver $23

million (direct), and $250m

(indirect) in productivity gains

per annum

• Several awards related to

efficiency and effectiveness

References:

• Boyle J, Jessup M, Crilly J, et al. Predicting emergency department admissions.

Emerg Med J. 2012 May;29(5):358-65.

• Boyle J, Ireland D, Webster F, O’Sullivan K, Predicting Demand for Hospital Capacity Planning,

Conf Proc IEEE Biomedical and Health Informatics. 2016: 328-331

• Jessup M, Crilly J, Boyle J, et al. Users' experiences of an emergency department patient

admission predictive tool: A qualitative evaluation. Health Informatics J. 2016 Sep;22(3):618-32

23

Case Study 3 : Predicting Demand

• Forms a regular component

of daily bed management

across major QLD public

hospitals

• Licensed in Australia and

overseas

• Estimated to deliver $23

million (direct), and $250m

(indirect) in productivity gains

per annum

• Several awards related to

efficiency and effectiveness

References:

• Boyle J, Jessup M, Crilly J, et al. Predicting emergency department admissions.

Emerg Med J. 2012 May;29(5):358-65.

• Boyle J, Ireland D, Webster F, O’Sullivan K, Predicting Demand for Hospital Capacity Planning,

Conf Proc IEEE Biomedical and Health Informatics. 2016: 328-331

• Jessup M, Crilly J, Boyle J, et al. Users' experiences of an emergency department patient

admission predictive tool: A qualitative evaluation. Health Informatics J. 2016 Sep;22(3):618-32

24

Case Study 3 : Making it WorkWHO?

• Bed manager, after hours co-

ordinator, hospital executive,

hospital executive on-call, Director

ED, Director Medicine, Director

Surgery, decision support services

WHY?

• Inform managers (nursing and

medical) re expected admissions

and discharges so they have

information to work proactively

HOW?

• PAPT Software and PAPT

Procedure Manual: guide to assist

communication processes with in-

built ‘triggers’ to inform decision

making (planning and functioning)

WHEN?

• Daily and Weekly Reference: Crilly J, Boyle J, Jessup M, et al. The Implementation and Evaluation of the Patient Admission Prediction Tool: Assessing Its Impact on

Decision-Making Strategies and Patient Flow Outcomes in 2 Australian Hospitals. Qual Manag Health Care. 2015 Oct-Dec;24(4):169-76

25

Case Study 3 : Making it WorkWHO?

• Bed manager, after hours co-

ordinator, hospital executive,

hospital executive on-call, Director

ED, Director Medicine, Director

Surgery, decision support services

WHY?

• Inform managers (nursing and

medical) re expected admissions

and discharges so they have

information to work proactively

HOW?

• PAPT Software and PAPT

Procedure Manual: guide to assist

communication processes with in-

built ‘triggers’ to inform decision

making (planning and functioning)

WHEN?

• Daily and Weekly Reference: Crilly J, Boyle J, Jessup M, et al. The Implementation and Evaluation of the Patient Admission Prediction Tool: Assessing Its Impact on

Decision-Making Strategies and Patient Flow Outcomes in 2 Australian Hospitals. Qual Manag Health Care. 2015 Oct-Dec;24(4):169-76

26

Case Study 4 : Evidence Driven KPIs

The National Emergency Access Target (NEAT)

“By 31 Dec 2015, 90% of all patients will physically leave the

Emergency Department (ED) within 4 hours”

Australian Institute of Health and Welfare Canberra, Australian

hospital statistics 2012–13, Emergency department care,

Health Services Series Number 52

64%

66%

72%77%

66%

67%

64%

57%

64%

66%

72%77%

66%

67%

64%

57%

• First study to deliver evidence driven

targets for public hospital ED patient flow

• Directly translated into government policy

• Several awards and endorsements

• Replicated in several other states, and in

the UK

Reference: Sullivan C, Staib A, Khanna S, et al. The National Emergency Access Target (NEAT)

and the 4-hour rule: time to review the target, Medical Journal of Australia 2016 May 16;204(9):354

27

Case Study 4 : Evidence Driven KPIs

Methodology :

• Focus on Emergency Admissions only

(i.e. eHSMR)

• Exclude Palliative Care and Short Stay

• Develop several predictive models for

eHSMR calculation

• Model relationship between NEAT

Compliance and eHSMR

• Check for confounding effect of palliative

care and short stayNo robust evidence regarding a clinically

significant mortality benefit above this threshold

Reference: Sullivan C, Staib A, Khanna S, et al. The National Emergency Access Target (NEAT)

and the 4-hour rule: time to review the target, Medical Journal of Australia 2016 May 16;204(9):354

28

Case Study 5 : Reducing Readmissions

Project Brief• To develop, implement and evaluate a

web-based risk stratification algorithm

that can be used in-hospital to identify

chronic disease patients with a high risk

of re-hospitalisation.

What are we predicting• Unplanned re-admission within 30

days of discharge from hospital

• Unplanned ED re-presentation within

30 days of discharge from hospital

Timeline• Trial - Apr 2018 to Mar 2019

• Evaluation - Apr 2019 to Jun 2019

Chronic Disease Patient Admitted to Hospital

Risk Score generated overnight

Risk score used by care teams for appropriate interventions and

care/discharge planning

Next morning

29

Case Study 5 : Reducing Readmissions

Patients who, over a 5 year period, as an

Emergency or Admitted Patient :

• Attended Logan Hospital, and

• Had at least one Chronic Disease visit

(any QLD hospital)

Patient Cohort

• Emergency Data (EDIS/FirstNet)

• Inpatient Data (QHAPDC/ePADT)

• Mortality Data (Death Registry)

• Pharmacy dispensing information (eLMS)

• Pathology test results (AUSLAB)

Data Used for Modelling & Validation

• Patients stays in prev. 180 days

• ED presentations in prev. 180 days

• Marital status

• Age

• Indigenous status

• SEIFA

• Admission source

• Admission unit

• Care type

• Elective status

• Planned same day status

• Binary flags for routine dialysis

• Number of medication records

• Binary flags for medication

• Binary flags for abnormal pathology

Predictor Variables in Final Models

• Logistic regression

• Naïve Bayes

• Neural Nets

• Random Forests

• Generalised Boosting

Modelling Techniques

30

Case Study 5 : Making it Work

Employing Intelligible Machine Learning

31

Case Study 5 : Making it Work

Employing Intelligible Machine Learning

32

- Engagement between with clinicians when designing solutions

- Focus on translation early on

- Frameworks, Standards and Governance

- Ensuring statistical rigor

- Choosing the right outcome measures – KPIs vs patient outcomes

- Translation is not the clients problem

- Innovative approaches

- Value of understanding the domain and the data

- Empathy with perspectives of the problem owner and the pain points

Recommendations

33

For more information, please contact:

www.csiro.us

Please remember to complete the online session evaluation

Questions

Justin Boyle

Principal Research Scientist

e [email protected]

Sankalp Khanna

Senior Research Scientist

e [email protected]

t @SankalpKhanna

l /in/sankalpk

http://www.csiro.us/

mailto:[email protected]

mailto:[email protected]

deriving value from patient flow analytics · deriving value from patient flow analytics session...

Documents