real-time monitoring and the data trap
TRANSCRIPT
Real-time monitoring and the data trap
Kate Silvester, Healthcare Systems Engineer, Kate Silvester Ltd
Chair: Andrew Hutchings, Assistant Professor, London School
of Hygiene and Tropical Medicine
Kate Silvester Ltd 2
Monitoring service performance
• Safety: • Right care • % patients receiving right care = process yield
• Flow • On time, every time, in full
• ‘Right care but too late’ • Flow governed by the rate limiting step (constraint) of the process • Cost of the time used to add-value/cost of total time the resources
are available = efficiency
• Quality • Service experience of patients, carers and staff
• System Productivity • = yield x efficiency
Simon Dodds MA MS FRCS
The Process for turning Data to Information:
Define the Reporting Window
Define the Data Window
Enter the patient level data
(Event start and finish date and
time)
Question the Request
Comparativedata cf other
services
Real time
time-series data for
the service
Return the data +/- report
Front line
staff perform
the work
Send the request to an information
analyst
Delay
Redefine the data window
Compare data with
Expectations and Reality
Adjust
our thinking (mental model)
Make a decision
+/- a change
Minimum 6 weeks with HES data
Kate Silvester Ltd
3
Kate Silvester Ltd 4
Issues with the process for turning Data to Information
1. Delay in the feedback loop. • Minimum of 6 weeks if we use HES data
• ‘’driving a car using the rear view mirror’’
2. Error in the method for accessing the data
Kate Silvester Ltd
5
Requesting Data for a Performance Report • “I need to write the monthly report for June
2015 to show the demand, activity, waiting times and numbers waiting for my endoscopy service”
• In a dynamic system, how do you get an accurate
picture of what is going on using data for a specific time period (e.g. month)?
01/06/2015 30/06/2015
Reporting window e.g. June 2015
Define the Reporting Window
Then define the Data Window:
Kate Silvester Ltd
6
All patients requested after beginning 01/06/15
and reported before end 30/06/15
?
What would the resulting charts ‘tell us’?
•
•
•
•
Waiting list numbers are increasing Lead times are getting longer Demand > Activity QED: ‘We need more resources’
800
700
600
500
400
300
200
100
0
Numbers waiting 900
June 2015: Endoscopy Waiting list (Invalid)
25
20
15
10
5
0
Days
30
Endoscopy: lead time in days for patients scoped in June 2015 (invalid)
Clinical and front line staff: Niggle: ‘Something odd’: ‘We know we have done more than 237 endoscopies in 22 days (10/day = 5/day/endoscopy room… we do double this number…) Lead times are under 30 days but we are not meeting our waiting time targets … for either new or surveillance patients ……the data must be wrong….’
60
50
40
30
20
10
0
01/
06/
201
5
02/
06/
201
5
03/
06/
201
5
04/
06/
201
5
05/
06/
201
5
06/
06/
201
5
07/
06/
201
5
08/
06/
201
5
09/
06/
201
5
10/
06/
201
5
11/
06/
201
5
12/
06/
201
5
13/
06/
201
5
14/
06/
201
5
15/
06/
201
5
16/
06/
201
5
17/
06/
201
5
18/
06/
201
5
19/
06/
201
5
20/
06/
201
5
21/
06/
201
5
22/
06/
201
5
23/
06/
201
5
24/
06/
201
5
25/
06/
201
5
26/
06/
201
5
27/
06/
201
5
28/
06/
201
5
29/
06/
201
5
30/
06/
201
5
Invalid Vitals Chat for June 2015: Endoscopy demand, activity
demand
Activity
Kate Silvester Ltd
7
The problem
30/06/15 01/06/15
The reporting window
Patient lead times
Request Date (demand)
Procedure Date (activity)
Long lead times: How far back should I go to capture all the demand and activity, especially for high risk patients under surveillance?
Kate Silvester Ltd
8
Due date
Due date
Due date
What is the longest lead time?
01/06/15 30/06/15
The reporting window
Patient lead times
Start of the data window
Kate Silvester Ltd
9
Now…
Or ‘I need the biggest possible data window’…..
Kate Silvester Ltd
10
Now what do we get now?
Same problem!
For the period of the reporting window, when does the valid data start and end?
The trap for Heffalumps
A.A. Milne
www.bookishnature.wordpress.com Kate Silvester Ltd
11
Recognise the Trap for Heffalumps: • All patients requested after
beginning 01/06/15
• Reported before end 30/06/15
Kate Silvester Ltd
12
08/03/2015 20/09/2015
01/06/2015
Data Window
Reporting window
e.g. June 2015
31/06/2015
1
3
4
6
Correct way: Counter-intuitive request
All patients requested before end 30/06/15
6
5
4
2
3
Counterintuitive request and reported after
01/06/15 Kate Silvester Ltd
13
Patient 3
Patient 4
Patient 5
Patient 6
Patient 2
We want the patients with Lead time types
Patient 1
Valid Vitals Charts® for June 2015
250
200
150
100
50
0
04/0
6/20
14
11/0
6/20
14
18/0
6/20
14
25/0
6/20
14
Number of requests,
procedures and
patients waiting
Valid Vitals Chart® for for Endoscopy Demand, Activity and Waiting list for June 2015
demand
activity
WiP = cumulative diff
What is the story now? Demand > Activity, WIP has gone up slightly
Surveillance Patients scoped at 1 year
New patients at 6/52 Suspected cancers at 2/52
Kate Silvester Ltd
14
Example of invalid updating of real time data
16
Tracking patients with long
Lengths of Stay in hospital
‘My patients are disappearing from the weekly reports when I know they are still in hospital! Why?’
Practice Nurse
Incorrect data window.
How do we monitor a service in real time?
• The patient level event data are there for real time monitoring
• Can we teach service managers and clinicians to: • Define the reporting window,
• Then define the data window,
• Counter-intuitive request
• Write their data queries,
• Update their time series charts in their performance reports,
• Interpret their charts correctly?
Kate Silvester Ltd
17
The Process for turning Data to Information:
Define the Reporting Window
Define the
Data Window
Enter the patient level data
(Event start and finish date and
time)
Real time
time-series data for the
service
Front line
staff perform
the work
Compare data with
Expectations and Reality
Adjust
our thinking (mental model)
Make a decision
+/- a change
Kate Silvester Ltd
18
Question the Request
Comparative data cf other
services
Return the data +/- report
Send the request to an information
analyst
Delay Redefine the data window
Minimum 6 weeks with HES data
Questions, discussion and further information
• Journal in Improvement Science (JOIS) • http://www.improvementscience.net/jois/
• Free
• Need to register • A Study of the Relative Value of Different Time-Series Charts for Proactive
Process Monitoring. S Dodds 2012
• Diagnosing the Flow Constraint in an Endoscopy Service. Part 1: Recognising
and Avoiding the Data Query Trap. K Silvester 2016
• Simon Dodd’s FISH and ISP Training programme • http://www.improvementscience.net/
Kate Silvester Ltd
19