is automatic data collection for quality indicators possible?
DESCRIPTION
IS AUTOMATIC DATA COLLECTION FOR QUALITY INDICATORS POSSIBLE?. 17.3.2011 Matti Reinikainen North Karelia Central Hospital Joensuu, Finland. CONFLICTS OF INTEREST STATEMENT:. MATTI REINIKAINEN, MD - PowerPoint PPT PresentationTRANSCRIPT
www.pkssk.fiPohjois-Karjalan sairaanhoito- ja sosiaalipalvelujen kuntayhtymä
IS AUTOMATIC DATA COLLECTION FOR QUALITY
INDICATORS POSSIBLE?
17.3.2011 Matti Reinikainen
North Karelia Central HospitalJoensuu, Finland
www.pkssk.fiPohjois-Karjalan sairaanhoito- ja sosiaalipalvelujen kuntayhtymä
CONFLICTS OF INTEREST STATEMENT:
MATTI REINIKAINEN, MD
- Position: Chief Physician, Dept of Intensive Care, North Karelia Central Hospital, Joensuu, Finland
- Position of responsibility: 1st Secretary, Finnish Society of Intensive Care
- Connection with the Finnish Intensive Care Consortium:
1. Using reports and analyses as department leader
2. Using database for research purposes, PhD thesis under preparation
- Economic interests in this subject: none
TERMS USED IN THIS PRESENTATION
• Clinical information system (CIS) – a computer system that replaces bedside paper
documentation– the system automatically collects data from patient
monitors and ventilators and the lab system and shows the data both numerically and as graphic trends
– bedside screen(s)
• Data collection software– a computer system that is linked to the CIS and
automatically transfers data to the centralised database of the Finnish Intensive Care Consortium
– data accuracy is checked before submission
QUESTIONS TO BE ANSWERED
• If automatic data collection were possible, would there be associated benefits?
• Is it possible?
• Are there drawbacks?
When you are taking care of a patient who is bleeding
…and who is haemodynamically unstable
Do you have time for careful documen-tation of blood pressures etc.?
Or would it be helpful if the data were collected automatically?
Automatic data capture into a clinical information system decreases the time spent by nurses on documentation and increases the time spent on patient care
• Wong DH et al. Changes in intensive care unit nurse task activity after installation of a third-generation intensive care unit information system. Crit Care Med 2003; 31: 2488-94.– Before and after installation of clinical information
system– The percentage of time ICU nurses spent on
documentation decreased by > 30% and the time spent on patient care increased
Automatic data capture into a clinical information system decreases the time spent by nurses on documentation and increases the time spent on patient care
• Bosman RJ et al. Intensive care information system reduces documentation time of the nurses after cardiothoracic surgery. Intensive Care Med 2003; 29: 83-90.
– Randomised controlled trial! – documentation on paper vs. into an information system
– A 30% reduction in documentation time (p < 0.001) was achieved, corresponding to 29 min per 8 h nursing shift
– This time was completely re-allocated to patient care
… or does it?
• Saarinen K, Aho M. Does the implementation of a clinical information system decrease the time intensive care nurses spend on documentation of care? Acta Anaesthesiol Scand 2005; 49: 62-5.
– After the implementation of a CIS, there was a small (statistically non-significant) increase in the time spent on documentation
– However, simultaneously there was a significant increase in the time spent on patient care
– ”…any plans to reduce the ICU staff with the aid of computers were not justified.”
Finland
Pirjo Kontio
North Karelia Central Hospital
Population in the district:
173 000 (+ 200-300 bears)
Finland
FINLAND
• Population 5,3 million
• Area 338 000 km2
- The Finnish Intensive Care Study, 1986-87- 25 ICUs- Niskanen M, Kari A, Halonen P. Five-year survival after
intensive care – comparison of 12 180 patients with the general population. Crit Care Med 1996: 24: 1962-1967.
- The Severity Study- Le Gall J-R et al. A new Simplified Acute Physiology Score (SAPS II) Based on a European / North American Multicenter Study. JAMA 1993: 270: 2957- 13 152 patients (720 from 7 Finnish hospitals)- Aarno Kari as country coordinator
1994
- A quality assurance project started in 1994 - Strong growth since 1998- university hospitals joined in 2000-2002
THE FINNISH INTENSIVE CARE CONSORTIUM
1994 2007
THE FINNISH INTENSIVE CARE CONSORTIUM
• 20 hospital districts on Finnish mainland• The main hospital in each district is called the
Central hospital• 15 non-university hospitals, all adult ICUs
participate in the Consortium• 5 university hospitals
– All ICUs from 3 of these participate– In 2 university hospitals: in addition to participating
units, some specialised units not participating
• Apart from 1 ICU, all units use clinical information systems and automatic data transfer into a centralised database
Data collected by clinical information systems, including laboratory test results
• … are automatically transferred by a data collection software to the centralised database of the Finnish Intensive Care Consortium
• The database is handled by Tieto Healthcare & Welfare (previously by Intensium)
• The results in key performance indicators are calculated and reported, many of them automatically
QUALITY INDICATORS
• DATA COMPLETENESS• ADEQUACY OF PATIENT SELECTION• OUTCOMES• RESOURCE CONSUMPTION
In the following slides:• Blue squares = North Karelia Central Hospital• Red circles = the rest of the Finnish Intensive Care
Consortium (i.e. ”the average ICU”)• Each square / circle is based on data from the
previous 6 months
EXAMPLES OF REPORTS THAT ARE UPDATED (SEMI)AUTOMATICALLY
DATA COMPLETENESS
The data completeness index: the second best performing unit (next to the one with the least missing data) gets the index figure 100. The second worst performer (next to the one with most missing data) gets the index figure 50. Other units get index figures based on how close their performance is to these two.
ADEQUACY OF PATIENT SELECTION
• Basic idea: the indication for intensive care is a temporary danger to life and a possibility to prevent death by intensive care
• ICU admissions may be ”inadequate” when– there is no danger to life and no care of high intensity is
needed – could these patients be managed elsewhere?
– patients are moribund – care is futile
High risk and intensive care
The percentage of patients with a high risk of death (> 0,3) and a high intensity of care (TISS score > 30/d)
Low risk – unnecessary ICU admission?
The percentage of patients who had a low risk of death (< 0,05), received care of low intensity (maximal TISS score < 15/d) and were discharge alive
OUTCOMES
• Hospital mortality rate (crude & standardised mortality ratios)
• Mortality after long ICU stays• Post-icu mortality
• (Also measured, though with a considerable amount of manual work: 6-month mortality & health-related quality of life at 6 months)
VLAD curve, the cumulative amount of ”extra lives saved”; curves for 3 ICUs
SMR
Standardised mortality ratios (O/E-ratio, the number of observed deaths divided by the number of expected deaths, the expected number here being based on the SAPS II model)
Hospital mortality after prolonged ICU care
Hospital mortality of patients who were treated in the ICU for > 6 days
Prolonged care – poor outcome
The percentage of patients who were treated in the ICU for > 6 days and who died in the ICU
Post-ICU hospital mortality
The percentage of patients who died in hospital after discharge from ICU
A problem here?
Readmissions within 48 hrs after discharge
The percentage of patients who were readmitted to the ICU within 48 hrs after ICU discharge
Shortage of beds?
• In relation to care days produced
• (Also measured: ”Cost of lives saved” – the amount of resources consumed per hospital survivor. However, comparisons are difficult because all ICU costs are not easily obtained in Finnish hospitals.)
RESOURCE CONSUMPTION
Care days / nurse / day
The number of patient days (24-h-periods) per each nurse / shift
IN PRACTICE, WE CAN SPEAK ABOUT SEMI-AUTOMATED DATA
COLLECTION
• Many data are entered manually into the clinical information system– Admission data– ICU and hospital discharge data (incl. outcome)– TISS items are documented partly automatically, partly
manually– Some physiological data need to be entered manually
Even automatically collected data are checked and validated,
… in North Karelia by this team
Admission data are checked for accuracy and for missing data
TISS items are checked
• Lab test results are mostly transferred without problems
Some derived parameters can be problematic
The PaO2/FIO2-ratio is correct only if both values are documented correctly
FIO2 is documented automatically but monitoring may not be on e.g. when inhalational drugs are given
(mmHg)
• Values of some physiological parameters have to be checked for possible technical artifacts– e.g. blood pressure (trends of systolic BP in
the next examples)
Unfiltered raw data
Patient 1
Median filtering (10 min) eliminates most technical artifacts
Patient 1
Patient 2
Unfiltered raw data
Patient 2
Median filtering eliminates most technical artifacts – but not all of them
Patient 2
Manual correction needed
HAS DATA COMPLETENESS CHANGED IN ASSOCIATION WITH THE AUTOMATION OF
DATA COLLECTION?
1998 2003 2008 p
Median (quartiles) 1 (1-2) 1 (0-2) 0 (0-0) < 0.001
Mean ± SD 2.0 2.3 1.2 1.6 0.25 0.85 < 0.001
Missing values of SAPS II parameters in the database of the Finnish Intensive Care Consortium:
43210-1-2-3-4-5-6-7-8-9
Years before and after IVT installation. Installation Year =0
100,00
80,00
60,00
40,00
20,00
0,00
Ra
tio
of
10
0%
co
mp
lete
re
co
rds
Mussalo P, Tenhunen J. Abstract presented at the ISICEM Congress, 2007.
- The proportion of 100% complete datasets was determined (incl. admission and discharge data and data on diagnoses)
- This proportion increased after installation of data collection software Installation of data collection software
Association of data completeness with installation of data collection software
ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen
How often is it possible to measure the outcome?
Mussalo, P. 1,2, Reinikainen, M.3, Karlsson, S.4, Ruokonen, E.5
For the Finnish Intensive Care Quality Consortium
1) Intensium Ltd, Kuopio, for the Finnish Intensive Care Quality Consortium, 2) Department of Computer Science, University Of Kuopio, Kuopio,
3) Department of intensive Care, North Karelia Central Hospital, Joensuu, Finland, 4) Department of Intensive Care, Tampere University Hospital, Tampere, Finland,
5) Department of Intensive Care, Kuopio University Hospital, Kuopio, Finland
ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen
The mean time from discharge to registration of hospital discharge data is 1 month
Delay to HDD registrationFrom ICU discharge From hospital discharge
STATUS N Mean CI95% Mean CI95%ALL 17146 38,5 (37,93-39,16) 31,2 (30,63-31,78)SURVIVOR 14297 40,8 (40,21-41,46) 32,6 (31,92-33,21)DEAD 2849 27,1 (25,20-28,96) 24,3 (23,14-25,56)
ESICM Lisbon, 24.9.2008, Mussalo, Reinikainen, Karlsson, Ruokonen
Outcome reporting is quick – situation 1st September, 2008
Does this hospital have a summertime problem?
ARE THERE DRAWBACKS?
The sensitivity of sophisticated computerized methods in detecting artifacts in blood pressure trends is inferior to that of experienced human observers
Care days / nurse / day
The number of patient days (24-h-periods) per each nurse / shift
IS THERE A POTENTIAL FOR ERRORS GOING UNNOTICED?
Care days / nurse / day
The number of patient days (24-h-periods) per each nurse / shift
This peak is caused by an error in the data: erroneous data submission after the fusion of two units
IS THERE A POTENTIAL FOR ERRORS GOING UNNOTICED?
• Morrison C et al. Electronic patient record use during ward rounds: a qualitative study of interaction between medical staff. Crit Care 2008; 12: R148 – The installation of an electronic patient record
system had a negative impact on multidisciplinary communication during ward rounds
• ”Pen & paper” is a very flexible system
• A highly sophisticated computer system may not always be that flexible
CONCLUSIONS
• Fully automatic data collection for quality indicators is not possible– Parts of data have to be entered manually
– (At least parts of) data collected automatically have to be checked and validated in order to assure good data quality
• Semi-automatic data collection is possible, it saves time used for documentation and improves data completeness