data & metrics; what can they tell us, how do we respond? nhs midlands & east stephen duncan...
TRANSCRIPT
Data & Metrics; What can they tell us, how do we respond?
NHS Midlands & East
Stephen DuncanHead of Intensive SupportUrgent & Emergency Care
NHS | Presentation to [XXXX Company] | [Type Date]2
Metrics are tools for supporting
actions that allow programs to evolve
toward successful outcomes, promote
continuous improvement, and enable
strategic decision making
But have we got our measurement systems right?
“We are flooded by data but starving for information”
Jiawei Han
a Hierarchy of profound Knowledge
data
wisdom
KNOWLEDGE
insight
information & evidence
• INFORMATION is data that are processed to provide answers to "who", "what", "where", and "when" questions – information is data that has been given meaning by the making of relational connection – this meaning can be useful, but does not have to be so.
• DATA are mere symbols – having no significance beyond their existence - usable or not. It has no meaning of itself. e.g. an un-interpreted spreadsheet.
Faced with meeting a Target?
•You can work to improve the system
•You can distort the system to give the illusion of improvement
•You can distort the data to give the illusion of improvement
Don Wheeler Understanding Variation
a good METRIC is..
M Measurable METRIC & MEASUREMENT PROCESS is operationally-defined, numerical if possible (though not necessarily)
and likely to detect systemic change, by separating noise from signals.
E Engaging,Ethical
Those involved are engaged by the continual learning that is likely to emerge. No one can be harmed by the measurement process.
T Time Sequenced,Time Real, Talked About
Chartable in TIME-SEQUENCE order,Observable in real time – by those who best understand the contextPrompting timely conversation for timely action.
R Relevant, Realistic
RELEVANT to scorecards & stakeholders – especially customers.REALISTIC in that it’s inexpensive to administer, and likely to stay that way.
I Inquiry enabling When analysed over time, the metric will tee-up inquiry by the people who best know the context, and can take the most appropriate action.
C Customer-led Metrics are related to intended system purpose and outcome – expressed in terms the customer would get.Sub-system/ Process metrics should link with outcomes.
7
Preventative/Predictive careDisease managementManaged populations
Alternatives to acute admission settings
Alternative access for diagnosis
Alternative settings for therapy
Alternative sites for discharge
Alternative sites for readmission
Health Promotio
n
General Practice
& GP OOH
Community
Support
Ambulance Service & GP OOH
A+EMAU/SAU/Short Stay
Focus on CDM and more effective responses to urgent care needs – ACS condition management
Clear operational performance framework and integrated in to primary care Improved integration with primary care
responders Front load senior decision process incl primary care
Redesign to left shift LOS
Inpatient Wards
Optimise ambulatory emergency care
Information flow converting the unheralded to the heralded
Discharge Process
Somethingvery important!
Lastmonth
Thismonth
What actionis appropriate?
Given two different numbers, one will always be bigger than the other!
NHS | Presentation to [XXXX Company] | [Type Date]9
Use Run charts and SPC
Measure yourself with yourself! Bench marking can be good, BUT… Avoid averages Avoid using data to make assumptions
without observation
Variation in a system is normal
• The variation is caused by factors that are inherent in the system over time
• They affect all outcomes
• This is ‘common cause’ variation or
• The causes are ‘unassignable’
• Common cause variation can be reduced by tackling things that affect the process all the time
Some variation may not be normal
• The factors are not present in the process all the time
• They do not affect everybody
• They arise because of specific circumstances
• This is ‘special’ or ‘assignable’ cause variation.
1. Roberts T. Understanding variation [Online] July 2005 [cited July 2009]; [17 Pages] Available from:http://www.evidencebasedpractice.org.uk/documents/presentations/spc_TEBPCJune2005.ppt
3 Dangers to Beware Of…
• Reacting to special cause variation by changing the process
• Ignoring special cause variation by assuming “its part of the process”
• Do not compare more than one process
Facts on Emergency Care Quality
People perceive things
Perception drives expectations
Expectations drive perception of quality
Definitions of quality vary
Different definitions of quality lead to confusion
Why we measure quality?
Internal “quality control”
To support improvement : if we do something differently, will we get better results?
To drive improvement
Performance management
Comparison
To inform commissioning
Measures
There are 3 types of metrics used to measure quality in urgent and emergency care:
Structure: Physical equipment and facilities & beds
Process: How the system works, (4 hour standard)
Outcome: The final product, results of the care delivered (e.g. mortality)
Structure and process are easier to measure; outcome is more important.
Where do we start?
• Be clear about your aim statement – with a definable system level improvement metric(s) –
• how much by when and how measured.
• Measure some key ‘Process’ metrics.
• Ignore balancing metrics at your peril!
• Use Statistical Process Control effectively
Setting Intent‘
Achieving the target without missing the point’• What does ‘good’ look like?
• Reduced LTC progression
• Reduced institutionalisation
• Increased independence
• Reduced ED attendances and emergency admissions
• Reduced occupancy of baseline adult non-elective beds
• Monitor 75+ year olds with LoS >14days
Examples of process measures
4 hour standard and CQIs
Response to calls in OOH care
Time to X-ray in dislocated shoulder
DKA to fluids
Door to balloon time
Good things about process measures
They measure processes !
They suit computers
Advantages over outcome measures
Can promote systems approach
Problems with process measures
Are they really linked with improved outcomes? (MC 2009) You need information systems to measure what you need
Are we allowing IT to determine our measures ?
They don’t measure everything
What process measures don’t measure
Clinical effectiveness “Failures”Quality in clinical decision makingQuality in training / educationQuality in researchCaringOutcomes
Problems with process measures
They often become a target
Improve the system, distort the system, distort the data
Winners and losers
Unforeseen consequences
Speed / quality trade-off
More meaningful metrics of quality? - Outcome measures
How many thrombolysable strokes were missed?
How many kids in pain in Emergency Departments were reassessed after analgesia?
What was the experience of bereaved families in Emergency Departments
Experience of out of hours care
Adoption / implementation of recent guidelines?
Our advice…..Principles of metrics and measuring qualityDevelop a set of simple and well defined measures
Make them useful, not perfection
Develop a metric strategy and a cascade of measures
Include balancing metrics or “unintended consequences”
Capture qualitative and quantitative information
Build measurement into daily work
Building a Cascade of Measures
L 1System
L 2Board & CEO
L 4
Outcome - system level eg admissions, death, harm, Institutionalisation etc
Process + Outcome
Process (+ Outcome)Microsystems: Units, Depts
L 5Physician & Patient
IndividualProcess Metrics
Adapted from Lloyd & Caldwell
L 3Service
Line
Impact – Beds occupied – TotalObjective – Hard Red Lines
Aim – Reduce Acute beds occupied to SPC mean of 600 or less + reduce crude in-hospital mortality rate by 10% + a fall in SCHMI by 31st March 2012Process measure – The whole system action plan etc etc ie holding the system to account not just the acute sector.Balancing – Deliver a decrease in Long term care ie more patients returning to live at home. No increase in 30 day re-admission rate
Understanding Demand – ED Attendance - Admitted vs Non Admitted - Trust
Aim – reduce emergency admission from ED to an SPC mean of < 50 per dayProcess – Deliver S+T, RAT, + Intermediate care + Mental Health improvementsBalancing – 7 day re-attendance and 30 day re-admission
Trust NEL Admissions and Discharges Day Profile
Aim – left shift discharges to the morningProcess – Board rounds, EDD, Criteria for discharge Balancing – don’t cheat and tip evening discharges to the next morning
Trust NEL Admissions and Discharges
Aim – Reduce emergency admissions – by 20 by 31st March 2012Processes – RAT in A+E, 10 Care improvements, improved EoL care etcBalancing - prevent any increase in institutional care
Zero LOS Discharges - TrustExcl paediatrics, midwifery and obstetrics
Aim – Increase zero LOSProcess – deliver AECBalancing – Reduce overall NEL admissions
2 midnights or less LOS Discharges - Trust
Aim – Increase short stay dischargesProcess – deliver AEC + short stay review processBalancing – Reduce overall NEL admissions
In-Patients with LOS 14 days or more - Trust
Aim – Reduce I/P with LOS 14 + to 75 or less by 31st March 2012Process – Early identification of at risk group, CGA, early supported discharge schemesBalancing – no increase in institutional care – aim for a reduction in over 75s in Long term Care
A word from the wise….
Most of what you can measure isn’t important
AND
Most of what is important can’t be measured
A final beware!!
What the human being is best at doing is interpreting all new information so that their prior conclusions remain intact.
Warren Buffett
