better information --> better decisions --> better health1

ScotPHO Training Day Hospital Standardised Mortality Ratios

Richard DobbieGavin MacColl

(ISD Quality Improvement Programme)

better information --> better decisions --> better health

Quality Improvement Programme

• Hospital Standardised Mortality Ratios (HSMR’s)

• Surgical / Medical Profiles

• Scottish Arthroplasty Project

• Complaints

• Incident Reporting Pilot

• ‘Better Together’ Inpatient Survey

• National Audits

•SICSAG / Stroke / STAG-Sepsis / Renal Registry

Routine Linked Data

Quality Strategy – Patient Centred

Clinical Engagement

better information --> better decisions --> better health3

What is an HSMR?

Observed Deaths

Predicted DeathsHSMR =

Observed Deaths = Deaths

Predicted Deaths = Predicted Probabilities

better information --> better decisions --> better health4

• Data source - SMR01/GRO linked data• Outcome - 30 day mortality• Patient indexing – Quarterly (CIS) • Explanatory variables

• Age, sex, deprivation• Type of admission (elective / non-elective)• Inpatient / day case• Place admitted from • Number of previous emergency admissions• Primary diagnosis• Prior morbidity – 1 and 5 years• Surgical / non-surgical specialty

HSMR Methodology

better information --> better decisions --> better health5

Patient Record Sets

SMR1 28-May-04 Emergency Edinburgh R.I. General Medicine Stroke 09-Jun-04

SMR1 09-Dec-06 Elective Ninewells Cardiac Surgery CABG AMI 10-Dec-06SMR1 10-Dec-06 Emergency Ninewells ICU Haemorrhage 13-Dec-06SMR1 13-Dec-06 Emergency Ninewells Cardiac Surgery Haemorrhage 18-Dec-06

SMR1 22-Aug-07 Elective Falkirk R.I Orthopaedics Hip Replacement Osteoarthritis 24-Aug-07SMR1 24-Aug-07 Transfer Stirling R.I Orthopaedics Remanipulation of Hip Dislocated Hip 30-Aug-07SMR1 30-Aug-07 Transfer Falkirk R.I Orthopaedics Blood Transfusion Infection 15-Sep-07

SMR1 17-Sep-08 Emergency Aberdeen R.I General Surgery Laparoscopy & Gastrectomy Stomach Cancer 28-Sep-08SMR6 17-Sep-08 Registration for Stomach Cancer

SMR1 03-Oct-08 Emergency Aberdeen R.I General Medicine DVT/PE 09-Oct-08

RG Death 12-Oct-08 Cause of Death: Cancer

HSMR’s SAP Profiles

better information --> better decisions --> better health6

Primary diagnosis group

30 day mortality rate Primary diagnosis group

30 day mortality rate

Cerebrovascular1 12.4 (8.3 – 13.8) Malignancy3 16.6 (14.7 – 30.3)

Cerebrovascular2 24.3 (19.3 – 40.1) Metabolic 3.2 (0.9 – 8.7)

CVS1 2.3 (1.3 – 7.2) Miscellaneous1 1.0 (0 – 2.83)

CVS2 11.0 (8.2 – 13.5) Miscellaneous2 5.6 (3.7 – 18.2)

CVS3 15.9 (13.7 – 31.3) Mortality 93.0 (92.9 – 100.0)

CVS4 68.0 (62.5 – 70.8) Neurological1 1.1 (0.6 – 2.9)

Gastrointestinal1 1.4 (0.7 – 4.1) Neurological2 4.5 (2.5 – 17.4)

Gastrointestinal2 6.5 (5.0 – 15.7) Renal 17.8 (8.3 – 22.2)

Gastrointestinal3 13.4 (10.1 – 23.4) Respiratory1 2.5 (0.7 – 4.3)

Haematology 2.7 (1.5 – 6.1) Respiratory2 6.6 (3.2 – 9.0)

Low risk 0.2 (0 – 0.6) Respiratory3 26.1 (8.9 – 47.8)

Malignancy1 3.9 (1.5 – 7.4) Trauma1 1.2 (0.7 – 3.0)

Malignancy2 11.4 (8.1 – 14.2) Trauma2 6.3 (3.1 – 16.7)

Primary Diagnostic Groupings

better information --> better decisions --> better health7

Condition Weight (1=least severe, 6=most severe)

Acute myocardial infarction 1

Congestive heart failure 1

Peripheral vascular disease 1

Cerebral vascular accident 1

Dementia 1

Pulmonary disease 1

Connective tissue disorder 1

Peptic ulcer 1

Liver disease 1

Diabetes 1

Diabetes complications 2

Paraplegia 2

Renal disease 2

Cancer 2

Metastatic cancer 3

Severe liver disease 3

HIV 6

Prior Morbidity

better information --> better decisions --> better health8

Scottish Case-Mix Model(selected pathways in model)

584,922 patients4.6% mortality within

30 days

Surgical specialties

328,834 (1.5%)

Non-surgical specialties

256,088 (8.6%)

Prim diag = Gastro2 or CVS2

2485 (10.4%)

Age <=44604 (1.0%)

Prim diag = Malig3

4676 (53.1%)

Non-elective admission

3251 (64.7%)

Previous emerg adms >0

1837 (69.2%)

Sex = Male992 (71.3%)

Predicted prob = 0.01

Predicted prob = 0.71

better information --> better decisions --> better health9

Timeline of HSMR Development

March 08 QIP starts developmentSurgical Profile

May 08 Learning Session 2Progress reported to senior NHS leadersRecommendation to establish design groupAligned to an extent with Dr Foster

December 08 First Results ReleasedAdmissions: October 2006 to September 2007 Communication: SPSA to Chief ExecutivesNo outliers on the funnel plot

January 09 Learning Session 3Methods and results of validation presented

better information --> better decisions --> better health10

Timeline of HSMR Development

May 09 Updated AnalysisAdmissions: October 2007 to September 2008Data passed to the SPSANo data circulated to Chief Executives

May 09 Learning Session 4

Sep – Dec 09 Model adapted for quarterly reporting

November 09 Learning Session 5Quarterly Model explained

December 09 Quarterly HSMR’s ReleasedTime series per hospital showing observed and expected deathsCommunication: ISD Director to CEOs and MDs Password protected area on ISD Website

better information --> better decisions --> better health11

Quarterly Release

L106HOctober 2006 - December 2009

Monklands District General Hospital

QuarterObserved deaths

(30 days)Predicted deaths

(30 days)

Standardised Mortality Ratio

(SMR)Number of

patientsCrude Mortality

Rate (%)

Oct-Dec 2006 220 229 0.96 7369 3.0

Jan-Mar 2007 271 247 1.10 7214 3.8

Apr-Jun 2007 296 226 1.31 7110 4.2

Jul-Sep 2007 265 210 1.26 6902 3.8

Oct-Dec 2007 229 230 1.00 7166 3.2

Jan-Mar 2008 274 228 1.20 7299 3.8

Apr-Jun 2008 228 221 1.03 7532 3.0

Jul-Sep 2008 232 211 1.10 7504 3.1

Oct-Dec 2008 277 234 1.18 7682 3.6

Jan-Mar 2009 259 241 1.07 8034 3.2

Apr-Jun 2009 223 229 0.97 7678 2.9

Jul-Sep 2009 234 233 1.00 7826 3.0

Oct-Dec 2009 262 239 1.09 7759 3.4

Source: ISD Scotland (SMR01) linked dataset

Table 1 - Quarterly Hospital Standardised Mortality Ratios in Monklands District General Hospital

Monklands District General Hospital

better information --> better decisions --> better health12

Quarterly Release

Figure 1b - Number of observed and predicted deaths; October 2006 - December 2009Scot

Source: ISD Scotland (SMR01) linked dataset

Quarterly Hospital Standardised Mortality Ratios in Monklands District General Hospital

0

50

100

150

200

250

300

350

400

450

500

Oct-Dec2006

Jan-Mar2007

Apr-Jun2007

Jul-Sep2007

Oct-Dec2007

Jan-Mar2008

Apr-Jun2008

Jul-Sep2008

Oct-Dec2008

Jan-Mar2009

Apr-Jun2009

Jul-Sep2009

Oct-Dec2009

Num

ber

of d

eath

s

Observed deaths (30 days)

Predicted deaths (30 days)

better information --> better decisions --> better health13

Quarterly Release

Figure 3 - Standardised Mortality Ratio with regression line; October 2006 - December 2009Scot

Source: ISD Scotland (SMR01) linked dataset

Quarterly Hospital Standardised Mortality Ratios in Monklands District General Hospital

0.0

0.5

1.0

1.5

2.0

Oct-Dec2006

Jan-Mar2007

Apr-Jun2007

Jul-Sep2007

Oct-Dec2007

Jan-Mar2008

Apr-Jun2008

Jul-Sep2008

Oct-Dec2008

Jan-Mar2009

Apr-Jun2009

Jul-Sep2009

Oct-Dec2009

Sta

ndar

dise

d M

orta

lity

Rat

io

Standardised Mortality Ratio(SMR)

Regression line

Baseline

better information --> better decisions --> better health14

Jul – Sep 10 Oct – Dec 10 Jan – Mar 11 Apr – Jun 11

4 Months

3 Months

Jan – Mar 09 Apr – Jun 09 Jul – Sep 09 Oct – Dec 09

9 Months

Addressing Timeliness

better information --> better decisions --> better health15

Timetable

Date Pre-release Access to Boards or into Public Domain

Reporting Final Data to: Provisional Results

Tues. 27th Jul 2010 Pre-release 31st Dec 2009 N/A

Tues. 31st Aug 2010 Public 31st Dec 2009 N/A

Tues. 26th Oct 2010 Pre-release 31st Mar 2010 Apr–Jun 2010

Tues. 30th Nov 2010 Public 31st Mar 2010 Apr–Jun 2010

Tues. 25th Jan 2011 Pre-release 30th Jun 2010 Jul-Sep 2010

Tues. 22nd Feb 2011 Public Release 30th Jun 2010 Jul-Sep 2010

Tues. 26th Apr 2011 Pre-release 30th Sep 2010 Oct-Dec 2010

Tues. 31st May 2011 Public Release 30th Sep 2010 Oct-Dec 2010

Tues. 26th Jul 2011 Pre-release 31st Dec 2010 Jan-March 2011

Tues. 30th Aug 2011 Public Release 31st Dec 2010 Jan-March 2011

Tues. 25th Oct 2011 Pre-release 31st Mar 2011 Apr–Jun 2011

Tues. 29th Nov 2011 Public Release 31st Mar 2011 Apr–Jun 2011

Tues. 31st Jan 2012 Pre-release 30th Jun 2011 Jul-Sep 2011

Tues. 28th Feb 2012 Public Release 30th Jun 2011 Jul-Sep 2011

Tues. 24th Apr 2012 Pre-release 30th Sep 2011 Oct-Dec 2011

Tues. 29th May 2012 Public Release 30th Sep 2011 Oct-Dec 2011

Tues. 31st Jul 2012 Pre-release 31st Dec 2011 Jan-March 2012

Tues. 28th Aug 2012 Public Release 31st Dec 2011 Jan-March 2012

better information --> better decisions --> better health16

better information --> better decisions --> better health17

What is a High HSMR?

• A high HSMR where variation greater than +3SDs above Scotland HSMR

– Also where HSMR >Scot HSMR but within control limits; Where HSMR is increasing or is steady over time

• Distribution of variation shown on a funnel plot

• In addition to published data, HSMR used as part of an internal process to highlight hospitals with exceptional variation (variation outwith norm)

• Data variability on a funnel plot is not published – risk of league tables

• Escalation protocol for potential quality/safety concerns flagged up by data – instigated by QIS (HIS) and ISD

better information --> better decisions --> better health18

HSMR Funnel Plot

Quarterly HSMR; October-December 2006

.00

.20

.40

.60

.80

1.00

1.20

1.40

1.60

1.80

2.00

0 50 100 150 200 250 300 350 400 450

Predicted deaths

Sta

nd

ard

ised

mo

rtal

ity

rati

o

ScotHSMRHospitals

HSMR

Control Limits Upper & Lower

CL+2SD CL+3SD

CL-2SD

CL-3SD

Unusual HSMR above CL(+3SD)

better information --> better decisions --> better health19

Information Support to Boards 1

• Provide data to support further investigation into high HSMR

• HSMR trend over time and control limits

better information --> better decisions --> better health20

Information Support to Boards 2HSMR within control limits over time

-0.5

0.5

1.5

2.5

Oct-Dec2006

Jan-Mar2007

Apr-Jun

2007

Jul-Sep2007

Oct-Dec2007

Jan-Mar2008

Apr-Jun

2008

Jul-Sep2008

Oct-Dec2008

Jan-Mar2009

Apr-Jun

2009

Jul-Sep2009

Oct-Dec2009

Jan-Mar2010

Apr-Jun

2010

Jul-Sep

2010p

Sta

ndar

dise

d M

orta

lity

Rat

io

Scot HSMR

HSMR withinControl limits

better information --> better decisions --> better health21

Information Support to Boards 2 HSMR reaching control limits

-0.5

0.5

1.5

2.5

Oct-Dec2006

Jan-Mar2007

Apr-Jun

2007

Jul-Sep2007

Oct-Dec2007

Jan-Mar2008

Apr-Jun

2008

Jul-Sep2008

Oct-Dec2008

Jan-Mar2009

Apr-Jun

2009

Jul-Sep2009

Oct-Dec2009

Jan-Mar2010

Apr-Jun

2010

Jul-Sep

2010p

Sta

ndar

dise

d M

orta

lity

Rat

io

Scot HSMR

Unusual HSMR above CL(+3SD)

better information --> better decisions --> better health22

Information Support to Boards 2

• Provide data to support further investigation into high HSMR

• HSMR trend over time and control limits

• HSMR stratified : Elective / non elective admissions, age group, specialty, diagnostic grouping

better information --> better decisions --> better health23

Information Support to Boards 2

•Elective / non elective, age group, specialty, diagnostic grouping

•Example HSMR elective / non elective, age group, specialty, diagnostic grouping

Specialty Quarter label Observed deaths Expected deaths Patients SMR Scot SMR

Surgical Oct - Dec 2006 43.00 28.94 2428 1.49 .96

Admission typeQuarter label Observed deaths Expected deaths Patients SMR Scot SMR

Elective Oct - Dec 2006 21.00 17.77 2210 1.18 .67

better information --> better decisions --> better health24

Information Support to Boards 2

Scot HSMR

HSMR

•Elective / non elective, age group, specialty, diagnostic grouping •Indicator of where to start to look •Indicate where Scot HSMR < lower 95% CI HSMR

Specialty Quarter label Observed deaths Expected deaths Patients SMR Scot SMR LCI UCI conditional (lci>scot_smr)

Surgical Oct - Dec 2006 43.00 28.94 2428 1.49 .96 1.04 1.93 1

Admission typeQuarter label Observed deaths Expected deaths Patients SMR Scot SMR LCI UCI conditional (lci>scot_smr)

Elective Oct - Dec 2006 21.00 17.77 2210 1.18 .67 0.68 1.69 1

better information --> better decisions --> better health25

Information Support to Boards 3 Case Listing

DOA CHI CASEREF HOSP SPEC SURG_MED DIAG1 DIAG_GRP SIMD AGE SEX admgrpdesc n_emerg death30 Prob20090609 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 18 2 Non_elective 0 1 0.0220090417 xxxxxxxxxx xxxxxxxxxx H1 AB 1 R268 Neuro2 2 84 2 Elective 1 1 0.0620090428 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E109 Metabolic 2 17 1 Non_elective 0 1 0.1120090417 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E162 Metabolic 3 19 1 Non_elective 0 1 0.1320090406 xxxxxxxxxx xxxxxxxxxx H1 A1 M R410 Neuro2 1 46 1 Non_elective 0 1 0.1420090411 xxxxxxxxxx xxxxxxxxxx H1 AB M R54X Miscel2 2 83 1 Non_elective 1 1 0.1520090606 xxxxxxxxxx xxxxxxxxxx H1 A1 1 R900 Neuro2 5 59 1 Non_elective 2 0 0.0120090421 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 4 25 1 Non_elective 0 0 0.0120090619 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 28 1 Elective 1 0 0.0220090421 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 1 16 1 Elective 1 0 0.0220090416 xxxxxxxxxx xxxxxxxxxx H1 AB 1 G819 Neuro2 5 79 2 Non_elective 0 0 0.0320090622 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 29 1 Non_elective 0 0 0.0320090416 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 1 17 1 Non_elective 2 0 0.0320090418 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E162 Metabolic 1 25 1 Non_elective 0 0 0.0620090520 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E162 Metabolic 1 29 2 Non_elective 0 0 0.0720090628 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 19 1 Elective 8 0 0.0720090615 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 17 2 Elective 1 0 0.0920090401 xxxxxxxxxx xxxxxxxxxx H1 AB 1 G819 Neuro2 5 83 1 Non_elective 2 0 0.1320090410 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E109 Metabolic 2 26 1 Elective 0 0 0.16

Deaths 30-Day Survivors

better information --> better decisions --> better health26

Information Support to Boards 3 Case Listing

DOA CHI CASEREF HOSP SPEC SURG_MED DIAG1 DIAG_GRP SIMD AGE SEX admgrpdesc n_emerg death30 Prob20090609 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 18 2 Non_elective 0 1 0.0220090417 xxxxxxxxxx xxxxxxxxxx H1 AB 1 R268 Neuro2 2 84 2 Elective 1 1 0.0620090428 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E109 Metabolic 2 17 1 Non_elective 0 1 0.1120090417 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E162 Metabolic 3 19 1 Non_elective 0 1 0.1320090406 xxxxxxxxxx xxxxxxxxxx H1 A1 M R410 Neuro2 1 46 1 Non_elective 0 1 0.1420090411 xxxxxxxxxx xxxxxxxxxx H1 AB M R54X Miscel2 2 83 1 Non_elective 1 1 0.1520090606 xxxxxxxxxx xxxxxxxxxx H1 A1 1 R900 Neuro2 5 59 1 Non_elective 2 0 0.0120090421 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 4 25 1 Non_elective 0 0 0.0120090619 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 28 1 Elective 1 0 0.0220090421 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 1 16 1 Elective 1 0 0.0220090416 xxxxxxxxxx xxxxxxxxxx H1 AB 1 G819 Neuro2 5 79 2 Non_elective 0 0 0.0320090622 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 29 1 Non_elective 0 0 0.0320090416 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 1 17 1 Non_elective 2 0 0.0320090418 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E162 Metabolic 1 25 1 Non_elective 0 0 0.0620090520 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E162 Metabolic 1 29 2 Non_elective 0 0 0.0720090628 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 19 1 Elective 8 0 0.0720090615 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E101 Metabolic 2 17 2 Elective 1 0 0.0920090401 xxxxxxxxxx xxxxxxxxxx H1 AB 1 G819 Neuro2 5 83 1 Non_elective 2 0 0.1320090410 xxxxxxxxxx xxxxxxxxxx H1 A1 1 E109 Metabolic 2 26 1 Elective 0 0 0.16

Unlikeliest Death According to Model

Unlikeliest Survivor According to Model

better information --> better decisions --> better health27

Information Support to Boards 3

Value of Case Listing

• Review of the patient journey

• Review of clinical records

• ICD 10 coding vs SMR01 linked file

• HSMR catalyst for improvement(s) in SMR01 coding accuracy

better information --> better decisions --> better health28

• Three funnel plots

Effect of Coding

better information --> better decisions --> better health29

Regression – Scotland, Crosshouse

The 15% ‘Target’

better information --> better decisions --> better health30

• Importance of Robust Clinical Coding– Individual probabilities of death based on ICD-10 coding

• Supporting Users– Developing data pack (case listings, aggregations etc)– Working (proactively with stakeholders locally to avoid surprises– Main contacts (Patient Safety Leads, Clinical Governance) – Refer to existing resources (Navigator, Surgical / Medical

Profile, ACaDMe)

• Data Timeliness– SMR01 and GRO data submissions– 30-day Follow-up– Reduced lag

Issues

better information --> better decisions --> better health

If you would like to contact us:

Richard DobbieRichard.Dobbie@nhs.net

Gavin MacCollGavin.Maccoll@nhs.net

Website address:www.isdscotland.org

Thank you very much for listening

better information --> better decisions --> better health32

Any Questions?