a systematic review of the clinical & economic literature...

0 | P a g e

A systematic review of the clinical & economic literature and a budget impact analysis of any new guideline recommendations to inform the planned update of National Clinical Guideline No. 1 - National Early Warning Score (NEWS) for the Irish health system.

This review has been prepared in response to a

request from the Department of Health to

conduct a review of literature to inform the

planned update of National Clinical Guideline No.

1 - National Early Warning Score (NEWS) for the

Irish health system.

Josephine Hegarty, Frances J. Drummond, Aileen

Murphy, Tom Andrews, Nuala Walshe, Bridie

McCarthy, Mohamad Saab, Mary Forde, Dorothy

Breen, Pat Henn, Jodi Cronin, Rosemary Whelan,

Jonathan Drennan, Eileen Savage.

5/23/2016

1 | P a g e

Title: A systematic review of the clinical & economic literature and a budget

impact analysis of any new guideline recommendations to inform the

planned update of National Clinical Guideline No. 1 - National Early Warning

Score (NEWS) for the Irish health system.

Team: Professor Josephine Hegarty(PI)1, Dr Frances J Drummond1, Dr Aileen

Murphy (Co-PI)4 Dr Tom Andrews1, Nuala Walshe1, Dr Bridie McCarthy1,

Mohamad Saab1, Mary Forde5, Dr Dorothy Breen2, Dr Pat Henn3, Jodi Cronin6,

Rosemary Whelan4, Professor Eileen Savage(Co-PI)1

External Collaborator and Advisor: Professor Jonathan Drennan7

1 School of Nursing and Midwifery, University College Cork 2 Cork University Hospital and ASSERT (Application of Science to Simulation-

based Education and Research on Training), University College Cork 3 School of Medicine, University College Cork 4 School of Economics, University College Cork 5 Bon Secours Hospital, Cork 6 Centre for Policy Studies, University College Cork 7 Professor of Healthcare Research within Health Sciences at the University of

Southampton

Citation details

This report should be referenced as follows:

Hegarty, J. , Drummond, F.J. , Murphy, A., Andrews, T., Walshe, N., McCarthy, B., Saab, M., Forde, M.,

Breen, D., Henn, P., Cronin, J. , Whelan, R., Drennan, J. , Savage, E., (2016). A systematic review of the

clinical & economic literature and a budget impact analysis of any new guidel ine recommendations to

inform the planned update of National Clinical Guideline No. 1 - National Early Warning Score (NEWS)

for the Irish health system. National Clinical effectiveness Committee, Department of Health: Dublin .

Accessible at http://health.gov.ie/patient-safety/ncec/national-cl inical-guidel ines-2/

http://health.gov.ie/patient-safety/ncec/national-clinical-guidelines-2/

2 | P a g e

National Clinical Effectiveness Committee (NCEC)

Clinical effectiveness is a key component of patient safety and quality. The integration of

best evidence in service provision, through clinical effectiveness processes, promotes

healthcare that is up to date, effective and consistent.

The National Clinical Effectiveness Committee (NCEC) is a Ministerial committee established

in 2010 as part of the Patient Safety First Initiative. The NCEC is supported by the Clinical

Effectiveness Unit (CEU), Department of Health. The NCEC is a partnership between key

stakeholders in patient safety and its mission is to provide a framework for national

endorsement of clinical guidelines and audit to optimise patient and service user care.

In December 2013, the first National Clinical Guideline (NCG) was published. This was NCEC

National Clinical Guideline No. 1 National Early Warning Score (NEWS). It relates to the

situation in an acute hospital setting where an adult patient’s physiological condition is

deteriorating. It was updated in August 2014 to ensure alignment with NCG No. 6 Sepsis

Management.

Invitations to tender were issued in July 2015 and a public procurement competition held

for a systematic literature review and budget impact analysis to support the update of NCG

No. 1 NEWS. Subsequently, this report was commissioned by the CEU/NCEC Department of

Health.

The NEWS is part of a suite of National Clinical Guidelines on Clinical Deterioration. The suite

currently consists of:

NCG No Title Date

NCG No.1 National Early Warning Score (NEWS) Feb 2013

with clinical update Aug 2014

NCG No.4 Maternity Early Warning Score

(IMEWS)

Nov 2014

NCG No.6 Sepsis Management Nov 2014

with NICE accreditation Mar 2015

NCG No.12 Paediatric Early Warning Score system

(PEWS)

Nov 2015

Emergency Department Monitoring

and Clinical Escalation tool for adults

(ED MACE)

Prioritised by the NCEC in Sept 2015

with a systematic literature review

currently in progress to support the

development of the guideline

Further information on the NCEC and National Clinical Guidelines is available at

www.health.gov.ie/patient-safety/ncec

http://www.health.gov.ie/patient-safety/ncec

3 | P a g e

Table of Contents

National Clinical Effectiveness Committee (NCEC) ................................................................................. 2

List of Abbreviations ............................................................................................................................... 6

Executive Summary ................................................................................................................................. 9

Chapter 1. Background and Methods relating to the sourcing of Clinical Literature Pertaining to the

Evaluation of Early Warning Scoring Systems ....................................................................................... 14

Introduction ...................................................................................................................................... 14

Summary ........................................................................................................................................... 16

Methods for Review of Clinical Literature Pertaining to the Evaluation of Early Warning Scoring

Systems ................................................................................................................................................. 17

Introduction ...................................................................................................................................... 17

Aim .................................................................................................................................................... 17

Objectives ......................................................................................................................................... 17

Search Processes ............................................................................................................................... 18

Search of Databases and Grey Literature ......................................................................................... 18

Inclusion/Exclusion Criteria............................................................................................................... 19

Review process ................................................................................................................................. 20

Audit Trail .......................................................................................................................................... 25

Data Extraction ................................................................................................................................. 26

Quality Assessment ........................................................................................................................... 26

External Validity/Transferability ....................................................................................................... 27

Data Synthesis ................................................................................................................................... 28

Summary ........................................................................................................................................... 28

Chapter 2. Results of the Review of Clinical Literature Pertaining to the Evaluation of Early Warning

Scoring Systems .................................................................................................................................... 29

Overview ........................................................................................................................................... 29

Characteristics of Clinical Papers ...................................................................................................... 29

National Early Warning Score ........................................................................................................... 32

Modified Early Warning Scoring System ........................................................................................... 35

Modification of Established and Newly Developed EWS (without NEWS) ....................................... 37

Direct Comparison of Established and Newly Developed EWS ........................................................ 40

EWS Chart Design .............................................................................................................................. 43

NEWSs and/or RRT Educational Interventions ................................................................................. 44

Implementation of EWS and/or RRT ................................................................................................. 46

Mortality as an Outcome Measure ................................................................................................... 51

4 | P a g e

Length of Stay (LOS) .......................................................................................................................... 62

Cardiac Arrest .................................................................................................................................... 64

Transfer to or admission to ICU ........................................................................................................ 71

Resource utilisation/documentation/clinical response .................................................................... 77

Clinical Validation .............................................................................................................................. 83

Barriers and facilitators to EWS system implementation ................................................................. 94

An Overview of NEWS: the Irish Context ........................................................................................ 103

Review of Clinical Guidance Published Internationally ................................................................... 105

Conclusion and Recommendations (from the review of clinical literature) ....................................... 109

Monitoring vital signs...................................................................................................................... 109

Escalation and response to abnormal NEWS .................................................................................. 111

Electronic systems........................................................................................................................... 112

The System ...................................................................................................................................... 113

Education ........................................................................................................................................ 114

Chart design .................................................................................................................................... 115

Research .......................................................................................................................................... 115

Perspectives on the Process ........................................................................................................... 115

Chapter 3. Economic Review Methods ............................................................................................... 117

Review Methods for the Economic Literature ................................................................................ 117

Selection Criteria for the Economic Studies ................................................................................... 117

Search Strategy ............................................................................................................................... 118

Search Results ................................................................................................................................. 119

Review Process................................................................................................................................ 119

Data Extraction ............................................................................................................................... 121

Study Results ................................................................................................................................... 121

Quality Appraisal ......................................................................................................................... 121

Transferability ............................................................................................................................. 121

Data Synthesis ............................................................................................................................. 122

Chapter 4. Economics Review Findings............................................................................................... 123

Introduction .................................................................................................................................... 123

Characteristics of Economics Papers .............................................................................................. 123

Systematic Search .......................................................................................................................... 125

Quality of Included Studies ......................................................................................................... 125

Transferability ............................................................................................................................. 125

5 | P a g e

Characteristics of Included Studies ................................................................................................. 130

Discussion ....................................................................................................................................... 133

References .......................................................................................................................................... 135

Appendix 1: Search Terms - Economic Search ............................................................................ 154

Appendix 2: Tool to Assist Reviewers in Application of Inclusion and Exclusion Criteria ........... 155

Appendix 3: Data Extraction Tool ............................................................................................... 157

Appendix 4a: Tables of Clinical Findings ..................................................................................... 158

Appendix 4b: Components of the Individual Early Warning Scoring Systems ............................ 299

Appendix 5a: BMJ Quality Checklist ............................................................................................ 314

Appendix 5b: EunetHTA Toolkit Economic Evaluations- Transferability .................................... 315

Appendix 6: Extraction Table – Economics Review .................................................................... 316

6 | P a g e

List of Abbreviations

ABCDE Airway, Breathing, Circulation, Disability, Expose/Examine

ADDS Adult Deterioration Detection System

AHF Acute Heart Failure

ALERT The Acute Life Threatening Events Recognition & Treatment

APACHE II Acute Physiology and Chronic Health Evaluation Score

AUC Area Under the Curve

AUROC Area Under the Receiver-Operated Curve

AVPU Alert, Voice, Pain, Unresponsive

AWSS Aggregate Weighted Scoring Systems

BEWS Bispebjerg Early Warning Score

BIA Budget Impact Analysis

BMJ British Medical Journal

BSS Braden Skin Score

CART Cardiac Arrest Risk Triage

CCI Charlson Comorbidity Index

CCOS Critical Care Outreach Services

CCOT Clinical Care Outreach Teams

CG Control Group

CHEC Consensus on Health Economic Criteria

CI Confidence Interval

CINAHL Cumulative Index to Nursing and Allied health Literature

COPD Chronic Obstructive Pulmonary Disease

CPR Cardiopulmonary Resuscitation

CREWS Chronic Respiratory Early Warning Score

CUH Cork University Hospital

DBP Diastolic Blood Pressure

DIST Distance-Based Scoring System

DNR Do Not Resuscitate

DOH Department of Health

DTEWS Decision-Tree Early Warning Score

DULK Dutch Leakage Score

ED Emergency Department

EHR Electronic Health Record

ELPQuiC Emergency Laparotomy Pathway Quality Improvement Care

EMR Electronic Medical Records

EPSS Electronic Physiological Surveillance System

E-RAPIDS Rescuing a Patient in Deteriorating Situations

ESI Emergency Severity Index

EWRS Early Warning and Response System

EWS Early Warning Score

FTR Failure to Rescue

FUP Follow-up

GCS Glasgow Coma Scale

GRADE Grading of Recommendations Assessment, Development and Evaluation

7 | P a g e

H+ Hydrogen

HCO3- Bicarbonate

HCP Healthcare Provider

HDU High Dependency Unit

HIQA Health Information and Quality Authority

HOTEL Hypotension, O2 Saturation, Low temperature, ECG change and Loss of independence

HR Heart Rate

HRB Health Research Board

HRV Heart Rate Variability

HTA Health Technology Assessment

ICER Incremental Cost-Effectiveness Ratio

ICU Intensive Care Unit

IG Intervention Group

IMEWS The Irish Maternity Early Warning System

Implementation Impl.

IOM Institute of Medicine

IQR Interquartile Range

ISBAR Identification, Situation, Background, Assessment, Recommendation

LDT Laboratory Decision Tree

LOE Level of Evidence

LOS Length of Stay

MA Meta-Analysis

MACE Major Adverse Cardiac Events

MDT Multidisciplinary Team

MEDS Mortality in the Emergency Department Sepsis

MEES Mainz Emergency Evaluation Score

MERIT Medical Early Response Intervention and Therapy

MET Medical Emergency Team

MEWS Modified Early Warning Score

MFS Multi-professional Full-scale Simulation

ML Machine Learning

MTS Manchester Triage System

MAELOR Multidisciplinary Audit and Evaluation of Outcomes of Rapid Response

NCEC National Clinical Effectiveness Committee

NEWS National Early Warning Score

NICE National Institute for Health and Care Excellence

NLH National Library of Health

NNE Number Needed to Evaluate

Nwulu PICO Prescribing, Information and Communication System (Nwulu et al. 2012)

O2 Oxygen

OR Odds Ratio

PaCO2 Partial Pressure of Carbon Dioxide

PAR Patient Acuity Rating

PARS Patient at Risk Score

PDA Personal Digital Assistants

8 | P a g e

Note: the acronyms are reported as per their utilisation in the studies reviewed.

PDSA Plan, Do, Study, Act

PEDS Prince of Wales Emergency Department Score

PICOCS Population, Intervention, Comparison, Outcomes, Context, Study Design

PICOS Patient/problem, Intervention, Comparison, Outcome, Setting

PIRO Predisposition, Infection, Response, Organ Dysfunction Score

PPV Positive Predictive Value

PRISMA Preferred Reporting Items for Systematic Reviews and Meta-Analyses

PTTS Physiological Track and Trigger System

RCT Randomised Controlled Trial

RED Resuscitation Events and Death

REMS Rapid Emergency Medicine Score

ROC Receiver Operating Characteristic

RR Relative Risk (as per context)

RR Respiratory Rate (as per context)

RRR Relative Risk Ratio

RRS Rapid Response System

RRT Rapid Response Team

RS Research Study

SAE Serious Adverse Events

SAP Simplified Acute Physiology Score

SBAR Situation, Background, Assessment, Recommendation

SBP Systolic Blood Pressure

SCS Simple Clinical Score

SD Standard deviation

SEWS Standardised Early Warning Score

SIGN Scottish Intercollegiate Guidelines Network

SMDP Semi-Markov Decision Process

SOFA Sepsis-related Organ Failure Assessment

SpO2 Peripheral oxygen saturation

SSSS Severe Sepsis and Septic shock

SUPER SpO2, Urinary volume, Pulse, Emotional state, Respiratory Rate

THERM The Resuscitation Management score

TIMI Thrombolysis in Myocardial Infarction Score

TPB Theory of Planned Behaviour

UCC University College Cork

UCD University College Dublin

ViEWS VitalPAC Early Warning Score

VSS Vital Sign Score

WOC Ward Observational Charts

Worthing PSS Worthing Physiological Scoring System

WSN Wireless Sensors Network

9 | P a g e

Executive Summary

Background

Patient safety and quality assurance is a priority across all components of the Irish

healthcare system. Acute physiological deterioration is a time-critical medical emergency

that affects millions of people worldwide. Early Warning Systems (EWS) (also termed ‘Track-

and-trigger systems’) have been established in acute care clinical settings to facilitate a

timelier response to, and assessment of, acutely ill patients by:

1. Classifying the severity of a patient’s illness;

2. Providing prompts and structured communications tools to escalate care;

3. Following a definitive escalation plan.

Standardization of early warning scores optimizes organizational delivery of safe, equitable

and quality care for all acutely ill patients. Such standardization has been achieved through

national consensus and the publication of national guidelines.

Aim

As specified by the Department of Health National Clinical Effectiveness Unit the aim of this

project was to complete an update of a systematic review of the clinical and economic

literature on early warning scores/systems or trigger systems used in adult (non-pregnant)

patients in acute healthcare settings for the detection of/timely identification of physical,

clinical deterioration.

Methods

A full search strategy was developed by the research team to include key terms and their

variations. Key terms included a combination of terms associated with “early warning

scoring systems”; such terms were associated with the PICOS (Patient/Problem,

Intervention, Comparison, Outcome, Setting) guidance framework. The published literature

was identified by searching the pertinent electronic databases and selected grey literature.

These included: Academic Search Complete, CINAHL (the Cumulative Index to Nursing and

Allied Health Literature), Medline, PsycINFO, PsycARTICLES, Psychology and Behavioral

Science Collection, SocINDEX, and UK/Eire Reference Centre), Cochrane library, Guideline

Websites, Business Source Complete, EconLit and NHS Economic Evaluation Database (NHS

EED).

All evidence sourced was graded according to Scottish Intercollegiate Guidelines Network

(SIGN) level of evidence (LOE) criteria and associated methodological checklists which

looked at internal validity and overall assessment of the study for each study type.

10 | P a g e

Findings

One hundred and twenty four (n=124) empirical and 5 economic papers were included in

the clinical and economic reviews respectively. The majority (92.7%) of the clinical evidence

was categorised as ≤2+ LOE categorisation using SIGN criteria.

Within the systematic review, a variety of EWSs were sourced. Even within each system

there were differences in the vital sign scoring and the weighting assigned to scores when

they were used in different studies, hospitals or with varying subpopulations. This made

comparisons difficult. However, this review summarises evidence that demonstrates that

NEWS has been shown to be an effective assessment tool to identify and triage the patient

for the most appropriate acute care assessments and interventions. The review of empirical

literature also identified that timely escalation remains an ongoing problem. Thus

understanding the organisational culture, systems, practices, barriers and facilitators and

the stakeholders’ perceptions and interactions with the NEWS is important.

An Early Warning System is a multi-component complex system as illustrated within Figure

1. Most studies reviewed failed to take into account that the overall performance of the

system depends on the performance of its individual parts, and the individuals interacting

with it which makes the interpretation of results difficult. High levels of adherence and

consistent adherence are necessary for the system to be effective. Many similarities exist

across guidance internationally pertaining to the early recognition of the deteriorating

patient. What is clear is the movement towards using a whole systems approach whilst

placing governance at the centre in the maintenance of organisation-wide recognition and

response systems. Such systems require the utilisation of data from the evaluation of EWS

to inform quality improvement activities.

Empowering health care professionals to act on their clinical judgment is also a critical

component of any system introduced. Across international guidelines the concept of

patients in high EWS trigger groups requiring immediate review by individual(s) with critical

care competencies and diagnostic skills was very evident.

The importance of multi-format, inter-professional training, regular reinforcement, case

reviews and an interactive in-person training process was reiterated in a number of studies

that explored educational interventions.

Given the absence of higher level evidence (e.g. trials, high quality case control or cohort or

studies) in the research literature sourced, it is important that research be conducted in

tandem with the implementation of EWS systems with particular emphasis on the

application of EWS to subgroups (e.g. older adults) and particular contexts of care (e.g. ED).

11 | P a g e

Figure 1 Diagrammatic representation of the general domains represented within

international guidance relating to Early Warning Systems.

The economic evidence from the literature on EWS used in adult patients in acute

healthcare settings for the timely detection of physical or clinical deterioration is limited.

Three of the five papers included in the economic review were Irish studies previously

commissioned by the Department of Health. This review re-affirms that there are no

published full economic evaluations of EWS used in adult patients in acute healthcare

settings for the timely detection of physical or clinical deterioration.

Considerations for Guideline Development Group emerging from the Systematic Review

Taking the evidence in its totality twelve key considerations have been highlighted to inform

the NCEC and the work of the NEWS update Guideline Development Group.

Consideration for Guideline Development Group number 1

Based on the findings of this review the team would consider it appropriate that the

National Early Warning (NEW)1 System continue to be used for adult non-pregnant patients

within the acute hospital system.

1 DOH 2014 National Clinical Guideline No. 1. National Early Warning Score. Published February 2013. Update

August 2014.

Governance

Early recognition of the deteriorating patient,

communication tool to escalate care (track and

trigger system)

Clinical emergency response systems

(graded response including access to

critical care competencies and diagnostic skills)

Education

(multi-disciplinary, continuous)

Evaluation

(feedback loops ensuring a

learning system)

12 | P a g e

Continue to use and develop a national approach to ensure the early identification and

effective escalation of the deteriorating patient.


Consider if patients in the following categories may benefit from different trigger scoring

thresholds and/or supplementary track and trigger/risk stratifying systems:

• Older adult patients; in particular frail older adults

• Patients with COPD or other conditions characterised by chronic hypoxemia

• Patients in the Emergency Department.


Consider the mechanisms that could be used to ensure that individuals with higher NEW

scores are reviewed promptly by health care professionals with critical care competencies

and diagnostic skills.


Reinforce the importance of monitoring respiratory rate as part of educational programmes

for health care professionals who monitor and interpret vital signs.


Consider carefully the potential use of electronic data capture, EWS triggering, notification

and tracking of outcomes in an Irish context, whilst being reflective of how such systems

would advance the science of early detection of the deteriorating patient within a complex

health care system.


A systems approach underpinned by appropriate governance is required. Such a systems

approach requires:

Consider NEWS as a system-level complex intervention.

Change the title of the Irish guideline to reflect a National Early Warning “System”

as opposed to National Early Warning “Score”.

Emphasize the importance of regular reinforcement, and auditing with a systems-

learning focus.

Promote high levels of adherence to NEWS to ensure effectiveness.

Introduce mechanisms to decrease errors and increase adherence. These could

include: clinical champions to promote NEWS, ward based education; multi-modal

education; the hiring of ‘observationalists’; publishing monthly audit reports; visual

imagery using posters; instilling confidence in practitioners in their ability to escalate

care for NEW score trigger points.

13 | P a g e

Use quality improvement methodologies and an understanding of human factors to

quantify the avoidable (associated with health care delivery system dysfunction) and

unavoidable delays (e.g. associated with diagnosis or treatment factors), as well as

their impact on care. Use pre-post-intervention analysis with longitudinal

measurement of predefined organisational, health care professional and patient

level outcomes with feedback sessions when implementing NEWS/changes to NEWS.


Ensure the education of all health care providers using NEWS; such education should

include: interdisciplinary in person simulations/case-studies; be multimodal, and include

regular reinforcement. This recommendation should also be read in conjunction with

recommendation 5 above.


Chart designs should avoid visual clutter and the use of overlapping graphical displays of

data.

Requirement for more research number 9

There may be a requirement for more research to be conducted in tandem with the

implementation of EWS systems with particular emphasis on the application of EWS to

subgroups (e.g. older adults) and particular contexts of care (e.g. ED). National consensus on

the tracking of key outcomes pertinent to NEWS will assist with this.

Consideration for NCEC and Guideline Development Group number 10

Prior to conducting a systematic review, for an update of a guideline, it is recommended

that the review team confer with the guideline development group to achieve clarity around

the key review question(s) in terms of emerging clinical practice/ utilisation of the guideline

in practice that need to be answered in the review update.


Ensure that guideline recommendations are very clear, with single messages to facilitate the

application of evidence to separate discrete areas/questions.


There may be a requirement for more research to be conducted in tandem investigating

the cost effectiveness of NEWS. Investigators could give consideration to the

appropriateness of using standard methods to investigate the cost effectiveness of NEWS.

Budget Impact Analysis could be undertaken with reference to the relevant NCEC and HIQA

guidance.

14 | P a g e

Chapter 1. Background and Methods relating to the sourcing of Clinical

Literature Pertaining to the Evaluation of Early Warning Scoring Systems

Introduction

Patient safety and quality assurance is a priority across all components of the Irish

healthcare system. The Report of the Commission on Patient Safety and Quality Assurance

‘Building a Culture of Patient Safety’ (Department of Health and Children 2008) emphasises

the need for good clinical effectiveness. A core element of this involves the standardisation

of definitions and terminology, developing clinical standards, guidelines and indicators that

enable healthcare professionals to monitor their performance at an individual, team and

organisation level against nationally, and internationally recognised comparative

parameters.

In 2010, the Minister for Health established The National Clinical Effectiveness Committee

(NCEC) with representation from a range of stakeholders. These stakeholders included

patients, clinicians, patient safety experts, the Department of Health (DOH), regulatory

bodies, public and private health service providers, and training and education bodies. One

of the principal roles of the NCEC is to provide guidance for improving the quality, safety

and cost effectiveness of healthcare in Ireland. This includes the recommendation of a suite

of National Clinical Guidelines to the Minister for Health for implementation in the Irish

healthcare system (http://health.gov.ie/patient-safety/ncec/national-clinical-guidelines-2/).

Similar government initiatives to support the development of clinical practice guidance have

been established in other countries. These include The National Institute for Health and

Care Excellence (NICE) in the UK (www.nice.org.uk); the Scottish Intercollegiate Guidelines

Network in the NHS in Scotland (www.sign.ac.uk); the Agency for Healthcare Research &

Quality in USA (http://www.ahrq.gov/professionals/index.html) and the Health Care

Committee of the National Health and Medical Research Council in Australia

(https://www.nhmrc.gov.au/guidelines/).

When patients and their families come in contact with the Irish healthcare system, they

expect care and treatments that are evidence based, effective, safe, and of high quality. As

critical care has evolved worldwide, attempts to identify early the clinically deteriorating

patient has led to the introduction of Early Warning Scores/Systems (EWS) in hospitals. In

Ireland, the National Early Warning Score (NEWS) as a work-stream of the National Clinical

Programme for Acute Medicine was agreed nationally for use in acute hospitals for adult

non-pregnant patients. The NEWS was the first National Clinical Guideline endorsed by the

NCEC and the Minister. It was published in February 2013. A subsequent update to the

http://www.nice.org.uk/

http://www.ahrq.gov/professionals/index.html%20)

https://www.nhmrc.gov.au/guidelines/

15 | P a g e

guideline to include additional practical guidance was approved by the NCEC in August

20142 .

Acute physiological deterioration is a time-critical medical emergency that affects millions of

people worldwide.

Early Warning Systems (also termed ‘Track-and-trigger systems’) have been established in

acute care clinical settings to facilitate a timelier response to, and assessment of, acutely ill

patients by:

1. Classifying the severity of a patient’s illness;

2. Providing prompts and structured communications tools to escalate care;

3. Following a definitive escalation plan.

Standardization of early warning scores optimizes organizational delivery of safe, equitable

and quality care for all acutely ill patients. Such standardization has been achieved through

national consensus and the publication of national guidelines; for example in the UK, NEWS

was introduced in 20123,4. The NEWS is based on an aggregate scoring system in which a

score is allocated to physiological measurements. Six simple physiological parameters form

the basis of the scoring system:

i) Respiratory rate

ii) Oxygen saturations

iii) Temperature

iv) Systolic blood pressure

v) Pulse rate

vi) Level of consciousness

A score is allocated to each as they are measured, the magnitude of the score reflecting how

extreme the parameter varies from the norm. The individual scores are then combined.

Depending on the NEW score the escalation of care is linked to recommendations on the

frequency of observations and the urgency of clinical review.

The use of NEWS in patients (n=2003) with sepsis in the Emergency Department (ED) (n=20

Scottish EDs) was evaluated (Corfield et al. 2014) revealing an association between

increased NEW scores on arrival and adverse outcomes (intensive care unit referral and

mortality). Similarly, in the Chinese context, NEW Scores of 7 or more were associated with

increased risk of death (OR=16.8; 95% CI 6.6-42.9) (Liu et al. 2015). Alam et al. (2015)

2 DOH 2014 National Clinical Guideline No. 1. National Early Warning Score. Published February 2013. Update

August 2014. 3 ROYAL COLLEGE OF PHYSICIANS. 2015. National Early Warning Score (NEWS). Available from

https://www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-new 4 SCOTTISH INTERCOLLEGIATE GUIDELINES NETWORK (SIGN). 2014. Care of deteriorating patients. Available

from http://www.sign.ac.uk

16 | P a g e

utilising a prospective observational design (n=274 patients) found that NEWS measured at

different time points was a good predictor of patient outcomes.

Earlier recognition of physiological deterioration has been shown to reduce mortality.

Analysis of data from a large UK hospital (n= 1,180,268 hospital admissions, of which 7,264

[0.6%] died) revealed that there was a significant fall in risk of death over the 7-year period

compared with baseline values (post implementation of an electronic track and trigger

system (Patientrack) (Bannard-Smith et al. 2015). A systematic review (n=7 papers) revealed

a positive trend towards better clinical outcomes following the introduction of the EWS

chart, sometimes coupled with an outreach service however the review authors noted that

the use of adapted forms of the EWS, different thresholds, poor or inadequate study

methodology made it difficult to draw definitive comparisons (Alam et al. 2014)

Some debate exists as to the appropriateness of various trigger points, for example, Abbott

et al. (2015) (based upon analysis of data from 453 patients) found that a NEWS of 3 or

more was associated with the primary outcome of death (odds ratio[OR]=7.03, p=0.003)

thus the authors argued that a lower threshold for triggering a clinical review in the UK

context should be considered (Abbott et al. 2015). A prospective observational study of 370

adult patients revealed that 18.9% (n=70 patients) of the NEW scores were calculated

incorrectly which in turn has implications for trigger actions and associated clinical care

(Kolic et al. 2015).

Research in the Irish context found that NEWS enhanced the nurse’s ability to identify

deteriorating patients. However, some issues have been noted in terms of implementation

of the NEWS which include: delayed response times, lack of training in the use of the tool,

and a failure by doctors to modify trigger parameters for patients with chronic conditions

(Fox & Elliott 2015). Therefore, successful implementation of a National Early Warning

System with escalation protocols must go hand in hand with appropriate education of staff

and increased awareness of the necessity of structural patient monitoring and systematic

evidence based actions. Systematic reviews on early warning scores/systems or trigger

systems can be useful in order to facilitate comparisons and inform guideline updates.

Summary

The key message from international literature is the potential for track and trigger systems

to support improvements in clinical monitoring, escalation of care and thus patient safety

and clinical outcomes for acutely ill patients within acute hospitals. In the Irish context, this

is achieved through the publication of clinical guidelines which provide guidance on the

standardisation of the assessment and scoring of physiological parameters with associated

escalation protocols and adopting such an approach across a health system. Systematic

reviews on early warning scores/systems or trigger systems can be useful in order to

facilitate comparisons and inform guideline updates.

17 | P a g e

Methods for Review of Clinical Literature Pertaining to the Evaluation of

Early Warning Scoring Systems

Introduction

Healthcare decision makers in their search for reliable information increasingly turn to

systematic reviews for a summary of the evidence. Such rigorous reviews help to identify,

select, assess, and synthesise the evidence, and can help clarify what is known and not

known about the potential benefits and harms of health care interventions inclusive of

various guidance types.

This desk-based secondary research was undertaken using a systematic review methodology

guided by the principles of conducting systematic reviews namely: The Cochrane Handbook

for Systematic Reviews5, the Institute of Medicine (IOM 2011) standards for systematic

reviews6 (in particular standard 8 on guideline updates), Health Information and Quality

Authority (HIQA) guidance on budget impact analysis7, the NCEC Developers Manual8, and

other related processes outlined by the NCEC in relation to guideline development9.

Aim

As specified by the DOH Clinical Effectiveness Unit the aim of this project was to complete

an update of a systematic review of the clinical and economic literature on early warning

scores/systems or trigger systems used in adult (non-pregnant) patients in acute healthcare

settings for the detection of/timely identification of physical, clinical deterioration.

Objectives

The specific objectives of the research are:

To source and describe any existing international empirical economic and clinical

literature pertinent to the implementation of warning score/systems or trigger

systems (including escalation protocols, communication tools and response

approaches) for the detection/timely identification of physiological deterioration in

adult (non-pregnant) patients in acute healthcare settings.

5 HIGGINS, J. P. T. & GREEN, S. 2011. Cochrane Handbook for Systematic Reviews of Interventions. Available

from http://handbook.cochrane.org/ 6 IOM. 2011. Standards for Systematic Reviews. Available from http://iom.edu/Reports/2011/Finding-What-

Works-in-Health-Care-Standards-for-Systematic-Reviews/Standards.aspx 7 HIQA. 2015a. Guidance on Budget Impact Analysis of Health Technologies in Ireland. Accessible from

https://www.hiqa.ie/system/files/Guidance_on_Budget_Impact_Analysis_of_Health_Technologies_in_Ireland.pdf 8 DOH (2013) Guideline Developer’s Manual. Available from

http://health.gov.ie/wp-content/uploads/2015/01/ncec_guideline_development_manual_january13.pdf HIQA & NCEC. 2015. National Quality Assurance Criteria for Clinical Guidelines - Version 2. Available from http://health.gov.ie/wp-content/uploads/2015/07/HIQA-NCEC-National-Quality-Assurance-Criteria-NCG-v2-April-2015.pdf 9 DOH. 2016. NCEC Processes and Templates. Available from http://health.gov.ie/patient-

safety/ncec/resources-and-learning/ncec-processes-and-templates/

http://health.gov.ie/wp-content/uploads/2015/01/ncec_guideline_development_manual_january13.pdf

18 | P a g e

To critically evaluate the included clinical literature in terms of the benefit/harm and

level of clinical validation.

To critically evaluate the included economic literature in terms of cost effectiveness,

cost impact and resources involved, including resources required for escalation of

care and implementation costs.

To complete a budget impact analysis to include the following: extraction of data

(where available) regarding potential resource impact of an intervention (e.g.

additional resources required, number of hours to train staff, number of pieces of

equipment required, offsets in terms of efficiencies, and so on).

Regarding any changes to the guideline to consider the resource required to

implement the change (e.g. in the Irish setting provide the Irish budget impact using

comparable Irish cost data).

To assess the relevance and potential transferability (external validity) of included

studies to the Irish healthcare setting.

To describe and critically evaluate education programmes which have been

established to educate healthcare professionals in the delivery of NEWS. In

particular, to describe the programme delivery systems used; level of evaluation

which has been used for these education programmes and the evaluation outcomes.

To appraise and synthesize the evidence to inform recommendations for updating

the NEWS clinical guideline

Search Processes

A full search strategy was developed by the research team to include key terms and their

variations. Key terms were associated with the PICOS (Patient/Problem, Intervention,

Comparison, Outcome, Setting) guidance framework10,11 (Table 1). The PICOS framework is

applicable when addressing a clearly defined clinical question relevant to a defined

population group and clinical context (Caldwell et al. 2012). Key terms included a

combination of terms associated with “early warning scoring systems” (Table 1).

Search of Databases and Grey Literature

The published literature was identified by searching the pertinent electronic databases and

grey literature.

Electronic databases were searched for primary and secondary empirical studies and

systematic reviews. EBSCO host Online Research Databases were used to simultaneously

search relevant health and psychosocial databases (Academic Search Complete, CINAHL (the

10

Note: This is the PICO framework as previously used and published for the clinical systematic review, additional items/key words that were deemed important were added to the pertinent search strings. A separate search string was added relating to educational programmes pertaining to early warning scores/systems or trigger systems. 11

PICOS for the clinical systematic literature review was agreed with the NCEC Steering group prior to utilisation within the review

19 | P a g e

Cumulative Index to Nursing and Allied Health Literature), Medline, PsycINFO, PsycARTICLES,

Psychology and Behavioral Science Collection, SocINDEX, and UK/Eire Reference Centre).

Cochrane Library (www.cochrane.org)

Guideline Websites searched included – e.g. Australian National Health and Medical

Research Council Clinical Practice Guidelines, Canadian Medical Association InfoBase of

Clinical Practice Guidelines, eGuidelines (UK), United States National Guideline

Clearinghouse (www.guideline.gov), the Guidelines International Network (www.g-i-n.net),

New Zealand Guidelines Group, NLH (National Library of Guidelines UK), NHS Evidence

database, NICE (www.nice.org.uk), the Institute for Healthcare Improvement (USA)

(http://www.ihi.org/resources/Pages/IHIWhitePapers/UsingCareBundles.aspx), and the

Scottish Intercollegiate Guidelines Network (www.sign.ac.uk).

Economic evaluation/Cost effectiveness:12 A search was performed in the Database of

Abstracts of Reviews of Effects, the NHS Economic Evaluation Database, the Health

Technology Assessment Database (www.crd.york.ac.uk/CRDWeb ), the Cochrane Central

Register of Controlled Trials and the Cochrane Database of Systematic Reviews

(www.thecochranelibrary.com), Health Technology Assessment Database. Search filters

(Economics, cost, cost analysis, markov chains, monte carlo) were used as search strings

(www.york.ac.uk/inst/crd/intertasc/econ.htm).

Inclusion/Exclusion Criteria

The inclusion and exclusion criteria relate to sourcing literature published since April 2011

relating to the 3 key foci of the review:

Empirical clinical papers relating to use of early warning scores/systems or track &

trigger systems used in adult (non-pregnant) patients in acute healthcare settings

for the detection of/timely identification of physical clinical deterioration.

Empirical economic papers relating to budget impact analysis of early warning

scores/systems or trigger systems used in adult (non-pregnant) patients in acute

healthcare settings for the detection of/timely identification of physical clinical

deterioration and escalation of care.

Evaluation of educational programmes relating to the education/training of health

care professionals relating to early warning scores/systems or track and trigger

systems used in adult (non-pregnant) patients in acute healthcare settings for the

detection of/timely identification of physical clinical deterioration and escalation of

care.

12 Note: The economic evaluation and budget impact analysis is described separately within Chapter 3 and

associated Appendices.

http://www.crd.york.ac.uk/CRDWeb

http://www.thecochranelibrary.com/

http://www.york.ac.uk/inst/crd/intertasc/econ.htm

20 | P a g e

The reference lists of the relevant primary and secondary publications retrieved in the

search were examined to identify further publications.

Review process

All potentially eligible papers identified in the search strategy were exported to Endnote

(Version 7) where duplicates were identified and removed. The papers were initially

screened by titles and abstracts independently by the research team (in pairs) to determine

whether the papers merit a full text review. Details of inclusion criteria are outlined within

Table 1 and Appendix 2. The full texts were obtained and independently evaluated by paired

members of the review team. All team members were involved in this process with a

relatively equal number of papers allocated to each pair. Disagreements were resolved by

consensus within each paired team and if necessary involved a third reviewer (JH). A record

was maintained of all decisions made during this process.

21 | P a g e

Table 1. A systematic literature search was performed using the PICOS framework

PICO Framework

Broad Areas Specific search terms Inclusion criteria Exclusion criteria

Population Adult patient No specifically applied search terms Adult acute patient (i.e. ≥16 years of age) (cared for within an acute hospital)

Pre-hospital settings Community Settings Patients with intellectual disability or psychiatric disorders cared for outside of the acute hospital setting. Acute Care: paediatric patients, obstetric patients.

Intervention Early Warning Scoring System(s)

General (in {Title/Abstract}) “Detection of deterioration” OR “clinical deterioration” OR “identification of deterioration” OR “physiological scoring system” OR “risk assessment report” OR “emergency response system” OR “early warning” OR “warning system” OR “warning scor*” OR “failure to rescue” OR ((“vital sign” {Title/Abstract}) N3 score {Title/Abstract})) OR ((“electronic system” {Title/Abstract}) N3 “early warning” {Title/Abstract})) OR ((tablet {Title/Abstract}) N3 “early warning” {Title/Abstract})) OR ((iPad {Title/Abstract}) N3 “early warning score” {Title/Abstract})) OR ((“escalation protocol” {Title/Abstract}) N3 “early warning” {Title/Abstract})) OR ((“communication” {Title/Abstract}) N3 “early warning” {Title/Abstract})) OR ((“response” {Title/Abstract}) N3 “early warning” {Title/Abstract})) OR ((“VIEWS” {Title/Abstract}) N3 “early warning” {Title/Abstract})) OR ((“NEWS” {Title/Abstract}) N3 “early warning” {Title/Abstract})) OR “medical emergency team” OR “rapid response team” OR “rapid response system” OR “emergency response system” OR “emergency response

Studies which addressed the effectiveness of early warning systems that have been developed to facilitate early detection of deterioration and escalation of care. Papers were included if the principal focus of the paper and its results is on evaluating the effectiveness of NEWS or validating the use of NEWS in the clinical context.

Early warning systems not suitable for measurement and reporting of acute clinical deterioration in the acute health care context. Studies which deal exclusively with the early development of an early warning system. Clinical studies which examine health care professionals’ responses to fictional/hypothetical cases e.g. vignettes.

22 | P a g e

PICO Framework


team” CINAHL BP “detection of deterioration” SH “patient safety”, “nursing assessment, “critical care” MEDline “risk assessment/methods” {Mesh}) OR “point-of-care systems”{Mesh}) OR “monitoring, Physiologic/ methods”{Mesh} Named Systems (in {Title/Abstract}) “Early warning system” OR “early warning score” OR “modified early warning score” OR “MEWS” OR “VitalPAC” OR “track and trigger system” OR Worthing OR SBAR OR “situation, background, assessment, recommendation” OR “situation, background, assessment and recommendation” OR “ISBAR” OR “Identify, Situation, Background, Assessment and Recommendation” OR“ Identify, Situation, Background, Assessment, Recommendation” OR “Manchester triage system” OR “biosignTM”, “Patient at Risk” OR “PAR score” OR “Physiological Scoring System” OR “Vital Sign Score” OR “Physiological Observation Track and Trigger System” OR “Between the flags”

Education programmes(s)

Education programmes (in {Title/Abstract}) ALERT™ OR COMPASS© OR Education OR Program* OR Training OR Course

Studies which address the effectiveness of education programmes that are used to educate/ train registered health care professionals in relation to early warning systems.

Studies which describe the development of education programmes used to educate/ train health care professionals in relation to early warning systems, with no outcome evaluation presented.

23 | P a g e

PICO Framework


Comparison Comparison against other interventions or with no intervention.

No specific search terms Studies looking at early warning systems and their implementation, clinical validation.

Outcome No specific search criteria See data extraction table

Setting No specific terms

No specific search criteria Acute hospital setting in countries categorized as either very high or high human development index (UNDP 2015) 13

Non-acute settings Acute hospital setting in countries categorized as either medium or low Human Development Index (UNDP 2015).

Publication type/level of evidence

Databases searched EBSCO host Online Research Databases were used to simultaneously search relevant health and psychosocial databases (Academic Search Complete, CINAHL (the Cumulative Index to Nursing and Allied Health Literature), Medline, PsycINFO, PsycARTICLES, Psychology and Behavioral Science Collection, SocINDEX, and UK/Eire Reference Centre). Embase and the Trip database were also searched. Cochrane Library (www.cochrane.org) Grey Literature: Guideline Websites were searched. As different study designs were required to meet the different objectives of this review, no study design limits

Time: Publication date within timeframe of April 2011- November 2015. Publication types: Systematic reviews, meta-analysis, meta-synthesis, meta-reviews. Studies which include analysis of data prospectively or retrospectively. The data were pre and post critical adverse clinical event(s) or pre-post EWS intervention. However, the analysis must

Publication quality Publication of study did not contain sufficient detail regarding intervention or outcome measures. Publication types: Surveys of health care professionals. Literature reviews, discussion papers, integrative reviews and opinion pieces.

13

The human development index is calculated using a composite of: life expectancy at birth (duration of life and pseudo measure of health); expected years of schooling (knowledge parameter); and gross national income per capita (indicator of standard of living) and is published by (UNDP 2015). Such countries have resource-limited health care settings, facilities, systems thus limiting the transferability of the research findings to the Irish context. UNDP. 2015. Human Development Report 2015. Available from http://report.hdr.undp.org/

24 | P a g e

PICO Framework


were applied thus ensuring that the likelihood of finding relevant studies irrespective of design was increased.

help to explicate the following: 1) Clinical effectiveness (harm/benefit) of Early Warning Systems 2) Clinical validation of Early Warning Systems In addition studies which evaluated the effectiveness of educational programmes preparing health care professionals for the implementation of Early Warning Systems were included.

Oral/poster conference abstracts (as limited data available for data extraction).

25 | P a g e

Figure 2. The identification, screening, eligibility of publications for inclusion in the review of

clinical literature

Audit Trail

Detailed notes were made to ensure transparency in terms of total number of papers

gleaned from each search strategy source i.e. databases, websites (an audit trail). All

searches were saved in an EBSCO account which facilitates additional searching for new

papers over the course of the review. All references were managed and categorized using

the bibliographic software Endnote to facilitate documentation of the search process,

streamline document management, remove duplications, and make the generation of

reference lists for the final report easier.

A summary of the search is provided as a PRISMA (Preferred Reporting Items for Systematic

Reviews and Meta-Analyses) flow chart in Figure 2. A total of 124 papers met the inclusion

criteria. Details of the papers included are outlined in the “Characteristics of Included

Papers” section of Chapter 2.

Records identified for clinical systematic review

(n=3,598)

Records Screened on title and abstract

(n=3,598)

Full-text papers from database search assessed for eligibility

(n=307)

Full-text papers from database search for inclusion

(n=124)

Full-text papers excluded

(n=183)

Records excluded on title and abstract

(n=3,291)

26 | P a g e

Data Extraction

The review questions guided data extraction. In addition, data were extracted on authors,

year and country of publication, type of study/review, and aim of study/review.

A Delphi process with experts14 was used to provide clarity in relation to the outcome

measures that would be appropriate to include within the review. Interestingly, the experts

included additional perspectives as follows: period of physiological instability, cardiac arrest

rates/1000, rapid response calls/1000 i.e. Medical Emergency Team (MET) call rate (dose),

unexpected death (without a Do Not Resuscitate[DNR] order) sepsis detection/treatment,

patient satisfaction and safety culture.

Separate data extraction tables were developed for:

Empirical clinical papers relating to use of early warning scores/systems or track &

trigger systems used in adult (non-pregnant) patients in acute healthcare settings

for the detection of/timely identification of physical clinical deterioration and

escalation of care ( template in Appendix 3 and extracted studies presented in Table

4a).

Empirical economic papers relating to budget impact analysis of early warning

scores/systems or track & trigger systems used in adult (non-pregnant) patients in

acute healthcare settings for the detection of/timely identification of physical

clinical deterioration and escalation of care (Appendices 5 and 6).

Evaluation of educational programs relating to the education/training of health

care professionals relating to early warning scores/systems or track & trigger

systems used in adult (non-pregnant) patients in acute healthcare settings for the

detection of/timely identification of physical clinical deterioration and escalation of

care (Appendix 4).

Data extraction was limited to two members of the team (JH and FJD) to ensure consistency

and all team members were involved in crosschecking the data.

Quality Assessment

All evidence sourced was graded by two team members independently according to Scottish

Intercollegiate Guidelines Network (SIGN) level of evidence (LOE) criteria and associated

methodological checklists which looked at internal validity and overall assessment of the

study for each study type. LOE was graded Grade 1-4, see Table 2. A colour coding system

was applied to the narrative display of data when presenting key outcome data.

14

Niels Egholm Pedersen, Herlev, Denmark; Chris Hancock and Rheolwr Rhaglen Rapid Response to Acute Illness Learning Set (RRAILS), Ken Hillman, Liverpool Hospital, University of New South Wales.

27 | P a g e

Table 2: Empirical literature characterised according to the SIGN level of evidence criteria15

1++ High quality meta-analyses, systematic reviews of randomized controlled trials (RCTs), or RCTs with a very low risk of bias

1+ Well-conducted meta-analyses, systematic reviews, or RCTs with a low risk of bias

1- Meta-analyses, systematic reviews, or RCTs with a high risk of bias

2++ High quality systematic reviews of case control or cohort or studies High quality case control or cohort studies with a very low risk of confounding or bias and a high probability that the relationship is causal

2+ Well-conducted case control or cohort studies with a low risk of confounding or bias and a moderate probability that the relationship is causal

2- Case control or cohort studies with a high risk of confounding or bias and a significant risk that the relationship is not causal

3 Non-analytic studies, e.g. case reports, case series

4 Expert opinion

Given the breath of outcomes, processes and the heterogeneity of interventions,

measurement tools included within this review, the use of GRADE (Grading of

Recommendations Assessment, Development and Evaluation) tool16 was not thought to be

appropriate.

External Validity/Transferability

The external validity of included studies was assessed using six items adapted from Foy et al.

2010)17 relating to the representatives of the study population, possibility of replication of

the intervention, sustainability of the intervention, appropriateness of outcome measures

for patients and clinical context, nature of follow up, clarity re details of the mechanism of

15

SIGN. 2014. Critical appraisal: notes and checklists. Available from http://www.sign.ac.uk/methodology/checklists.html 16

Assessing the strength of recommendations: using GRADE criteria for assessing the quality of evidence (risk of bias/study limitations, directness, consistency of results, precision, publication bias, magnitude of the effect, dose-response gradient, influence of residual plausible confounding and bias “antagonistic bias”) and the overall quality of evidence can be assessed for each important predetermined outcome and expressed using four (e.g. high, moderate, low, very low) categories based on definitions for each category that are consistent with the definitions used by the GRADE Working Group. GRADE Working Group. 2014. Grading the quality of evidence and the strength of recommendations. Available from http://www.gradeworkinggroup.org/intro.htm 17

FOY, R., HEMPEL, S., RUBENSTEIN, L., SUTTORP, M., SEELIG, M., SHANMAN, R., & SHEKELLE, P. G. 2010. Meta-analysis: Effect of interactive communication between collaborating primary care physicians and specialists. Annals of Internal Medicine, 152, 247-258.

28 | P a g e

action of the intervention. In addition, consideration was given to overall application of the

intervention to the Irish health care context.

Data Synthesis

Similarly measured outcomes (low level of heterogeneity) were combined where

appropriate – decisions regarding meta- analysis were dependent on the review question,

research approach, quality of research, and types of data. Heterogeneity may be attributed

to clinical, methodological or statistical heterogeneity.

A qualitative data-synthesis, which takes methodological differences between primary

studies into account (if the studies reviewed are heterogeneous), were completed. The

overall picture of the evidence was presented.

A record was made of specific examples of good practices internationally which have been

demonstrated to support implementation of NEWS.

Summary

In summary, a systematic review was conducted which sought to appraise and synthesise

the evidence to support a framework for the development of standards for the early

identification and care of the deteriorating patient. The key findings from the review of the

clinical literature are presented in the next chapter.

29 | P a g e

Chapter 2. Results of the Review of Clinical Literature Pertaining to the

Evaluation of Early Warning Scoring Systems

Overview

Studies investigating the development and

efficacy of a number of Early Warning

Scoring systems were identified during the

literature search. These systems differ in

terms of: the vital sign parameters

recorded; the weightings given to each vital

sign; whether they were single-parameter

or aggregate EWS; paper-and-pen based or electronic; the chart design used; the escalation

protocol and the associated emergency response system. The EWSs also differed as to

whether they were aimed at identifying clinical deterioration in all patients, or in a

(sub)group of patients (e.g. patients within

the ED, or non-monitored wards).

Furthermore, the studies differed in whether

they investigated EWSs scoring tools in

isolation or whether they have been

implemented with a rapid response team

(RRT), the type of RRT and escalation

processes associated with them and whether

communication tools, for example ISBAR

(identify, situation, background, assessment,

and recommendation) were implemented to

facilitate the transfer of information

between health care professionals. The

experimental designs of the studies and the

primary and secondary outcomes also

differed.

The overview of the studies is presented

using the following categorisations: NEWS; Modified Early Warning Scoring System (MEWS);

modification of established and newly developed EWS (without NEWS); direct comparison

of established and newly developed EWS; EWS chart design; EWSs and/or RRT educational

interventions; implementation of EWS and/or RRT. This is followed by a summary of data in

relation to key identified outcomes.

Characteristics of Clinical Papers

Papers published from April 2011 to November 2015 relating to the evaluation of EWS

(n=124 papers) were included. Countries represented across these papers were

The review of the literature identified the

heterogeneous nature of empirical studies

relating to the evaluation of Early Warning

Systems.

The study of Early Warning Systems differs

in terms of: the vital sign parameters

recorded; the weightings given to each

vital sign; whether it is single-parameter or

aggregate EWS; paper-and-pen based or

electronic; the chart design used; the

escalation protocol and the associated

emergency response system; the samples

studied; methodological approaches and

the variables measured.

Key points and recurrent themes

emanating from the review of empirical

literature are highlighted as extracted text

within boxes.

30 | P a g e

predominantly the UK (n=37), USA (n=32), the Netherlands (n=8), Australia (n=7), Singapore

(n=7) and Denmark (n=6) (Figure 2). Two studies undertaken in Ireland were identified, the

remainder were performed in other European (n=10) and Asian (n=10) countries; South

Africa (n=2), Israel (n=1) and New Zealand (n=1). With regards to study design the majority

were retrospective analyses of data (n=42), and prospective studies (n=41), pre- and post-

intervention studies (n=19) and systematic reviews (n=5). Other study designs were mixed

methods (n=4), surveys (n=1), trials (n=5), qualitative (n=2), proof of concept (n=2), quasi

experimental (n=2) or reviews of the literature (n=3). The vast majority of studies were

single once off studies which limits the ability to draw conclusions.

Figure 3. Diagrammatic representation of the countries reflected in papers sourced.

The focus of the papers sourced differed and were broadly characterised as papers

addressing: NEWS exclusively; MEWS exclusively; newly developed EWS and/or modified

established EWSs; comparisons of EWS; evaluation of chart design; educational

interventions; implementation of EWS/RRTs (Figure 4).

The Level of Evidence (LOE) categorisation of studies is presented in Figure 5. This

categorisation is used to present a narrative synthesis of the included papers. However

despite these broad headings being used it must be reiterated that there is a lack of

standardised definitions and /or terminology in terms of early warning systems (using the

same named titles e.g. NEWS). The systems differed in a number of ways for example: the

vital sign parameters recorded; the weightings given to each vital sign; whether they were

single-parameter or aggregate EWS; paper-and-pen based or electronic; the chart design

utilised; the escalation protocol and the associated emergency response system used.

0 5 10 15 20 25 30 35 40

UK

USA

Netherlands

Australia

Singapore

Denmark

Other Euopean

Other Asian

South Africa

Israel

New Zealand

Ireland

31 | P a g e

Figure 4. Diagrammatic representation of the numbers of papers sourced relating to each

category.


Level of Evidence (LOE) Category.

0 5 10 15 20 25 30

NEWS

MEWS

Newly developed EWS and/or modified…

Comparisons of EWS

Chart design

Educational interventions

Implementaton of EWS/RRTs

0 10 20 30 40 50 60

LOE 1++

LOE 1++

LOE 1-

LOE 2++

LOE 2+

LOE 2-

LOE 3

32 | P a g e

National Early Warning Score

There were seventeen studies that had NEWS

as the primary focus. While NEWS stands for

National EWS, differences in the NEWS

systems used were identified between studies.

One of the biggest differences was in the

scoring of inspired oxygen (O2); in Ireland ‘Any

O2’ receives a score of 3, whilst in the UK, a

score of 2 is given, see Appendix 4b.

These studies were performed in the UK (n=8),

the Netherlands (n=3), Ireland (n=2), USA (n=2), Denmark (n=1) and Finland (n=1). The

majority of these studies used the standard NEWS (Appendix 4). Studies were prospective

(n=7), retrospective (n=9) and qualitative in design (n=1).

The level of evidence18 varied; LOE2+ (n=5),

LOE2- (n=6) and LOE3 (n=7).

The findings reported are as follows:

(i) A systematic review concluded that

EWS tools perform reasonably well in

predicting cardiac arrest and death

within 48 hours, but not length of stay

(LOS). RRT use improved EWS scoring

consistency (n=1; LOE 2++) (Smith et al.

2014).

(ii) Among patients presenting at the ED,

the NEWS score measured at different

time points can further risk stratify ED

patients within higher Emergency

Severity Index (ESI) risk categories, for

hospital admission, death and need for

Intensive Care Unit (ICU) admission.

The NEWS score can be used to

longitudinally monitor patients

throughout their stay in the ED and in the hospital (n=1; LOE 2+) (Alam et al. 2015).

(iii) A high rate of NEWS scores calculated incorrectly for medical patients on wards

which adversely affected clinical response; a trend towards increased mortality for

18

SIGN. 2014. Critical appraisal: notes and checklists. Available from http://www.sign.ac.uk/methodology/checklists.html

The NEWS implemented in different

hospitals/settings varied in the weighting

given to vital sign parameters.

Many of the barriers to the

implementation of NEWS are

sociocultural.

Incorrect calculation of NEWS and

documentation is common. A trend

towards increased mortality was observed

in patients with incorrect NEWS

documentation.

Clinical response to NEWS is worse out of

hours.

Note: Within the narrative summaries, a

colour coding system is applied to visually

represent the Level of Evidence (LOE) of

pertinent narrative synthesis (light green

equates to LOE1, light orange to LOE2

and light grey shading was applied for

LOE 3).

33 | P a g e

patients who received an incorrect response to a NEWS score was observed. (n=1;

LOE 2+) (Kolic et al. 2015).

(iv) Clinical response to NEWS scores is significantly worse at weekends compared to

weekdays, which has implications for standards of care for patients out of hours

(n=1; LOE 2+) (Kolic et al. 2015).

(v) Poor compliance with the NEWS protocol and level of care was observed when faced

with a deteriorating patient (n=1; LOE 2-) (Petersen et al. 2014).

(vi) Many of the barriers to the implementation of NEWS were related to sociocultural

aspects of introducing a new system into current practice. It was highlighted that

these sociocultural issues may affect non-compliance and must be addressed in

order to improve detection of the clinical deterioration of patients (n=1; LOE 2-)

(Lydon et al. 2015).

(vii) NEWS discriminates high risk patients in a heterogenic general ward population

independently of multiple confounding factors, and could predict 30- and 60-day

mortality. The conventional dichotomised activation criteria were not able to detect

high risk patients (n=1; LOE 2+) (Tirkkonen et al. 2014).

(viii) NEWS was superior to PARS (Patient at Risk Score) at identifying general medical

patients at risk of critical care/ICU admission or death within 48 hours (n=1; LOE 2-)

(Abbott et al. 2015).

(ix) NEWS and sepsis:

a. Among patients with sepsis a higher NEWS score on arrival at ED was

associated with higher odds of adverse outcome, including ICU admission and

30-day mortality. Use of NEWS could facilitate patient pathways to ensure

triage to a high acuity area of the ED and senior clinician involvement earlier;

a NEWS ≥7 among these patients may benefit from a review by senior ED

clinical staff or a critical care outreach team (CCOT) (n=1; LOE 2-) (Corfield et

al. 2014). A NEWS ≥3 at ED triage may be used to trigger a systematic screen

for septic shock and to obtain an early serum lactate and initiate treatment

(n=1; LOE 3) (Keep et al. 2015).

b. In another study comparing ESI (Emergency Severity Index), MEWS and MEDS

(Mortality in the Emergency Department Sepsis), the authors found that the

systematic use of the MEDS score in the ED could lead to the detection of

critically ill patients with sepsis. (ESI and MEWS do not identify patients with

sepsis with accuracy, but they could be amended by disease-specific risk

stratification tools like MEDS or ESI) (n=1; LOE 2-) (Geier et al. 2013).

(x) NEWS lacks specificity for patients with chronic hypoxia. CREWS (Chronic Respiratory

Early Warning Score) a variant of NEWS, is superior in this patient group and could

reduce unnecessary triggers and alarm fatigue for patients with chronic hypoxaemia,

whilst identifying the sickest patients. This is likely to be due to the respiratory

variables in NEWS (n=1; LOE 2-) (Eccles et al. 2014) and (n=1; LOE 3) (Lobo et al.

2015).

34 | P a g e

(xi) Both NEWS and MEWS are limited in predicting oncology patients at risk of clinical

deterioration or adverse outcomes e.g. ICU admission and 30-day mortality, with the

physiological parameters predicting clinical deterioration in this cohort not captured

(n=1; LOE 3) (Cooksley et al. 2012).

(xii) In a study from the UK, an aggregate NEWS score of ≥5 was associated with higher

risk of adverse events than at an aggregate NEWS score of 3 or 4 with or without a

single vital sign score of 3. The UK recommended NEWS escalation guideline

(escalating of care at a score of 3 or 4 and a vital sign of 3) increases workload with a

modest increase in detection of adverse effects and benefit to patients. However,

there may be a case for defining extreme values for each vital sign at which

escalation is required, irrespective of the aggregate NEWS score, but such scores

should be more severely deranged (extreme) that those currently scored 3 in NEWS

(n=1; LOE 3) (Jarvis et al. 2015a).

(xiii) Semi-Markov decision process (SMDP) models, which include a mathematical

framework for modelling decision making, may be used to (a) monitor patient over

time in hospitals with electronic systems and (b) define the optimum management

of deterioration in patient subgroups: e.g. a SMDP concluded that a highly frail

surgical patient (Braden skin score [BSS] ≤11) without previous deterioration events

would benefit from RRT activation at

a NEWS score of 1-4 or single extreme

value; while RRT activation at NEWS

≥7 is most appropriate for a

moderately frail medical patient. In

addition, any surgical patient

(regardless of BSS at admission)

would benefit from RRT activation at

a NEWS score of 1-4, or with a single

extreme value (n=2; LOE 3) (Capan et

al. 2015a; Capan et al. 2015b).

(xiv) Decision-Tree Early Warning Scores

(DTEWS) validated NEWS among

medical patients (n=1; LOE 3)

(Badriyah et al. 2014).

(xv) ViEWS (VitalPAC Early Warning Score)

and MEWS were validated as being

predictive of mortality upon

admission and at any point during

hospitalization (n=1; LOE 3) (Bleyer et

al. 2011)

NEWS may be used to further risk stratify

patients within high ESI risk categories in

the ED. It can also be used to identify

patients with sepsis who need clinical

review or a systematic screen for septic

shock.

However, NEWS may not be the optimum

scoring system for all patient subgroups,

for example:

CREWS may be superior to NEWS in

risk stratifying patients with chronic

hypoxia

Both NEWS and MEWS maybe

limited in ability to predict

deterioration among patients with

cancer.

35 | P a g e

Modified Early Warning Scoring System

There were sixteen studies where MEWS was the primary focus. The MEWS while based on

that described by Subbe et al. (2001) varied in the vital sign parameters included and their

scoring, see Appendix 4b. These studies were performed in the USA (n=6), UK (n=2),

Singapore (n=2) and one each in the Netherlands, Turkey, South Korea, Austria, Italy and

Hong Kong.

The quality of evidence from these studies varied; 1- (n=2), 2++ (n=1), 2+ (n=2), 2- (n=5) and

3 (n=7).

I. MEWS tools’ scoring of physiologic findings, including vital signs has a positive

relationship with earlier detection of clinical deterioration, but the development of

established criteria for validating MEWS scoring system criteria, organisational-

specific reliability testing and multi-site trials are recommended (n=1; LOE 1-) (Roney

et al. 2015).

II. No studies investigating the use of MEWS among septic patients were identified

(n=1; LOE 1-) (Roney et al. 2015).

III. MEWS can be used to predict need for hospitalisation in the ED (n=1; LOE 3) (Urban

et al. 2015).

IV. Rapid Emergency Medicine Score (REMS) was superior to MEWS as a predictor of in-

hospital mortality and hospitalisation in medical and surgical patients admitted to ED

(n=1; LOE 2-) (Bulut et al. 2014).

V. MEWS can be used as a routine

screening tool and to monitor the

clinical course of patients with

pancreatitis on a general surgical

ward (n=1; LOE 2-) (Suppiah et al.

2014). MEWS may assist in

reducing ICU admissions post-

emergency surgery (n=1; LOE 2++)

(Peris et al. 2012).

VI. MEWS score ≥5 predicted patient

death in surgical patients who

have experienced cardiac arrest

(n=1; LOE 3) (Stark et al. 2015).

VII. The MEWS score was significantly

different between patients

experiencing cardiac arrest and

control patients by 48 h prior to

the event however MEWS

includes poor predictors of

cardiac arrest such as

The MEWS systems investigated varied.

The reported discriminatory ability of

MEWS to predict cardiac arrest, ICU

admission and mortality varied between

studies.

The discriminatory ability of MEWS to

predict adverse outcomes was lower in

older patients.

Respiratory rate is an important vital sign

as predictor of adverse events on the ward.

Issues with implementation identified

included poor documentation of MEWS

and underutilisation of RRTs. These

problems were worse out of hours.

36 | P a g e

temperature and omits significant predictors such as diastolic blood pressure (DBP)

and pulse pressure index. Respiratory rate was the best vital sign predictor of

cardiac arrest on the ward. However, the ideal cut-off is unknown, partly because it

is often inaccurately measured and poorly documented in hospitalised patients (n=1;

LOE 2+) (Churpek et al. 2012a).

VIII. The accuracy of MEWS to predict cardiac arrest decreases with the age of the

patient; almost all vital signs more

accurately detected cardiac arrest

in nonelderly compared to elderly

patients. Patient age should be

considered when interpreting vital

sign data. There is also a need to

identify other cardiac arrest

predictors (e.g. comorbidities,

medications) in the elderly to

supplement EWSs (n=1; LOE 2+)

(Churpek et al. 2015).

IX. Variation in the optimum MEWS trigger scores in predicting each outcome, and in

different patient populations was observed. MEWSs scores of 4 had a predictive

value for serious adverse events (SAEs) in patients post-ICU discharge, up to three

shifts prior to the SAE. But the sensitivity of MEWS is relatively low, therefore

clinical judgement is crucial. Intensive care specialists can use MEWS or Simplified

Acute Physiology Score (SAPS) 3 to assess patients at risk of an SAE post-ICU

discharge (n=1; LOE 2+) (De Meester et al. 2013a).

X. In another study MEWS score was not above the trigger score of 5 for the majority of

patients who suffered a cardiac arrest (n=1; LOE 3) (Harris 2013). Sensitivity of

MEWS was also lower in critically ill Asian patients attending the ED and the MEWS

score did not perform well in predicting poor patient outcomes i.e. 30-day mortality

or transfer to ICU or High Dependency Unit (HDU) in this patient group (n=1; LOE 3)

(Ho et al. 2013). In contrast, MEWS scores between 24 and 8 hours prior to the

event were associated with in-hospital mortality at each time point monitored.

MEWS also predicted cardiac arrest, however, although almost half of the patients

investigated had a low MEWS (<4) up to 8 hours prior to their cardiac arrest (n=1;

LOE 3) (Kim et al. 2015).

XI. MEWS at ICU admission can predict ICU mortality, 30-day mortality and LOS in ICU,

but not as well as ICU specific tools (Sepsis-related Organ Failure Assessment [SOFA]

and SAPS III) (n=1, LOE 2-) (Reini et al. 2012). In another study an aggregate MEWS

score ≥5 was significantly associated with death or ICU admissions among unselected

general medical patients among Asian patients. However, MEWS was not associated

Barriers to the implementation of MEWS

include; lack of skills among healthcare

assistants, clinical judgement overruling

MEWS score, no agreement on optimum

frequency of monitoring overnight, and

underutilisation of the Critical Care Teams

or Medical Emergency teams.

37 | P a g e

with excess LOS; functional status and altered mental status were independent

predictors of excess LOS (n=1, LOE 2+) (Huggan et al. 2015).

XII. MEWS may improve MET response to deteriorating patients (n=1, LOE 3) (Parham

2012).

XIII. The Rothman Index (RI), a patient acuity score based upon summation of excess risk

functions that utilize additional data (26 items) from the electronic medical records

(EMR) outperforms MEWS in identifying mortality within 24 h in hospitalised

patients (n=1; LOE 3) (Finlay et al. 2014).

XIV. Problems with MEWS implementation were reported. The majority of observations,

including MEWS were recorded by healthcare assistants who may not have the skills

to recognise a deteriorating patient, document observations accurately, calculate

MEWS correctly or initiate an appropriate response. Critical care teams were

underutilised, especially out of hours, despite the presence of a clear response

strategy (n=1; LOE 3) (Harris 2013).

XV. Optimum frequency of night time MEWS monitoring for low risk medical patients is

uncertain (n=1; LOE 2-) (Yoder et al. 2013). Other barriers included the perception

by experienced nurses that they used it less as they relied on their own judgement

(n=2; LOE 2-) (Pattison & Eastham 2012; Stewart et al. 2014); ward busyness/time

consuming (n=2; LOE 2-) (Pattison & Eastham 2012; Mok et al. 2015), and

misjudgement by healthcare providers (HCPs) of their ability to handle patients’

condition. Referral to outreach may threaten trust between ward nurses and doctors

who had been managing the situation on the ward (n=1; LOE 2-) (Pattison & Eastham

2012).

XVI. Nurse clinical judgements improved following the introduction of MEWS

introduction (n=1; LOE 2-) (Shuk-Ngor et al. 2015).

XVII. No independent association was found between a composite outcome of death or

ICU admission and the Charleston Comorbidity Index (CCI) or admission modified

Barthel Index among patients presenting to the ED (n=1; LOE 2-) (Huggan et al.

2015).

Modification of Established and Newly Developed EWS (without NEWS)

There were 22 studies which described newly developed EWS and/or modified established

EWSs. These were undertaken in the USA (n=7), Singapore (n=3), UK (n=2), Denmark (n=2)

and one each in Switzerland, Germany, Australia, South Africa, The Netherlands, Spain,

China and a combined study in centres in the USA and UK.

The level of evidence from these studies varied; LOE2+ (n=5), LOE2- (n=8) and LOE3 (n=9).

I. A Vital Sign Score (VSS) based on MET calling criteria was a significant predictor of

mortality in patients assessed by the MET. Increasing MET utilisation coincided with

a decrease in cardiac arrest calls (n=1; LOE 2-) (Etter et al. 2014).

38 | P a g e

II. While developing the Critical Vital Sign scoring system, the authors concluded that

the simultaneous presence of ≥3 critically abnormal vital signs any time during

hospitalisation was associated with very high in-hospital mortality, which occurs

more frequently in the first 48 hours of hospitalisation (n=1; LOE 3) (Bleyer et al.

2011).

III. Addition of a lactate measurement can increase the sensitivity of ViEWS in predicting

mortality and could equal that of more complex, currently accepted scoring systems

in a critically ill cohort (n=1; LOE 2-) (Jo et al. 2013). The ViEWS-L score out-

performed ViEWS, HOTEL (Hypotension, O2 Saturation, Low temperature, ECG

change and Loss of independence), and the APACHE II (Acute Physiology and Chronic

Health Evaluation) score as well as SAP II and SAP III in predicting mortality up to 4

weeks post-admission, of a mixed

cohort of unselected critically ill

medical patients admitted to the

medical ICU through the ED (n=1,

LOE 2) (Jo et al. 2013).

IV. A 7-item EWS from readily available

physiological parameters is a simple

and valid tool for identifying

patients at low, intermediate and

high risk of dying within 30 days of

an acute stroke (n=1, LOE 3)

(Liljehult & Christensen 2015)

V. In ED patients with chest pain,

the12-lead ECG combined with

heart rate variables (HRV) and vital

signs were strongly associated with

acute cardiac complications within

72 h. The Ensemble-Based Scoring

System (ESS) has been proposed to

integrate these multiple sources of

predictors for risk stratification of

patients. The ESS model had

superior performance compared to existing methods including the Thrombolysis in

Myocardial Infarction Score (TIMI), MEWS and Distance-Based Scoring System (DIST)

(n=1; LOE 2-) (Liu et al. 2014a).

VI. The SUPER score (consisting of peripheral oxygen saturation [SpO2], urinary volume,

pulse, emotional state and RR) may be used for predicting the onset of acute heart

failure in high risk patients. It may also be used to stratify patients into low-,

moderate-, high- and extremely-high risk. It was superior to MEWS in patients in an

acute heart failure unit (n=1; LOE 3) (Bian et al. 2015).

Many studies were identified which aimed at

improving the discriminatory ability of EWS.

These included supplementing current EWSs

systems with additional parameters. The

additional parameters (diastolic blood

pressure, a 12-lead ECG and/or arterial blood

gas, lactate, real time laboratory results)

increased the discriminatory ability to predict

adverse effects in certain patient

subpopulations.

Integrating EWS into the electronic medical

record facilitated the use of more informative

parameters, automated calculations, and

improved RRT alerting. This coupled with

different analytical techniques (using trend &

change data, clinical prediction models) may

facilitate the development of disease-specific

and population-specific EWSs.

39 | P a g e

VII. The CART (Cardiac Arrest Risk Triage) score derived from EHRs is more accurate in

detecting cardiac arrest and ICU transfer than MEWS. Implementing CART may

decrease RRT resource utilisation and lead to improved patient outcomes. However,

calculation may require electronic calculation (n=1; LOE 3) (Churpek et al. 2012b),

(n=1; LOE 2-) (Churpek et al. 2014b).

VIII. Machine Learning (ML) scores incorporating HRV parameters, age and vital signs

were more accurate than the MEWS in predicting cardiac arrest within 72 hours.

There is potential to develop bedside devices for risk stratification based on cardiac

arrest prediction for the risk stratification of critically ill patients in the ED (n=2; LOE

2-) (Ong et al. 2012; Liu et al. 2014b) (n=2, LOE 2+) (Liu et al. 2014a; Liu et al. 2014b).

IX. A ML devised model could better discriminate patients with Major Adverse Cardiac

Events (MACE) within 72 hours of ED admission from those without MACE than TIMI

and MEWS. Inclusion of only 3 variables (Systolic Blood Pressure [SBP], avHR and

aRR) achieved the best prediction scores (n=1; LOE 2-) (Liu et al. 2014a).

X. Decision tree analysis is a faster method of deriving a EWS than human ‘trial and

error’ method and may be employed in the future to develop disease-specific EWS.

However, clinical input will be required (n=1; LOE 3) (Badriyah et al. 2014).

XI. A laboratory decision tree (LDT)-EWS could discriminates risk of death and using

commonly measured laboratory tests collected soon after hospital admission can

also be represented in a simple, paper-based EWS (LDT-EWS) to help discriminate in-

hospital mortality (n=1; LOE 2-) (Jarvis et al. 2013).

XII. A prediction tool for ward patients was developed and validated using EHR data

(including vital signs, laboratory results, demographics and location) which

simultaneously predicted risk of cardiac arrest and ICU transfer. This model was

more accurate than ViEWS for both outcomes. The authors suggested that this

model could be implemented in the EHR and used in real-time to detect critically ill

patients (n=1; LOE 3) (Churpek et al. 2014a).

XIII. An automated clinical prediction model harnessing EMR data outperformed MEWS

and human judgement-based RRT, in predicting SAEs outside the ICU (n=2; LOE 2-)

(Escobar et al. 2012; Alvarez et al. 2013). This technology may be investigated for its

ability to discriminate by individual diseases. To use these EMR-based models,

hospitals must have EMRs and longitudinal data.

XIV. The Early Warning and Response System (EWRS) designed to monitor vital signs and

laboratory results in real time with an automated alert resulted in an improvement

in early sepsis care; accurate identification of non-ICU inpatients at increased risk of

deterioration or death, improved documentation, and a suggestion of reduced

mortality (n=1; LOE 2+) (Umscheid et al. 2015).

XV. Tarassenko et al. (2011) described the development of a centile-based EWS

(analysing the frequency distribution of physiological observations and thereby

defining scores purely by the degree of the difference from the statistical mean).

(n=1; LOE=3) (Tarassenko et al. 2011). However, this EWS performed worse of all

40 | P a g e

other aggregate scoring EWS in predicting adverse events (n=2; LOE 3) (Churpek et

al. 2012b, Jarvis et al. 2015c).

XVI. The specificity and Negative Predictive Value of the Bispebjerg Early Warning Score

(BEWS) are high and can be used to both categorise patients in the ED into low and

high risk and activate MET response (n=1; LOE 3) (Christensen et al. 2011).

XVII. Consideration of comorbidities and biochemical data (with the exception of c-

reactive protein) did not increase the discriminative ability of the Worthing

Physiological Scoring System (PSS) to predict mortality within 72 hours of ED

patients. Its performance was similar to NEWS (n=1; LOE 2-) (Dawes et al. 2014).

XVIII. Introducing an 8-item EWS system in the form of an Adult Deterioration Detection

System (ADDS) chart in addition to an existing cardiac arrest team response appears

to have decreased the number of in-hospital cardiac arrest responses during the 12-

month implementation period, without significantly increasing the number of

medical emergency calls, in one tertiary hospital (n=1; LOE 2-) (Drower et al. 2013).

XIX. A prediction tool based on routinely measured ‘real world’ vital signs and nursing

measurements (Shock Index, RR, SaO2 and BSS) can serve as a very early warning

system for adverse events within 12 hours among hospitalised medical patients (n=1;

LOE 2+) (Kirkland et al. 2013).

XX. Patients at higher risk of triggering an RRT activation could be identified through

higher RR and HR in the ED (n=1; LOE 2+) (Mora et al. 2015).

XXI. TREWS (mobility, resting rate, heart rate, systolic BP, temp, Alert, Voice, Pain,

Unresponsive (AVPU), Trauma) modified from MEWS could be used to risk stratify

patients in the ED (n=1; LOE 3) (Naidoo et al. 2014).

XXII. The DULK (Dutch Leakage) score may allow earlier diagnosis of anastomotic leakage

3.5 days earlier than routine clinical judgment alone (n=1; LOE 2-) (Martin et al.

2015).

Direct Comparison of Established and Newly Developed EWS

There were sixteen studies which directly compared a number of EWSs. These studies were

performed in the USA (n=6), UK (n=3), Singapore (n=2) and one each in Australia, Turkey,

Germany, Israel and South Korea.

The level of evidence from these studies varied; LOE1- (n=1), LOE2++ (n=1), LOE2- (n=9) and

LOE3 (n=5).

41 | P a g e

I. A systematic review concluded that EWS tools perform reasonably well in

predicting cardiac arrest and death within 48 hours, but not LOS. RRT use

improved EWS scoring consistency (n=1, LOE 2++) (Smith et al. 2014).

II. NEWS has the best discrimination ability of 35 EWSs in the prediction of death

within 24 hours among hospitalised patients who were subsequently discharged.

MEWS was the second best performing EWSs in all analyses (n=1, LOE 3) (Jarvis

et al. 2015c). NEWS had the best

discriminatory ability of 35 EWSs

to predict unanticipated ICU

admission or death but not

cardiac arrest among medical

admissions (n=1, LOE 3) (Smith et

al. 2013).

III. NEWS ranked sixth, behind

THERM, the Worthington EWS,

MEES (Mainz Emergency

Evaluation Score), PEDS (Prince of

Wales Emergency Department

Score) and MEWS in predicting

the composite output of ICU

admission or death within 7 days

of critically ill ED patients. THERM is a new score, derived and validated in an ED

setting, using readily available variables (data). THERM has advantages over

NEWS and MEWS which makes it more applicable to the ED; (i) THERM does not

include use of supplemental O2 (ii) NEWS does not discriminate between degrees

of reduced consciousness; anything below ‘alert’ on the AVPU and therefore has

little discriminatory value in the ED (n=1; LOE 2-) (Cattermole et al. 2014).

IV. NEWS ranked third behind SOFA and PICO in predicting clinical deterioration in

non-ICU patients in general medical wards, during 0 and 12 hours prior to the

deterioration. However, differences were not significant. SOFA identified clinical

deterioration earlier than other scoring systems. Scoring models that consider

trends in scores over time may have increased prognostic value over models that

use only a single set of measurements (n=1, LOE 2-) (Yu et al. 2014).

V. Aggregate weighted scoring systems outperformed single parameter systems for

most outcomes among medical and surgical hospitalised patients, with the

Standardised Early Warning Score (SEWS), MEWS, ViEWS, and CART scores being

the most accurate for detecting cardiac arrest, mortality, ICU transfer and a

composite of the three outcomes. However there was a wide range of accuracy

across outcomes for a given system and across systems (Note NEWS was not

included) (n=1; LOE 3). (Churpek et al. 2013).

Aggregate EWSs outperform single

parameter scoring systems.

Compared to other established EWS

systems, NEWS has relatively good

discriminatory ability to predict adverse

events among hospitalised patients.

Sensitivity and Positive Predictive Values

are more useful metrics to compare EWSs

than AUROC from a clinical perspective.

42 | P a g e

VI. MEWS, Simple Clinical Score (SCS) and REMS scoring systems were appropriate

for the detection of early death (1-5 day mortality) in patients with sepsis.

However, SCS and REMS were the most appropriate mortality prediction models

for patients with sepsis admitted to general medicine departments, for all time

points investigated (1-60 day mortality). (Note NEWS was not included) (n=1; LOE

3) (Ghanem-Zoubi et al. 2011).

VII. A binary NEWS (defined as 0 for ‘all vital sign parameters which were normal’ or

1 if ‘abnormal i.e. any parameter with a score of ≥1’) performs better than 35

other standard EWS, except for the standard aggregated NEWS. Therefore,

these simplified EWSs may be used to identify patients at increased risk of

adverse events. Binary NEWS may result in fewer errors, but a higher workload

for Rapid Response Systems (RRSs) (n=1, LOE 3) (Jarvis et al. 2015b).

VIII. Age at admission is not useful for discriminating changes in risk during an

episode of care, but it is useful for discriminating risk between two episodes of

care in which the patients have different ages. EWS which weight patient age

heavily function less well than EWSs which do not weight age. Age is a more

useful discriminator of death when only one observation per episode of care is

included in the Area Under the Receiver-Operated Curve (AUROC) calculation,

because it reduces the bias of more observations per episode for older patients,

when all observations are included (n=1, LOE 3) (Jarvis et al. 2015c).

IX. Both MEWS and ViEWS are easy to use and can predict discharge, hospitalisation

and in-hospital mortality among geriatric patients attending the ED with similar

performances (n=1, LOE 2-) (Dundar et al. 2015).

X. ICU scoring systems (APACHE II/III and SAPS II) out-performed ED scoring systems

(MEWS, REMS, PEDS) in predicting mortality in critically ill patients admitted

directly to the ICU from the ED. (Note NEWS was not included) (n=1, LOE 2-)

(Moseson et al. 2014).

XI. To compare EWSs it is important to report metrics that incorporate the

extremely low prevalence. Such metrics could include: positive predictive value

(PPV), the number needed to evaluate (NNE) and/or the estimated rate of alerts

combined with sensitivity to evaluate each of the plausible score cut-off values.

Including two of these metrics in a graph allows for easy evaluation that is of

practical clinical usefulness both in absolute terms and for comparison of two or

more EWSs. Evaluating EWSs in this way demonstrates the balance between the

benefit of detecting and treating very sick patients with the associated clinical

burden on providers and patients. Clinically, EWSs should not replace clinical

judgment and decision-making but should serve as a safety net”(p5) (n=1, LOE 3)

(Romero-Brufau et al. 2015).

43 | P a g e

EWS Chart Design

There were six studies investigating the impact of chart design (either pen and paper or

electronic charts) on documentation of EWS. These were conducted in Australia (n=3), UK

(n=2) and Denmark (n=1)

The level of evidence from these studies varied; LOE2+ (n=5), LOE2- (n=1).

Findings:

I. The ADDS chart outperformed the other designs investigated (including those with

which the participants had experience), with faster speeds and fewer errors

recorded (n=1; LOE 2+) (Christofidis et al.

2013).

II. Charts where blood pressure and heart

rate observations are plotted separately

resulted in fewer errors and faster

response times (even among participants

trained in the ‘seagull sign’19), precluding

use of the Seagull Sign. Future

observation charts should include an

effective early EWS, and should be

designed based on evidence (n=1, LOE

2+) (Christofidis et al. 2014).

III. Vital sign placement affects the speed

and accuracy of chart completion. Faster and more accurate EWS scoring was

obtained on chart formats which did not require the recording of individual vital sign

scores. This may be explained by having less visual switches, and visual clutter.

Empirical evaluations of chart designs are essential (n=1, LOE 2+) (Christofidis et al.

2015)

IV. Graphical data display is superior to numerical data display in terms of faster and

more accurate interpretation of information (n=1, LOE 2+) (Fung et al. 2014)

V. Implementation of an electronic observation chart was feasible, although variation in

engagement in different wards was observed and could not be explained.

Traditional gaps in observation, e.g. RR were observed in the electronic capture of

vital signs. It is unrealistic to expect a complete set of vital signs to be measured

each time. Vital signs pertinent to the clinical concern of individual patients should

be reviewed. Nurses were initially anxious about using the new electronic chart,

fearing increased workload and monitoring. The success of new interventions

depends on human interaction with the systems and variable organisational

practices (n=1, LOE 2+) (Nwulu et al. 2012).

19

The Seagull Sign occurs when a graphical representation of the heart rate and blood pressure is plotted and the Heart Rate is above the Systolic Blood Pressure.

Chart design affects the speed and

accuracy of documentation.

Researchers (based on their

findings) recommended the use of

graphical display and avoiding

visual clutter, and the use of

overlapping graphical displays of

data.

44 | P a g e

VI. Chart design investigated as part of EWS bundle (monitoring practice, EWS, and

observational chart (i.e. colour-coding in green, yellow, orange or red on top of the

chart used for each parameter score and range), and an algorithm for bedside

management, implemented by inter-professional teaching, training and optimization

of communication and collaboration). Implementation of the bundle linked to

decreases in unexpected patient mortality (LOW 2-, Bunkenborg et al. 2013)

NEWSs and/or RRT Educational Interventions

A number of formal education programmes have been developed to support the roll out of

the EWS. One such programme is COMPASS© which is used in the Irish context. The idea

behind the ‘Compass’ name is to point an individual in the right direction, with the aim of

improving the early recognition and timely management of deteriorating patients. The

COMPASS programme developed in 2007 explains the rationale for the EWS, how to

calculate the EWS score and how to act on a rising score (Mitchell, McKay, Van Leuvan et al.,

2010). The COMPASS© education program was designed to be flexible with online pre-

learning and limited face to face time. However, limited research exits which tests the

impact of COMPASS in the general adult context20; whilst some literature in the paediatric

context21 exists- such literature was outside the scope of this review.

Nine studies (in ten publications) evaluated

educational interventions aimed at

improving the recognition of and/or

response to clinical deterioration. These

were performed in the USA (n=3), UK (n=2),

South Africa (n=1), The Netherlands (n=1)

and Singapore (n=2). Design varied regarding

design, content and follow-up time of

evaluation.

The level of evidence from these studies

varied; LOE2+ (n=2), LOE2- (n=7).

The findings

i. A multi-modal educational intervention to core nursing staff resulted in

improvements in the (a) documentation and (b) use of an electronic MEWS score

20

LIAW et al. (2011) in a review of publications from 2000-2010 found four existing programmes that educated nurses in the recognition and management of deteriorating patients within ward settings i.e. The Acute Life Threatening Events Recognition & Treatment (ALERT), Multi-professional Full-scale Simulation (MFS), COMPASS an inter-professional course for both qualified medical and nursing staff working in hospitals and the Acute Illness Management (AIM) used in the context of health professionals and students. 21

MCKAY, H., MITCHELL, I. A., SINN, K., MUGRIDGE, H., LAFFERTY, T., VAN LEUVAN, C., MAMOOTIL, S., ABDEL-LATIF M. E. 2013. Effect of a multifaceted intervention on documentation of vital signs and staff communication regarding deteriorating paediatric patients. Journal of Paediatrics & Child Health, 49, 48-56.

Interdisciplinary, multimodal and follow-up

educational programmes are most effective.

Few of the studies evaluating educational

programmes investigated the long-term

impact of the intervention. Those that did:

did not see a significant change in

behaviour one-year post-intervention.

45 | P a g e

as a clinical decision making tool to engage the RRT by core staff and improved

patient rescue strategies up to three months post-implementation (n=1, LOE 2-)

(Rose et al. 2015).

ii. Multidisciplinary training led to more accurate identification of deteriorating

patients, up to 6 months post-intervention, with implications for subsequent

care and outcome (n=1, LOE 2+) (Merriel et al. 2015).

iii. There was a significant increase in knowledge and performance in assessing,

managing and reporting clinical deterioration following participation in a web-

based educational programme developed for hospital nurses one week post-

intervention (n=1, LOE 2-) (Liaw et al. 2015a &b). In contrast a virtual simulation

intervention tested among students did not lead to improved performance.

Authors commented that hands on simulation coupled with social interaction

underline collaborative and deeper learning experiences (Liaw et al., 2014)

iv. MEWS education increased the confidence of nursing staff working in mental

health inpatient wards and their ability to recognise and manage physically

deteriorating patients (n=1, LOE 2-) (Shaddel et al. 2014).

v. Clinical stimulation was effective in improving student knowledge and clinical

judgement, specifically concerning RRSs when assessed immediately followed

the education (n=1, LOE 2-) (Lindsey & Jenkins 2013).

vi. An educational intervention consisting of e-learning, simulation and debriefing,

with SBAR (Situation, Background, Assessment, Recommendation)

communication, can improve instability recognition and communication,

resulting in improved knowledge and decreased time to critical actions among

nurses on surgical wards, when assessed immediately (n=1; LOE 2-) (Ozekcin et

al. 2015).

vii. A combination of a new MEWS chart and the Cape Town MEWS training

programme and manual enhanced recording of all parameters, and nurses’

knowledge, but not nurses’ responses to patients who triggered the MEWS

reporting algorithm. There is no evidence that MEWS improved patient clinical

outcomes. The authors concluded that MEWS did not replace clinical judgement

in detecting deteriorating patients (n=1; LOE 2+) (Kyriacos et al. 2015).

viii. Nurses who received simulation training (with other interventions including

posters, feedback sessions, face-to-face conversations and small posters) in

MEWS and SBAR could identify a patient who was deteriorating and react more

appropriately than nurses who were not trained. However, these tools were

rarely used one year post-implementation despite rigorous implementation of

this methodology, communication remained suboptimal and there was little

change in behaviour (n=1; LOE 2-) (Ludikhuize et al. 2011).

46 | P a g e

Implementation of EWS and/or RRT

Thirty four studies described and/or evaluated the implementation of EWSs and/or RRTs.

These were performed in the USA (n=5), UK (n=13), The Netherlands (n=3), Denmark (n=2),

Australia (n=3), Belgium (n=2), and one each in Sweden, Scotland, Singapore, and New

Zealand.

The level of evidence from these studies varied; LOE1++ (n=1), LOE1+ (n=1), LOE2+ (n=8),

LOE2- (n=18) and LOE3 (n=6).

I. For RRSs to be effective a ‘whole system’ approach should be adopted and aggregate

weighted scoring systems are more effective than single parameter systems. RRSs

are most effective for patients with predictable clinical decline than among patients

who are post-operative or acute haematology patients (n=1, LOE 1+)(McNeill and

Bryden, 2013) (n=1, LOE 1-) (Alam et al. 2014), (n=1, LOE 3) (Subbe & Welch 2013).

II. RRSs have a positive effect on cardiopulmonary arrest rates, but total hospital

mortality was not significantly reduced. Cultural systems problems, including

education and support for nurses, education and teamwork need to be addressed if

RRSs are to be effective in increasing detection of patient deterioration and reducing

preventable deaths. Also, optimum team composition remains to be elucidated (n=1,

LOE 1+)(Winters et al. 2013).

III. A National RRS consisting of MEWS, SBAR and an RRT implemented over 24 months

in the Netherlands was associated with a 15% adjusted risk reduction in the

composite endpoint of cardiopulmonary arrest, unplanned ICU admission, or death;

but MEWS/SBAR may not be effective on their own. The authors suggested

mandatory MEWS measurements and bedside patient evaluation by physician rather

than telephone consultations (n=1; LOE 2+) (Ludikhuize et al. 2015).

IV. An ethnographic study of the implementation of an RRS identified three themes

illustrating the nature of rescue work within the field and collective rules which

guided associated occupational distinction practices; (1) the ‘dirty work’ of vital sign

recording and its distinction from diagnostic (higher order) interpretative work; (2)

the moral order of legitimacy claims for additional help; and (3) professional

deference and the selective managerial control of rescue work (p. 233).

Responsibility for rescue is distributed across the whole organisation and is not

limited to the frontline staff (n=1; LOE 3) (Mackintosh et al. 2014).

V. A number of studies described the feasibility and successful implementation of

various EWS & RRT systems:

a. Implementation of a three-component clinical intervention: (i) MEWS, (ii) an

observational chart and (ii) an algorithm for bedside action, in medical and

surgical wards in one hospital, implemented by inter-professional teaching,

training and optimization of communication and collaboration, may have

significantly reduced unexpected in-hospital mortality. However, changes in the

hospital organisation occurred simultaneously and results from individual

47 | P a g e

elements of the intervention could not be elucidated (n=1; LOE 2-) (Bunkenborg

et al. 2014).

b. Adverse patient outcomes (mortality, cardiac arrests and transfer to ICU

following cardiopulmonary resuscitation [CPR]) decreased significantly in the four

years post implementation of MEWS charts with Critical Care Outreach Services

(CCOS) (introduced with educational programmes). This intervention had a

positive influence but other factors may have also impacted on outcomes and it

is difficult to quantify specific effects (n=1; LOE 2-) (Moon et al. 2011). Similar

results were observed following a protocolled multi-stage intervention in the

USA (n=1; LOE 2-) (Mathukia et al. 2015). The authors conclude that high levels of

adherence to MEWS is necessary to ensure effectiveness.

c. The staged implementation of an ED-specific RRS (with an escalation criteria and

a new observation chart) resulted in a decrease in the frequency of unreported

clinical deterioration. Each element of the RRS had an independent and

cumulative effect on the outcome but the effectiveness of the RRS plateaued and

ongoing strategies to ensure clinical engagement with the aims, structure and

function of RRSs are needed to maintain ongoing improvements in the reporting

of clinical deterioration (n=1; LOE 2-) (Considine et al. 2015).

d. Implementation of systems involving the (i) automatic identification of patients

at risk of clinical deterioration using EWS from existing EMR databases calculated

using ML algorithms, and (ii) the real-time detection of clinical event based on

real-time vital sign data collected from on-body wireless sensors network (WSN)

technology attached to those high-risk patients and (iii) automatic notification of

nurses through the hospital’s paging system was feasible (n=1; LOE 2+)

(Hackmann et al. 2011), as was the implementation of nurse led electronic EWS

(n=1; LOE 3) (Jones 2013). An association with risk of clinical deterioration i.e.

ICU transfer was observed (n=1; LOE 2+) (Hackmann et al. 2011).

e. A review also concluded that a multifaceted approach of EWSs that interface

with EMRs and are supplemented with decision aides and clinical support

systems produce an effective screening system for early identification of

deteriorating patients which decreased unplanned ICU admissions and hospital

mortality (n=1; LOE 1-) (Mapp et al. 2013).

f. A complex intervention (an intervention which included training, new

observation charts and a new track and trigger system) had a positive impact on

the self-assessed knowledge and confidence of registered and unregistered

nurses (n=1; LOE 2-) (McDonnell et al. 2013).

g. Introduction of a 5-component evidence-based care bundle, emergency

laparotomy pathway quality improvement care (ELPQuiC) which included EWS

use augmented by senior clinical input, sepsis and fluid algorithms led to a

significant reduction in risk-adjusted 30-day mortality following an emergency

laparotomy (n=1; LOE 2+) (Huddart et al. 2015).

48 | P a g e

h. An Electronic physiological surveillance system (EPSS) specifically designed to

increase the reliability of the collection, calculation of EWS, documentation and

display of vital signs in hospital was associated with a significant reduction in

hospital mortality. The results being mirrored in two hospitals support this claim

i.e. a reduction of 7.75% (2168/27 959) to 6.42% (1904/29 676) in one hospital

(estimated 397 fewer deaths), and from 7.57% (1648/21 771) to 6.15% (1614/26

241) at the second (estimated 372 fewer deaths) (n=1; LOE 2+) ( Schmidt et al.

2015).

i. Introduction of SBAR increased perception of effective communication and

collaboration in nurses. Using SBAR items in patient records, nurses were more

prepared prior to calling a physician and more able to make recommendations

based on thorough assessment, there was a shift towards earlier detection,

trigger and response, a decrease in unexpected deaths and an increase in

unplanned ICU transfers (n=1, LOE 2+) (De Meester et al. 2013b).

j. Introducing a standard nurse observation protocol with a MEWS score calculated

after ICU discharge increased the observation frequency, particularly in the non-

monitoring areas, and decreased the number of SAEs. The effect was obtained

by implementing the afferent limb of the RRS without introducing a RRT apart

from the existing cardiac arrest team (n=1, LOE 2+) (De Meester et al. 2013a).

k. Vital signs and MEWS determination three times daily, resulted in better

detection of physical abnormalities, significantly more frequent call outs and

more reliable activation of RRT. A trend towards a decrease in adverse effects

was also observed, especially in the protocol wards where MEWS was calculated

regularly (n=1, LOE 2-) (Ludikhuize et al. 2014).

l. In the UK, evaluation from a three-phase implementation found evidence of

consultant involvement in only 51% of cases of adult patients. In the sickest adult

patients, observations often improved following initial medical intervention and

that early review within working hours may prevent deterioration and need for

escalation of the out of hours service. Solutions included monthly audit results

communicated verbally and via e-mail to each ward in addition to placing them

on performance boards, staff dedicated to supporting the implementation and

tracking of the EWS. Obtaining staff engagement and involving clinical champions

at the beginning and throughout the process contributed to the sustained

improvement especially amongst nursing staff, as did identifying and defining

ward culture and a no-blame approach whilst maintaining transparency in order

to maximise learning from case reviews (n=1, LOE 2-) (Wood et al. 2015).

m. In the US, initial results post implementation of RRT showed no difference.

However, results in the second-year post-implementation of RRT revealed a

decrease in the failure to rescue (FTR) measure as well as an increase in the

unplanned ICU transfer rate occurring corresponding to an increase in the

49 | P a g e

number of RRT calls per month. Effect of RRT was not visible in terms of patient

level data until 18 months post intervention (n=1, LOE 2-) (Moriarty et al. 2014).

n. A qualitative study investigating the experiences of using an MET, four

qualitative themes included: (1) sensing clinical deterioration; (2) resisting and

hesitating; (3) pushing the button; and (4) support and leadership. The authors

concluded that recognising, and managing the deteriorating patient is complex

and challenging (n=1) (Massey et al. 2014).

o. In hospitals is one NHS Trusts there was a non-statistically significant decrease in

mortality four years post-implementation of MEWS with CCO service in a trauma

unit. MEWS may not be sensitive enough to identify physiological deterioration

in the elderly orthopaedic patients (n=1, LOE 3) (Patel et al. 2011).

VI. Errors including incomplete documentation and incorrect scoring of

EWS/SEWS/MEWS are common and are associated with missed alerts particularly

when a patient first becomes unstable and at night. Missed alerts are more common

when previous observation sets are incorrect suggesting that clinical staff

‘outperform EWS’ by detecting changes before the EWS system by using information

not encoded within it. There is a need to understand how to capture this

information, including clinician concern, and incorporating it into future EWS is

important”(n=2, LOE 2-) (Clifton et al. 2015) (Ludikhuize et al. 2012)(n=1., LOE 2+)

(Gordon & Beckett 2011), (n=1, LOE 3)(Hands et al. 2013). In a successful

multicomponent EWS implementation in Denmark, the main component which was

lacking was the documentation of the actions resulting from an abnormal MEWS

(n=1, LOE 2-) (Niegsch et al. 2013).

VII. Similar results were found for ViEWS, with large variations in observational sampling

frequency throughout the day, with identical patterns every day, and lower

frequency at night. Sicker patients had higher frequency of vital sign observations

during the day and at night, but less than that recommended by the escalation

protocol. The pattern of vital sign monitoring is likely to impact on the effectiveness

of RRT activation. Therefore, despite clear protocols adherence was low (n=1, LOE 3)

(Hands et al. 2013).

VIII. The optimum threshold value for a site-specific EWS was ≥3. Lower thresholds

resulted in lower sensitivity, and an increased workload, at the risk of making staff

less cautious (n=1, LOE 2-)(van Rooijen et al. 2013).

IX. The optimum threshold value for a site-specific EWS was ≥3 among surgical patients

(n=1, LOE 2+)(Smith et al. 2012).

50 | P a g e

X. Use of handheld electronic devices may assist in the improvement of EWS

documentation and a decrease in errors (n=1, LOE 3) (Subbe & Welch 2013).

XI. The most widely used weighted track-and-trigger scores did not offer good

predictive capabilities for use as criteria for an automated alarm system. For the

implementation of an automated

alarm system, better criteria need

to be developed and validated

before implementation (p 549)

(n=1, LOE 2-) (Romero-Brufau et al.

2014).

XII. Clinical judgement of patient

stability can be reliably quantified

with Patient Acuity Rating (PAR).

Implementation of PAR could

improve the communication regarding at risk patients between healthcare

professionals during handoffs (n=1, LOE 2-) (Edelson et al. 2011). PAR increases the

accuracy of MEWS (n=1, LOE 2-) (Patel et al. 2015b).

XIII. “Score to Door” Time seemed to be largely independent of illness severity and, when

combined with qualitative feedback

from centres, suggests that admission

delays could be due to organisational

issues, rather than patient factors.

Score to Door Time could act as a

suitable benchmarking tool for Rapid

Response Systems and help to

delineate avoidable organisational

delays in the care of patients at risk of

catastrophic deterioration” (p1) (n=1,

LOE 2-) (Oglesby et al. 2011).

XIV. The Multidisciplinary Audit and

Evaluation of outcomes of rapid

response (MAELOR) tool can be used

to classify RRT episodes with readily

available information, identify and

target areas performing sub-

optimally and facilitate comparison of

RRTs (n=1, LOE 2-) (Morris et al.

2013).

A ‘whole system’ approach incorporating a

EWS, well designed chart, ISBAR

communication tool, decision aides,

evidence based care bundles, RRT, bedside

evaluation, education, reinforcement and

audit is most effective at identifying and

responding to deteriorating patients.

The implementation of the EWS as a system

requires significant time and effort as the

ecosystem (environment, processes, culture,

teamwork) need to be carefully considered

and studied. A high level of adherence to

EWS is necessary to ensure effectiveness.

However, when implemented as a whole it

is difficult to assess the impact of each

element on patient outcomes.

Time period between the NEWS trigger and

arrival to the patient (Score to Door Time)

could act as a suitable benchmarking tool

for Rapid Response Systems and help to

delineate avoidable organisational delays.

51 | P a g e

Mortality as an Outcome Measure

One of the key outcomes that a health system is concerned with is mortality. The

implementation of an early warning system seeks to identify clinical deterioration early and

thus reduce adverse outcomes. The following is the evidence which highlights the impact of

early warning systems on mortality (retrospectively) and the association of EWS scores with

mortality (i.e. predictive of mortality). Papers sourced which helped to answer this question

were: LOE 1++ (n=1), LOE 1+ (n=1), LOE 1- (n=2), LOE 2++ (n=1), LOE 2+ (n=6), LOE 2- (n=17),

LOE 3 (n=14) (Figure 6).

Figure 6. Diagrammatic representation of the numbers of papers, sourced categorised by

Level of Evidence i.e. LOE), in relation to mortality

LOE 1++

(Re colour scheme-please note foot note below22)

Winters et al. (2013) in a systematic review (n=43 included papers) sought to

establish the effectiveness and implementation of RRSs (Rapid Response System) in

acute care settings. Conflicting effects of RRS on mortality were reported. RRS was

associated with a reduced but not statistically significant impact on rates of

mortality, ((n=1 SR); RR 0.96 (95%CI 0.84, 1.09)). RRS had a positive effect on total

hospital mortality (n=18 of 23 studies; 7 of these were significant). Benefit of RRS

may depend on the patient population.

22

A colour coding system is applied to visually represent the LOE of pertinent narrative synthesis (light green equates to LOE1, light orange LOE2 and light grey LOE 3).

0 5 10 15 20

LOE1++

LOE1+

LOE1-

LOE2++

LOE2+

LOE2-

LOE3

52 | P a g e

LOE 1+

McNeill & Bryden (2013) in a systematic review of 43 studies noted that there was

no evidence that the implementation of a single parameter EWS triggering system

alone has a positive effect on hospital survival; no change (n=1) or a reduction were

observed (n=2). In contrast the introduction of aggregate weighted scoring systems

was associated with improved hospital survival/reduction in hospital mortality (n=2),

or no change (n=1). MET teams introduction was associated with a decrease in

mortality/improved survival (n=9) or no difference (n=4). Multidisciplinary outreach

team introduction was associated with a decrease in mortality/improved survival

(n=3) or no difference (n=3).

LOE 1-

Alam et al. (2015) in a systematic review of 7 studies encompassing 486,237 patients

found conflicting results in six of the studies that investigated the effect of the

introduction of EWSs on mortality as the endpoint finding. These included: a

significant reduction in in-hospital mortality (n=2); a trend toward decreased

mortality (n=2); and no significant differences in in-hospital mortality (n=2) following

the introduction of EWSs. Authors noted that in general there was a positive trend

towards better clinical outcomes (improved survival, lower ICU mortality and a

decrease in SAE numbers) after the introduction of a EWS system.

Mapp et al. (2013) in a review examined EWSs and the incorporation of clinical

support on their effectiveness in predicting a patient’s potential for deterioration

and whether they prevent unplanned ICU admissions and/or death. A post-EWS

implementation decrease in-hospital mortality was observed (n=2).

LOE 2++

Peris et al. (2012) using a prospective, before-and-after intervention study to

determine if MEWS calculation can help the anaesthetist select the correct level of

care to avoid inappropriate admission to the ICU and to enhance the use of the HDU

after emergency surgical procedures; patients with a MEWS of 3 or 4 were

transferred to the HDU, whereas a MEWS score of ≥5 was considered criteria for ICU

admission. Mortality rate analysis did not differ between the two groups.

LOE 2+

Alam et al. (2015) in a Dutch prospective observational study (ED patients (n=274))

found that 30 day mortality was significantly correlated with NEWS on arrival (T0),

an hour after arrival (T1) and at transfer to the general ward/ICU (T2) (P<0.05). T0

AUROC = 0.768 (95% confidence interval (CI) 0.618, 0.919); T1 AUROC = 0.867 (95%

CI 0.769, 0.964); T2 AUROC = 0.767 (95% CI 0.568, 0.916).

Schmidt et al. (2015) utilising a retrospective, observational, pre-post intervention

design sought to establish if the introduction of EPSS would reduce hospital

mortality. During implementation of EPSS across site 1 and site 2 respectively, crude

mortality fell from 7.75% (2168/27,959) at baseline (2004) to 6.42% (1904/29 676)

53 | P a g e

after implementation (2010) (p<0.0001), with an estimated 397 fewer deaths i.e.

7.57% (1648/21 771) at baseline (2006) to 6.15% (1614/26 241) (2010) (p<0.0001)

(estimated 372 fewer deaths).

Tirkkonen et al. (2014) in a prospective point prevalence trial using prospective data

(n=615 patients on general wards) found that NEWS score ≥ 5 or if the weighted

score for any individual vital sign was 3 was associated with an increased odds of

mortality at 30 and 60 day: NEWS score ≥5 30-day mortality: OR 11.8 (95%CI 4.26,

32.6); NEWS score ≥7 30-day mortality: OR 11.4 (95%CI 4.40, 29.6); NEWS score ≥5

60-day mortality OR 5.55 (95%CI 2.91–10.6); NEWS score ≥7 60-day mortality: OR

6.42 (95%CI 2.92, 14.1).

Mora et al. (2015) in a retrospective, case-controlled study found that patients who

trigger an RRT call within 24 h are at 4-times higher risk of in-hospital mortality.

Cases who triggered an RRT within 24 h had a significantly higher odd of death; OR

4.65 (95%CI 1.86, 11.65). Patients at higher risk of triggering an RRT activation could

be identified through higher RR and HR in the ED.

Ludikhuize et al. (2015) in a pragmatic, prospective multicentre, before and after

trial sought to describe the effect of implementation of a RRS on the composite

endpoint of cardiopulmonary arrest, unplanned ICU admission, or death. There was

a significant reduction in in-hospital mortality per 1000 admitted patients post-

implementation compared to pre-implementation (adjusted OR=0.802 (95%CI 0.644,

1.000); p=0.05).

Huddart et al. (2015) compared risk-adjusted 30-day mortality after emergency

laparotomy pre- and post-implementation of the ELPQuiC bundle including EWS and

escalation. There was a significant 3.5% (95%CI -1.4, 8.4) decrease in overall crude

30-day mortality post-implementation (P=0.152): Pre-implementation: 14.0% (95%CI

10.1, 18.0%); Post-implementation: 10.5% (95%CI 7.6, 13.5%).

LOE 2-:

Abbott et al. (2015) using a prospective, observational cohort study design (n=431;

n=16 met the primary outcome) found that NEWS was more strongly associated with

the composite endpoint of critical care admission or death within the first 48hours of

the hospital stay than PARS; NEWS OR 1.54, (95% CI 1.26–1.91, p <0.001). Every one-

point increase in NEWS was associated with a 55% increased risk. Analysis of

individual NEWS thresholds identified that a score of ≥3 was associated with the

composite end point.

Cattermole et al. (2014) in a prospective observational study (n=234 ED patients)

validation study which compared the performance of various EWS found that AUROC

for each EWS for predicting the composite output of ICU admission or death within 7

days was: THERM: 0.84 (95% CI 0.786 to 0.884); Worthing 0.78 (95% CI 0.72-0.83);

MEES: 0.75 (95% CI 0.69-0.80); PEDS: 0.75 (95% CI 0.69-0.80); MEWS: 0.73 (95% CI

0.67-0.79); NEWS: 0.71 (95% CI 0.64-0.76); REMS: 0.70 (95% CI 0.64-0.76); SCS: 0.70

(95% CI 0.64-0.76); MEDS: 0.59 (95% CI 0.52-0.6). THERM (which included 3 items:

54 | P a g e

GCS (Glasgow Coma Scale), HCO3– and SBP) outperformed NEWS in both derivation

and validation in patient datasets. NEWS is entirely physiological, not requiring

bedside blood tests. Thus the scoring of some of the parameters in NEWS limits its

usefulness in the ED e.g. use of supplemental O2 scores highly, which is reasonable

for stable inpatients. But many ambulance and resuscitation-room patients are

routinely given oxygen initially, with subsequent titration or removal. In addition

NEWS does not discriminate between degrees of reduced consciousness.

Corfield et al. (2014) using a prospective national audit (n=2,003 patients with sepsis)

noted that for each rise in NEWS category there was an associated increased risk of

mortality when compared to the lowest category: for 30-day mortality, the age-

adjusted ORs for NEWS categories compared to the baseline category (≤4) 5–6: OR

1.95, 95% CI 1.21 to 3.14 (p=0.01); 7–8: OR 2.26, 95% CI 1.42 to 3.61 (p<0.00); 9–20:

OR 5.64, 95% CI 3.70 to 8.60 (p<0.00). NEWS AUROC characteristics for the

combined endpoint of ICU and/or mortality were NEWS: 0.71; AUROC age-adjusted

NEWS: 0.70. The positive predictive value illustrates that 27% of patients with a

NEWS of 7 were admitted to the ICU within 2 days and/or died within 30 days. For a

NEWS of 9 this rose to 35%.

Eccles et al. (2014) in a prospective, observational cohort study (n=196 admissions of

whom 78 had chronic hypoxia) found that 30-day mortality for chronic hypoxia

patients: NEWS AUROC during stability/at discharge = 0.876 (95%CI 0.788, 0.963);

CREWS AUROC during stability/at discharge = 0.913 (95%CI 0.845, 0.981). Mean

NEWS score was consistently higher for patients with chronic hypoxia (6 ± 3). The

percentage of patients with chronic hypoxia reaching triggering thresholds with

NEWS, was higher than the percentage using CREWS at two thresholds (≥5 and ≥6)

when patients were stable.

Dawes et al. (2014) using a prospective observational study, post-intervention

(introduction of The Worthing physiological scoring system) (n=3,184 patients):

hospital mortality decreased from 8.3% to 5.2% over 5 years post-intervention

(P=0.29, NS). Prediction of death within 72 hours: Worthing PSS AUROC (2010): 0.74

(95% CI: 0.69, 0.78); NEWS AUROC: 0.76, (95% CI: 0.72, 0.80). Worthing PSS ≥2 was

associated with a mortality of >10%; NEWS ≥4 was associated with a mortality of

>10%. The predictive performance of the Worthington PSS “was not enhanced by

the addition of biochemical variables and co-morbidities” (p.603).

Alvarez et al. (2013) using a retrospective cohort study sought to derive and validate

an automated prediction model based on near real-time electronic medical record

(EMR) data to identify patients at high risk of a composite outcome of resuscitation

events and death (RED) outside of ICU. The automated clinical prediction model had

good discriminatory for the prediction of RED events, and was significantly better

than MEWS; Derivation dataset AUROC curve =0.87 (95%CI 0.85, 0.89); Validation

dataset AUROC=0.85 (95%CI 0.82-0.87), MEWS AUROC=0.75 (95%CI 0.71-0.78). The

55 | P a g e

Automated clinical prediction model outperformed MEWS and human judgement-

based RRT, in predicting SAEs.

Churpek et al. (2014b) developed and validated an electronic risk score Cardiac

Arrest Risk Triage (eCART) score using commonly collected EHR data in an

observational cohort study. Predictor variables were vital signs (temperature, heart

rate, blood pressure, respiratory rate, oxygen saturation), mental status [AVPU] and

laboratory results (white cell count, haemoglobin, platelets, sodium, potassium,

chloride, bicarbonate, anion gap, blood urea nitrogen, creatinine, glucose, calcium,

total protein, albumin, total bilirubin, aspartate aminotransferase, alanine

aminotransferase, and alkaline phosphatase), these and age were obtained

electronically. (Final model components not stated (16-item from another paper)).

eCART cut-off of ≥50 would detect; 51% cardiac arrests, 44% ICU transfers, and 83%

deaths. Death: MEWS AUROC: 0.88 (95%CI 0.88-0.88); eCART AUROC: 0.93 (95%CI

0.93-0.93).

Ong et al. (2012) using a prospective, non-randomised, observational cohort study

design sought to validate a novel ML score, incorporating HRV against MEWS for the

risk stratification of critically ill patients in the ED. The ML score had better

discriminatory ability to predict in-hospital death within 72 h than MEWS; ML score

AUROC=0.741; MEWS AUROC=0.693. In the low, intermediate and high risk ML score

groups, the rate of death within 72 hours increased from 2.3%, to 29.1% and 68.6%,

respectively. A cut-off ML score ≥ 60 predicted death within 72 h with a:

Sensitivity=69.8%, Specificity=73.9%, PPV=21.5%, and NPV=96.0%.

Dundar et al. (2015) evaluated the utility of MEWS and ViEWS in predicting both

hospitalisation and in-hospital mortality among geriatric patients presenting to the

ED with non-traumatic acute surgical or medical diseases. In-hospital mortality rate

of elderly patients presenting to the ED was 8.5%. In-hospital mortality was

predicted by both MEWS and ViEWS, with similar high discriminatory ability (not

statistically different). Optimal cut-off scores were 4 and 8 for MEWS and ViEWS,

respectively: MEWS AUROC=0.891 (95% CI 0.844, 0.937); ViEWS AUROC=0.900 (95%

CI 0.860, 0.941).

Geier et al. (2013) in a prospective observational study investigated the diagnostic

and prognostic accuracy of the ESI, MEWS and MEDS regarding severe sepsis and

septic shock (SSSS) for patients presenting to the ED. MEDS had the highest in-

hospital 28-day mortality of patients with suspected sepsis (Prognostic accuracy: ESI

AUROC=0.617 (95%CI 0.479, 0.755); MEWS AUROC=0.642 (95%CI 0.517, 0.768);

MEDS AUROC=0.871 (95%CI 0.796, 0.945); CCI: AUROC=0.673 (95%CI 0.558, 0.787)).

A high MEWS score was not associated with mortality in this patient group.

Ghanem-Zoubi et al. (2011) in a prospective study compared the prognostic value of

4 scoring systems in patients with sepsis upon admission to general internal

medicine departments and found overall in-hospital mortality prognostic value;

MEDS AUROC=0.73 (95%CI 0.70, 0.77); REMS AUROC=0.77 (95%CI 0.73, 0.80); MEWS

56 | P a g e

AUROC=0.69 (95%CI 0.65, 0.73); SCS AUROC=0.77 (95%CI 0.74, 0.80). The best 28-

day mortality was predicted by REMS and SCS.

Jo et al. (2013) investigated (i) whether the predictive value of ViEWS in unselected

critically ill patients could be improved by including rapid lactate levels (ViEWS-L).

The ViEWS-L score had significantly better predictive value than the ViEWS, HOTEL

and APACHE II scores for the four mortality outcomes. Hospital mortality (p=0.009):

ViEWS-L AUROC=0.802 (95%CI 0.729, 0.875); ViEWS AUROC=0.742 (95%CI 0.661,

0.823; p=0.009); HOTEL AUROC=0.662 (95%CI 0.577, 0.747; p<0.001); APACHE II

AUROC=0.689 (95%CI 0.577, 0.747; p=0.024); SAP II AUROC=0.799 (95%CI 0.726,

0.872; p=0.944); SAP III AUROC=0.803 (95%CI 0.729, 0.878; p=0.972). Addition of

lactate measurement can increase sensitivity of VIEWS in predicting mortality.

Moseson et al. (2014) compared the ability of ICU and ED scoring systems to predict

mortality within 60 days in critically ill patients. There were significant differences in

the discriminatory ability of ED and ICU scoring systems (P=0.01), with ICU scores

outperforming the ED scoring systems regarding 60-day mortality and in-hospital

mortality.

Bunkenborg et al. (2013) evaluated the “short and long term effects of systematic

inter-professional use of EWS, structured observational charts, and clinical

algorithms for bedside action” on unexpected in-hospital death (i.e. sudden, no

resuscitation), death after cardiopulmonary resuscitation, and death within 24h of

admission for intensive care) death. Unexpected patient mortality decreased post-

intervention: 25 (1st post-intervention) vs 61 (pre-intervention) per 100 adjusted

patient years (P=0.053), rate ratio: 0.404 (95%CI 0.161-1.012); 17 (2nd post-

intervention) vs 61 (pre-intervention) per 100 adjusted patient years (P=0.013), rate

ratio: 0.271 (95%CI 0.097-0.762). The number of SAEs decreased post-intervention

(n=21) compared to pre-intervention (n=31).

Mathukia et al. (2015) described the experience and impact of implementing MEWS

in a protocolised way. The overall mortality decreased post-implementation

compared to pre-implementation; 2011: 2.3%, 2012: 2.0%, 2013: 1.5%, 2014: 1.2%.

There was a significant increase in RRT calls post-implementation, compared to pre-

implementation (P<0.01): 2011: 0.24 per 100 PD, 2012: 0.25 per 100 PD, 2013: 0.38

per 100 PD, 2014: 0.48 per 100 PD. There was a non-significant trend towards better

survival of non-ICU ‘Code Blues’ since MEWS implementation: 2011: 65%, 2012:

43%, 2013: 65%, 2014: 71%.

Moon et al. (2011) determined whether cardiac arrest calls, the proportion of adult

patients admitted to ICU after CPR and their associated mortalities were reduced in

the 4 year period after the introduction of a 24/7 CCOS and MEWS Charts. There was

a significant 7.1% decrease in in-hospital mortality decreased post- (697/year)

compared to pre-implementation (750/year; P<0.0001). Deaths per adult admission

decreased significantly post- (1.2%) compared to pre-implementation (1.4%;

57 | P a g e

P<0.0001). There was a significant decrease in in-hospital mortality following ICU

admission post-implementation (42% v 52%, P=0.05).

Moriarty et al. (2014) sought to determine the effect of RRT implementation on

Failure To Rescue (FTR). Overall hospital mortality in the pre RRS implementation

period was 1.5% compared with 1.6% in the full post-implementation period (P =

0.299). No significant decreases were observed pre- and post-implementation for

cardiopulmonary resuscitation events. Initial results post implementation of RRT

showed no difference. However, results in the second-year post-implementation of

RRT revealed a decrease in the FTR measure as well as an increase in the unplanned

ICU transfer rate corresponding to an increase in the number of RRT calls per month.

Effect of RRT was not visible in terms of patient level data until 18 months post-

intervention.

LOE 3

Jarvis, Kovacs, Briggs et al. (2015a) in a retrospective study of an electronic database

using 10,000 observation sets randomly selected for analysis to minimise any biases

found that for all outcomes an aggregate NEWS score of 5 was associated with a

significantly higher risk than that of an aggregate score of 3 (with one vital sign of 3);

risk of death and any adverse outcome was significantly higher for a NEWS score of 5

than an aggregate score of 4 or 3 (with one vital sign of 3). Odds of death increased

(almost doubled) with each increase of 1 point in the aggregate NEWS scores.

Where a single vital sign had a score of 3, the odds increased, but not significantly.

Authors’ noted that escalation of care to a doctor when any component of NEWS

scores 3 compared to when aggregate NEWS values ≥5, would have increased

doctors workload by 40% with only a small increase in detected adverse outcomes

from 2.99 to 3.08 per day (a 3% improvement in detection).” Jarvis, Kovacs, Briggs et

al. (2015c) also found that NEWS performed the best of all 35 EWSs when predicting

risk of death within 24 hours. The top three EWS using ‘All observations’ were NEWS

AUROC=0.898; MEWS AUROC=0.862; Worthing AUROC=0.861 and the lowest was

Centiles AUROC=0.783. Similarly Jarvis, Kovacs, Briggs et al. (2015b) noted that the

PPV and sensitivity of the standard weighted NEWS was better than binary NEWS;

NEWS aggregate score ≥5, sensitivity=69.7%, specificity=94.2%, PPV=11.8%,

NVP=99.6% and in contrast the binary NEWS score ≥3 , sensitivity=67.7%,

specificity=92.9%, PPV=9.6%, NVP=99.6%.

Jarvis, Kovacs, Briggs et al. (2013) used a retrospective approach to construct/test

LDT-EWS using a derivation dataset (n=3496 to 4093 approx. per discrete validation

set). Biochemical and haematology blood test parameters, with acceptable ranges

were included (Hb, WCC, U (0.4-107.1 mmol/L), Alb (10-70 g/L), Cr (8.8-2210

umol/L), Na (100-200 mmol/L) and K (1-15 mmol/L). A LDT-EWS score of 4 would

trigger a response in 40.7% of all laboratory test datasets; 79.7% of all patients

having a trigger would subsequently die. For all patients, the min and max AUROCs

58 | P a g e

were given but these differences are not statistically significant; Q9 AUROC= 0.801

(95%CI 0.776, 0.826); Q16 AUROC= 0.755 (95%CI 0.727, 0.783).

Badriyah et al. (2014) using a retrospective analysis of a database of vital signs i.e.

198,755 vital signs from 35,585 consecutive acute adult (>16 years) admissions

within 24 hours of a given vital sign observation, outcomes of DTEWS and NEWS

were similar for mortality: NEWS AUROC=0.894 (95% CI 0.887, 0.902); DTEWS

AUROC = 0.899 (95% CI 0.982, 0.907).

Cooksley et al. (2012) using retrospective analysis (n=840 patients reviewed by the

Outreach Team) found that both EWSs were significant in predicting 30-day

mortality: MEWS score = P=0.004; NEWS score = P=0.0003. Receiver operating

characteristic curves demonstrate that both scores have poor predictive

discriminatory value; the 30 day mortality AUROC score of MEWS equalled 0.60 and

for NEWS 0.62. Key EWS variables predictive of 30-day mortality were respiratory

rate (P=0.0001), temperature (P<0.0001).

Churpek at al. (2013) in a retrospective study of 59,643 hospital admissions found

that aggregate weighted scoring systems outperformed the other systems for most

outcomes, with the SEWS, MEWS, ViEWS, and CART score being the most accurate

for detecting cardiac arrest, mortality, ICU transfer and a composite of the three

outcomes. Single-parameter scoring systems had the lowest predictive accuracy.

AUROCs were highest for mortality: SEWS, VIEWS, and CART were similar in their

prediction of mortality (AUROC=0.88 for all). CART score was best for predicting

cardiac arrest: AUROC=0.83 (95%CI 0.79-0.86).

There were 47 deaths (6.6%) in the MEWS <4 group as compared with 53 (17.0%)

deaths in the MEWS ≥4 group (P<0.001); MEWS Score cut off were poor predictors

of mortality; MEWS <4 AUROC=0.68; MEWS <5 AUROC=0.66 (Ho et al. 2013).

Kim et al. (2015) in a retrospective analysis of prospectively collected data found that

MEWS (adjusted for age and sex) was associated with in-hospital mortality at each

time point prior to cardiac arrest (P=0.01): MEWS(24 h prior to cardiac arrest)

OR=1.14 (95%CI 1.17, 1.70); MEWS (16 hours) OR=1.14 (95%CI 1.17, 1.70); MEWS (8

hours) OR=1.23 (95%CI 1.07, 1.40).

Stark et al. (2015) in a retrospective study investigated the ability of a MEWS to

identify patients at higher risk of death. In multivariate analysis, Max MEWS was a

significant predictor of in-hospital death: maximum MEWS OR=1.39, 95% CI 1.04,

1.85; P=0.025), AUROC=0.7827; MEWS score ≥5 OR=1.39.

Bleyer et al. (2011) in a longitudinal analysis of retrospective data to examine the

association of EWS scores occurring at any time during the hospitalisation with

mortality using MEWS, ViEWS and authors created a ‘Critical vital sign’ scoring

system with different weightings for age. 1 vs 3 simultaneous critically abnormal vital

sign, with increasing age, was associated with a 19-fold increase in mortality (all

ages: 0.92% and 23.6%: age>70 years, 0.62 and 42%, respectively. VIEWS detected

more deaths at the same sensitivity as the ‘Critical vital sign’ scoring system.

59 | P a g e

Liljehult and Christensen (2015) in a retrospective cohort study found that a EWS

created from readily available physiological parameters is a simple and valid tool for

identifying patients at low, intermediate and high risk of dying after acute stroke.

Mortality at 30 days increased with EWS score on admission, but only those in the

high risk group had a significantly high mortality rate than the overall mortality rate

(P values); low (EWS 0-1): 2.4% mortality

rate (P=0.004); medium (EWS 2-4); 14%

mortality rate (P=0.56); high (EWS ≥5);

64% mortality rate (P<0.001).

Churpek et al. (2012a) compared the

efficacy of multiple risk scoring systems

to predict adverse events. AUROCs were

highest for mortality. SEWS, VIEWS, and

CART were similar in their prediction of

mortality (AUROC=0.88 for all). MERIT (Medical Early Response Intervention and

Therapy) AUROC = 0.74 (95%CI 0.71-0.76); Modified MERIT = 0.79 (95%CI 0.76-0.81);

Multiple parameter (Bleyer) = 0.84 (95%CI 0.82-0.87); Centile-based system = 0.83

(95%CI 0.80-0.86); MEWS = 0.87 (95%CI 0.84-0.89); SEWS = 0.88 (95%CI 0.86-0.90);

VIEWS = 0.88 (95%CI 0.86-0.91); CART Score = 0.88 (95%CI 0.86-0.90).

Smith et al. (2013) tested the ability of NEWS to discriminate patients at risk of

cardiac arrest, unanticipated ICU admission or death within 24h of a NEWS value and

compared its performance to that of 33 other EWSs, using the AUROC curve and a

large vital signs database. The AUROCs for NEWS for death within 24 h, 0.894 (95%CI

0.887–0.902). Similarly, the ranges of AUROCs (95% CI) for the other 33 EWSs were

0.813 (95%CI 0.802–0.824) to 0.858 (95%CI 0.849–0.867) (death).

Patel et al. (2011) sought to investigate the effect of the implementation of MEWS

with a critical Care Outreach service on mortality in a trauma unit. The in-hospital

mortality rate decreased for all patients, males and females post-implementation

compared to pre-implementation, but not statistically significantly.

In summary Is an aggregate weighted scoring systems (when compared with single parameter

systems) was associated with improved hospital survival/reduction in hospital mortality?

Aggregate weighted scoring systems (when compared with single parameter systems) were

associated with improved hospital survival/reduction in hospital mortality: (LOE 1+ (n=1,

McNeill & Bryden, 2013; LOE 3 (n=2, Churpek et al. 2012b; 2013).

How does the introduction of an Early Warning System impact on mortality as an

outcome?

If focusing on evaluating the impact of introducing EWSs on mortality as the endpoint, then

the studies reviewed found conflicting results: LOE 1- (n=2, Mapp et al. 2013; Alam et al.

Using mortality as an outcome:

aggregate weighted scoring systems

performed better when compared

with single parameter systems.

60 | P a g e

2015): evidence from a systematic review found a significant reduction in in-hospital

mortality (n=2 papers); a trend toward decreased mortality (n=2 papers); and no significant

differences in in-hospital mortality (n=2 paper) following the introduction of EWSs) (Alam et

al. 2015). A review assessing the impact of EWS implementation found that in two papers

decreases in-hospital mortality were observed (n=2 papers) (Mapp et al. 2013).

LOE 2++ (n=1, Peris et al. 2012): mortality rate analysis did not differ according to MEWS (3

or 4) and ≥5 groups.

LOE 2+ (n=1, Schmidt et al. 2015): crude mortality fell over 1% following implementation of

EWS (p<0.0001).

LOE 2- (n=4, Moon et al. 2011; Dawes et al. 2014; Bunkenborg et al. 2013; Mathukia et al.

2015). Dawes et al. 2014 found that hospital mortality decreased from 8.3% to 5.2% over 5

years post EWS intervention (P=0.29, NS), prediction of death within 72 hours: Worthing PSS

AUROC (2010): 0.74 (95% CI: 0.69, 0.78); NEWS AUROC: 0.76, (95% CI: 0.72, 0.80).

Bunkenborg et al. (2013) found that the use of systematic inter-professional EWS,

structured observational charts, and clinical algorithms for bedside action led to a reduction

in unexpected patient mortality (P=0.013, rate ratio: 0.271 (95%CI 0.097-0.762). The overall

mortality decreased post-implementation compared to pre-implementation of MEWS

(Mathukia et al. 2015). A significant 7.1% decrease in in-hospital mortality in the 4 year

period after the introduction of a 24/7 CCOS and MEWS Charts (Moon et al. 2011).

LOE 3 (n=1, Patel et al. 2011): the implementation of MEWS with a critical Care Outreach

service reduced in-hospital mortality rate but not statistically significantly.

Are Early Warning Systems predictive of mortality?

Studies seeking to answer this question focus on reviewing the effectiveness of EWSs and

correlating EWS to mortality (i.e. modelling the

patient level data). These studies found:

The 30-day mortality was significantly correlated

with NEWS at various time points (LOE 2+ (n=1,

Alam et al. 2015) with AUROC varying from 0.767

(95% CI 0.568, 0.916) to 0.867 (95% CI 0.769,

0.964); NEWS was significant in predicting 30-

day mortality (LOE 3 (n=1, Cooksley et al. 2012).

The 30-day mortality ROC score of MEWS

equalled 0.60 and for NEWS 0.62 (Cooksley et al.

2012).

NEWS was more strongly associated with the composite endpoint of critical care admission

or death within the first 48hours of the hospital stay than PARS (NEWS OR 1.54, (95% CI

1.26–1.91, p <0.001), LOE 2, n=1, Abbott et al. 2015). NEWS is associated with the

composite endpoint of critical care admission or death within the first 48hours of the

Higher NEWS scores can be

correlated with increased mortality

as measured at various time

points; every one-point increase in

NEWS is associated with an

increased risk of mortality.

61 | P a g e

hospital stay (LOE 2-, n=1, Abbott et al. 2015); the composite output of ICU admission or

death within 7 days AUROC 0.71 (95% CI 0.64-0.76) (LOE 2-, n=1, Cattermole et al. 2014).

Mortality within 24 hours of a given vital sign observation (LOE 3, n=2, Badriyah et al. 2014;

Cooksley et al. 2012). The ability of NEWS to discriminate patients at risk of cardiac arrest,

unanticipated ICU admission or death within 24h of a NEWS value was compared to that of

33 other EWSs, the AUROCs for NEWS for death within 24 h, 0.894 (95%CI 0.887–0.902)

(LOE 3, (n=1, Smith et al. 2013).

NEWS score ≥ 5 or if the weighted score for any individual vital sign is 3 was associated with

an increased odds of mortality at 30 and 60 day (LOE 2+,n=1, Tirkkonen et al. 2014). Risk of

death and any adverse outcome was significantly higher for a NEWS score of 5 than an

aggregate score of 4 or 3 (with one vital sign of 3) (LOE 3, n=1, Jarvis, Kovacs, Briggs et al.

2015a). Every one-point increase in NEWS was associated with an increased risk of mortality

(LOE 2- , n=2, Abbott et al. 2015; Corfield et al. 2014).

Some conflicting results on the mortality predictive value of MEWS (LOE 3, n=2 predictive,

Kim et al. 2015; Stark et al. 2015; n= 1 not predictive, Ho et al. 2013) was noted. VIEWs

predictive of mortality (LOE 3, n=1; Bleyer et al. 2011). LOE 3, (n=1, Churpek et al. 2012b)

SEWS, VIEWS, and CART were similar in their prediction of mortality (AUROC=0.88 for all).

62 | P a g e

What is the impact of the NEWS on mortality? In terms of NEWS: LOE 2+ (n=1, 30 day mortality was significantly correlated with NEWS); LOE 2+ (n=1, NEWS score ≥ 5 or if the weighted score for any individual vital sign is 3 was associated with an increased odds of mortality at 30 and 60 day); LOE 2- (n=1, NEWS was more strongly associated with the composite endpoint of critical care admission or death within the first 48 hours of the hospital stay than PARS, every one-point increase in NEWS was associated with a 55% increased risk). LOE 2- (n=1, the performance of various EWS found that AUROC for each EWS for predicting the composite output of ICU admission or death within 7 days was: THERM (which included 3 items: GCS, HCO3– and SBP): 0.84 (95% CI 0.786 to 0.884); NEWS: 0.71 (95% CI 0.64-0.76); 30-day mortality for chronic hypoxia patients: LOE 2- (n=1, NEWS AUROC during stability/at discharge = 0.876 (95%CI 0.788, 0.963); CREWS AUROC during stability/at discharge = 0.913 (95%CI 0.845, 0.981); LOE 2- (n=1) NEWS prediction of death within 72 hours, NEWS AUROC: 0.76, (95% CI: 0.72, 0.80), NEWS ≥4 was associated with a mortality of >10%; LOE 3 (n=1, Odds of death increased (almost doubled) with each increase of 1 point in the aggregate NEWS scores); LOE 3 (n=1, NEWS performed the best of all 35 EWSs when predicting risk of death within 24 hours); LOE 3 (n=1, PPV and sensitivity of the standard weighted NEWS is better than binary NEWS); LOE 3 (n=1 , NEWS significant in predicting 30-day mortality: MEWS score = P=0.004; NEWS score = P=0.0003); LOE 3 (n=1, The AUROCs for NEWS for death within 24 h, 0.894 (95%CI 0.887–0.902)). NEWS is entirely physiological not requiring bedside blood tests. Thus the scoring of some of the parameters in NEWS limits its usefulness in the ED e.g. use of supplemental O2 scores highly, which is reasonable for stable inpatients. But many ambulance and resuscitation-room patients are routinely given oxygen initially, with subsequent titration or removal. In addition, NEWS does not discriminate between degrees of reduced consciousness. NEWS score is consistently higher for patients with chronic hypoxia reaching triggering thresholds even when patients are stable.

Length of Stay (LOS)

Fourteen studies described and/or evaluated the effect of the implementation of EWSs on

hospital, ED or ICU LOS. These were from the UK (n=5), Netherlands (n=2), USA (n=2),

Belgium (n=1), Singapore (n=2), Sweden (n=1) and Australia (n=1).

The level of evidence of these studies varied: LOE 1+ (n=1), LOE 1- (n=1), LOE 2++ (n=1), LOE

2+ (n=5), LOE 2- (n=4), LOE 3 (n=2).

LOE 1+

Introduction of aggregate weighted scoring systems was associated with reduced

LOS (n=1). MET teams introduction was not associated with reduced LOS (n=1). MDT

outreach team introduction was associated with a possible increase in LOS (n=1)

(McNeill, Bryden (2013).

63 | P a g e

LOE 1-

Differences in the effect of EWSs on LOS was observed. A non-significant trend

towards a shorter LOS was observed following the introduction of EWSs (n=1), while

in another study median hospital LOS increased significantly for patients admitted to

ICU or HDU (n=1). A higher EWSs score was significantly correlated with hospital LOS

(n=1) (Alam, Hobbelink, van Tienhoven et al., 2014).

LOE 2++

No difference in LOS in ICU 1-2 years post-implementation (n=2). Decreased in LOS

between 47 pre- and 38 days post-implementation (n=1; p<0.001). Increase in LOS

between 4 months pre- vs 4 months post-implementation (n=1) (Smith Chiovaro

O’Neill et al., 2014).

LOE 2+

LOS was significantly correlated with NEWS, at all measured time points (P<0.05)

Median LOS more than doubled for a NEWS score >7 compared with a score of 0–4

(Alam, Vegting, Houben et al., 2015).

No significant difference in hospital LOS was observed with the Early Warning and

Response System (EWRS) pre- and post-intervention Umscheld, Betesh, Van

Zandbergen 2014).

No significant difference in hospital LOS pre- and post- an intervention involving

MEWS and a coloured observation chart (De Meester, Verspuy, Monsieurs, et al.,

2013)

There was a decrease in LOS between baseline and alert phase: 9.7 v 6.9 days

(P<0.001) post-implementation of a 5-item EWS and Patientrack (Jones, Mullally

Ingleby et al., 2011)

No decrease in LOS was observed post-implementation of an evidence-based

ELPQuiC However, Improvements were observed within existing resources, without

resulting in an increase in LOS (Huddart, Peden, Swart, 2014)

LOE 2-

Both NEWS and PARS were poor predictors of hospital LOS (Abbott, Vaid, Ip et al.,

2015).

Median LOS was 5 days (IQR 3–7) with 99 (25%) in this group staying ≥7 days. MEWS

aggregate scores ≥5 was not associated with excess LOS (OR 1.26, 95%CI 0.51, 3.14,

P=0.616. Excess LOS was associated with a Modified Barthel Index (MBI) ≤17 (OR

1.93, 95%CI 1.01, 3.7, P=0.048) and altered mental status (AVPU score ≥1 (OR 4.39,

95%CI 1.09, 17.71, P=0.038) at presentation. The excess LOS can largely be

accounted for by frail elderly patients. Therefore the need of short-and long-term

hospital stay patients are different (Huggan, Akram, Christen, 2015).

64 | P a g e

Admission Worthington PSS was correlated with hospital LOS. Mean LOS decreased

from 4 to 2 days, but this reflected an increase in short stay admissions given the

decrease in patients attending with less physiological derangement on admission

Dawes, Cheek, Bewick, et al., (2014)

LOS in ICU: MEWS >6 vs <6: OR 2.30 (95%CI 1.40-3.76) (Reini, Fredrikson, Oscarsson

(2012).

LOE 3

The average LOS for the MEWS <4 group

was 6.97 days and for the MEWS ≥4

group was 7.75 days (Ho, Li, Shahidah

(2013)

Patients whose MET response was within

24 hours of emergency admission were significantly more likely to have a shorter

hospital LOS (7 days) compared to those whose MET response was beyond 24 hours

of admission (11 days; P=0.039). ED LOS was not significantly different for patients

whose MET response was within 24 hours of emergency admission and those whose

MET response was beyond 24 hours of admission (Considine, Charlesworth, Currey

et al., 2014).

Cardiac Arrest

A cardiac arrest is a sentinel event in healthcare. Avoidance of cardiac arrest is one of the

key outcomes that EWSs seek to address through early identification of clinical

deterioration. Papers (n=21) which addressed this issue were categorised as follows: LOE ++

(n=1), LOE 1+ (n=1), LOE 1- (n=1), LOE2++ (n=1), LOE 2+ (n=4), LOE2- (n=7), LOE 3 (n=6)

(Figure 7).

0 1 2 3 4 5 6 7

LOE1++

LOE1+

LOE1-

LOE2++

LOE2+

LOE2-

LOE3

The evidence is not conclusive in

terms of the evaluation of the

impact of introducing EWSs on

length of stay as the endpoint.

65 | P a g e


category (Level of Evidence i.e. LOE) in relation to cardiac arrest

LOE 1++

Winters et al. (2013) in a systematic review with 43 included articles sought to

establish the effectiveness and implementation of RRSs in acute care settings. RRS

had a favourable effect on cardiorespiratory arrest in the majority of studies. RRS is

associated with reduced rates of cardiorespiratory arrest outside ICU (n=1 SR); RR

0.66 (95%CI 0.54, 0.80). RRS had a positive effect on cardiorespiratory arrest (n=19

of 20 studies). Benefit of RRS may depend on the patient population.

LOE 1+

McNeill & Bryden (2013) in a systematic review of studies (n=43) noted MET teams

introduction was associated with decrease in cardiopulmonary arrest (n=7 papers) or

no difference (n=5 papers). The effect of the introduction of Aggregate Weighted

Scoring Systems (AWSS) was variable; no difference (n=1 paper) and a reduction

(n=1 paper) in cardiac arrest rates was observed. The proportion of patients with a

delayed RRS activation decreased significantly over time (40.3% v 22.0%; P<0.001)

(n=1).

LOE 1-

Systematic review of studies investigating the effect of the introduction of EWSs on

cardiopulmonary arrests reported conflicting results (n=2); a decrease in

cardiopulmonary arrests (and mortality) among patients who had CPR (n=1), and an

increase in cardiopulmonary arrests (n=1) was observed. The latter study contained

a more heterogeneous population and a higher number of sick patients compared

with the control group. SAEs (i.e. number of deaths without an attempt to

resuscitation and readmission to ICU within 5 days of ICU discharge) decreased, but

was not statistically significantly (Alam et al. 2014).

LOE 2++

Smith et al. (2014) in a systematic review found that for Prediction: EWS (NEWS;

MEWS); CART and a novel electronic EWS) had consistently good predictive

performance in predicting cardiac arrest within both 24 and 48 hours of recording

vital signs (n=4 papers). AUROC was highest for the electronic EWS (AUROC=0.88)

and NEWS (AUROC=0.857) and lowest for VIEWS (AUROC=0.74). In terms of impact

of EWS implementation: (n=4 papers): a decrease in number of cardiac arrest calls

per admission (n=1; p<0.0001) and at the time of a ‘code blue’ call (n=1 paper;

p=0.0024); Increase in cardiac arrest in moderate risk patients (score 3/4), (n=1

paper; p<0.016); no difference in low- (score ≤2) or high-risk (score 5-15); no

difference (n=1 paper), note: no difference in RCT paper.

LOE 2+

66 | P a g e

Elderly patients had a significantly higher cardiac arrest rate (p<0.001); elderly: 2.2

cardiac arrests/1,000 ward admissions; non elderly: 1.0 cardiac arrests/1,000 ward

admissions. Elderly patients had a significantly lower MEWS score than nonelderly

patients 4 hours prior to cardiac arrest (P<0.001); elderly median MEWS = 2

(interquartile range [IQR], 1-3); nonelderly median MEWS = 3 (IQR, 2-5). MEWS was

significantly more accurate for detecting cardiac arrest in the ward in nonelderly,

than elderly patients (P<0.001); elderly MEWS AUROC: 0.71 (95%CI 0.88, 0.75),

nonelderly MEWS AUROC: 0.85 (95%CI 0.82, 0.88); older elderly (≥75 years) MEWS

AUROC: 0.71 (95%CI 0.66, 0.75), nonelderly MEWS AUROC: 0.81 (95%CI 0.78, 0.83).

Thus more accurate methods for risk stratification of elderly patients are necessary

to decrease the occurrence of cardiac arrest (Churpek et al. 2015).

Mean MEWS scores were significantly higher in cases, with increasing disparity

leading up to the event; 24 h prior to cardiac arrest (P<0.001), Cases: 2.3±1.3,

Control: 1.5±0.9; 48 h prior to cardiac arrest (P<0.005), Cases: 2.1±1.0, Control:

1.6±1.0. In the 48 h preceding cardiac arrest, maximum MEWS was the best

predictor: AUROC: 0.77 (95% CI, 0.71-0.82) (Churpek et al. 2012a).

Patel et al. (2015b) in a prospective, blinded validation study found that the

combination of PAR (i.e. a subjective 7-point Likert scale) assessing physician clinical

judgement) and MEWS was more accurate at predicting cardiac arrest within 24

hours: PAR-MEWS AUROC=0.70 (95%CI 0.63, 0.78); PAR AUROC=0.68 (95%CI 0.60,

0.75); MEWS AUROC=0.67 (95%CI 0.61, 0.74).


trial sought describe the effect of implementation of a RRS on the composite


a significant reduction in cardiopulmonary arrest per 1000 admitted patients post-


0.937); p=0.018).

LOE 2-

Churpek et al. (2014b) developed and validated an electronic risk score Cardiac

Arrest Risk Triage (eCART) score using commonly collected EHR data in an

observational cohort study. eCART cut-off of ≥50 would detect; 51% cardiac arrests,

44% ICU transfers, 83% deaths. Cardiac arrest: MEWS AUROC: 0.71 (95%CI 0.70-

0.73); CART AUROC: 0.83 (95%CI 0.82-0.83.

Etter et al. (2014) in a retrospective cohort study sought to review the preceding

factors, patient characteristics, process parameters and their correlation to patient

outcomes of the MET calls since its introduction. Researchers found that there was a

significant decrease in cardiac arrests post MET-implementation: 1.6 to 0.8 per 1000

hospital admissions (2008 and 2013, respectively; P<0.001).


design sought to validate a novel ML score, incorporating HRV against MEWS for the

risk stratification of critically ill patients in the ED. The ML score had a significantly

67 | P a g e

better discriminatory ability to predict cardiac arrest within 72 h than MEWS

(P=0.037): ML score AUROC= 0.781; MEWS AUROC=0.680. In the low, intermediate

and high risk ML score groups, the rate of cardiac arrest within 72 hours increased

from 0%, to 1.6% and 13.1%, respectively. Median MEWS score was significantly

higher for patients experiencing cardiac arrest (4 (IQR 2 to 5) than those who did not

(2 (IQR 1 to 4; P<0.001). A cutoff MEWS ≥ 3 predicted cardiac arrest within 72 h with

a: Sensitivity=74.4%, Specificity=54.2%, PPV=7.4%, NPV=97.8%.

Mapp et al. (2013) in a review examined EWSs and the incorporation of clinical

support on their effectiveness in predicting a patient’s potential for deterioration

and whether they prevent unplanned ICT admissions and/or death. Post-EWS

implementation variation in the effect of patient cardiac arrest was reported; a 50%

decrease in cardiopulmonary arrest were observed (n=1 paper); no change in

cardiopulmonary arrest (n=2 papers). Changes in number of RRT calls was reported

(n=5 papers). A 50% increase in RRTs was reported (n=3 papers), with concomitant

decrease in cardiac arrest scores was decreased (n=1). Decrease in medical team

calls was reported by another study, and a 43% decrease was predicted if the MEWS

and RRT were implemented hospital-wide (n=1 paper).

Drower at al. (2013) evaluated the introduction of a hospital specific EWS system

(patient observational chart with escalation protocol) on the incidence in-hospital

adult (≥16 years) cardiac arrest. The rate of cardiac arrests per admission decreased

by 38% post-intervention implementation (P=0.005): Pre-intervention: 4.67 per 1000

admissions; Post-intervention: 2.91 per 1000 admissions. The number of cardiac

arrest responses decreased post-intervention implementation: Pre-intervention: 8.5

arrests per month; Post-intervention: 5.5 arrests per month. There was a non-

significant increase in the number of emergency calls post-intervention

implementation: Pre-intervention: 7.5 per month; Post-intervention: 9.1 per month.

Moon et al. (2011) determined whether cardiac arrest calls, the proportion of adult

patients admitted to ICU after CPR and their associated mortalities were reduced in

the 4 year period after the introduction of a 24/7 CCOS and MEWS Charts. The

number of cardiac arrest calls relative to total adult hospital admissions decreased

significantly post- (n=584; 0.2%) compared to pre-implementation (n=767; 0.4%;

P<0.0001).

Stewart et al. (2014) investigated the impact of MEWS on (i) the decision-making

process to trigger the RR system, and (ii) cardiopulmonary arrests. Cardiopulmonary

arrests decreased post-MEWS introduction intervention compared to pre-MEWS

introduction (P=0.878): post-intervention: n=11; pre-intervention: n=14.

LOE 3

68 | P a g e

In terms of cardiac arrest: NEWS AUROC = 0.722 (95% CI 0.685-0.759), (decision tree-

DTEWS AUROC = 0.708 (95% CI 0.669-0.747) (Badriyah et al. 2014).

Odds of cardiac arrest increased

(approx. doubled) with each

increase of 1 point in the

aggregate NEWS scores. Where a

single vital sign had a score of 3

within an aggregate score, the

odds increased, but not

significantly. NEWS 5 OR =1.00

(0.59, 1.44); NEWS 3 (with a

component=3) OR=0.24 (95%CI

0.00, 0.55); NEWS 4 (with a


0.17, 1.26); NEWS 3 (no


0.07, 0.36); NEWS 4 (no


0.14, 0.74) (Jarvis et al. 2015a).

Mean MEWS score within 24 h

prior to cardiac arrest was 2.24

and MEWS score 1 = 45% (n=15) of

patients; MEWS score 2 = 21.2%

(n=7) of patients; MEWS score 3 =

12.1% (n=4) of patients; MEWS

score 4= 9.1% (n=3) of patients;

MEWS score 5= 9.1% (n=3) of

patients; MEWS score 6= 3% (n=1)

of patients. Final observation set

prior to cardiac arrest were

recorded by staff nurses (n=13;

39.9%) or healthcare assistants

(n=20; 60.6%). MEWS score was

not above the trigger score of 5 for the majority of patients who suffered a cardiac

arrest. The majority of cardiac arrests occurred out of hours (69.7%) (Harris 2013).

Critical care team was underutilised, especially out of hours, despite the presence of a clear

response strategy.

Churpek et al. (2012b) sought to develop a Cardiac Arrest Risk Triage (eCART) score

using ward vital signs to predict cardiac arrest, and compare its accuracy to MEWS.

eCART included RR, HR, DBP, and age. eCART score was significantly more accurate

than MEWS for predicting cardiac arrest (P=0.001): eCART AUROC=0.84; MEWS

The evidence is not definitive in terms of

the evaluation of the impact of introducing

EWSs on cardiac arrest as the endpoint;

however, there is evidence of a reduction in

numbers of cardiac arrests, whether this is

directly attributable to EWS is not clear.

LOE 1++ (n=1): ↓ of cardiorespiratory

arrest; LOE 1+ (n=1): ↓ of

cardiorespiratory arrest and no difference;

LOE 2+ (n=2): ↓ of cardiorespiratory

arrest, LOE 2+ (n=1): not conclusive. LOE 2-

(n=4): ↓ of cardiorespiratory arrest; LOE 2-

(n=2): ↓ of cardiorespiratory arrest and no

difference.

Many of the papers reflect results of the

concurrent implementation of RRTs or MET

teams which from the papers reviewed are

linked to reduced cardiac arrest rates.

Higher EWS scores can be correlated with

cardiac arrest. EWS have good predictive

performance in predicting cardiac arrest

within 24 and possibly 48 hours of

recording vital signs. Such predictive value

can be altered in different patient

populations and may be more uncertain in

older adult patients. Such predictive

capability may be enhanced by inclusion of

physician clinical judgement Likert scale.

69 | P a g e

AUROC=0.76. The addition of DBP instead of SBP was useful in improving the

accuracy of CART over MEWS in predicting cardiac arrest.

Churpek et al. (2012a) compared the efficacy of multiple risk scoring systems to

predict adverse events. CART score was best for predicting cardiac arrest: CART

Score = 0.83 (95%CI 0.79-0.86); VIEWS = 0.77 (95%CI 0.72-0.82). CART was also the

score which was best for predicting a composite outcome of cardiac arrest, ICU

transfer and mortality.




large vital signs database. The AUROCs for NEWS for cardiac arrest, within 24 h, was

0.722 (95%CI 0.685–0.759). Similarly, the ranges of AUROCs for the other 33 EWSs

were 0.611 (95%CI 0.568–0.654) to 0.710 (95%CI 0.675–0.745) (cardiac arrest).

In summary

What is the impact of EWS on cardiac arrest figures?

LOE 1++: (n=1, Winters et al. 2013) in a systematic review found that RRS were associated

with reduced rates of cardiorespiratory arrest outside ICU (n=1 SR); RR 0.66 (95%CI 0.54,

0.80). RRS had a positive effect on cardiorespiratory arrest (n=19 of 20 studies). Benefit of

RRS may depend on the patient population.

LOE 1+: (n=1, McNeill & Bryden, 2013) in a systematic review of studies (n=43) noted MET

teams introduction was associated with decrease in cardiopulmonary arrest (n=7 papers) or

no difference (n=5 papers).

LOE 1-: (n=1, Alam et al. 2014) in a systematic review of studies investigating the effect of

the introduction of EWSs on cardiopulmonary arrests reported conflicting results (n=2); a

decrease in cardiopulmonary arrests (and mortality) among patients who had CPR (n=1),

and an increase in cardiopulmonary arrests (n=1) was observed.

LOE 2+: (n=3, Smith et al. 2014; Churpek et al. 2012a; Ludikhuize et al. 2015) in a systematic

review of studies investigating the effect of the introduction of EWSs on cardiopulmonary

arrests: (n=4 papers): a decrease in number of cardiac arrest calls per admission (n=1;

p<0.0001) and at the time of a ‘code blue’ call (n=1 paper; p=0.0024); Increase in cardiac

arrest in moderate risk patients (score 3/4), (n=1 paper; p<0.016); no difference in low-

(score ≤2) or high-risk (score 5-15); no difference (n=1 paper), note: no difference in RCT

paper. Mean MEWS scores were significantly higher in cases of cardiac arrest 24 and 48

hour prior to arrest (Churpek et al. 2012a). Ludikhuize et al. (2015) found a significant

reduction in cardiopulmonary arrest per 1000 admitted patients post-implementation of

RRS compared to pre-implementation (adjusted OR=0.607 (95%CI 0.393, 0.937); p=0.018).

70 | P a g e

LOE 2+ :(n=2, Churpek et al. 2015; Patel et al. 2015) Churpek et al. (2015) found that MEWS

was significantly more accurate for detecting cardiac arrest in the ward in nonelderly, than

elderly patients (P<0.001); elderly MEWS AUROC: 0.71 (95%CI 0.88, 0.75), nonelderly

MEWS AUROC: 0.85 (95%CI 0.82, 0.88); older elderly (≥75 years) MEWS AUROC: 0.71

(95%CI 0.66, 0.75), nonelderly MEWS AUROC: 0.81 (95%CI 0.78, 0.83). The combination of

PAR (i.e. a subjective 7-point Likert scale) assessing physician clinical judgement) and MEWS

was more accurate at predicting cardiac arrest within 24 hours (Patel et al 2015).

LOE 2- (n=5, Moon et al. 2011; Drower at al., 2013; Mapp et al. 2013; Etter et al. 2014;

Stewart et al. 2014))

Moon et al. (2011) determined whether cardiac arrest calls, the proportion of adult patients

admitted to ICU after CPR and their associated mortalities were reduced in the 4 year period

after the introduction of a 24/7 CCOS and MEWS Charts. The number of cardiac arrest calls

relative to total adult hospital admissions decreased significantly post implementation.

Drower at al. (2013) evaluated the introduction of a hospital specific EWS system (patient

observational chart with escalation protocol) on the incidence in-hospital adult cardiac

arrest. The rate of cardiac arrests per admission decreased by 38% post-intervention

implementation (P=0.005). Researchers found that there was a significant decrease in

cardiac arrests post MET-implementation: 1.6 to 0.8 per 1000 hospital admissions (2008 and

2013, respectively; P<0.001) (Etter et al. 2014). Mapp et al. (2013) in a review examined

EWSs and the incorporation of clinical support on their effectiveness in predicting a

patient’s potential for deterioration and whether they prevent unplanned ICT admissions

and/or death. Post-EWS implementation variation in the effect of patient cardiac arrest was

reported; a 50% decrease in cardiopulmonary arrest were observed (n=1 paper); no change

in cardiopulmonary arrest (n=2 papers). Stewart et al. (2014) investigated the impact of

MEWS on (i) the decision-making process to trigger the RR system, and (ii) cardiopulmonary

arrests. Cardiopulmonary arrests decreased post-MEWS introduction intervention

compared to pre-MEWS introduction (P=0.878).

What is the impact of EWS on predicting cardiac arrest?

LOE 2++ (n=1 Smith et al. 2014). Smith et al. (2014) in a systematic review found that for

Prediction: EWS (NEWS; MEWS); CART and a novel electronic EWS) had consistently good

predictive performance in predicting cardiac arrest within both 24 and 48 hours of recording

vital signs (n=4 papers). AUROC was highest for the electronic EWS (AUROC=0.88) and

NEWS (AUROC=0.857) and lowest for VIEWS (AUROC=0.74).

LOE 2- (n=2, Churpek et al. 2014b; Ong et al. 2012). Churpek et al. (2014b) developed and

validated an electronic risk score Cardiac Arrest Risk Triage (eCART) score, eCART cut-off of

≥50 would detect; 51% cardiac arrests. Ong et al. (2012) validated a novel ML score,

incorporating HRV against MEWS for the risk stratification of critically ill patients in the ED.

71 | P a g e

The ML score had a significantly better discriminatory ability to predict cardiac arrest within

72 h than MEWS (P=0.037.

LOE 3 (n=3, Churpek et al. 2012b; Harris 2013; Smith et al. 2013; Jarvis et al 2015a). eCART

score included RR, HR, DBP, and age was significantly more accurate than MEWS for

predicting cardiac arrest (P=0.001) (Churpek et al. 2012b). Odds of cardiac arrest increased

(approx.. doubled) with each increase of 1 point in the aggregate NEWS scores Jarvis et al

2015a). MEWS score was not above the trigger score of 5 for the majority of patients who

suffered a cardiac arrest (Harris 2013). Smith et al. (2013) tested the ability of NEWS to

discriminate patients at risk of cardiac arrest, unanticipated ICU admission or death within

24h of a NEWS value and compared its performance to that of 33 other EWSs and found

that the AUROCs for NEWS for cardiac arrest, within 24 h, was 0.722 (95%CI 0.685–0.759).

Transfer to or admission to ICU

Transfer to or admission to ICU is one of the potential chains of events that can happen on

the patients’ journey if they have triggered an escalation within a EWS. Twenty one studies

included data pertinent to this outcome: LOE1+ (n=1), LOE 1- (n=1), LOE 2++ (n=1), LOE2+

(n=4), LOE2- (n=6), LOE3 (n=8).

LOE 1+

McNeill & Bryden (2013) in a systematic review of studies (n=43) noted MET teams

introduction was associated with decrease in cardiopulmonary arrest (n=7 papers) or

no difference (n=5 papers). The effect of the introduction of AWSS was variable; no

difference (n=1 paper) and a reduction (n=1 paper) in cardiac arrest rates was

observed. Introduction of AWSS was associated with decreased ICU admission (n=2)

or no change (n=1). MET teams introduction was associated with decreased ICU

admission (n=4) or no difference (n=2). AWSS implementation improves hospital

survival and decreases unplanned ICU admission and cardiac arrest. Effect on ICU

and hospital LOS is uncertain. MET teams were found to improve hospital survival,

decrease unplanned ICU admission and cardiac arrests, but the effect of MET on

hospital LOS and ICU mortality is unclear. Ongoing review of the activation process

and education programmes is required. Authors noted that for RRSs to be effective a

‘whole system’ approach should be adopted and aggregate weighted scoring

systems are more effective than single parameter systems. RRSs are most effective

for patients with predictable clinical decline than among patients who are post-

operative or acute haematology patients.

LOE 1-

Systematic review: A significant increase in HDU admissions was observed following

the introduction of the EWSs in emergency surgical patients (14 to 21%; P=0.0008)

(n=1 study) (Alam et al. 2014).

LOE 2++

72 | P a g e

Peris et al. (2012) using a prospective, before-and-after intervention study to

determine if MEWS calculation can help the anaesthetist select the correct level of

care to avoid inappropriate admission to the ICU and to enhance the use of the HDU

after emergency surgical procedures, patients with a MEWS of 3 or 4 were

transferred to the HDU, whereas a MEWS score of ≥5 was considered criteria for ICU

admission. After MEWS introduction, the HDU admission increased from 14 %

(control) to 21 % (experimental) (P=0.0008; sensitivity 0.4457, 95% CI 0.3725-

0.5206; specificity 0.4187, 95%CI 0.3862-0.4517). The predictive values for an

appropriate HDU admission (78.6% with 95% CI from 74.7% to 82.3%). After MEWS

introduction, number of ICU admissions decreased from 11 % in the control group to

5 % in the MEWS experimental group (P=0.0010; sensitivity 0.7204, 95%CI 0.6178-

0.8086; specificity 0.4570, 95%CI 0.4256-0.4887). The predictive values for an

avoidable ICU admissions (94.6% with 95% CI from 92.1% to 96.4%).

LOE 2+

Risk of SAEs: II (i.e. cardiac arrest, emergency ICU admission or death): NEWS score

≥5 OR 18.1 (95% CI 4.51, 72.8); NEWS score ≥7 OR 11.5 (95%CI 3.40, 38.6) (Tirkkonen

et al. 2014).

Umscheld et al. (2015) developed, implemented and validated an electronic sepsis

detection and response system to improve patient outcomes; The EWRS using

retrospective development and prospective validation, and pre-, post-intervention

study. Authors noted that an automated prediction tool identified at-risk patients

and prompted a bedside evaluation resulting in more timely sepsis care, improved

documentation, and a suggestion of reduced mortality. No significant difference in

hospital LOS pre- and post-intervention. EWRS score of ≥4 were 7 times more likely

to experience a RRT. Number of alerts decreased post-implementation (3.8% to

3.5%). No significant difference in the number of patients transferred to ICU pre- and

post-intervention but the proportion of patients transferred to ICU within 6 h of the

alert increased post-intervention (10% vs 7%, P=0.06).

Hackmann et al. (2011) investigated implementation of a two-tier EWS to identify

the signs of clinical deterioration and provide early warning of serious clinical events

i.e. transfer to ICU. Automatic identification of patients at risk of clinical

deterioration using EWS from existing EMR databases calculated using machine

learning algorithms, and (2nd) the real-time detection of clinical event based on real-

time vital sign data collected from on-body WSN technology attached to those high-

risk patients. Data was sent to the EMR and EWS scores are assigned to patients in

real time using ‘machine-learning techniques’ to analyse the data. 398 variables

were used to predict patient outcome i.e. transfer to ICU. The model had an AUROC

= 0.8834. Using a ‘real-time simulator’ of the model, the predictive value for transfer

to ICU was; AUROC = 0.7293. Predict ICU transfer: Retrospective data: Specificity=

0.9500, Sensitivity=0.4877, PPV= 0.3138, NPV= 0.9753, Accuracy= 0.9292; Real time

simulator data: Specificity= 0.9492, Sensitivity=0.4127, PPV= 0.2955, NPV= 0.9691,

73 | P a g e

Accuracy= 0.9229. Authors noted that this integrated approach for EWS monitoring

and response is feasible.


trial sought describe the effect of implementation of a RRS on the composite


a significant reduction in cardiopulmonary arrest per 1000 admitted patients post-


0.937); p=0.018). However there was a non-significant decrease in the number of

unplanned ICU admission per 1000 admitted patients post-implementation

compared to pre-implementation (adjusted OR=0.878 (95%CI 0.755, 1.021);

p=0.092). Authors noted that nationwide (in the Netherlands) implementation of a

RRS was associated with a 15% adjusted risk reduction in the composite endpoint of

cardiopulmonary arrest, unplanned ICU admission, or death.

LOE 2-

For ICU admission within 2 days the age-adjusted ORs for NEWS categories

compared to the baseline category (≤4) 5–6: OR 1.22, 95% CI 0.59 to 2.54 (p=0.59);

7–8: OR 2.01, 95% CI 1.02 to 3.97 (p=0.04); 9–20: OR 5.76, 95% CI 3.22 to 10.31

(p<0.01); AUROC NEWS: 0.67, AUROC age-adjusted NEWS: 0.61 (Corfield et al. 2014).

REMS performed significantly better at predicting admission to ICU/HDU of patients

presenting to ED (P<0.001): REMS AUROC=0.589 (95%CI 0.567-0.611) (P<0.001);

MEWS AUROC area under the curve (AUC)=0.538 (95%CI 0.516-0.560) (P=0.009)

(Bulut et al. 2014).

Huggan et al. (2015) in an observational cohort study found that MEWS aggregate

scores ≥5 was significantly associated with death or ICU admission (HR 5.50, 95%CI

1.77, 17.07, P=0.003). The excess LOS was largely accounted for by frail elderly

patients.

Yoder et al. (2013) sought to investigate whether the MEWS could identify low-risk

patients who might forgo overnight vital sign monitoring. The median evening MEWS

was 2 (IQR, 1-2). The adverse event rate (defined as ICU transfers or cardiac arrests

within the 24-hour period) increased with higher evening MEWS, from a rate of 5.0

per 1000 patient-days (when the MEWS was ≤1) to 157.3 per 1000 patient-days

(when the MEWS was ≥7) (P = 0.003 for trend).

Bunkenborg et al. (2013) evaluated the “short and long term effects of systematic

inter-professional use of EWS, structured observational charts, and clinical

algorithms for bedside action” on unexpected in-hospital death (i.e. sudden, no

resuscitation), death after cardiopulmonary resuscitation, and death within 24h of

admission for intensive care) death. Unexpected patient mortality and the number

of SAEs decreased post-intervention compared to pre-intervention. However, there

was no difference in the number of ICU admissions observed (n=17 both pre and

post-intervention) which may be attributable to improved patient monitoring and

earlier intervention.

74 | P a g e

Moon et al. (2011) determined

whether cardiac arrest calls,

the proportion of adult

patients admitted to ICU after

CPR and their associated

mortalities were reduced in the

4 year period after the

introduction of a 24/7 CCOS

and MEWS Charts. There was a

24.5% increase in annual

admission to ICU post-

implementation. The number

of patients admitted to ICU

having undergone CPR in

hospital decreased significantly

post- (2%) compared to pre-

implementation (3%; P=0.004).

LOE 3

In terms of unanticipated admission to ICU: NEWS AUROC = 0.857 (95%CI 0.847-

0.868), DTEWS AUROC = 0.862 (95CI 0.852-0.872) (Badriyah et al. 2014).

Both EWSs were significant in predicting CCU admission: MEWS = P=0.037, NEWS =

P=0.00046. Median EWS for patients admitted and not admitted to CCU was: MEWS,

5 and 4, respectively; NEWS, 8 and 7, respectively. MEWS AUROC = 0.55, NEWS

AUROC = 0.59 (Cooksley et al. 2012). Key EWS Variables predictive of CCU admission:

Respiratory rate (P=0.0003), Temperature (P=0.033).

Odds of unanticipated ICU transfer increased (doubled) with each increase of 1 point

in the aggregate NEWS scores. Where a single vital sign had a score of 3, the odds

increased, but not significantly; NEWS 5 OR =1.00 (0.55, 1.49); NEWS 3 (with a

component=3) OR=0.23 (95%CI 0.00, 0.52); NEWS 4 (with a component=3) OR=0.46

(95%CI 0.00, 0.99); NEWS 3 (no component=3) OR=0.22 (95%CI 0.09, 0.38); NEWS 4

(no component=3) OR=0.45 (95%CI 0.13, 0.80) (Jarvis et al. 2015a).

In a retrospective cohort study, 267 patients were admitted to HD/ICU (37.4%) in the

MEWS <4 group as compared with 86 (27.7%) patients in the MEWS ≥4 group.

MEWS Score cut offs were poor predictors of ICU/HD transfer; MEWS <4 AUROC=

0.49; MEWS <5 AUROC= 0.47. The average LOS for the MEWS <4 group was 6.97

days and for the MEWS ≥4 group was 7.75 days (Ho et al. 2013).




In summary

The evidence is not conclusive in terms of the

evaluation of the impact of introducing EWSs on

admission to intensive care unit as the

endpoint, however some data suggests that ICU

admission maybe more timely.

Three key parameters are used to record such

data: HDU admission, ICU admission and

unanticipated ICU admission.

75 | P a g e

than MEWS for predicting ICU transfer (P<0.001): eCART AUROC=0.71; MEWS

AUROC=0.67.

Churpek et al. (2012a) compared the efficacy of multiple risk scoring systems to

predict adverse events. eCART was also the score which was best for predicting a

composite outcome of cardiac arrest, ICU transfer and mortality. eCART was also the

score which was best for predicting ICU transfer. CART Score = 0.77 (95%CI 0.76-

0.78). Aggregate weighted scoring systems outperformed the other systems for most

outcomes.

Churpek et al., (2014) using retrospective cohort, single centre design found that a

derived models for specfic outcomes were significantly more accurate than ViEWS

at detecting ICU transfer ever and within 24 hours (P<0.001) and the derived cardiac

arrest model was more sensitive in the detection of cardiac arrest than ViEWS at the

same specificity. Routinely collected laboratory values added to the model were

significant predictors of both outcomes.




large vital signs database. The AUROCs for NEWS for unanticipated ICU admission

within 24 h, was 0.857 (95%CI 0.847–0.868). Similarly, the ranges of AUROCs for the

other 33 EWSs were 0.570 (95%CI 0.553–0.568) to 0.827 (95%CI 0.814–0.840)

(unanticipated ICU admission).

76 | P a g e

In summary Impact of EWS on transfer to or admission to ICU LOE 1+ (n=1 systematic review Introduction of AWSS was associated with ↓ ICU admission (n=2) or no change (n=1), MET introduction was associated with ↓ ICU admission (n=4) or no difference (n=2), for RRSs to be effective a ‘whole system’ approach should be adopted). LOE 1- (n=1, ↑ in HDU admissions observed following the introduction of the EWSs), LOE 2++ (n=1, after MEWS introduction ↑ in HDU admissions, number of ICU admissions ↓). LOE 2+ (n=1, risk of SAE: II (ie. cardiac arrest, emergency ICU admission or death): NEWS score ≥5 OR 18.1 (95% CI 4.51, 72.8); NEWS score ≥7 OR 11.5 (95%CI 3.40, 38.6), (n=1, no significant difference in the number of patients transferred to ICU pre- and post-intervention but the proportion of patients transferred to ICU within 6 h of the alert ↑ post-intervention (10% vs 7%, P=0.06), (n=1, integrated approach for EWS monitoring with automatic identification of patients at risk of clinical deterioration, using a ‘real-time simulator’ of the model, the predictive value for transfer to ICU was; AUROC = 0.7293), (n=1, a non-significant ↓ in the number of unplanned ICU admission per 1000 admitted patients post-implementation of RRS compared to pre-implementation), (n=1, REMS performed significantly better at predicting admission to ICU/HDU of patients presenting to ED), (n=1, MEWS aggregate scores ≥5 was significantly associated with death or ICU admission), (n=1, the adverse event rate (defined as ICU transfers or cardiac arrests within the 24 hour period) ↑ with higher evening MEWS), (n=1, no difference in the number of ICU admissions observed post EWS), (n=1, introduction of a 24/7 CCOS and MEWS Charts – the number of patients admitted to ICU having undergone CPR in hospital ↓ significantly, however an ↑ in annual admission to ICU post-implementation), (n=1, MEWS Score cut offs were poor predictors of ICU/HD transfer), (n=1, eCART (included RR, HR, DBP, and age) score was significantly more accurate than MEWS for predicting ICU transfer (P<0.001): eCART AUROC=0.71; MEWS AUROC=0.67)). NEWS specific LOE 2- (n=1, for ICU admission within 2 days the age-adjusted ORs for NEWS categories compared to the baseline category (≤4) 5–6: OR 1.22; NEWS 7–8: OR 2.01 (p=0.04); NEWS 9–20: OR 5.76, (p<0.01); AUROC NEWS: 0.67, AUROC age-adjusted NEWS: 0.61). LOE 3 (n=1, in terms of unanticipated admission to ICU: NEWS AUROC = 0.857 (95%CI 0.847-0.868), DTEWS AUROC = 0.862 (95CI 0.852-0.872)), (n=1, both EWSs were significant in predicting CCU admission: MEWS = P=0.037, NEWS = P=0.00046, NEWS AUROC = 0.59, Key EWS Variables predictive of CCU admission: Respiratory rate (P=0.0003), Temperature (P=0.033). ), (n=1, odds of unanticipated ICU transfer increased (doubled) with each increase of 1 point in the aggregate NEWS scores), (n=1, The AUROCs for NEWS for unanticipated ICU admission within 24 h, was 0.857 (95%CI 0.847–0.868), slightly better than 33 other EWS).

77 | P a g e

Resource utilisation/documentation/clinical response

LOE 2+

Researchers sought to assess the scoring accuracy and the adequacy of the

prescribed clinical responses to

NEWS: 74.1% of patients had an

appropriate clinical response,

25.9% had an inappropriate

response. There was a trend

towards increased mortality for

patients who received an incorrect

response to a NEWS score.

Adequacy of response did not

change significantly with time of

day (correct response during day

and night were 75.9% and 72.1%, respectively; P=0.404). Patients admitted at the

weekend had a worse clinical response (correct response during weekday and

weekend were 79.3% and 53.3%, respectively; P<0.0001).; adjusted OR 4.15 (95%CI

2.24, 7.69). The clinical response worsened with increasing NEWS score. Adjusted

ORs and P values are compared to baseline NEWS score of 0);NEWS 0; % correct

response=92%; NEWS 1-4; % correct response =67.5% (P<0.0001); adjusted OR 6.13

(95%CI 3,08, 12.16); NEWS 5-6; % correct response =0% (P<0.0001); adjusted OR

177 (95%CI 20.72, 1510); NEWS 7; % correct response =25% (P<0.0001); adjusted OR

40.64 (95%CI 7.04, 234.7). In terms of accuracy of NEWS score scoring: accuracy

decreased significantly with increasing score or worsening physiological

derangement. P values are compared to baseline NEWS score of 0); NEWS 0; %

correct score=87.9%; NEWS 1-4; % correct score=78.0% (P=0.018); NEWS 5-6; %

correct score=69.2% (P=0.072); NEWS 7; % correct score=50% (P=0.008) (Kolic et al.

2015). The authors recommend us of automated observation calculation across the

UK.

Patel et al. (2015) in a prospective, blinded validation study found that the

combination of PAR (i.e. a subjective 7-point Likert scale) assessing physician clinical

judgement) and MEWS was more accurate at predicting RRT activation within 24

hours: PAR-MEWS AUROC=0.72 (95%CI 0.66, 0.77); PAR AUROC=0.68 (95%CI 0.62,

0.74); MEWS AUROC=0.67 (95%CI 0.62, 0.73). Thus the PAR score which quantifies

physician worry, when used in combination with MEWS increases the accuracy of

MEWS to predict adverse events within 24 hours.

Umscheld et al. (2015) developed, implemented and validated an electronic sepsis

detection and response system to improve patient outcomes; The EWRS using

retrospective development and prospective validation, and pre-, post-intervention

Factors such as poor recording of vital

signs, incorrect calculations, non-

adherence to the escalation protocol will

reduce the sensitivity of the triggering

system and the overall effectiveness of the

EWS system

78 | P a g e

study. A ‘text page’ alert was sent to RRT for patients meeting EWRS criteria and

nurses were alerted by ‘pop-up’ notification on HER. Authors noted that an

automated prediction tool identified at-risk patients and prompted a bedside

evaluation resulting in more timely sepsis care, improved documentation, and a

suggestion of reduced mortality. Post implementation, there was a significant

increase in ordering antibiotics, intravenous fluid boluses and lactate and bolus

cultures within 3 h of the trigger. Discharge to home increased significantly post-

intervention (64% v 58%, P=0.04). Sepsis discharge diagnosis increased post-

intervention (45% v 39%; P=0.02). No significant difference in hospital LOS pre- and

post-intervention. EWRS score of ≥4 were 7 times more likely to experience a RRT.

Number of alerts decreased post-implementation (3.8% to 3.5%).

LOE 2-

Nurses escalated care and contacted physicians for events with EWS ≥ 2 and EWS ≥ 3

in 64% and 60% of events of unanticipated intensive care unit admission (UICU) and

58% and 55% for cardiac arrest (CA) events. On call physicians provided adequate

care (defined as attended the patient immediately and implemented an appropriate

treatment) in 49% of cases of UICU and 29% of cases of the CA when EWS exceeded

5 points. Senior staff was involved according to protocol (with EWS ≥ 9), in 53% and

36% of cases of UICU and CA, respectively (pg 1699). At least two full sets of EWS

were recorded in 87%, 94% and 75% of UICU, CA and UD respectively in the 24 h

preceding the event (Petersen et al. 2014). Overall authors noted the poor

compliance with the escalation protocol was commonly found. Only in 12 events

(8%) was the escalation protocol strictly adhered to; five of these had a EWS < 2 in

the 24 h prior to the event. In 132 events (92%) non-adherence to the escalation

protocol at one or several levels was noted.

Pattison and Eastham (2012) using a mixed methods explanatory design found that

for 23.8% of referrals to a critical care outreach team (CCOT), there was a delay

between the point at which patients deteriorated and the time patients were

referred. Notably the average delay was 2.96 h (95% CI 1.97–3.95; Standard

Deviation [SD] 9.56). Mean of MEWS at referral: 3.76 (95% CI 3.49, 3.99); at

deterioration: 3.96 (95% CI 3.67, 4.18). Referral would often be provoked by the

culmination of various factors, including blood results, MEWS, and how patients felt.

Untimely referrals were associated with lower survival to discharge (χ2p = 0.004) and

3 and 6 month mortality (χ2p = 0.004; P= 0.026) [n = 309). Three- and 6-month

mortalities were significantly associated with a higher MEWS at referral (P= 0.022, Z

= 2.119; p = 0.010, Z = −2.575).

Yoder et al. (2013) sought to investigate whether the MEWS could identify low-risk

patients who might forgo overnight vital sign monitoring. The frequency of vital sign

disruptions was unchanged, with a median of 2 vital sign checks per patient per night

and at least 1 disruption from vital sign collection 99.3% of the nights regardless of

MEWS category. Almost half of all night-time vital sign disruptions (45.0%) occurred

79 | P a g e

in patients with a MEWS of 1 or less. Authors note that the researchers suggests that

the night-time frequency of vital sign monitoring for low-risk medical inpatients

might be reduced positively impacting on patient sleep and could also have

significant health care resource implications. Vital sign data does not incorporate

more nuanced markers of clinical status.

Shuk-Ngor et al. (2015) using a prospective observational design compared the

performances of detecting patient deterioration with and without using the MEWS

for a group of patients who are waiting for in-patient beds in a public ED. In the

MEWS group, there was approximately 1 episode of activation in every 10 patients

(equating to 100% sensitivity and 100% specificity) but it was 1 in 20 patients in the

Usual Observation group (equating to 98.3% sensitivity and 97.8% specificity).

Dawes et al. (2014) in a prospective observational study, post-intervention found

that the introduction of an electronic alerting system (the Worthing PSS), calculated

using VitalPACTM) did not have a clinical impact on patient outcomes. However, the

speed, appropriateness and compliance of the response to alerts was not measured.



outcomes of the MET calls since its introduction. The MET consisted of intensive

care specialists and was available 24/7. Researchers found that there was a

significant increase in MET calls from 5.2 to 16.5 per 1000 hospital admissions

(P<0.001) post MET implementation. 14% of MET calls had no vital signs recorded in

the 24 h before the MET call. Implementation of MET did not result in a significant

decrease in the delay between the first vital sign abnormality and the MET call.

There is variability in the assessment and recording of vital signs prior to MET events

and delays between vital sign instability and subsequent MET call. These factors

could reduce the sensitivity of the triggering system.

Rose et al. (2015) investigated whether use of an electronic MEWS (eMEWS) as a

clinical decision tool would improve eMEWs documentation would result in more

appropriate the faster activation of RRT by nurses. A decrease in RRT calls was

observed post-implementation (17/90 days) compared to pre-implementation

(23/90 days; 100% survival). A decrease in CB calls was observed post-

implementation (1/90 days) compared to pre-implementation (6/90 days; 1 patient

died). The mean eMEWS score at RRT activation increased post-implementation (2.3

± 1.79; range 0-6) compared to pre-implementation (3.2 ± 1.79; range 1-6). The

number of undocumented eMEWS scores decreased post-implementation (0/17 RRT

events; 0%) compared to pre-implementation (11/23 RRT events; 49%). According to

the authors changes incorporated and recommended as a result of study findings

included; (a) eMEWS training will be incorporated into the training of all new

employees, (b) frequent brief eMEWS educational items will become part of monthly

staff meetings, (c) core staff will be included in monthly reviews of RRT and CB

events and (d) a bidirectional level of communication will be established regarding

80 | P a g e

problem identification and solving, doing and reflecting, and uniform buy-in from all

stakeholders establishing the ultimate goal of eliminating failure to rescue.

LOE 3

The total expected resource intensity, including nurse and RRT utilization increases

as a patient’s condition deteriorates, but time to stabilisation and resource time

differ depending on the subpopulation (Capan et al. 2015a; Capan et al. 2015b).

Retrospective application of the CREWS in patients with chronic hypoxaemic with a

NEWS score ≥7 would have reduced the number of reviews by 70.3 % (Lobo et al.

2015) (Irish Study).

Large variations in the pattern of documentation, within the 24 hour period with

peaks in the morning (6:00-6:69; 13.58%) and evening (21:00-21:59; 8.58%) but little

variation in this pattern between days of the week. Results show that staff continue

to take vital signs according to predetermined patterns that may prefer to use

clinical judgment rather than operating by protocol (Hands et al. 2013). Patients with

higher aggregate ViEWS scores were more likely to have their vital signs measured at

night. Sicker patients with higher VIEWS score got more frequent vital signs but not

consistently and less so at night. Adherence to escalation of monitoring contrasted

between day and night-time; 73.1% of the vital sign observation sets had a

subsequent set recorded within 6 hours during 08:00-11:59, compared with 25.32%

during 20:00-23:59. This percentage difference was true for all ViEWS scores,

including scores ≥9 (08:00-11:59 (86.65%) and 20:00-23:59 (68.78%)). 47.42% of

patients with ViEWS=7/8 and 31.22% of patients with a ViEWS ≥9 in the period

20:00-23:59 did not have vital signs recorded in the following 6 hours. Time to next

observation decreased with increasing ViEWS value, but less than expected by the

monitoring protocol e.g. 20:00-23:59hrs: ViEWS 3-6 : mean=7.88 hours wait; ViEWS

7-8 : mean=6.59 h; ViEWS ≥9: mean=5.17 h.

Electronically generated, automated scoring using RI -a patient acuity score based

upon summation of excess risk functions that utilize additional data (26 items) from

the vital signs, laboratory test results, Braden Scale, cardiac rhythm and nursing

assessments. RI had superior discrimination of 24 h mortality compared to MEWS:

24 hour mortality, RI AUROC = 0.93 (95%CI 0.92, 0.93); MEWS AUROC = .82 (95%CI

0.82, 0.33). At a MEWS score of 4, and a corresponding RI score of -16, has similar

sensitivity, but RI has twice the likelihood ratio (positive) and reduces false positive

alarms by 53%. At a score of 30, RI captures 54% more of the patients who will die

within 24 hours: MEWS Score= 4: likelihood ratio (positive)=7.8, likelihood ratio

(negative)=0.54, Sensitivity=49.8%, Specificity=93.6%, PPV=5.2%, NPV=99.6%; RI

Score =-16:likelihood ratio (positive) = 16.9, likelihood ratio (negative)=0.54,

Sensitivity=48.9%, Specificity=97.1%, PPV=10.6%, NPV=99.6%; RI Score =30:

81 | P a g e

likelihood ratio (positive) = 7.9, likelihood ratio (negative)=0.26, Sensitivity=76.8%,

Specificity=90.4%, PPV=5.3%, NPV=99.8% (Finlay et al. 2014).

Parham (2012) in a retrospective analysis found that 85% (n=17) of MET responses

were within 1 min; 15% (n=2) had a delay in the MET response of more than 1 min. 2

met criteria which merited a MET response prior to the observations which resulted

in a MET response. 25% (n=5) patients had MEWS>4 within 180 min of the MET call;

mean time between MEWS>4 and MET call = 113 min (range 5 to 210 min).

Churpek et al. (2014) sought to derive and validate a prediction model for cardiac

arrest while treating ICU transfer as a competing risk, using EHR data. ViEWS was

chosen as the best performing system of 33 tested (Kellett, Kim 2012) for

comparative purposes. Use of ViEWS would have resulted in 5,500 more ‘false

alarms’ than the derived model. Routinely collected laboratory values added to the

model were significant predictors of both outcomes.

Jarvis, Kovacs, Briggs et al. (2015b) in a retrospective analysis compared 36 published

EWSs and investigated whether EWS if truncated to a binary score of 0 ‘normal’ or 1

‘abnormal’ resulted in a decrease in errors associated with weighting or scoring EWS.

The researchers found that PPV and sensitivity of the standard weighted NEWS is

better than binary NEWS. Binary NEWS trigger score ≥3 would detect more adverse

outcomes than NEWS score at a trigger score ≥5, but would require a 15% higher

triggering rate. The percentage of observation sets that trigger a response is higher

for binary (11.8%) than standard (10.2%) NEWS. The number of unique patients that

trigger a response daily is higher for binary (n=145 (SD 24)) than standard (n=118 (SD

20)) NEWS.

82 | P a g e

In summary regarding adherence to system

LOE 2+ (n=1, re scoring accuracy and the adequacy of the prescribed clinical responses to NEWS: 74.1% of patients had an appropriate clinical response, 25.9% had an inappropriate response, there was a trend towards increased mortality for patients who received an incorrect response to a NEWS score, the clinical response worsened with increasing NEWS score, in terms of accuracy of NEWS score scoring: accuracy decreased significantly with increasing score or worsening physiological derangement, the authors recommend use of automated observation calculation across the UK).LOE2- (n=1, nurses escalated care and contacted physicians for events in between 55-64% of events; on call physicians provided adequate care in 29-49% of cases studied; senior staff were involved according to protocol in 36-53% of cases; at least two full sets of EWS were recorded in 75- 94% in the 24 h preceding the event; authors noted the poor compliance with the escalation protocol was commonly found, only 8% of cases was the escalation protocol strictly adhered to), (n=1, delay between the point at which patients deteriorated and the time patients were referred, the average delay was 2.96 h, untimely referrals were associated with lower survival to discharge). LOE 3 (n=1, large variations in the pattern of documentation, time to next observation decreased with increasing ViEWS value, but less than expected by the monitoring protocol).

In summary regarding resource utilisation

LOE2- (n=1, in the MEWS group, there was approximately 1 episode of activation in every 10 patients (equating to 100% sensitivity and 100% specificity) but it was 1 in 20 patients in the Usual Observation group ), (n=1, Researchers found that there was a significant increase in MET calls from 5.2 to 16.5 per 1000 hospital admissions (P<0.001) post MET implementation. 14% of MET calls had no vital signs recorded in the 24 h before the MET call. Implementation of MET did not result in a significant decrease in the delay between the first vital sign abnormality and the MET call), (n=1, the number of undocumented MEWS scores decreased post-implementation of electronic MEWS. LOE 3 (n=1, the total expected resource intensity, including nurse and RRT utilization increases as a patient’s condition deteriorates, but time to stabilisation and resource time differ depending on the subpopulation), (n=1,retrospective application of the CREWS in patients with chronic hypoxaemic with a NEWS score ≥7 would have reduced the number of reviews by 70.3 % ), (n=1, electronically generated, automated scoring using RI reduced false positive alarms by 53% in comparison to MEWS), (n=1, mean time between MEWS>4 and MET call = 113 min (range 5 to 210 min)), (n=1, use of ViEWS would have resulted in 5,500 more ‘false alarms’ than the derived model using routinely collected laboratory values), (n=1, the number of unique patients that trigger a response daily is higher for binary (than standard (n=118 (SD 20)) NEWS .

83 | P a g e

Clinical Validation

LOE 1-

In a systematic review Roney et al. (2015) sought to evaluate the evidence reporting

outcomes from modified early warning scoring system tools utilisation to prevent

failure to rescue in hospitalised adult medical- surgical/telemetry patients. Authors

found no MEWS assessment tool which combined nursing assessment findings

adjusted for systemic inflammatory response syndrome (SIRS) vital sign criteria and

laboratory values to aid in the identification of both the at-risk and septic patient

was identified within the review. No tool was sourced which incorporated all four

SIRS components of sepsis screening. Findings from the review of literature suggest

MEWS tools’ scoring of physiologic findings; including vital signs has a positive

relationship with earlier detection of clinical deterioration.

LOE 2+

After adjusting for confounding factors, conventional dichotomised activation (i.e. If

one or more of the vital signs met the agreed activation threshold), criteria were not

associated with outcome and discriminated high risk patients poorly. However,

NEWS cumulative score was

able to detect high risk ward

patients regardless of multiple

factors affecting patient

outcome (Tirkkonen et al.

2014).

In the 48 h preceding cardiac

arrest, maximum MEWS was

the best predictor: AUROC:

0.77 (95% CI, 0.71-0.82).

Other predictors of cardiac

arrest: Max RR: AUROC 0.72 (95% CI, 0.65-0.78); Max HR: AUROC 0.68 (95% CI, 0.61-

0.74); Max pulse pressure index: AUROC 0.61 (95% CI, 0.54-0.68); Min DBP: AUROC

0.60 (95% CI, 0.53-0.67). As pulse pressure is less intuitive than other vital signs and

requires a calculation, automated derivation in the electronic medical record may be

necessary for this predictor to be most effective (Churpek et al. 2012a). Respiratory

rate was the best vital sign predictor of cardiac arrest on the ward. However, the

ideal cut-off is unknown, partly because it is often inaccurately measured and poorly

documented in hospitalised patients.

Kirkland et al. (2013) in a retrospective case-control and cohort chart review study

using a multivariate regression analysis found that the significant predictors of

clinical deterioration within 2-12 hours were; The Braden Scale OR=0.91 (95%CI

0.84,0.98; P=0.01); RR OR=1.08 (95%CI 1.04-1.13; P< 0.01); SaO2 OR=0.97 (95%CI

Respiratory rate was the best vital sign

predictor of cardiac arrest on the ward.

However, the ideal cut-off is unknown,

partly because it is often inaccurately

measured and poorly documented in

hospitalised patients.

84 | P a g e

0.96-0.99; P< 0.01); Shock index OR = 2.37 (95%CI 1.14-3.98; P< 0.01). In the

validation dataset, the predictive ability of the model to identify patient

deterioration within 2-12 hours was; Model AUROC=0.71 (95%CI 0.68, 0.74). Authors

noted that their tool created using routinely collected clinical and nursing

measurements (Shock Index, RR, SaO2 and the Braden Scale) can serve as a very

early warning system for adverse events within 12 hours among hospitalized medical

patients.

Mora et al. (2015) in a

retrospective, case-controlled

study found that patients who

trigger an RRT call within 24 h are

at 4-times higher risk of in-hospital

mortality. Patients at higher risk of

triggering an RRT activation could

be identified through higher RR

and HR in the ED. There were 154

RRT calls with 24 h of admission.

Cases had a significantly higher HR at triage, after 3 h in the ED and at discharge than

controls: OR: 1.02 (95%CI 1.02, 1.12) for each beat/min increase in HR prior to ward

transfer. RR was higher in cases than controls: OR: 1.07 95%CI 1.002, 1.030) for each

1 breath/min increase in RR.

Smith et al. (2012) using a prospective cohort design sought to investigate the

relationship between the EWS and the occurrence of major adverse events in

surgical patients during their stay on a general and trauma surgical ward. The

cumulative incidence of adverse events during hospitalization was 8·0 %. Patients

with an EWS ≥3 were shown to have a significantly higher risk of reaching the

combined endpoint (death, reanimation, unexpected ICU admission, emergency

operations and severe complications) i.e. EWS ≥3 compared with patients with an

EWS ˂3: (OR 12·9, 95%CI 6·4, 25·7) and when adjusted for baseline ASA

classification, the odds ratio was 11·3, 95 % CI: 5·5 to 22·9). The AUROC was 0·87 (95

%CI: 0·81 to 0·93). The negative predictive value of an EWS ≥3 was 97%. An EWS ≥3

as a positive test result equated to a sensitivity of 74 % and specificity of 82 %.

Whilst an EWS≥4 as a positive test result equated to a sensitivity of 54% and

specificity of 94%.

LOE 2-

Combined outcome i.e. patients with sepsis admitted to the ICU within 2 days and/or

died within 30 days; NEWS of ≥7: Sensitivity = 72%, Specificity = 54%, PPV =27%,

NPPV = 90%; NEWS of ≥9: Sensitivity = 52%, Specificity = 77%, PPV = 35%, NPPV =

88%. The authors argue that at a NEWS of ≥7 “an argument can be made for

mandating senior ED clinical review for all these patients. There could also be an

Corfield et al. (2014, p.486) argue that at a

NEWS of ≥7 “an argument can be made for

mandating senior ED clinical review for all

these patients. There could also be an

argument for mandatory review by a

critical care outreach team, regardless of

ultimate destination

Corfield et al. (2014, p.486) argue that at a

NEWS of ≥7 “an argument can be made for

mandating senior ED clinical review for all

these patients. There could also be an

argument for mandatory review by a

critical care outreach team, regardless of

ultimate destination.

85 | P a g e

argument for mandatory review by a critical care outreach team, regardless of

ultimate destination” (Corfield et al. 2014, p.486).

Altering EWS thresholds: In hospital mortality was 0% for EWSmax=0 and increased

almost logarithmically to 1% for EWS=3 and 24% for EWS max ie. ≥ 6. One-year

overall mortality was 3%, 12% and 40% for EWSmax=0, EWSmax=3 and EWSmax ≥ 6

respectively; when hospital mortality was excluded this was 3%, 11% and 16% for

the respective EWS values. However raising EWS cut-off for all patients could lead to

unacceptable decreases in sensitivity. By modelling the in the study data- a cut-off

value when raised to EWS ≥4 impacted upon the calculated sensitivity changing it to

74%, i.e. below the predefined 90%. Sensitivity decreased even further to 52% if EWS

≥ 5 was used within the model. Authors noted that lower thresholds result in

increased workload, at the risk of making staff less cautious (van Rooijen et al. 2013).

Eccles et al. (2014) in a prospective, observational cohort study (n=196 admissions of

whom 78 had chronic hypoxia) found that 30-day mortality for chronic hypoxia

patients: NEWS AUROC during stability/at discharge = 0.876 (95%CI 0.788, 0.963);

CREWS AUROC during stability/at discharge = 0.913 (95%CI 0.845, 0.981). Mean

NEWS score was consistently higher for patients with chronic hypoxia (6 ± 3). The

percentage of patients with chronic hypoxia reaching triggering thresholds with

NEWS, was higher than the percentage using CREWS at two thresholds (≥5 and ≥6)

when patients were stable.

A prospective, observational cohort study, MEW scores ≥1 for SBP had a 5-times

increased risk of ICU/HDU admission or death (HR 4.78 95%CI 1.74, 13.15, P = 0.002);

MEW scores ≥2 for RR had a 7.5-

times higher risk of ICU/HDU

admission or death (HR 7.54 95%CI

2.74, 20.77, P<0.001) (Huggan et al.

2015). Authors note that patients

with abnormal SBP or RR should be

‘flagged’ for increased monitoring

and assessment.

Suppiah et al. (2014) sought to

assess accuracy of MEWS and

determine an optimal MEWS value in predicting severity in acute pancreatitis (AP).

The optimal highest MEWS per 24 hours period (hMEWS) and mean MEWS per 24

hour period (mMEWS) in predicting severe acute pancreatitis as determined by ROC

were 2.5 ((AUC 0.924, 95% CI: 0.849 – 0.998)) and 1.625 (AUC 0.91 4, 95% CI: 0.835–

0.993) respectively; with hMEWS ≥3 and mMEWS > 1 utilised in this cohort as MEWS

scores are whole numbers pg. 569. On admission the sensitivity, specificity and

accuracy, of: hMEWS ≥3 was 95.5%, 90.8%, 92% and for mMWES > 1 was 95.5%,

87.5%, 88.7%. The accuracy of hMEWS ≥3 and mMEWS > 1 increased over the

subsequent 72 hours from 92 to 96%, and 89% to 94%. According to the authors:

The percentage of patients with chronic

hypoxia reaching triggering thresholds

with NEWS, was higher than the

percentage using CREWS at two

thresholds when patients were stable.

86 | P a g e

MEWS is suitable for all pancreatitis patients as a routine screening tool on a general

surgical ward and can easily be reassessed to reflect changes in clinical course (p

575).

Shuk-Ngor et al. (2015) using a prospective observational design compared the

performances of detecting patient deterioration with and without using the MEWS

for a group of patients who are waiting for in-patient beds in a public ED. Using

MEWS -- 100% sensitivity and a 98.3% specificity in detecting patient deterioration,

100% sensitivity and a 97.8%) specificity in the comparison control group. Overall,

nurses were found to be able to significantly discriminate between stable patients

and patients at risk of deterioration in the study. The improvement in the validity of

nurses decision making after the introduction of the MEWS system was statistically

significant (p<0.00001).

Alvarez et al. (2013) sought to derive and validate an automated prediction model

based on near real-time electronic

medical record (EMR) data to

identify patients at high risk of a

composite outcome of RED

outside of ICU. Automated

prediction model included 14

variables which were predictors of

RED events in multivariate

analysis; older age (>54 years),

abnormal vital signs: DBP max

<120 mmHg, SpO2 max ≤86%,

Abnormal Laboratory tests.

Aspartate Aminotransferase (AST) >250U/L, white blood cell count >11x103

cells/mm3, platelets <100x103 cells/mm3, potassium >5.1 mEq/L, abnormal arterial

blood gas (ABG) results, Partial pressure of carbon dioxide (PaCO2) (max) ≤22 mmHg,

PaCO2 mmHg (max) >70mmHg, automated physician orders including ABG,

Electrocardiogram (ECG), Stat Physician Order (reflecting a physician’s escalating

concern about patient’s stability). The automated clinical prediction model was more

sensitive than MEWS i.e. Automated clinical prediction model: Sensitivity=51.6%,

Specificity=94.3%, PPV=10%, NPV=99.4%: MEWS: Sensitivity=42.2%,

Specificity=91.3%, PPV=5.6%, NPV=99.2%. The automated clinical prediction model

had good discriminatory for the prediction of RED events (a composite outcome of

resuscitation events and death), and was significantly better than MEWS; Derivation

dataset AUROC curve =0.87 (95%CI 0.85, 0.89); Validation dataset AUROC=0.85

(95%CI 0.82-0.87), MEWS ROC=0.75 (95%CI 0.71-0.78).

Churpek et al. (2014) developed and validated a risk score (eCART) using commonly

collected EHR data in an observational cohort study. Predictor variables were vital

signs (temperature, heart rate, blood pressure, respiratory rate, oxygen saturation),

Automated prediction models can include

multiple and varied parameters e.g. older

age (>54 years) abnormal vital signs:

Abnormal Laboratory tests, ECG, Stat

Physician Order, which in turn can increase

the predictive value of the EWS particularly

if tailored to particular patient

populations.

87 | P a g e

mental status [AVPU] and laboratory results (white cell count, hemoglobin, platelets,

sodium, potassium, chloride, bicarbonate, anion gap, blood urea nitrogen,

creatinine, glucose, calcium, total protein, albumin, total bilirubin, aspartate

aminotransferase, alanine aminotransferase, and alkaline phosphatase) and age

were obtained electronically. (Final model components not stated (16-item from

another paper)). eCART was more accurate than MEWS in predicting all outcomes

(P<0.01). eCART cut-off of ≥50 would detect; 51% cardiac arrests, 44% ICU transfers,

83% deaths.

Escobar et al. (2012) in a retrospective case-control study the EMR-based model

performed better than the calculated MEWS for predicting unplanned ICU transfer

(or death on ward). EMR-based detection of impending deterioration outside the ICU

is feasible in integrated healthcare delivery systems. Future models must investigate

discrimination by individual diseases more thoroughly as the model performed

differently in different disease subpopulations.



outcomes of the MET calls since its introduction. In multivariate analysis, RR and

GCS were significantly correlated with in-hospital mortality: RR: OR 1.043 (95%CI

1.019, 1.068; P<0.0001); GCS: OR 0.886 (95%CI 0.820, 0.958; P=0.002).

Liu et al. (2014b) in a prospective observational study which compared MEWS, TIMI

Score and a ML-based EWS system selection found that the ML model with the top

three variables had the highest predictive ability for MACE within 72 hours, than all

other models with different number of variables. ML score (3 variable model; SBP,

aHR, aRR) AUROC=0.812 (95%CI 0.716, 0.908); ML score (23 variable model)

AUROC=0.736 (95%CI 0.630, 0.841); TIMI AUROC=0.637 (95%CI 0.526, 0.747); MEWS

AUROC=0.622 (95%CI 0.511, 0.733).

Liu et al. (2014a) sought to develop a novel intelligent scoring system for the early

identification of patients at high risk of cardiac arrest within 72 hours, using ECG and

vital signs, and to compare it with

established EWS. The ESS predictor

parameters which had the best

discriminatory ability were; *ECG,

HRV, Vital signs, AUROC=0.837

(95%CI 0.724, 0.949). ECG, HRV,

AUROC=0.812 (95%CI 0.694, 0.930).

ECG, Vital signs, AUROC=0.815

(95%CI 0.697, 0.932). HRV, Vital signs, AUROC=0.759 (95%CI 0.632, 0.886). ESS had

better combination of sensitivity and specificity for the prediction of acute cardiac

complications within 72 hours. The low PPV and NPV is due to imbalanced data i.e.

there are more patients without complications.

Ong et al. (2012) noted that there is

potential to develop bedside devices to

assist in risk stratification of patients.

88 | P a g e


design sought to validate a novel Machine Learning (ML) score, incorporating HRV

against MEWS for the risk stratification of critically ill patients in the ED. The ML

score had a significantly better discriminatory ability to predict cardiac arrest within

72 h than MEWS (P=0.037): ML score AUROC= 0.781; MEWS AUROC= 0.680. In the

low, intermediate and high risk ML score groups, the rate of cardiac arrest within 72

hours increased from 0%, to 1.6% and 13.1%, respectively. Median MEWS score was

significantly higher for patients experiencing cardiac arrest (4 (IQR 2 to 5) than those

who did not (2 (IQR 1 to 4; P<0.001). A cut-off MEWS ≥ 3 predicted cardiac arrest

within 72 h with a: Sensitivity=74.4%, Specificity=54.2%, PPV=7.4%, NPV=97.8%. ML

scores incorporating HRV parameters, age and vital signs were more accurate than

the MEWS in predicting cardiac arrest within 72 hours. Authors noted that there is

potential to develop bedside devices for risk stratification based on cardiac arrest

prediction.

Cattermole et al. (2014) in a prospective observational study (n=234 ED patients)

validation study which

compared the performance of

various EWS found that AUROC

for each EWS for predicting the

composite output of ICU

admission or death within 7.

NEWS is entirely physiological

not requiring bedside blood

tests. Thus the scoring of some

of the parameters in NEWS

limits its usefulness in the ED e.g. use of supplemental O2 scores highly, which is

reasonable for stable inpatients. But many ambulance and resuscitation-room

patients are routinely given oxygen initially, with subsequent titration or removal. In

addition NEWS does not discriminate between degrees of reduced consciousness.

Max THERM score is 37. THERM, high risk cut-off (≤30): Sensitivity = 0.57 (95% CI

0.40-0.73), Specificity = 0.89 (95% CI 0.84-0.93), PPV = 0.50 (95% CI 0.34-0.66), NPP =

0.92 (95% CI 0.87-0.95). NEWS, high risk cut-off: Sensitivity = 0.65 (95% CI 0.48-0.80),

Specificity = 0.71 (95% CI 0.64-0.77), PPV = 0.29 (95% CI 0.20-0.40), NPP = 0.91 (95%

CI 0.86-0.95).

Geier et al. (2013) in a prospective observational study investigated the diagnostic

and prognostic accuracy of the ESI, MEWS and MEDS regarding severe sepsis and

SSSS for patients presenting to the ED. MEDS had the highest in-hospital 28-day

mortality of patients with suspected sepsis (Prognostic accuracy). MEDS also the

highest sepsis diagnostic accuracy (i.e. MEDS AUROC=0.778 (95%CI 0.704, 0.853);

MEDS (≥8): Sensitivity=0.857, Specificity=0.682, PPV=0.305, NPV=0.967. MEDS score

is based on clinical criteria that consider organ dysfunction (e.g. tachypnea or

Geier et al., (2013) noted that EWS tools

could be amended by addition of disease-

specific risk stratification or EWS could be

extended by including disease-specific

parameters instead of using two tools.

89 | P a g e

hypoxia, presence of septic shock, platelet count <150,000/mm3, and altered mental

status) as well as age, nursing home resident, presence of lower respiratory infection

and rapidly terminal comorbid illness “which adds to its diagnostic accuracy in this

patient group. Authors noted that ESI and MEWS could be amended by disease-

specific risk stratification tools like MEDS or ESI and MEWS could be extended by

including disease-specific parameters instead of using two tools.

Jo et al. (2013) investigated (i) whether the predictive value of ViEWS in unselected

critically ill patients could be

improved by including rapid lactate

levels (ViEWS-L). The ViEWS-L score

had significantly better predictive

value than the ViEWS, HOTEL and

APACHE II scores for the four

mortality outcomes. Hospital

mortality (p=0.009): ViEWS-L

AUROC=0.802 (95%CI 0.729, 0.875.

Addition of lactate measurement

can increase sensitivity of VIEWS in

predicting mortality.

Romero-Brufau et al. (2014)

evaluated a number of EWS

systems (MEWS, SEWS, GMEWS,

Worthing, ViEWS and NEWS) and RRT single parameter activation criteria in use in

the institution. Positive predictive values ranged from less than 0.01 (Worthing, 3 h)

to 0.21 (GMEWS, 36 h). Sensitivity ranged from 0.07 (GMEWS, 3 h) to 0.75 (ViEWS,

36 h). Thus MEWS had the best specificity, but missed many events; VIEWS detected

more events, but identified many false positive alerts. Used in an automated fashion,

these would correspond to 1040–215,020 false positive alerts per year. The MEWS,

SEWS and the researchers own institution’s RRT criteria demonstrated less of a

decay in specificity; however, they did not reach the same degree of sensitivity

through time (data represented diagrammatically without figures, no figures for

NEWS). Authors noted that when the evaluation is performed in a time-sensitive

manner, the most widely used weighted track-and-trigger scores do not offer good

predictive capabilities for use as criteria for an automated alarm system. For the

implementation of an automated alarm system, better criteria need to be developed

and validated before implementation (p 549).

Yu et al. (2014) examined and compared the ability of nine prediction scores (SOFA,

Predisposition/ Infection/Response/Organ Dysfunction Score (PIRO), ViEWS, SCS,

MEDS, MEWS, SAPS II, APACHE II and REMS) to estimate the risk of clinical

deterioration. At the 0- to 12-hour interval before clinical deterioration, all scores

except REMS (AUC 0.67 95%CI 0.62, 0.71) performed with acceptable discrimination

Romero-Brufau et al., (2014) noted that

when the evaluation of a patient is

performed in a time-sensitive manner, the

most widely used weighted track-and-

trigger scores do not offer good predictive

capabilities for use as criteria for an

automated alarm system. For the

implementation of an automated alarm

system, better criteria need to be

developed and validated before

implementation (p. 549).

90 | P a g e

(i.e. AUC ≥0.70) and had roughly equivalent AUC. However, at the 12- to 72-hour

intervals, all scores, with the exception of MEDS (AUROC=0.69 (95%CI 0.63-0.74) at

24 to 48 hours and AUROC=0.71(95%CI 0.64-0.78) at 48-72 hours), no longer

performed with acceptable discrimination for mortality (AUROC <0.70). For all

models, average scores of cases increased closer to time of clinical deterioration (P

<0.05). For the MEWS, SAPS II, APACHE II, and REMS scoring models, this increase

was detected as early as 12 to 24 hours before deterioration (P <0.05). For SOFA this

increase can be detected even

earlier at 24 to 48 hours before

clinical deterioration. That is, the

average SOFA score of cases

during the 24- to 48-hour interval

was significantly higher than

during the 48- to 72-hour interval

(P = 0.01). In contrast, average

scores of controls did not increase closer to the index time. Scoring models that take

advantage of score’s change over time may have increased prognostic value over

models that use only a single set of physiologic measurements. Compared with

controls, cases were generally older, more likely to be male, and more likely to be

admitted from a nursing home.

Considine et al. (2015) evaluated the effect of the staged implementation of a RRS

on reporting of clinical deterioration in ED patients. Unreported clinical deterioration

decreased from pre- to post-implementation; this was clinically significant but not

statistically significantly (P=0.141): 2009-10: 17.9% decrease; 2010-11: 13.5%

decrease; 2011-12: 1.3% decrease; Overall decrease 32.7% (2009-12; P=0.198). RR

>30 breaths/min, HR >120 beats/min and SBP <90 mmHg were most commonly

unreported. Between 2009 and 2012 unreported RR and HR decreased by 57.6%

and 15.9%, respectively. SBP decreased between 2009 and 2011 (45.7%) and

increased during 2011 and 2012 (10%). Patients who were significantly more likely to

deteriorate (p<0.001) if they arrived by ambulance, were triaged to Australasian

Triage Scale (ATS) categories 1 or 2, and had a mean 2.8 hour longer median stay

compared to patients who did not deteriorate. Patients who deteriorated in the ED

were also significantly more likely to be admitted than those who did not (31.9%;

P<0.001).

Edelson et al. (2011) developed and tested a judgement-based scale for conveying

the risk of clinical deterioration A PAR score cut-offs had the following characteristics

for predicting cardiac arrest or patient transfer to ICU within 24 hours: PAR≥4,

Sensitivity=82,4%, Specificity=68.3%; PAR≥5, Sensitivity=84.6%, Specificity=62.2%.

Implementation of PAR could improve the communication regarding at risk patients

between healthcare professionals during handoffs. However the ability of PAR to

Optimum NEWS triggering thresholds for

RRT activation at different NEWS scores

differ for different subpopulations of

patients e.g. older adult, frail patient.

91 | P a g e

predict cardiac arrest or transfer to ICU within 24 hours differed significantly

depending on the user (P=0.01).

LOE 3

A highly frail surgical patient (defined using surrogate marker of BSS ≤11 without

previous deterioration events would benefit from RRT activation when the NEWS is

[1-4] or with a single extreme value, whereas the threshold is NEWS ≥7 for a

moderately frail medical patient. In addition, any surgical patient (regardless of BSS

at admission) would benefit from RRT activation when the NEWS is [1-4], or with a or

single extreme value (Capan et al. 2015a).

The optimum policies for RRT activation at different NEWS scores differ for different

subpopulations of patients (Capan et al. 2015a; Capan et al. 2015b). Two critical

sub-populations were identified; (a) for surgical patients with low risk of

deterioration (ROD) who may be healthier and able to recover from deterioration

without RRS intervention, and therefore should only activate an RRS at NEWS>4; and

(b) escalation should be considered at for patients in ‘slightly concerning’ or worse

health states-NEWS>0 for all other subpopulation (Capan et al. 2015b).

A score of 3 for a single vital sign in NEWS is too low by itself to indicate imminent

risk of adverse effect (with the exception of temperature ≤35°C). An alternative

protocol would be to increase frequency of observation within these patients, but

not to escalate only on one vital sign score of 3 (Jarvis et al 2015a).

An individual score of 3 for low temperature (≤35°C) was the only single vital sign

that significantly increased risk of cardiac arrest above that of an aggregate score of

5. But this is rare therefore loss of consciousness as a single vital sign is a better

measure of risk; however risk was not significantly higher that an aggregate score of

5 (Jarvis et al. 2015a).

An aggregate NEWS score of ≥3 performed best for the identification of patients

with SS. For the identification of a patient at risk of SS; NEWS AUROC = 0.89 (95%CI

0.84, 0.94) (Keep et al. 2015). NEWS ≥3: Sensitivity=92.6%, Specificity=77%,

PPV=18.7%, NPV=99.5% for sepsis; NEWS ≥4, Sensitivity=74.1%, Specificity=86.5%,

PPV=23.8%, NPV=98.3% (Keep et a.l 2015). Authors noted that a NEWS ≥3 at ED

triage may be the trigger to systematically screen for septic shock, obtain an early

serum lactate and where appropriate start fluid resuscitation and antibiotic therapy”

(Keep et al. 2015, p.4).

Retrospective application of the CREWS in patients with chronic hypoxaemic with a

NEWS score ≥7 would have reduced the number of reviews by 70.3 % (Lobo et al.

2015) (Irish Study). The two parameters with individual scores of 3 contributing to an

aggregate score of ≥7 were O2 supplementation (89.9% of cases) and O2 saturation

(31.6 % of cases). 19.7 % of patients were receiving home oxygen therapy which

would also give them a score of 3. Patients with chronic hypoxia conditions

92 | P a g e

frequently have scores of 3, even though their condition is stable. Therefore

applying this NEWS trigger score would lead to persistent triggering in these

patients.

Mean MEWS score within 24 h prior to cardiac arrest was 2.24. MEWS scores within

24 hours of the cardiac arrest corresponded to an appropriate response of referral

pathways to doctors, but critical care outreach team service is underutilised: MEWS

score 1-2 = 10.3% (n=3) of patients had a doctor informed; MEWS score 3-4 = 57.2%

(n=8) of patients had a doctor informed; MEWS score ≥5 = 83.3% (n=5) of patients

were referred to a doctor urgently. For 1 patient the critical care outreach team was

called. The majority of patients had their MEWS score recorded (n=25; 78.8%). Final

observation set prior to cardiac arrest were recorded by staff nurses (n=13; 39.9%)

or healthcare assistants (n=20; 60.6%). The majority of cardiac arrests occurred out

of hours (69.7%) (Harris 2013).

In a retrospective cohort study, 267 patients were admitted to HD/ICU (37.4%) in the

MEWS <4 group as compared with 86 (27.7%) patients in the MEWS ≥4 group.

MEWS Score cut offs were poor predictors of ICU/HD transfer; MEWS <4 AUROC=

0.49; MEWS <5 AUROC= 0.47. The average LOS for the MEWS <4 group was 6.97

days and for the MEWS ≥4 group was 7.75 days (Ho et al. 2013).MEWS had low

sensitivity for predicting mortality or ICU admission.

Stark et al. (2015) in a retrospective study investigated the ability of a MEWS to

identify patients at higher risk of death. In-hospital mortality: MEWS score of ≥4,

Sensitivity=91%, Specificity=48%, PPV =71%, NPP=80%; MEWS score of ≥5,

Sensitivity=68%, Specificity=68%, PPV=74%, NPP=61%.

Bian et al. (2015) sought to find a scoring system to predict the onset of Acute Heart

Failure (AHF) in patients in the acute heart failure unit. SUPER scoring system

included SpO2, Urinary volume, Pulse, Emotional state and RR, each scored between

0 and 2 points. SUPER was significantly better than the MEWS at identifying patients

at risk of AHF (P<0.05). There was no statistical significance of adding age to the

SUPER score: SUPER + Age AUROC=0.820; SUPER AUROC=0.811; MEWS

AUROC=0.662. Incidence of AHF by hours increased significantly with higher SUPER

scores. Authors noted that in patients at high risk of AHF, the SUPER scoring system

could predict the onset of AHF 2 to 6 hours earlier. Pre-emptive treatment according

to the SUPER score may prevent or delay AHF occurrence to improve quality of life,

reduce mortality and waste of medical resources

Churpek et al. (2014) sought to derive and validate a prediction model for cardiac

arrest while treating ICU transfer as a competing risk, using EHR data. ViEWS was

chosen as the best performing system of 33 tested (Kellett, Kim 2012) for

comparative purposes. Derived cardiac arrest model: Time (hours): time from first

ward vital sign, Prior ICU stay, HR (beats/min), DBP (mmHg), RR (breaths/min),

Temperature, Supplemental O2 use, age (years), Blood urea nitrogen (mg/dL), Anion

gap (mEq/L), Platelet count (K/uL), White blood cell count (K/uL). Derived ICU

93 | P a g e

transfer model: Time (hours): time from first ward vital sign, Prior ICU stay, HR

(beats/min), DBP (mmHg), RR (breaths/min), O2 saturation (%), Temperature,

Mental status (AVPU), Supplemental O2 use, age (years), Blood urea nitrogen

(mg/dL), Anion gap (mEq/L), Haemoglobin (g/dL), Platelet count (K/uL), potassium

(mEq/L), White blood cell count (K/uL). Cardiac arrest within 24 hours: Derived

cardiac arrest model AUROC=0.88 (95%CI 0.88-0.89); ViEWS AUROC=0.74 (95%CI

0.72-0.75). Use of ViEWS would have resulted in 5,500 more ‘false alarms’ than the

derived model. Routinely collected laboratory values added to the model were

significant predictors of both outcomes.




than MEWS for predicting cardiac arrest (P=0.001): eCART AUROC=0.84; MEWS

AUROC=0.76. CART score was significantly more accurate than MEWS for predicting

ICU transfer (P<0.001): eCART AUROC=0.71; MEWS AUROC=0.67. The addition of

DBP instead of SBP was useful in improving the accuracy of CART over MEWS in

predicting cardiac arrest.

Jarvis et al. (2013) built an EWS exclusively on routine laboratory tests using DT

analysis and noted that commonly measured laboratory tests collected soon after

hospital admission can be represented in a simple, paper-based EWS (LDT-EWS) to

discriminate in-hospital mortality. Male and female LDT-EWS developed using

Biochemical and haematology blood test parameters, with acceptable ranges were

included (Hb, WCC, U (0.4-107.1 mmol/L), Alb (10-70 g/L), Cr (8.8-2210 umol/L), Na

(100-200 mmol/L) and K (1-15 mmol/L). A 0-3 weighting system for risk bands was

developed. A LDT-EWS score of 4 would trigger a response in 40.7% of all laboratory

test datasets. 79.7% of all patients having a trigger would subsequently die.

Different trigger scores were observed for males and females.

Liljehult and Chirstensen (2015) in a retrospective cohort study found that an EWS

from readily available physiological parameters is a simple and valid tool for

identifying patients at low, intermediate and high risk of dying after acute stroke.

Patients with a EWS≥5 or whose EWS score increases during the admission period

indicates closer observation and monitoring and increased risk of death in the latter

case among patient with acute stroke. This EWS could be used as a tool to select

patients who need to be moved to the stroke unit. “Sensitivity was highest for the

lowest scores of both admission EWS and max EWS and decreased with rising scores,

indicating that the risk of false-negative test results was greatest with the higher

scores. Overall, the sensitivity was best for max EWS. Specificity was low at lower

scores and increased with rising scores, indicating that the risk of false-positive test

results was greatest with lower scores. Overall specificity was best in admission EWS.

PPVs were low at lower scores in both admission EWS and max EWS, but increased

with rising scores, whereas NPVs were high at all levels. Of the individual vital signs

94 | P a g e

only RR (AUROC 0.673; P 0.005) and AVPU (AUROC 0.721; P < 0.001) were

significantly better at distinguishing between survivors and non-survivors than pure

chance.

Tarassenko et al. (2011) sought to develop an alerting system using the hypothesis

that an EWS of 3 (which, in most systems, initiates a review of the patient) should be

generated when a vital sign is below the 1st centile or above the 99th centile for that

variable (for a double-sided distribution)” and the assumption that alerts occur

whenever a “score of 3 is assigned to a single variable and a score of ≥4 for the

multivariate case. Authors highlight issues with choosing mortality or ICU admission

as the outcome noting that there is no obvious binary outcome for early

deterioration. The new EWS differs most in respect of respiratory rate (≤7) and

systolic blood pressure (≤85) trigger values. However if used in clinical practice based

on data used in the development of the tool (with high risk patients), with four-

hourly observations in a 12-h shift, about 1 in 8 at-risk patients would trigger the

alerting system during the 12 hour shift.

Jarvis, Kovacs, Briggs et al. (2015b) in a retrospective analysis compared 36 published

EWSs and investigated whether EWS if truncated to a binary score of 0 ‘normal’ or 1

‘abnormal’ resulted in a decrease in errors associated with weighting or scoring EWS.

The researchers found that PPV and sensitivity of the standard weighted NEWS is

better than binary NEWS; NEWS aggregate score ≥5, sensitivity=69.7%,

specificity=94.2%, PPV=11.8%, NVP=99.6% and in contrast the binary NEWS score ≥3,

sensitivity=67.7%, specificity=92.9%, PPV=9.6%, NVP=99.6%. Binary NEWS trigger

score ≥3 would potentially detect more adverse outcomes than NEWS score at a

trigger score ≥5, but would in turn require a 15% higher triggering rate. The

percentage of observation sets that trigger a response is higher for binary (11.8%)

than standard (10.2%) NEWS. The number of unique patients that trigger a response

daily is higher for binary (n=145 (SD 24)) than standard (n=118 (SD 20)) NEWS.

Barriers and facilitators to EWS system implementation

Although the beneficial effects of implementing EWS are becoming clearer as such systems

are more universally applied, not all the potential benefits of EWS are realised. There are

several potential explanations for this and understanding the barriers and facilitators maybe

helpful in this regard. The barriers and facilitators can be categorised as follows:

management/organisational/setting specific issues; education/ training matters; issues

with the EWS system (complexity of system, observation tool, tools (paper/electronic) and

escalation) (Table 2, Figure 8).

Understanding the organisational culture, systems, practices, and stakeholders pre-

implementation is extremely important and is a unifying concept among many of the studies

reviewed. A change in culture from one of blame to a supportive one (Winters et al. 2013;

Moriarty et al. 2014; Lydon et al. 2015) is the hallmark of a learning system. The success of

new interventions depends on human interaction with the system and understanding the

95 | P a g e

variable organisational practices is critical; this involves understanding how the nursing staff

incorporate the EWS system into their daily work routines and how they feel the system

works for them (Nwulu et al. 2012). Quality improvement methodology is helpful in

supporting implementation of EWS (Huddart et al 2015).

The importance of seeking the engagement and support of key stakeholders is critical;

clinical champions may be helpful in this regard (Woods et al 2015). Continuous

engagement and communication with all stakeholders identifies barriers early.

Understanding that the EWS System is a complex intervention is critical. Maturation of the

intervention (i.e. over time) and improved implementation strategies can lead to improved

results in and across institutions (Winters et al. 2013; Moriarty et al. 2014). However, higher

levels of adherence to EWS are necessary to ensure effectiveness and long-term patient

level benefits (Lindsey & Genkins 2013; Mathukia et al 2015). Measures to support

adherence include continuous education, clinical champions, audits, publication of findings,

and dedicated specifically trained observationalist23. Multidisciplinary EWS training may

foster team working and clarify roles, responsibilities and workloads of members and

increase support for EWS implementation by senior doctors, thus improving adherence to

protocol (Lydon et al. 2015).

Charts need to be well designed, integrating human factors24 perspective. Automating the

identification of a deteriorating patient through continuous monitoring and a directly

activated response team (the automated alert) potentially would both improve sensitivity

and fidelity and mitigate cultural barriers (Subbe & Welch 2013; Winters et al. 2013). Such a

system could help to reduce potential errors in the calculation of summative scores.

Escalation of care is one of the key cornerstones of the EWS yet healthcare practitioners

delay activating the escalation protocol (Massey et al. 2014). Capan et al. (2015b, LOE 3),

using a stochastic model of acute-care decisions noted that using differing activation

thresholds for different subpopulations the RRT can be activated on average 5.2 hours

earlier. Earlier recognition of the need for activation of an emergency response system can

improve the stabilization of patients, and potentially reduce the incidence of undesired

outcomes. Thus it is important that organisations promote awareness of the significance of

early referrals and explore barriers to delays in escalation especially out of hours. Track and

trigger systems that interface with emergency response systems which are supplemented

with decision aides (algorithms) and clinical support systems produce a more effective

screening system for early identification of deteriorating patients (Mapp et al. 2013)

23

An observationalist is a person who is specifically employed to observe, document, report, benchmark practice using their real-world assessments (observational comment) about the implementation of EWS with a focus on ongoing targeted quality improvements. 24

Human factors relates to the understanding of interactions among humans and other elements of a system, with the integration of theory, principles, data and methods and the aim of optimising human well-being and overall system performance.

96

Table 2. Summative perspective on barriers and facilitators and barriers to EWS system

Domain Enablers/Facilitators Barriers

Management/organisational/setting specific Understanding the organisational culture, systems, practices and the stakeholders pre-implementation.

Achieving surveillance through EWS for the whole patient journey (Schmidt et al. 2015) Understanding that the success of new interventions depends on human interaction with the systems and variable organisational practices (Nwulu et al. 2012). Identifying and defining ward culture (Woods et al. 2015). A change in culture from one of blame to a supportive one, address sociocultural issues before implementation (Winters et al 2013; Moriarty et al. 2014; Lydon et al. 2015). Organisations need to address power hierarchy between medial teams to reduce delays in response to deteriorating patients (Mackintosh et al. 2014).

Hierarchical system (Wood et al. 2015). Conferring with the nurse in charge before MET activation reinforcing the understanding that a culture of seeking support to validate clinical decisions exits in some institutions (Massey et al. 2014). The majority of observations, including MEWS were recorded by healthcare assistants who may not have the skills to recognise a deteriorating patient, document observations accurately, calculate MEWS correctly or initiate an appropriate response (Finlay et al. 2014).

Seek engagement and support of key stakeholders. Clinical champions.

Engagement and support of key stakeholders (Woods et al 2015). Clinical champions (Woods et al. 2015).

Nurses were initially anxious about using the new electronic chart, fearing increased workload and monitoring (Nwulu et al. 2012).

Support implementation using quality improvement methodology-- includes pre-post-intervention analysis with longitudinal measurement of predefined organisational, HCP, and patient level outcomes with feedback sessions.

Support implementation using quality improvement methodology (Huddart et al. 2015). A pre-post-intervention analysis reveals barriers to implementation and allows teams to address these (e.g. ordering handheld devices, identifying processes for repair of malfunctioning devices) (Rose et al. 2015). Feedback sessions with participants identified some barriers (Ludikhuize et al. 2011). Communication of the results of studies (Christofidis et al. 2013). Research on implementation strategies is required (Ludikhuize et al. 2011) A more mandatory nature of implementation and measurement of outcomes would assist in the continual

EWS had a negative impact on interns’ perceptions of intern-nurse teamwork, possibly due to separate training and lack of clarity re roles and responsibilities in the RRS (Lydon et al. 2015). Negative responses when escalating (Wood et al 2015). A lack of support from senior doctors, who have a low engagement with NEWS and poor attendance at education and training sessions (Lydon et al. 2014).

97


optimization and research into RRS (Ludikhuize et al. 2015). Assessments of EWS performance should report proportions of alerts missed or generated in error, instead of the overall error rate (Clifton et al. 2015).

EWS System is a complex intervention including: The EWS system

Systematic monitoring practice/protocols

EWS triggers, and observational chart

An algorithm for bedside management

Escalation protocol with emergency response system

Implemented by inter-professional teaching, training and optimization of communication and collaboration

High levels of adherence to EWS is necessary to ensure effectiveness

Measures to support adherence (e.g. education, clinical champions, audits, publication of findings, observationalist)

Clinical Interventions comprising systematic monitoring practice, EWS, and observational charts, and an algorithm for bedside management, implemented by inter-professional teaching, training and optimization of communication and collaboration, can reduce unexpected in-hospital mortality. (Bunkenborg et al. 2013) EWSs that interface with emergency response systems and are supplemented with decision aides (algorithms) and clinical support systems produce an effective screening system for early identification of deteriorating patients (Mapp et al 2013). High levels of adherence to EWS are necessary to ensure effectiveness (Lindsey & Jenkins 2013; Mathukia et al. 2015). High compliance with all elements of the guidelines are necessary for them to be effective (Niegsch et al. 2013). Specifically trained ‘observationists’ (Lindsey & Jenkins 2013) Monthly audit results communicated verbally and via e-mail to each ward in addition to placing them on performance boards (Wood et al 2015). Mapping outreach episodes of care and patient outcomes can help highlight areas for improvement (Pattison & Eastham 2012).

Effectiveness of complex interventions e.g. RRT are not visible in terms of patient level data for some time e.g. until 18 months post intervention (Moriarty et al. 2014). Poor compliance with EWS system protocols (Peterson et al. 2014; Kolic et al. 2015).

Education/ training (of all providers involved)

Multidisciplinary training

Multidisciplinary EWS training may foster team working and clarify roles, responsibilities and workloads of members and increase support for EWS implementation

Lack of multidisciplinary training. Once off training. Online training exclusively.

98


Continuous information/ updates Support for afferent link (informed

clinical judgment) Maintaining transparency in order

to maximise learning from case reviews

by senior doctors, thus improving adherence to protocol (Lydon et al. 2015). Continuous information and education of all healthcare professionals involved in the care of at-risk patients (Winters et al. 2013; Etter et al. 2014; Kyriacos et al. 2015). Training improves patient outcomes (Alam et al. 2014). A mechanism to encourage nurses to use their clinical judgement in their perception of clinical deterioration instead of just a ‘blind adherence’ to the NEWS is required (Lydon et al. 2015). Maintaining transparency in order to maximise learning from case reviews (Wood et al 2015).

Observation tool: Observation protocol which helps

to risk stratify patients at risk for increased frequency of observation

User-friendly, well-designed chart which integrates human factors perspective

Graphical data display Enhance patient monitoring and

recording of respiratory rate Evaluate compliance with a

standard EWS observation protocol

Presence of nurse observation protocol (Kyriacos et al 2015). The multi parameter system introduced (using a ‘protocol’ defining the measurement of MEWS three times daily (Ludikhuize et al. 2014) resulted in more comprehensive vital sign measurements. Implementing and evaluating compliance with a standard protocol for frequency of physiological observations (Kyriacos et al. 2015) Risk stratification of patients at the beginning of night shifts may help to identify those in need of more frequent observations (De Meester et al. 2013a). Knowledge of activation criteria (Winters et al 2013; Kyriacos et al 2015) Enhance patient monitoring and recording of respiratory rate (Shuk-Ngor et al. 2015). Use of automated respiratory monitoring devices to assist nurses in RR monitoring helps with this (Liaw et al. 2015). Assigning a weight to clinician concern within the EWS may improve the accuracy of the EWS (Clifron et al. 2015). Participants made significantly fewer errors and

Predetermined times to make ‘observation rounds’ (Hands et al. 2013) Different versions of the track and trigger systems used in different settings- many of which have not been validated (Patel et al. 2015).

99


responded significantly faster when using a novel, user-friendly, well-designed chart which integrated human factors perspectives (Christofidis et al. 2013)--- suggesting that blood pressure and heart rate observations are plotted separately, precluding use of the Seagull Sign (Christofidis et al. 2013). Chart design informed by multidisciplinary teams of human factors specialists and clinicians with empirical evaluations of proposed designs Christofidis et al. 2015). Graphical data display is superior to numerical data display in terms of faster and more accurate interpretation of information (Fung et al. 2014). It is unrealistic to expect a complete set of vital signs to be measured each time. Vital signs pertinent to the clinical concern of individual patients should be reviewed (Nwulu et al. 2012). An automated model harnessing available EMR data automatically offers great potential for identifying adverse events (Churpek et al. 2012a; Alvarez et al. 2013; Churpek et al. 2014).

Tools (electronic) Support Standardisation and

automation Facilitate electronic bedside

capture of EWS data

Standardisation and automation (Subbe & Welch 2013) Handheld electronic equipment/ Personal Digital Assistants and associated EWS software to do the calculations and escalations (Bleyer et al. 2011; Ludikhuize et al. 2011; Jones et al. 2011; Hands et al. 2013; Badriyah et al. 2014; Jarvis et al. 2015a). Electronic bedside capture of EWS data reduces errors (Smith et al. 2013). Use of an electronic EWS may improve standardisation, calculation documentation and escalation (Niegsch, et al. 2013). Use of PDAs (personal digital assistants) and tablets with Wi-Fi capacity can improve many points on the implementation chain (Subbe & Welch 2013). Use of additional technologies such as ‘natural language

Insufficient and malfunctioning handheld recording units (Rose et al. 2015). Not possible to determine errors in vital sign entry in the electronic system (Bleyer et al. 2011). Need to manually enter vital sign data into the electronic medical record. No mechanism to alert nurses to missing or inaccurate data (Stewart et al. 2014).

100


processing’ and ‘adverse drug event detection software’ may improve prediction of poor hospital outcomes (Alvarez et al. 2013). Potential to include a greater number of variables/tools/ disease specific parameters (Geier et al. 2013); blood test results (Jarvis et al. 2013).

Tools (paper based) Consider complexity of tool if

paper based Potential errors in calculation of

summative score.

For paper based systems a simplified binary ESW is more advantageous (Jarvis et al. 2015a). Consider complexity of system (Carmichael et al. 2011).

Variability in the assessment and recording of vital signs, incomplete vital sign recording (Christensen et al. 2011; Gordon & Beckett 2011; Parham 2012; Escobar et al. 2012; Etter et al. 2014; Clifton et al. 2015). The electronically calculated score differed to the AWS score (PARS) manually recorded on the patient observation chart e.g. 27% of cases (Abbott et al. 2015) and 18.9% of patients had an incorrectly calculated NEWS score (Kolic et al. 2015). High rate of false alarm is a disadvantage, and may be improved with the move to an electronic form of MEWS (Mathukia et al. 2015)

Escalation Promote awareness of the

significance of early referrals The automated alert Clear and concise referral pathway

to emergency response system (RRT, physicians) with support from all members of the team

Consider barriers to delays in escalation especially in out of hours

Awareness by all health care professionals of the significance of early referrals (Pattison & Eastham 2012) Communication tools (e.g. SBAR) may facilitate the referral process by enabling nurses to give an accurate, clear and concise telephone referral (Harris 2013) PDAs used to record RRT responses (Morris et al. 2013) Clinical expertise (Winters et al 2013; Kyriacos et al. 2015) Availability of senior medical expertise (e.g. consultants) during out of hours (Kolic et al. 2015) Support by medical and nursing staff(Winters et al. 2013) Communication and teamwork (Winters et al. 2013) Non critical, judgemental attitude of outreach team. Outreach- trusted resource; add weight to nurses’ opinions; outreach-supportive relationship (Pattison & Eastham 2012).

Current education, especially regarding the documentation of actions and escalations may be deficient (Niegschet et al. 2013). Inexperience with new system (Christofidis et al. 2013) Reasons for delays in activating the MET included, not knowing if it was the right thing to do, not wanting to appear an idiot, fear of getting into trouble, fear of reprisals or punishments (Massey et al. 2014). Delays in activation and clinical response (also termed Score to Door Time) (Oglesby et al. 2011). The belief that use of NEWS leads to a failure to use clinical judgement (Pattison & Eastham 2012; Lydon et al. 2015). Perception by experienced nurses that they used it less as they relied on their own judgement (Pattison & Eastham 2012). Misjudgement by HCPs of their ability to handle

101


The automated alert (Bleyer et al 2011; Umschield et al. 2015; Kolic et al. 2015). Bedside entry of electronic clinical observations and matching of aggregated EWS to automated alerting logic using the Patientrack system significantly improved timely clinical attendance to acutely ill adult medical patients with EWS score ≥3 (e.g. clinical attendance to patients with EWS 3, 4 or 5 increased from 29% at baseline to 78% with automatic alerts (P<0.001)); an associated reduction in critical care use was also reported (Kolic et al. 2015). RRT use improved EWS scoring consistency (Smith et al. 2014). Education regarding the purpose and function of RRS (Winters et al 2013). Increasing diagnostic services out of hours (Kolic et al. 2015).

patients’ condition (Pattison & Eastham 2012). Referral to outreach may threaten trust between ward nurses and doctors who had been managing the situation

on the ward (Pattison & Eastham 2012).

Higher EWS were significantly less likely to be monitored according to protocol (Escobar, et al. 2012; Peterson et al. 2014; Kolic et al. 2015). Sicker patients with higher VIEWS score got more frequent vital signs monitoring but not consistently and less so at night (Hand et al. 2013). Delays between vital sign instability and subsequent MET call (Etter et al. 2014). Escalation protocol was not achievable given the resources (Hands et al. 2013). A lack of critical care beds (Peterson et al 2014) Lack of knowledge of who was in the RRT (Rose et al. 2015) Clinical response to NEWS scores is significantly worse at weekends (Kolic et al. 2015) and at night (Lindsey & Jenkins 2013). The impact of false positives; EWSs have low sensitivity and there is insufficient staffing levels, especially at night, therefore interns especially encounter many false alarms. High false alarm rates are exasperated by inappropriate use of escalation protocols by nurses and patients with abnormal baseline vital signs not having their NEWS parameters altered by senior doctors (Lydon et al. 2015). A feeling among staff that the situation was under control in the ward setting (Peterson et al 2014).

102

Figure 8. A summative perspective on enabling optimum EWS system implementation

MANAGEMENT/ORGANISATIONAL/SETTING SPECIFIC.

Understanding the organisational culture.

Seek engagement and support of key stakeholders.

Support implementation using quality improvement methodology.

EDUCATION/ TRAINING

Multidisciplinary training.

Continuous information/ updates.

Support for afferent limb (informed clinical judgment).

Maintaining transparency in order to maximise learning from case reviews.

THE SYSTEM

Support standardisation and automation.

Promote awareness of the significance of early referrals.

Ensure an optimum emergency response system is implemented for the health care setting.

Consider barriers to delays in escalation especially during out of hours

PROMOTE HIGH LEVELS OF ADHERENCE TO EWS TO ENSURE EFFECTIVENESS

EWS System is a complex intervention.

Measures to support adherence (e.g. education, clinical champions, audits, publication of findings, observationalist).

Measure longitudinal impact.

THE EWS SYSTEM

Systematic monitoring practice/protocols

EWS triggers, and observational chart

An algorithm for bedside management Communication tool

Escalation protocol with emergency response system

Implemented by inter-professional teaching, training and optimization of communication and collaboration

103

An Overview of NEWS: the Irish Context

The National Early Warning Score and COMPASS© Education programme is a work stream

of the National Acute Medicine Programme25. The NEWS system is used for the adult

patient population in Ireland. It was introduced in 2011 in response to a policy initiative and

patient safety programme from the Health Service Executive (HSE) (Health Service Executive

2011; Health Service Executive 2012, NCEC 2013). Other early warning scores are in place

specifically for the paediatric and obstetric population. Some ongoing research is also

underway to look at Early Warning Scores in the context of the Emergency Department

(Deasy & McAuliffe)26. Research in the Irish context includes: MSc dissertations (Flanagan

2013; McLoughlin 2015, n=2); retrospective audit of charts (Lobo et al. 2015); surveys (Fox &

Elliott 2015; Neary, Regan, Joyce, Callanan, 2015, n=2) and an unpublished RCT (Breen et al.

personal communication).

Flanagen (2013) noted that the initial success of the NEWS initiative was heavily dependent

on substantial local resources in the form of training, development and support. However,

authors commented that in the longer term the sustainability of NEWS will necessitate the

“allocation of specific resources, ensuring context driven interventions, training and

evaluation” (pg. 9). Six months following the implementation of the Irish NEWS, general

surgery doctors in training (n=27) and nurses (n=13) were surveyed about the NEWS

programme’s perceived impact (Neary, Regan, Joyce, Callanan, 2015). The NEWS

programme was thought to improve patient care by 69% of nurses, 11% registrars, 50%

SHOs and 58% of interns. All nurses and interns surveyed felt that they complied with

NEWS, whilst only 74% and 50% of registrars and SHOs, respectively, felt they comply.

Participants perceived that the NEWS did not correlate well with the clinical status of

patients within the first 24 hours post-surgery. The absence of an RRT as part of the

escalation protocol was also commented on by the majority of participants.

Fox and Elliott (2015) in a survey of 74 medical-surgical nurses working in a regional acute

hospital found that NEWS helped to identify patients who needed to be monitored more

closely, and that nurses used NEWS to supplement rather than replace their own clinical

judgment. However, they differentiated between those staff who benefited most noting

that it was a most useful decision-making tool for newly qualified and nursing students. The

majority (85%) of those surveyed noted delays in response times of doctors (i.e. outside the

recommended timeline), which they attributed to low out of hours medical staffing levels or

25

The National Early Warning Score and COMPASS© Education programme project is a work stream of the National Acute Medicine Programme, HSE, in association with the National Critical Care Programme, HSE, the National Elective Surgery Programme, HSE, the National Emergency Medicine Programme, HSE, the Quality and Patient Safety Directorate, HSE, Patient Representative Groups, Nursing and Midwifery Services Directorate, HSE, the Clinical Indemnity Scheme (State Claims Agency), the Irish Association of Directors of Nursing and Midwifery, and the Therapy Professionals Committee. 26

RED-ACE – Implementation of ED escalation protocol in Cork University Hospital (CUH) (HRB funded) Led by Dr. Conor Deasy, CUH with collaborators from CUH ED, UCC, National Emergency Medicine programme and UCD (Prof. Eilish McAuliffe).

104

the doctors being busy. Nurses noted that doctors were reluctant to modify trigger points

for patients with chronic illnesses e.g. COPD which meant that such patients continued to

trigger the NEWS whilst being stable. The authors acknowledged the complexity of the

system and noted that successful and full implementation of NEWS in clinical practice

depends on: an appropriate response structure; having medical staff available to respond to

activation calls; use of communications technology to enhance transfer of patient

information, and possibly the provision of out-of-hours medical cover by a centralised

multidisciplinary team. Similarly, Lydon et al., (2015) examined the perceptions among

nurses and doctors of NEWS and sought to identify variables that impact on intention to

comply with protocols. The barriers identified related to sociocultural aspects of introducing

a new system into current practice e.g. the belief that use of NEWS leads to a failure to use

clinical judgement; a perceived negative impact on interns’ perceptions of intern-nurse

teamwork; low engagement of senior doctors with the system. These sociocultural issues

must be addressed in order to improve detection of the clinical deterioration of patients.

Lobo et al. (2015), in an Irish study, sought to determine whether a NEW score of ≥7 in

medical patients resulted in a change in clinical management, as a measure of clinical

relevance. The NEWS which was introduced in March 2012 used ViEWS track and trigger

parameters. The researchers conducted a retrospective review of the medical charts of

consecutive patients admitted during 1st April 2012 and 14th June 2012. Just over two thirds

(64.6 %, n=51) of medical patients with a NEWS ≥7 had no change in their clinical

management. A retrospective application of the CREWS to the data of patients with chronic

hypoxemia with a NEWS score ≥7 would have reduced the number of reviews by 70.3 %

(i.e. NEWS ≥7: n=35; CREWS ≥7: n=11). Therefore, the authors noted that applying the Irish

NEWS trigger score leads to persistent triggering in potentially stable patients with chronic

hypoxemia. Smith et al. (2016) in an opinion article relating to Lobo et al. (2015 paper) also

commented that “the early warning score used throughout Ireland is actually the VitalPAC

Early Warning Score (ViEWS) not NEWS” basing their comments on the “differences in the

weighting bands of the physiological variables used in the two systems (specifically, for

supplemental oxygen, pulse and BP)” (pg. 267). Kellet and Murray (2014) analysed the

trajectories of previously published data on National Early Warning Score after admission

and found that they are of prognostic importance and escalation protocols should relate

changes in the score to its initial value on admission, highlighting the importance of changes

in NEWS as a parameter for monitoring by heath care professionals.

The Irish NEWS is supported by an online education programme titled COMPASS. The

COMPASS© Education Programme was modified to suit the Irish healthcare system and

covers key topic areas including categorisation of patients’ severity of illness, early detection

of patient deterioration, use of the ISBAR tool, triggers points that should prompt early

medical review and use of an escalation plan (Health Service Executive 2011b). In an Irish

trial researchers sought to determine if a proficiency- based progression (PBP) simulation

training programme results in superior performance in the use of the ISBAR communication

105

tool when compared to the both (i) The national e-learning NEWS education (COMPASS)

programme alone and (ii) the e-learning (COMPASS) programme combined with standard

simulation (without PBP). Participants were randomised to one of three groups: Group A:

n=36, e-learning alone; Group B: n=19 e-learning plus a 90 min training sessions which

consisted of simulated phone calls based on 2-4 paper cases; Group C: n=17 e-learning plus

a 90 minute training session with the same paper cases and simulated phone calls except

that trainees were trained to a proficiency benchmark (Breen, Gallagher and Walshe

2016)27. All participants underwent a formal assessment of ISBAR performance by

undergoing a video-recorded high fidelity simulation of a deteriorating patient in which they

were required to escalate care using the NEWS tool. Performance was scored by two

blinded assessors using previously defined metrics as a series of steps, errors and critical

errors. Analysis of variance showed a significant difference between the three groups (F (2,

68) = 7.95, p = 0.001). Group C, the PBP trained group completed significantly more steps

(mean (SD) = 7.4 (2.2)) than either group B (mean (SD) = 6.0 (1.4) p=.016) or Group C (mean

(SD) = 5.4 (1.7) p<.000). Nobody in Group A or B reached the proficiency benchmark whilst

22% of Group C (4/17) demonstrated the benchmark. The mean inter-rater reliability for all

participants was 0.93 (SD = 0.12) (i.e., 93% agreement between raters). Given the relatively

small number of participants within the study, it is worthwhile repeating it with larger

numbers to confirm study findings. However, the findings of the study raise serious

questions about the impact of training programmes on the proficiency of programme

participants in the utilisation of NEWS on exit from the programmes. It would also be

worthwhile examining proficiency using a longitudinal approach to data collection to see if

proficiency is sustained over time.

In summary, a limited body of published research exists in the Irish context. Thus, more

research needs to be conducted to report on both implementation issues & solutions and

the impact of NEWS on the patient, health care professionals and Irish Health System.

Review of Clinical Guidance Published Internationally

Clinical Guidelines are systematically developed statements, based on a thorough evaluation

of the evidence. The ultimate aim of clinical guidance is to assist practitioners and service

users’ in making clinical decisions which are consistent across the entire health system.

Many countries have well defined processes underpinning guideline development and a

number of countries have published guidance in relation to early detection of the

deteriorating patient.

NICE (UK) in their clinical guidance28 requires that, hospitals must have (i) a clear written

monitoring plan specifying which vital signs should be recorded (and at what frequency for

all adult hospitalised patients), (ii) a physiological EWS for documenting vital signs and (iii) a 27

BREEN, D., GALLAGHER, T., WALSHE, N., et al., 2016, personal communication. 28

NICE. 2007. Acute illness in adults in hospital: recognising and responding to deterioration. Available from https://www.nice.org.uk/guidance/cg50

106

graded response strategy. The graded response strategy according to NICE for patients

identified as being at risk of clinical deterioration should be agreed and delivered locally

(low score group -increased observations, charge nurse alerted; medium score group-

urgent call to team with primary medical responsibility for the patient and simultaneous call

to personnel with core competencies for the management of acute illness which can be

delivered by a variety of means; high-score group- emergency call with immediate response

to team with critical care competencies and diagnostic skills).

SIGN (2014)29 developed consensus recommendations to underpin a national approach to

the care of adult deteriorating patients. Similarly, the SIGN system makes reference to

patients with high NEWS score requiring immediate action from staff with an emergency call

to the team with critical care competencies and diagnostic skills. SIGN noted that as a first

step in implementing any new recommendation an understanding of current clinical

practice is required. In addition, acute hospitals should consider the introduction of

electronic track, trigger and alert systems.

In 2010 the National Consensus Statement relating to the “Recognising and Responding to

Clinical Deterioration” was endorsed by Health Ministers as the national approach for

recognition and response systems in Australian acute care facilities30. It encompassed

guidance on: the measurement and documentation

of observations; escalation of care; rapid response

systems, and clinical communication. An Australian

guidance document was particularly concerned

regarding implementation strategies and noted that

governance arrangements need to be in place “to

support the development, implementation, and

maintenance of organisation-wide recognition and

response systems” (Standard 9.1)31 . Such a

governance system includes: the identification of a

suitable individual, group or committee to take

responsibility for governance; development and implementation of processes for collecting,

analysing and reporting feedback from the workforce; identification of system failures

through data collection systems which review deaths and cardiopulmonary arrest; routine

and timely provision of relevant data about recognition and response systems to the clinical

29

SIGN. 2014. Care of deteriorating patients. Available from http://www.sign.ac.uk 30

AUSTRALIAN COMMISSION ON SAFETY AND QUALITY IN HEALTH CARE. 2010. National Consensus Statement: Essential Elements for Recognising and Responding to Clinical Deterioration. Available from http://www.safetyandquality.gov.au/wp-content/uploads/2012/01/national_consensus_statement.pdf 31

AUSTRALIAN COMMISSION ON SAFETY AND QUALITY IN HEALTH CARE. 2012. Safety and Quality Improvement Guide Standard 9: Recognising and Responding to Clinical Deterioration in Acute Health Care. Available from http://www.safetyandquality.gov.au/wp-content/uploads/2012/10/Standard9_Oct_2012_WEB.pdf

The ultimate aim of clinical

guidance is to assist

practitioners and service

users’ in making clinical

decisions which are consistent

across the entire health

system.

107

workforce; utilisation of the data from evaluation of recognition and response systems to

inform quality improvement activities.

The Between the Flags (BTF) system32 is a 'safety net' for patients who are cared for in the

public hospitals of New South Wales and health care facilities which sought to deliver on the

National Consensus Statement mentioned previously: Essential Elements for Recognising

and Responding to Clinical Deterioration (Australian Commission on Safety and Quality in

Health Care, 2010)30. BTF is designed as a series of interventions that act synergistically and

which are kept as simple as possible. Again governance is pivotal to the whole system.

Standard calling criteria includes the fundamental steps in clinical practice for the taking and

recording of vital sign observations. Clinical judgement remains paramount and BTFs focuses

on giving clinicians the confidence to call for help when it is required. The third domain of

the BTF initiative is universal clinical emergency response systems with minimum standards.

The whole system is underpinned by education and evaluation. Evaluation provides

feedback and lessons learned for continuous cyclical

improvement. The BTFs initiative is underpinned by

the “Recognition and Management of Patients who are

Clinically Deteriorating policy”32. Observation charts

are colour coded: blue zone (criteria for increasing

observations), yellow zone (early warning signs of

deterioration and the criteria for which a clinical

review may be required) and red zone (late warning

signs of deterioration and the criteria for which a rapid

response call is required). The policy makes reference

to a Clinical Emergency Response System which is

defined as “a formalised system for obtaining urgent

assistance when a patient is clinically deteriorating,

and ensures that the required skills, knowledge and

equipment are available to the deteriorating patient as

needed” pg. 10. For patients in the red zone the

responder (RRT or designated responder) must have advanced level of competence in the

management of the clinically deteriorating patient.

The US Institute for Clinical Systems Improvement33 published a RRT protocol (2011), which

makes reference to the adult criteria for RRT activation. Within such criteria, practitioners

are particularly asked to consider acute significant changes in patients’ baseline vital sign

criteria. In addition, a mechanism for patients and/or families to directly activate the RRT is

32

NEW SOUTH WALES CLINICAL EXCELLENCE COMMISSION. 2013. Between the flags keeping patients safe Initiative. And the Recognition and Management of Patients who are Clinically Deteriorating policy directive. Available from http://www.cec.health.nsw.gov.au/programs/between-the-flags 33

INSTITUTE FOR CLINICAL SYSTEMS IMPROVEMENT. 2011. Health Care Protocol: Rapid response team. Available from https://www.icsi.org/_asset/8snj28/RRT.pdf

Guidelines internationally include:

The escalation protocol

which embraces the concept

of alerting staff with critical

care competencies and

diagnostic skills.

Ensuring governance

arrangements are in place

to assure a system wide

implementation, response,

evaluation and learning

system.

108

provided. Currently, most RRTs in the US are triggered by one parameter at a time e.g. a significant

change in a particular vital sign whilst approximately 40% of calls to the RRT are generated because

the caller feeling there is “something just not right” with the patient34. The utilisation of early

warning scores to trigger RRTs is offered as an option with the clarifying statement that such track

and trigger systems have not been well validated. The protocol notes that if considering initiating

a RRT system the health care providers need to consider: team composition; criteria for

calling the RRT; the mechanism for calling the team (e.g., team pagers, overhead page,

other); education and training to senior leaders, physicians, team members, health care

facility staff members, patients, visitors and families; documentation tools/forms;

communication and feedback processes.

Figure 9. Diagrammatic representation of the general domains represented within the

international guidance pertaining to early detection of the deteriorating patient

In summary regarding international clinical guidance

Many similarities exist across guidance internationally pertaining to the early recognition of

the deteriorating patient. What is clear is the movement towards using a whole systems

approach whilst placing governance at the centre in the maintenance of organisation-wide

recognition and response systems (Figure 9). Such systems require the utilisation of data

34

INSTITUTE FOR HEALTHCARE IMPROVEMENT. 2016. Early Warning Systems: Scorecards That Save Lives. Available from http://www.ihi.org/resources/pages/improvementstories/earlywarningsystemsscorecardsthatsavelives.aspx

Governance

Early recognition of the deteriorating

patient,

communication tool to escalate care

(track and trigger system)

Clinical emergency response systems

(graded response including access to

critical care competencies and diagnostic skills)

Education

(multi-disciplinary, continuous)

Evaluation

(feedback loops ensuring a

learning system)

109

from the evaluation of recognition and response systems to inform quality improvement

activities.

Empowering health care professionals to act on their clinical judgment is also a critical

component of any system. Family and patient worry is also mentioned within some of the

guidance.

Across the guidelines the concept of patients in high score groups requiring immediate

action/review by individual(s) with critical care competencies and diagnostic skills are very

evident.

Conclusion and Recommendations (from the review of clinical

literature)

Monitoring vital signs

The evidence reviewed identified that EWSs potentially reduce adverse events due to the

earlier detection of clinical deterioration in patients and the associated escalation of care.

Within the systematic review, a variety of EWSs were reviewed. Even within each system

there were differences in the vital sign scoring and the weighting assigned to scores when

they were used in different studies/hospitals/subpopulations (Appendix 4b). Within the

review variations in the scoring of NEWS was observed. Varying NEW trigger parameters

were observed as follows: RR ≤ 8 (score 3), RR≤ 9 (score 3); SpO2 ˂92 or 91 (score 3);

Temperature > 39.0C/≥30.0C, ≥39.1C (score 2), ≥39.1C (score 3); Supplemental yes O2

(Score 2 or 3); SBP ≥220 mmHg (score of 2 or 3); ≥250 mmHg (score 1). In addition binary

NEW scores varied across all parameters. This made comparisons difficult.

In addition, national reviews in both the UK and New Zealand have shown that multiple

systems and varying parameters have been used. Patterson et al. (2011) compared clinical

practice in London and Scotland against national guidelines and found that a number of

different combinations of trigger parameters were in use in the UK at the time: London

(n=11) and Scotland (n=5). Whilst in New Zealand large variance was found to exist in the

criteria used to detect deteriorating patients; researchers drew attention to the differences

which existed in the extreme parameter values assigned maximal scores (Psirides et al.

2013) i.e. bradypnoea (abnormally slow breathing rate) had the widest range of trigger

values with nine systems (43%) assigning a maximal score to a respiratory rate ≤8 and three

(14%) not doing so until the respiratory rate was scored ≤4. Notably, respiratory rate is one

of the least documented vital signs, yet is identified as one of the most predictive of

physiological decline.

Aggregate weighting scoring systems (e.g. NEWs, MEWS, ViEWS, CART) were reported to be

superior to single parameter systems at predicting adverse events. But using different

trigger scores may alter the sensitivity, specificity and PPV for different outcomes in

110

different patient subpopulations thus compromising the ability of the authors to make

comparison using the empirical literature. However, the review highlighted that for sub-

populations the NEWS system, whilst beneficial, lacks sensitivity and specificity. These

subpopulations included: older adult patients (with/without comorbidity, high frailty index),

patients with chronic obstructive pulmonary diseases (COPD) and patients in the Emergency

Department. Scoring systems which do not incorporate age as a variable are were found to

be less accurate in assigning scores to older adults.

Use of supplemental oxygen scores highly, which is reasonable for stable inpatients. But

many ambulance and ED patients are routinely given O2 initially, with subsequent titration

or removal. To include this as part of the first-look score in the ED limits its discriminatory

function. NEWS does not discriminate between degrees of reduced consciousness; anything

below ‘alert’ on the alert, to voice, to pain, or unconscious (AVPU) scale indicates medium

risk, regardless of other parameters. The inclusion of additional parameters such as: a

particular focus on alterations in a discrete vital sign; knowledge of the individual’s

presenting complaint; age; concurrent availability of multiple assessment results, and a fine-

grain aggregated score which takes account of trend-analysis all offer potential in terms

increasing the specificity/sensitivity of the EWS and its ability to pick up on early

deterioration in the ED context. EWS should be used along- side, rather than instead of

other scoring systems e.g. GCS. For patients in the ED department for example a holistic

assessment using additional parameters such as GCS, blood glucose, serum bicarbonate

(HCO3–), lactate, leukocyte count and a history of metastases can increase sensitivity and

specificity of the ability to predict deterioration.

There is a need for more accurate risk stratification of patient trigger scores related to the

older person e.g. the odds of an adverse event increased with increasing NEWS score but

the magnitude of the increase is greater for older persons. Thus alterations in the NEWS

thresholds for specific patient groups may offer benefit although the evidence is not strong.

However, one of the important consequences of an alteration in the EWS thresholds and

associated EWS’ sensitivity and specificity is the workload that it creates for an organisation.

Furthermore, younger patients were under-represented in the included studies, so the

authors are unable to specify if differences exist in the sensitivity of EWS systems this

cohort.

Patients with COPD may need separate SpO2 weighting scales for patients with or without a

risk of hypercapnic respiratory failure. Chronic physiological abnormalities such as those

found in chronic obstructive pulmonary disease can lead to potentially higher trigger rates.

The use of a modified score for COPD to account for oxygen administration and saturation

scoring chronically (e.g. CREWS) should be investigated.

Nurses and nurse assistants play an important role in vital sign collection and

documentation, however, some evidence suggests compliance and accuracy in performing

this role may be lacking (Tysinger, 2015). Some of the most common issues in the reviewed

111

literature included: insufficient time, insufficient resources, equipment issues, distractions,

multi-tasking, numeracy, and individual judgments about the importance of vital signs. The

importance of clinical intuition and practitioner concern as an integral component of the

EWS is not disputed however it may also be seen to undermine the importance of vital signs

as trigger parameters. Technology has been reported to lead to improvements in vital sign

accuracy, however, there is also a concern that in many instances nurses are relying too

heavily on technology to collect vital sign data. Thus there is a need to carefully review how

to best utilize technologies in this area to best inform future direction.

The NEWS has been shown to be an effective assessment tool to identify and triage the

patient for the most appropriate acute care assessments and interventions; however, the

authors caution, based on the evidence, against the use of the tool to estimate risk of death

or risk of other outcomes. Such estimations should be based upon the totality of the clinical

information available to a clinician and their clinical judgment and discussions with the

patient and family.


Based on the findings of this review the team would consider it appropriate that NEWS2

continue to be used for adult non-pregnant patients within the acute hospital system.

Continue to use and develop a national approach to ensure the early identification and

effective escalation of the deteriorating patient.


Consider if patients in the following categories may benefit from different trigger scoring

thresholds and/or supplementary track and trigger/risk stratifying systems:

• Older adult patients; in particular frail older adults

• Patients with COPD or other conditions characterised by chronic hypoxemia

• Patients in the Emergency Department.

Escalation and response to abnormal NEWS

The studies reviewed identified that timely escalation remains an ongoing problem. Higher

NEWS scores were correlated with increased mortality as measured at various time points;

every one-point increase in NEWS was reported to be associated with an increased risk of

mortality. Time to activation can be reduced using automatically generated alerts through

electronic systems; but specificity of EWS may be lowered. Supporting practitioner

articulation of their worry/ use of their clinical judgement is important. However,

introducing automated advanced assessments/interventions for specific patient groups and

symptom clusters could lead to more rapid risk stratification of patients for example taking

an arterial blood gas at specific trigger points, and taking 12 lead ECG for those with chest

pain may help to further risk stratify patients at risk of clinical deterioration. Providing

112

direction as to the action that may be taken when the EWS is triggered means that whilst

help is being sought active interventions are being taken to minimise further deterioration

or to aid more accurate diagnosis of the cause of deterioration.

The early detection of clinical deterioration provides an opportunity for the prevention of

cardiopulmonary arrest and its attendant mortality; to this end, the use of a rapid response

team (RRT, i.e. practitioners with critical care competencies) has been advocated as a means

of reducing in-hospital mortality. The evidence supporting the impact of RRTs shows a

reduction in cardiac arrest but the effect on mortality is less certain with the quality of

evidence reviewed being low (McNeill& Bryden, 2013; Jones et al., 2009; England and Bion

2008). Nonetheless, across international guidelines the concept of patients in high EWS

trigger groups requiring immediate action/review by individual(s) with critical care

competencies and diagnostic skills was very evident.

For an individual parameter which scores ≤3, it is recommended to increase the frequency

of observations. However, practitioners need to be particularly vigilant for changes in

respiratory rate, low temperature (hypothermia) and loss of consciousness. Respiratory rate

was found to be one of the best vital sign predictors of adverse events. However, the ideal

cut-off (i.e. RR/minute) is unknown, partly because respiratory rate is often inaccurately

measured and poorly documented in hospitalised patients. The importance of respiratory

sign documentation needs to be reiterated to practitioners.


Consider the mechanisms that could be used to ensure that individuals with higher NEW

scores are reviewed promptly by health care professionals with critical care competencies

and diagnostic skills.


Reinforce the importance of monitoring respiratory rate as part of educational programmes

for health care professionals who monitor and interpret vital signs.

Electronic systems

Incorporating NEWS into an electronic system can improve the documentation of vital signs,

reduce errors (e.g. incorrect scores calculated), increase the number of parameters that can

be considered when monitoring or risk stratifying patients and reduce false alarm rates.

Electronic systems also have the potential to predict outcomes from discrete disease states,

and adjust trigger scores for different outcomes. Thus electronic systems also have the

potential to support the implementation of disease specific EWSs.

Electronic data capture makes the recorded vital signs readily available for further

processing. This coupled with different analytical techniques (e.g. using trend and change

data) and software algorithms that can identify patients at risk of deterioration with greater

113

sensitivity and specificity, and make intelligent alerting and remote patient surveillance

possible. However Bonnici, Tarassenko, Clifton et al., (2013) caution that the potential of

these technologies depends on implementation, user acceptance and the cultural and

organisational factors surrounding the introduction of these systems with poor-quality

deployment likely to worsen patient care. In addition, Romero-Brufau et al., (2014) noted

that when the evaluation of a patient is performed in a time-sensitive manner, the most

widely used weighted track-and-trigger scores do not offer good predictive capabilities for

use as criteria for an automated alarm system. Thus for the implementation of an

automated EWS, better criteria need to be developed and validated before implementation.


Consider carefully the potential use of electronic data capture, EWS triggering, notification

and tracking of outcomes in an Irish context, whilst being reflective of how such systems

would advance the science of early detection of the deteriorating patient within a complex

health care system.

The System

Understanding the organisational culture, systems, practices and the stakeholders’

perceptions and interactions with the NEWS is important. However, the reality is patients

have died within health care systems because health care providers have been failing to

recognize and act upon acute changes in their patient’s clinical status. Systems theory posits

that human error is a reality but that it is the systems we work in that primarily contribute

to failures and adverse events. Individual culpability is minimised with a shift to a systems

orientated philosophy of patient safety. Whilst analysing the effectiveness of a EWS system

is complex and time consuming, it is necessary.

A much more strategic, systems engineering, analytic, systems based approach is required.

Most studies reviewed failed to take into account that the overall performance of the

system depends on the performance of its individual parts, and the individuals interacting

with it which makes the interpretation of results difficult. High levels of adherence and

consistent adherence are necessary for the system to be effective. Notable issues include:

incomplete vital sign observation sets; incorrect calculation of NEW score; incorrect alerts;

lack of documentation of alerts/actions; accuracy of recording of NEW scores decreases as

the patient deteriorates (i.e. as NEW score increases); delayed escalation on EWS trigger;

review of patient outside of recommended timeframes, and lack of critical care

competences by healthcare staff who review patients at more extreme EWS trigger points.

Any of these variables have the potential to impact on the sensitivity of the triggering

system and the overall effectiveness of the EWS system.

Many similarities exist across guidance internationally pertaining to the early recognition of

the deteriorating patient. What is clear is the movement towards using a whole systems

approach whilst placing governance at the centre in the maintenance of organisation-wide

recognition and response systems (Figures 8 and 9). Such systems require the utilisation of

114

data from the evaluation of recognition and response systems to inform quality

improvement activities. Empowering health care professionals to act on their clinical

judgment is also a critical component of any system introduced.


A systems approach underpinned by appropriate governance is required. Such a systems

approach requires:

Consider NEWS as a system-level complex intervention.

Change the title of the Irish guideline to reflect a National Early Warning “System”

as opposed to National Early Warning “Score”.

Emphasize the importance of regular reinforcement, and auditing with a systems-

learning focus.

Promote high levels of adherence to NEWS to ensure effectiveness.

Introduce mechanisms to decrease errors and increase adherence. These could

include: clinical champions to promote NEWS, ward based education; multi-modal

education; the hiring of ‘observationalists’; publishing monthly audit reports; visual

imagery using posters; instilling confidence in practitioners in their ability to escalate

care for NEW score trigger points.

Use quality improvement methodologies and an understanding of human factors to

quantify the avoidable (associated with health care delivery system dysfunction) and

unavoidable delays (e.g. associated with diagnosis or treatment factors), as well as

their impact on care. Use pre-post-intervention analysis with longitudinal

measurement of predefined organisational, health care professional and patient

level outcomes with feedback sessions when implementing NEWS/changes to NEWS.

Education

When educating for optimal clinical judgement educators need to consider reinforcing the

importance of other parameters which have been shown to predict clinical deterioration

and adverse outcomes. These include; patient age, urinary output, emotional state, frailty,

diastolic blood pressure, pulse pressure index, prior ICU visit and comorbidities.

The importance of multi-formats, inter-professional training, regular reinforcement, case

reviews and an interactive- in person training process was reiterated in a number of studies

that explored educational interventions.


Ensure the education of all health care providers using NEWS; such education should

include: interdisciplinary in person simulations/case-studies; be multimodal, and include

regular reinforcement. This recommendation should also be read in conjunction with

recommendation 5 above.

115

Chart design

Chart design can positively and negatively influence documentation, error rate and time to

complete observations. Graphically depicted charts were identified as being more effective

than numerical ones and were completed faster. Graphs which overlap should be avoided

which may preclude use of the seagull- sign. Unfortunately in the reviewed literature,

incorrect calculation of EWS and documentation of EWS was noted, particularly out of

hours. Thus in auditing EWS particular attention needs to be given to ensuring the accurate

documentation of EWS.


Chart designs should avoid visual clutter and the use of overlapping graphical displays of

data.

Research

Given the absence of higher level evidence (e.g. trials, high quality case control or cohort or

studies) in the research literature sourced, it is important that research be conducted in

tandem with the implementation of EWS systems with particular emphasis on the

application of EWS to subgroups (e.g. older adults) and particular contexts of care (e.g. ED).

Some authors have expressed concern over the utility of the area under the receiver

operator characteristic curve (AUROC) and the C-statistic citing the fact that it is not

informative in the evaluation of early warning scores given the relative low frequency of

adverse events (Romero Brufau et al. 2015). Others have quoted the PPV, sensitivity and

specificity as much more informative statistics. Using comparable approaches to data

collection and reporting of presentation aids in the conduction of systematic review; whilst

other statistics may aid practitioners in terms of clinical interpretation.


There may be a requirement for more research to be conducted in tandem with the

implementation of EWS systems with particular emphasis on the application of EWS to

subgroups (e.g. older adults) and particular contexts of care (e.g. ED). National consensus on

the tracking of key outcomes pertinent to NEWS will assist with this.

Perspectives on the Process

Conduction of systematic reviews is a resource and time consuming process. Therefore, it is

recommended that the time allocated for conduction of systematic reviews be increased to

allow for synthesis of research findings to better inform guideline recommendations. It is

also recommended that prior to conducting a systematic review for an update of a guideline


the key question in terms of emerging clinical practice and the issues with the utilisation of

the guideline in practice that need to be answered in the review update.

116

Guideline recommendations need to be very clear, with single messages to facilitate the

application of evidence to separate discrete areas/questions. Guidelines which contain a

large number of recommendations in essay format create less certainty in terms of the

focus of the evidence synthesis.


Prior to conducting a systematic review, for an update of a guideline, it is recommended


the key review question(s) in terms of emerging clinical practice/ utilisation of the guideline

in practice that need to be answered in the review update.


Ensure that guideline recommendations are very clear, with single messages to facilitate the

application of evidence to separate discrete areas/questions.

117

Chapter 3. Economic Review Methods

Review Methods for the Economic Literature

As specified by the Department of Health Clinical Effectiveness Unit, the aim of this

review was to update the current evidence from the economic literature on early

warning scores/systems used in adult patients in acute healthcare settings for the

timely detection of physical or clinical deterioration.

The specific objectives of the economic arm of the research were as follows:

To source and describe any existing international empirical economic literature

pertinent to the implementation of warning score/systems or trigger systems

(including escalation protocols, communication tools and response approaches) for

the detection/timely identification of physiological deterioration in adult (non-

pregnant) patients in acute healthcare settings.

To critically evaluate the included economic literature in terms of cost effectiveness,

cost impact and resources involved, including resources required for escalation of

care, implementation costs and relevant education costs.

The review was guided by: the HIQA guidelines for the retrieval and interpretation of

economic evaluations of health technologies in Ireland35 (HIQA, 2014); the Cochrane

Handbook for Systematic Reviews of Interventions36 (Higgins & Green, 2011); and the HIQA

guidelines for budget impact analysis of health technologies in Ireland37(HIQA, 2015a).

Selection Criteria for the Economic Studies

As different study designs were required to meet the different objectives of this

review, no study design limits were applied thus ensuring that the likelihood of finding

relevant studies irrespective of design was increased;

Documents or reports published or unpublished as grey literature;

Studies written in the English language, published since 1st April 2011 (so as to be

consistent with clinical literature search);

Studies focused on adults’ aged ≥16 years.

The following types of studies were not considered for inclusion in the economic literature:

Any study that did not adhere to the clinical criteria;

Theses, case studies, discussion or opinion papers that did not present research

findings;

Studies published in a foreign language.

35

HIQA. 2014. Guidelines for the Retrieval and Interpretation of Economic Evaluations of Health Technologies in Ireland. Available from https://www.hiqa.ie/system/files/Guidelines-Retrieval-and-Interpretation-of-Econ-Lit.pdf 36

HIGGINS, J. P. T. & GREEN, S. 2011. Cochrane Handbook for Systematic Reviews of Interventions - Chapter 15: Incorporating economics evidence. Available from http://handbook.cochrane.org/ 37

HIQA. 2015a. Guidance on Budget Impact Analysis of Health Technologies in Ireland. Accessible from https://www.hiqa.ie/system/files/Guidance_on_Budget_Impact_Analysis_of_Health_Technologies_in_Ireland.pdf

118

The PICOS framework (as described previously) is used to support inclusion criteria but with

a minor adaption to include context (Davies, 2011), PICOCS (Population, Intervention,

Comparison, Outcomes, Context, Study design)

Table 4. A systematic literature search was performed using the PICOCS framework

Population: Adult acute patient, Adult patient, Medical patient (≥16 years)

Interventions:

Early Warning Score, Modified Early Warning Score, VitalPAC™ (VIEWS), Track and Trigger System, ALERT™, COMPASS©

Comparator: Comparison against each other or with no intervention

Outcomes:

Cost utility analysis – generic outcome measures – quality-adjusted life years (QALYs)/ health years equivalent (HYE)/ disability-adjusted life years (DALYs) etc.; Cost effectiveness analysis – cost per unit of effect (cost per life – years gained (LYG)) or effects per unit of cost ( LYG per Euro spent); Cost-benefit ratios; Incremental cost-effectiveness ratio (ICERs); Incremental cost-per QALY. Any measure of economic outcomes; Resource use – length of stay - hospital/ICU/HDU stay; ICU/HDU admissions; unexpected ICU/HDU admissions; use of RRS and MET Costs – implementation costs; escalation costs; service utilisation costs; direct medical costs; indirect medical costs; education costs and cost savings.

Contexts: Acute hospital setting

Studies:

No study design limits were applied thus ensuring that the likelihood of finding relevant studies irrespective of design was increased.

Search Strategy

The aim of the search was to accumulate all the current economic evidence on early

warning systems used in adult patients in acute healthcare settings. As previously

mentioned, there were no study limits applied to the search, thus increasing the prospect of

identifying all the relevant economic evidence on Early Warning Systems. An economic

filter was applied to the clinical search terms in order to perform each search, details of

which can be found in Appendix 1.

The systematic literature search was completed in the following databases: Academic

Search Complete, Business Source Complete, CINAHL, EconLit, PsycARTICLES, Psychology

and Behavioral Sciences Collection, PsycINFO, SocINDEX with Full Text, UK & Ireland

Reference Centre, (via EBSCO) and EMBASE (via Elsevier). Searches were also performed in

the Database of Abstracts of Reviews of Effects (DARE) and the NHS Economic Evaluation

119

Database (NHS EED) via the CRD website, along with the Cochrane Methodology Register

(via Wiley).

The search was conducted on the 7th December 2015, searches were limited by dates

01/04/2011 – 07/12/2015 and EndNote was used to store all references.

The economic search for grey literature was conducted in the following repositories: Open

Grey, New York Academy of Medicine, Open Doar, National Institutes of Health (NIH),

Health Service Executive (HSE), HIQA, Health Research Board (HRB), Lenus, World Health

Organisation (WHO), National Institute for Health and Care Excellence (NICE), Centre for

Health Economics and Policy Analysis (CHEPA), Institute of Health Economics (Alberta

Canada), Department of Health UK, National Health Service UK (NHS), Public Health Agency

of Canada, Google Scholar and Google. The search terms used can be found in Appendix 1.

Search Results

Figure 10 outlines the study identification process. Further to the systematic and grey

literature searches, 4,156 references were imported into EndNote Web and 1,261

duplicates were removed automatically. A total of 2,895 records were screened by title and

abstract and 34 studies were identified for full text review. The systematic search yielded 1

study that met the inclusion criteria for the review and 4 papers were identified from the

grey literature search.

Review Process

The data selection was performed in three stages:

Stage 1: All potentially relevant papers yielded by the search were screened by abstracts.

These were then assessed against the inclusion and exclusion criteria by two reviewers

specialised in the field of health economics.

Stage 2: For studies that seemed to meet the inclusion criteria according to their abstract, or

in cases when there was ambiguity concerning the economic data in the abstract, the full

paper was retrieved for an in depth assessment against the inclusion criteria independently

by the two economic reviewers.

Stage 3: The papers that met the economic criteria were subjected to a further screen by a

reviewer with knowledge in the medical field, so as to ensure the studies satisfied the

clinical criteria.

120

Total number of records identified during initial searches

(n=4,156)

EBSCO (Academic Search Complete, Business Source Complete. CINAHL Plus with Full Text, EconLit with Full Text,

MEDLINE, PsychARTICLES, Psychology and Behavioral Sciences Collection, PsychINFO, SocINDEX with Full Text, UK

& Ireland Reference Centre) (n=560)

Embase (n=470)

CRD-DARE, NHS, EED & HTA (n=34)

Cochrane Library (n=56)

Grey Literature (n=3,036)

Records Screened on title and abstract

(n=2,895)

Full-text papers assessed for eligibility from:

Database search (n=12)

AND

Grey Literature search (n=22)

Full-text papers for inclusion:

Database search (n=1) AND

Grey Literature search (n=4)

Duplicates removed

(n=1,261)

Full-text papers excluded from: Database search (n=11) AND

the Grey Literature search

(n=18)

Grey Literature search:

Partial economic evaluation (n=4)

Database search:

Partial economic evaluation (n=1)

Records excluded on title and abstract

(n=2,861)

Figure 10. Flow chart of the search process and results – Economic Arm

121

Data Extraction

The data extraction was performed by the two health economists on the team and the

evidence is provided in tabular format to support reliability of reporting, reproducibility and

to reduce bias (CRD 2008). A summary of the full text read extraction table can be found in

Appendix 6.

As per HIQA (2014) data extraction included the following elements:

Study question, population, intervention and type of EWS, comparator and setting

Modelling methods

Sources and quality of clinical data

Sources and quality of cost data

Cost data

Resource usage

Study outcomes, and methods used in synthesis

Outcomes and benefits

Methods for dealing with uncertainty

Study Results

Quality Appraisal

A critical evaluation of the methodological quality was conducted to assess the quality of

evidence from the included studies and was performed independently by the two economic

reviewers. As recommended by the Cochrane handbook (Schemilt et al. 2011, cited HIQA

2014) “the BMJ checklist” (Drummond et al. 1996) and the CHEC - list were applied to

inform a critical appraisal of the methodological quality of the economic evidence of the

papers yielded from the systematic search. Copies of these checklists are provided in

Appendix 5a.

Transferability

With a growing demand for evidence based decision making, but limited resources and a

paucity of evidence on EWS from the Irish setting, the review includes five studies, of which,

three are Irish and the remaining two studies are from the UK and the Netherlands. The

EUnetHTA transferability tool provided in economic evaluation questions 27-29 (EUnetHTA

2013) was applied following the HIQA (2014) guidelines to determine transferability of the

UK and Dutch studies to the Irish setting. A copy of the EUnetHTA transferability tool is

included in Appendix 5b.

122

Data Synthesis

The evidence was compiled and condensed using a narrative synthesis; this is supported by

evidence tables in Appendix 6 which give more detail on the findings.

123

Chapter 4. Economics Review Findings

Introduction

As per the findings of previous systematic literature reviews, the economic evidence on

early warning scores/systems used in adult patients in acute healthcare settings for the

timely detection of physical or clinical deterioration is limited (HIQA, 2015b & NCEC, 2013).

The search of the economic literature pertaining to EWS did not yield any full economic

evaluation; however, one health technology assessment (HIQA, 2015b), two budget impact

analyses (NCEC, 2013 & NCEC 2014) and two cost descriptions (Simmes et al. 2014 & Subbe

et al. 2014) were accumulated by the search. In addition, the systematic search of the

literature produced the HIQA (2015b) Health technology assessment (HTA) of the use of

information technology for early warning and clinical handover systems, whilst the

remaining four papers were sourced via the grey literature search. The three Irish studies

(HIQA, 2015b; NCEC, 2013 & NCEC 2014) included in the review, consider the NEWS system

in their analyses, however the NCEC (2014) Sepsis Management Guideline, considers the

NEWS in the context of a management framework, where NEWS is recommended as the

first step in escalating care for medial review and timely diagnosis of sepsis, thereby its

inclusion in this review is questionable. A Dutch study, Simmes et al. (2014) considered a

single parameter EWS (unspecified) as part of a RRT and reported the financial

consequences of implementing such a system on a surgical ward. Whilst the fifth study

within the review, Subbe et al. (2014), considered the impact of an advanced triage system

on length of stay in a medical emergency admission unit hospital in Wales. Due to the

heterogeneity of these studies, the findings for each study are presented individually and

the evidence is synthesised in the discussion section of the review.

Characteristics of Economics Papers

Five papers were sourced through the review of literature relating to economic evaluations

of EWS/NEWS. Countries evaluated were Ireland (3), the Netherlands (1), and the UK (1).

With regards to study design one was a health technology assessment (HTA) (HIQA, 2015),

two were budget impact analyses (BIA) (NCEC, 2013 & NCEC, 2014), and two were cost

descriptions (Simmes et al. 2014 & Subbe et al. 2014). The Irish studies included a HTA of

electronic implementation of an EWS (HIQA, 2015b); a BIA for the original guideline on EWS

(NCEC, 2013) and a BIA on the additional cost implications from the implementation of the

Sepsis management guideline (NCEC, 2014). Figures 11 and 12 present an overview of the

characteristics of these papers.

124

Figure 11. Countries represented in the economic review

Figure 12. Studies included in the analysis

0 1 2 3

Ireland

Netherlands

UK

0 1 2

HTA

BIA

Cost Descriptions

125

Systematic Search

Quality of Included Studies

As proposed by HIQA (2014a, the BMJ checklist (Drummond et al. 1996) and the Consensus

on Health Economic Criteria (CHEC)-list were applied to assess the quality of each of the

studies yielded from the systematic search. Given the studies were not full economic

evaluations a number of the criteria were not applicable. With this in mind, the criteria

which were relevant were considered to be of good quality. These results are documented

in Tables 5 and 6.

Transferability

As for HIQA (2014a) guidelines, the EUnetHTA toolkit for transferability (EUnetHTA, 2013)

was applied employed to the UK and Dutch studies (Simmes (2014) and Subbe (2014).

Differences are centred on pre- and post-intervention care and the integration of

technology in health care system. Table 5 provides further detail on the transferability of

these studies to the Irish setting.

126

Table 5. Quality appraisal using the British Medical Journal (BMJ) Checklist

Item HIQA

(2015b)

NCEC

(2013)

NCEC

(2014)a

NCEC

(2014)b

Simmes

(2014)

Subbe

(2014)

Extract Study design. Y Y Y Y Y Y

1. The research question is stated. Y Y Y Y Y Y

2. The economic importance of the research question is stated. Y Y Y Y Y Y

3. The viewpoint(s) of the analysis are clearly stated and justified. Y n/a Y Y Y Y

4. The rationale for choosing alternative programmes or interventions compared is stated. Y n/a Y Y Y Y

5. The alternatives being compared are clearly described. Y n/a Y Y Y Y

6. The form of economic evaluation used is stated. Y Y N/A Y Y Y

7. The choice of form of economic evaluation is justified in relation to the questions addressed. Y Y N/A Y Y Y

Data collection.

8. The source(s) of effectiveness estimates used are stated. Y Y Y Y Y Y

9. Details of the design and results of effectiveness study are given (if based on a single study). Y N N/A Y Y Y

10. Details of the methods of synthesis or meta-analysis of estimates are given (if based on a

synthesis of a number of effectiveness studies). N/A N/A Y N/A N/A N/A

11. The primary outcome measure(s) for the economic evaluation are clearly stated. Y N/A N/A Y Y Y

12. Methods to value benefits are stated. Y N/A N/A Y Y Y

13. Details of the subjects from whom valuations were obtained were given. N/A N/A N/A N/A N/A N/A

14. Productivity changes (if included) are reported separately. N/A N/A N/A N/A N/A N/A

15. The relevance of productivity changes to the study question is discussed. N/A N/A N/A N/A N/A N/A

16. Quantities of resource use are reported separately from their unit costs. Y N Y Y Y Y

17. Methods for the estimation of quantities and unit costs are described. Y Y Y Y Y Y

18. Currency and price data are recorded. Y Y Y Y Y Y

19. Details of currency of price adjustments for inflation or currency conversion are given. Y N/A Y Y Y Y

20. Details of any model used are given. N/A N/A Y N/A N/A N/A

21. The choice of model used and the key parameters on which it is based are justified. N/A N/A N N/A N/A N/A

127

Analysis and interpretation of results.

22. Time horizon of costs and benefits is stated. Y Y Y Y Y Y

23. The discount rate(s) is stated. N/A N/A Y N N N

24. The choice of discount rate(s) is justified. N/A N/A N N/A N/A N/A

25. An explanation is given if costs and benefits are not discounted. N N/A N N/A N/A N/A

26. Details of statistical tests and confidence intervals are given for stochastic data. Y N/A Y N/A N/A N/A

27. The approach to sensitivity analysis is given. N N/A Y N/A N/A N/A

28. The choice of variables for sensitivity analysis is justified. Y N/A N N/A N/A N/A

29. The ranges over which the variables are varied are justified. Y N/A N N/A N/A N/A

30. Relevant alternatives are compared. Y N/A Y N/A Y Y

31. Incremental analysis is reported. N N/A N/A Y N N

32. Major outcomes are presented in a disaggregated as well as aggregated form. Y N/A N/A Y Y Y

33. The answer to the study question is given. Y Y Y Y Y Y

34. Conclusions follow from the data reported. Y Y Y Y Y Y

35. Conclusions are accompanied by the appropriate caveats Y Y Y Y Y Y

128

Table 6. Quality appraisal using the Consensus on Health Economic Criteria (CHEC)

Item

HIQA

(2015b)

NCEC

(2013)

NCEC

(2014)a

NCEC

(2014)b

Simmes

(2014)

Subbe

(2014)

1. Is the study population clearly described? Y Y Y Y Y Y

2. Are competing alternatives clearly described? Y N Y Y Y Y

3. Is a well-defined research question posed in answerable form? Y Y Y Y Y Y

4. Is the economic study design appropriate to the stated objective? Y Y Y Y Y Y

5. Is the chosen time horizon appropriate to include relevant costs and consequences? Y Y Y Y Y Y

6. Is the actual perspective chosen appropriate? Y Y Y Y Y Y

7. Are all important and relevant costs for each alternative identified? Y Y N/A N/A Y Y

8. Are all costs measured appropriately in physical units? Y N/A N/A Y Y Y

9. Are costs valued appropriately? Y Y Y Y Y Y

10. Are all important and relevant outcomes for each alternative identified? Y N/A N/A N/A Y Y

11. Are all outcomes measured appropriately? Y N/A N/A Y

Y

12. Are outcomes valued appropriately? Y N/A N/A Y N Y

13. Is an incremental analysis of costs and outcomes of alternatives performed? N N/A N/A N N N

14. Are all future costs and outcomes discounted appropriately? Y N/A N/A N N/A N

15. Are all important variables, whose values are uncertain, appropriately subjected to

sensitivity analysis? Y N/A N/A N N N

16. Do the conclusions follow from the data reported? Y Y N/A Y Y Y

17. Does the study discuss the generalizability of the results to other settings and

patient/ client groups? N N/A N N N N

18. Does the article indicate that there is no potential conflict of interest of study

researcher(s) and funder(s)? Y Y Y Y N Y

19. Are ethical and distributional issues discussed appropriately? N/A N/A N/A N/A N/A N/A

NCEC (2014)a (Sepsis) - economic literature review NCEC (2014)b (Sepsis) -BIA of the National Clinical Guideline

129

Table 7. Transferability using EUnetHTA ToolKit Transferability Q27-29

Item Simmes (2014) Subbe (2014)

27 How generalizable and relevant are the results, and validity of the data and model to the relevant jurisdictions and populations?

28 Are there any differences in the following parameters?

a) I Perspective x x

II Preferences n/a n/a

III Relative costs

IV Indirect costs n/a n/a

V Discount rate n/a n/a

VI Technological context x x VII Personnel characteristics

VIII Epidemiological content (including genetic variants) x x

IX Factors which influence incidence and prevalence x x

X Demographic context

XI Life expectancy x x

XII Reproduction x x

XIII Pre- and post-intervention care

XIV Integration of technology in health care system

XV Incentives

b) If differences exist, how likely is it that each factor would impact the results?

In which direction? n/a n/a

Of what magnitude? n/a n/a

c) Taken together, how would they impact the results and of what magnitude? n/a n/a

d) Given these potential differences, how would the conclusions likely change in the target setting? n/a n/a

Are you able to quantify this in any manner? n/a n/a

29 Does the economic evaluation violate your national/regional guidelines for health economic evaluation?

n/a n/a

130 | P a g e

Characteristics of Included Studies

A summary of studies from the systematic search is provided in Appendix 6 and an overview

of these studies in terms of the research question is discussed below.

1) HIQA (2015b) “Health technology assessment of the use of information technology for

early warning and clinical handover systems”

This HTA estimated resources gains and the investment required to implement an electronic

early warning system into a representative Model 4 (530 bed) Irish teaching hospital.

Benefits were estimated using extrapolated results from the systematic review and

measured as resource gains. Using Irish average LOS data and evidence from Jones et al.

(2011) estimated potential reductions in general average LOS was 28.9% and ICU average

LOS was 40.3%. It was estimated that these reductions in LOS translated into just over

802,000 bed days per annum in general wards and 30,628 ICU bed days per annum.

However, this was considered as an efficiency rather than a monetary saving. Other

potential benefits presented were efficiencies owing to a reduction in vital sign recording

time (up to 1.6 times faster than the paper system).

In terms of the investment required to move from paper based to an electronic EWS, a core

model without continuous monitoring was included in the analysis. Resources considered

over a five-year period were classified as technology based (software, hardware and

integration fees) and implementation (project management staff, staff education and

clinical leadership). Note that two different licensing agreements were considered in the

analysis. Type 1 involved a fee for a definitive time period plus additional hardware and

maintenance costs per annum. Whereas, type 2 required a one-off license payment, but

maintenance and hardware costs were on-going. Prices were estimated using indicative

costs from suppliers and hospitals in the UK. Total cost for type 1 (including implementation

costs) over five years was €1.0 million and type 2 was €1.3 million per site. The authors

highlighted that this amounted to a national cost of €40.1 million for type 1 and €51.4

million for type 2 over 5 years.

2) NCEC (2013) “National Early Warning Score, National Clinical Guideline No. 1”

This report included a budget impact analysis (BIA) to assess the economic impact of

introducing NEWS and the COMPASS education programme. In assessing the budget impact

of employing NEWS and COMPASS two cost categories were considered, those that applied

to the initial implementation phase and the on-going intervention costs. Initial costs,

included staff costs (trainers and trainees), which amounted to €7.47million and non-staff

costs of €18,000 for materials. On-going intervention costs, which included staff and non-

staff costs, were estimated to be €425,000 per annum. The report acknowledged that

additional resources were likely due to the expected increase in the response rate to

triggers; however an estimate for this was not provided. In addition, efficiency savings were

likely owing to reduced ICU days (estimated at €4.2 million using Irish ICU LOS data and cost

per diem and assumptions regarding reduction in ICU admissions informed by Mitchell et al

131 | P a g e

2010). Other efficiency savings were gained from replacing the previously used ALERT

system with COMPASS, realising a saving of approximately €6,000 in annual licence fees per

annum, also disability treatments avoided due to the reduction in cardiac arrest were

expected but the potential value attributed to this saving was not given.

3) NCEC (2014) “Sepsis Management, National Clinical Guideline No 6”

Due to the paucity of economic investigation into the cost effectiveness of EWS, evidence

from the Sepsis Management Guideline of the National Clinical Effectiveness Committee,

published by the Department of Health in 2014 was included in the review.

The use of NEWS as a screening tool, to detect deteriorating in-patients, under the

framework of Sepsis management, was recommended as the first step in escalating care for

medical review and timely diagnosis of sepsis. The guideline prescribed that the Sepsis 6

protocol is completed within one hour of diagnosis, and was a key recommendation for the

initial treatment of sepsis. The Guideline Development Group recommended the Surviving

Sepsis Campaign Guideline and the Sepsis 6 bundle as the guide to the management of

sepsis in Ireland. They gave evidence that the implementation of integrated sepsis

protocols reduced variation in clinical care and demonstrated a reduction in mortality and in

ICU costs of 35% (Shorr et al. 2007)

The NCEC (2014) report highlighted the significance of the timely recognition of sepsis and

incorporated the NEWS system as part of a suite of guidelines to detect the acutely

deteriorating in-patient in the correct management of sepsis.

The report included evidence from a UK study, conducted by the Sepsis Trust, who found

that compliance with the Sepsis 6 protocol reduced the relative risk of sepsis by 46.6%

(Richards, 2013). The study also showed that patients in receipt of the protocol reduced

their LOS by an average of 2 days in critical care with a total reduction of 3.4 hospital days,

which equated to a cost saving of €4,500 per patient (Richards, 2013).

NCEC (2014) also conducted a BIA which considered the additional cost implications that

could arise further to implementation of the guideline. They outlined the costs involved in

introducing point of care lactate testing, and, in their BIA considered the costs of the device,

education and staff. The BIA showed an estimated cost of €1.9 million (€1.4 million incurred

in the initial set-up and on-going annual costs of €0.5 million) leading to a saving of €12

million per annum.

4) Simmes et al. (2014), “Financial consequences of the implementation of a rapid

response team on a surgical ward”

This analysis, performed from the Dutch health payer’s perspective, compared costs before

and after implementation of an RRS system on a surgical ward. The RRS in this

investigation, involved a single parameter track and trigger system (EWS) and a MET, led by

a doctor. The study identified a non-significant reduction in cardiac arrest by 0.25%; a

132 | P a g e

significant increase in unplanned ICU admissions from 2.5% to 4.1%, without a decrease in

severity of illness (APACHE II score 17.5 v’s 17.6) and a median ICU LOS (3.5 v’s 3 days).

Hospital LOS did not change after RRS implementation.

To estimate the financial consequences of the RRS, the authors determined the mean costs

of RRS per patient day and also tested the hypothesis that an increase in unplanned ICU

admissions with less severely ill patients would subsequently reduce the mean RRS cost per

patient per day. Mean RRS costs per patient, per day were €26.87, of which the extra

unplanned ICU day was the most costly component (85%). The authors found that further to

implementation of RRS there was little difference in the mean APACHE II scores of patients

for unplanned ICU admission but identified a delayed or absent MET consult in half of cases

prior to an adverse event.

In the scenario analysis, the APACHE II was reduced to 14, which significantly reduced the

mean RRS costs per patient per day by 62% even when one-third extra MET consults and

one-fifth extra ICU admissions were also considered. The authors however pointed out that

their scenario analysis did not correct for the possible reduction in costs for avoiding

unplanned ICU admissions and unexpected death as a consequence of timely MET consults

and unplanned ICU referrals of less severely ill patients.

They concluded that further to the implementation of RRS, the costs for extra unplanned

ICU days were relatively high but the remaining RRS costs (implementation and

maintenance, training, nurse time and MET consults) were relatively low. By reducing the

APACHE II score to 14 in the scenario analysis, they confirmed that costs for the number of

unplanned ICU admissions could be reduced if less severely ill patients are referred to the

ICU despite the considerable expected increases in MET consults and unplanned ICU

admissions.

5) Subbe et al. (2014) - “A pragmatic triage system to reduce length of stay in medical

emergency admission: Feasibility study and health economic analysis”

The authors examined a triage system to detect low risk patients based on their theory

that identification and efficient management of these patients yield better patient

outcomes, whilst, simultaneously reducing health care costs. Their analysis considered a

cohort in an Acute Medical Unit (AMU) of a District General Hospital in Wales and

compared a six month baseline phase to a 6 month intervention phase.

Their AMU’S Electronic Point of Care (EPOC) system used physiological bed-side

observations, past medical history and functional measures to appoint patients into risk

groups of in-hospital death using the Simple Clinical Score (SCS).

During the intervention, the Clinical Frailty Scale (CFS) was applied to facilitate faster

classification and identification of patients. Patients identified by the algorithm as “very

low risk” (SCS 0-3) with no frailty issues (CFS 1–3) on admission were interviewed and

133 | P a g e

examined by the Navigator (an advanced practitioner with prescribing competencies).

The Navigator established a working diagnosis and accelerated treatment pathways for

the patient.

In the very low risk group, control phase patients were compared with intervention

phase patients. LOS was reduced by a mean of 1.85 days thus a reduction in the cost of

care. The authors indicated a reduction in the mean cost per low risk patient of £482,

which translated to an overall cost saving of £250,158 for the intervention period.

The authors concluded that the implementation of an advanced triage system (An EPOC

system, CFS for identification of low risk patients and a Navigator to manage patient

care) led to significant reductions in LOS, hospital costs with better outcomes for

patients.

Discussion

As per previous systematic literature reviews the economic evidence from the literature

on EWS used in adult patients in acute healthcare settings for the timely detection of

physical or clinical deterioration is limited (HIQA, 2015b & NCEC, 2013). Three of the five

studies considered for this review are Irish studies previously commissioned by Department

of Health. We also included two studies that did not specifically examine an EWS system.

The NCEC (2014) Sepsis Management Guideline was considered as it recommends the use of

NEWS as a screening tool to detect deteriorating in-patients, under the sepsis management

framework. The guideline recommended NEWS as the first step in escalating care for

medical review and timely diagnosis of sepsis. Secondly, evidence from Subbe et al. (2014)

was incorporated into the review based on their findings that a computer-assisted triage

system had a measurable impact on cost of care for patients with very low risk of death.

This review re-affirms the conclusions of previous studies (NCEC, 2013 & CRD, 2014) that

there are no published full economic evaluations of EWS used in adult patients in acute

healthcare settings for the timely detection of physical or clinical deterioration. Also, where

partial evaluations have been conducted, estimates of costs averted are not always

presented and when they are, they are often based on clinical literature from a hospital or

trial that may not be directly transferable to every clinical setting given the heterogeneity of

clinical environments.

The results of this review indicate that implementing some form of track and trigger

system resulted in a reduced LOS. The HTA, BIAs’ and cost descriptions reviewed within

this analysis, suggest that a EWS system leads to cost and/or efficiency savings. If this

trend is continuous and savings can be realised, one could hypothesis that EWS may

indeed be cost effective. However, despite the widespread investment in EWS throughout

the developed world, there is currently no formal evidence using economic evaluation

techniques to support this proposition.

134 | P a g e

There have been recommendations and calls to assess the cost effectiveness of EWS

systems (CRD, 2014). However, to date, no economic evaluations have been published.

This paucity of evidence may be due to practical difficulties as exposed by the

abandonment of the costing component of the COMET study in the Netherlands owing to

difficulties in collecting reliable data (Ludkihuize, 2016). Other practicalities in conducting a

full economic evaluation of such complex interventions, in line with standard guidelines,

include difficulties surrounding the measurement of intangibles like culture, creating

champions and other socio-cultural aspects. Also, while mortality estimates are available

these may not capture the benefit of early warning scores/systems. The patients under

consideration are acute and prone to serious escalation. Thus to capture the benefit of

EWS, a quality of life (and dying) measurement is warranted. Nevertheless, isolating

improvements to quality of life (and dying) as being attributable to EWS too has practical

implications.

This review demonstrates the potential for EWS to reduce LOS. It is apparent that further

research is needed to investigate the cost effectiveness of EWS and the appropriateness of

using standard methods to do so.


There may be a requirement for more research to be conducted in tandem investigating

the cost effectiveness of NEWS. Investigators could give consideration to the

appropriateness of using standard methods to investigate the cost effectiveness of NEWS.

Budget Impact Analysis could be undertaken with reference to the relevant NCEC and HIQA

guidance.

135 | P a g e

References ABBOTT, T. E. F., VAID, N., IP, D., CRON, N., WELLS, M., TORRANCE, H. D. T. & EMMANUEL, J. 2015. A single-centre observational cohort study of admission National Early Warning Score (NEWS). Resuscitation, 92, 89-93. ALAM, N., HOBBELINK, E. L., VAN TIENHOVEN, A. J., VAN DE VEN, P. M., JANSMA, E. P. & NANAYAKKARA, P. W. B. 2014. The impact of the use of the Early Warning Score (EWS) on patient outcomes: A systematic review. Resuscitation, 85, 587-594. ALAM, N., VEGTING, I. L., HOUBEN, E., VAN BERKEL, B., VAUGHAN, L., KRAMER, M. H. H. & NANAYAKKARA, P. W. B. 2015. Exploring the performance of the National Early Warning Score (NEWS) in a European emergency department. Resuscitation, 90, 111-115. ALVAREZ, C. A., CLARK, C. A., SONG, Z., HALM, E. A., SHANNON, J. J., GIROD, C. E., COOPER, L. & AMARASINGHAM, R. 2013. Predicting out of intensive care unit cardiopulmonary arrest or death using electronic medical record data. BMC Medical Informatics & Decision Making, 13, 1-11. ANTMAN E, COHEN M, BERNINK P. J. L. M., MCCABE, C. H., HORACEK, T., PAPUCHIS, G., MAUNTER, B., CORBALAN, R., RADLEY, D. & BRAUNWALD, E. 2000. The TIMI risk score for unstable angina/non-ST elevation MI—A method for prognostication and therapeutic decision making. JAMA, 284, 835-842. AUSTRALIAN COMMISSION ON SAFETY AND QUALITY IN HEALTH CARE. 2010. National Consensus Statement: Essential Elements for Recognising and Responding to Clinical Deterioration. Available from http://www.safetyandquality.gov.au/wp-content/uploads/2012/01/national_consensus_statement.pdf AUSTRALIAN COMMISSION ON SAFETY AND QUALITY IN HEALTH CARE. 2012. Safety and Quality Improvement Guide Standard 9: Recognising and Responding to Clinical Deterioration in Acute Health Care. Available from http://www.safetyandquality.gov.au/wp-content/uploads/2012/10/Standard9_Oct_2012_WEB.pdf BADRIYAH, T., BRIGGS, J. S., MEREDITH, P., JARVIS, S. W., SCHMIDT, P. E., FEATHERSTONE, P. I., PRYTHERCH, D. R. & SMITH, G. B. 2014. Decision-tree early warning score (DTEWS) validates the design of the National Early Warning Score (NEWS). Resuscitation, 85, 418-423. BANNARD-SMITH, J., ABBAS, S., INGLEBY, S., FULLWOOD, C., JONES S. & EDDLESTON J. 2015. Use of an electronic early warning scote a motality for patients admitted out of hours to a large teaching hospital. Critical Care, 19, P408. BERGSTROM, N., BRADEN, B. J., LAGUZZA, A. & HOLMAN, V. 1987. The Braden Scale for predicting pressure sore risk. Nursing Research, 36, 205-210.

136 | P a g e

BIAN, Y., XU, F., LV, R.-J., WANG, J.-L., CAO, L.-J., XUE, L., ZHENG, W., QIAO, Y.-X., YAN, X.-L., LIU, Z.-F., ZHANG, Y. & CHEN, Y.-G. 2015. An early warning scoring system for the prevention of acute heart failure. International Journal Of Cardiology, 183, 111-116. BLEYER, A. J., VIDYA, S., RUSSELL, G. B., JONES, C. M., SUJATA, L., DAEIHAGH, P. & HIRE, D. 2011. Longitudinal analysis of one million vital signs in patients in an academic medical center. Resuscitation, 82, 1387-1392. BONNICI, T., TARASSENKO, L., CLIFTON, D. & WATKINSON, P. 2013. The digital patient.

Clinical Medicine, 13, 252-257.

BULUT, M., CEBICCI, H., SIGIRLI, D., SAK, A., DURMUS, O., TOP, A. A., KAYA, S. & UZ, K. 2014. The comparison of modified early warning score with rapid emergency medicine score: a prospective multicentre observational cohort study on medical and surgical patients presenting to emergency department. Emergency Medicine Journal, 31, 476-481. BUNKENBORG, G., SAMUELSON, K., POULSEN, I., LADELUND, S. & ÅKESON, J. 2014. Lower incidence of unexpected in-hospital death after interprofessional implementation of a bedside track-and-trigger system. Resuscitation, 85, 424-430. CAHILL, H., JONES, A., HERKES, R., COOK, K., STIRLING, A., HALBERT, T., YATES, A., LAL, S., GARDO, A., DONNELLY, R. & GATTAS, D. J. 2011. Introduction of a new observation chart and education programme is associated with higher rates of vitalsign ascertainment in hospital wards. BMJ Quality and Safety, 20, 791-796. CALDWELL, P. H. Y., BENNETT, T. & MELLIS, C. 2012. Easy guide to searching for evidence for the busy clinician. Journal of Paediatric Child Health, 48, 1095-1100. CAPAN, M., IVY, J. S., ROHLEDER, T., HICKMAN, J. & HUDDLESTON, J. M. 2015a. Individualizing and optimizing the use of early warning scores in acute medical care for deteriorating hospitalized patients. Resuscitation, 93, 107-112. CAPAN, M., IVY, J. S., WILSON, J. R. & HUDDLESTON, J. M. 2015b. A stochastic model of acute-care decisions based on patient and provider heterogeneity. Health Care Management Science, 1-20. CATTERMOLE, G. N., MAK, S. K., LIOW, C. H., HO, M. F., HUNG, K. Y., KEUNG, K. M., LI, H. M., CRAHAM, C. A., RAINER, T. H. 2009. Derivation of a prognostic score for identifying critically ill patients in an emergency department resuscitation room. Resuscitation, 80, 1000-1005. CATTERMOLE, G. N., LIOW, E. C. H., GRAHAM, C. A. & RAINER, T. H. 2014. THERM: the Resuscitation Management score. A prognostic tool to identify critically ill patients in the emergency department. Emergency Medicine Journal, 31, 803-807. CHRIST, M., GROSSMANN, F., WINTER, D., BINGISSER, R. & PLATZ, E. 2010. Modern triage in the emergency department. Deutsches Ärzteblatt International, 107, 892-898.

http://www.ncbi.nlm.nih.gov/pubmed/?term=Christ%20M%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

http://www.ncbi.nlm.nih.gov/pubmed/?term=Grossmann%20F%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

http://www.ncbi.nlm.nih.gov/pubmed/?term=Winter%20D%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

http://www.ncbi.nlm.nih.gov/pubmed/?term=Bingisser%20R%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

http://www.ncbi.nlm.nih.gov/pubmed/?term=Platz%20E%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

https://www.google.ie/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0ahUKEwjkxLbwkfDLAhVCuw8KHaRFD5sQFgghMAE&url=https%3A%2F%2Fwww.aerzteblatt.de%2Fint&usg=AFQjCNF-3qyE4jvH9hvdO4DhVVPioLZOFw&sig2=b4qLQEaJP8VSu3iDbmF2sQ&bvm=bv.118443451,d.ZWU

137 | P a g e

CHRISTENSEN, D., JENSEN, N. M., MAALØE, R., RUDOLPH, S. S., BELHAGE, B. & PERRILD, H. 2011. Nurse-administered early warning score system can be used for emergency department triage. Danish Medical Bulletin, 58, A4221-A4221. CHRISTOFIDIS, M. J., HILL, A., HORSWILL, M. S. & WATSON, M. O. 2013. A human factors approach to observation chart design can trump health professionals’ prior chart experience. Resuscitation, 84, 657-665. CHRISTOFIDIS, M. J., HILL, A., HORSWILL, M. S. & WATSON, M. O. 2014. Observation charts with overlapping blood pressure and heart rate graphs do not yield the performance advantage that health professionals assume: an experimental study. Journal of Advanced Nursing, 70, 610-624. CHRISTOFIDIS, M. J., HILL, A., HORSWILL, M. S. & WATSON, M. O. 2015. Less is more: the design of early-warning scoring systems affects the speed and accuracy of scoring. Journal of Advanced Nursing, 71, 1573-1586. CHURPEK, M. M., YUEN, T. C. & EDELSON, D. P. 2013. Risk Stratification of Hospitalized Patients on the Wards. CHEST, 143, 1758-1765. CHURPEK, M. M., YUEN, T. C., HUBER, M. T., PARK, S. Y., HALL, J. B. & EDELSON, D. P. 2012a. Predicting cardiac arrest on the wards: a nested case-control study. CHEST, 141, 1170-1176. CHURPEK, M. M., YUEN, T. C., PARK, S. Y., MELTZER, D. O., HALL, J. B. & EDELSON, D. P. 2012b. Derivation of a cardiac arrest prediction model using ward vital signs*. Critical Care Medicine, 40, 2102-2108. CHURPEK, M. M., YUEN, T. C., PARK, S. Y., GIBBONS, R. & EDELSON, D. P. 2014a. Using electronic health record data to develop and validate a prediction model for adverse outcomes in the wards*. Crit Care Medicine, 42, 841-848. CHURPEK, M. M., YUEN, T. C., WINSLOW, C., HALL, J. & EDELSON, D. P. 2015. Differences in Vital Signs Between Elderly and Nonelderly Patients Prior to Ward Cardiac Arrest. Critical Care Medicine, 43, 816-822. CHURPEK, M. M., YUEN, T. C., WINSLOW, C., ROBICSEK, A. A., MELTZER, D. O., GIBBONS, R. D. & EDELSON, D. P. 2014b. Multicenter development and validation of a risk stratification tool for ward patients. American Journal of Respiratory & Critical Care Medicine, 190, 649-655. CLIFTON, D. A., CLIFTON, L., SANDU, D.-M., SMITH, G. B., TARASSENKO, L., VOLLAM, S. A. & WATKINSON, P. J. 2015. 'Errors' and omissions in paper-based early warning scores: the association with changes in vital signs--a database analysis. BMJ Open, 5, e007376-e007376. CONSIDINE, J., CHARLESWORTH, D. & CURREY, J. 2014. Characteristics and outcomes of patients requiring rapid response system activation within hours of emergency admission. Critical Care Resuscitation, 16, 184-189.

138 | P a g e

CONSIDINE, J., RAWET, J. & CURREY, J. 2015. The effect of a staged, emergency department specific rapid response system on reporting of clinical deterioration. Australasian Emergency Nursing Journal: AENJ, 18, 218-226. COOKSLEY, T., KITLOWSKI, E. & HAJI-MICHAEL, P. 2012. Effectiveness of Modified Early Warning Score in predicting outcomes in oncology patients. QJM: An International Journal of Medicine, 105, 1083-1088. CORFIELD, A. R., LEES, F., ZEALLEY, I., HOUSTON, G., DICKIE, S., WARD, K. & MCGUFFIE, C. 2014. Utility of a single early warning score in patients with sepsis in the emergency department. Emergency Medicine Journal, 31, 482-487. CRETIKOS, M., CHEN, J., HILLMAN, K., BELLOMO, R., FINFER, S., & FLABOURIS, A. 2007. The objective medical emergency team activation criteria: a case-control study. Resuscitation, 73, 62-72. CRD. 2008. CRD’s guidance for undertaking reviews in health care. Available from https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf DAWES, T. R., CHEEK, E., BEWICK, V., DENNIS, M., DUCKITT, R. W., WALKER, J., FORNI, L. G. & VENN, R. 2014. Introduction of an electronic physiological early warning system: effects on mortality and length of stay. BJA: The British Journal of Anaesthesia, 113, 603-609. DE MEESTER, K., DAS, T., HELLEMANS, K., VERBRUGGHE, W., JORENS, P. G., VERPOOTEN, G. A. & VAN BOGAERT, P. 2013a. Impact of a standardized nurse observation protocol including MEWS after Intensive Care Unit discharge. Resuscitation, 84, 184-188. DE MEESTER, K., VERSPUY, M., MONSIEURS, K. G. & VAN BOGAERT, P. 2013b. SBAR improves nurse-physician communication and reduces unexpected death: A pre and post intervention study. Resuscitation, 84, 1192-1196. DEPARTMENT OF HEALTH AND CHILDREN. 2008. Building a Culture of Patient Safety: Report on the Commission of Patient Safety and Quality Assurance. Available from http://health.gov.ie/wp-content/uploads/2014/03/en_patientsafety.pdf DOH. 2014. National Clinical Guidelines. Available from http://health.gov.ie/patient-safety/ncec/national-clinical-guidelines-2/ DOH. 2016. NCEC Processes and Templates. Available from http://health.gov.ie/patient-safety/ncec/resources-and-learning/ncec-processes-and-templates/ DROWER, D., MCKEANY, R., JOGIA, P. & JULL, A. 2013. Evaluating the impact of implementing an early warning score system on incidence of in-hospital cardiac arrest. New Zealand Medical Journal, 126, 26-34.

https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf

139 | P a g e

DRUMMOND, M.F. & JEFFERSON, T.O. 1996. Guidelines for authors and peer reviewers of

economic submissions to the BMJ. BMJ, 313(7052), 275-283.

DUCKITT, R. W., BUXTON-THOMAS, R., WALKER, J., CHEEK, E., BEWICK, V., VENN, R. & FORNI, L. G. 2007. Worthing physiological scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study. British Journal of Anaesthesia, 98, 769-774. DUNDAR, Z. D., ERGIN, M., KARAMERCAN, M. A., AYRANCI, K., COLAK, T., TUNCAR, A., CANDER, B. & GUL, M. 2015. Modified Early Warning Score and VitalPac Early Warning Score in geriatric patients admitted to emergency department. European Journal Of Emergency Medicine: Official Journal Of The European Society For Emergency Medicine. ECCLES, S. R., SUBBE, C., HANCOCK, D. & THOMSON, N. 2014. CREWS: Improving specificity whilst maintaining sensitivity of the National Early Warning Score in patients with chronic hypoxaemia. Resuscitation, 85, 109-111. EDELSON, D. P., RETZER, E., WEIDMAN, E. K., WOODRUFF, J., DAVIS, A. M., MINSKY, B. D., MEADOW, W., HOEK, T. L. V. & MELTZER, D. O. 2011. Patient acuity rating: quantifying clinical judgment regarding inpatient stability. Journal Of Hospital Medicine, 6, 475-479. ENGLAND, K., BION, J.F., 2008, Introduction of medical emergency teams in Australia and New Zealand: a multicentre study. Crit Care 12:151. ESCOBAR, G. J., LAGUARDIA, J. C., TURK, B. J., RAGINS, A., KIPNIS, P. & DRAPER, D. 2012. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. Journal Of Hospital Medicine, 7, 388-395. ETTER, R., TAKALA, J. & MERZ, T. M. 2014. Intensive care without walls - introduction of a Medical Emergency Team system in a Swiss tertiary care centre. Swiss Medical Weekly, 144, w14027-w14027. EUnetHTA. 2013. HTA core model for medical and surgical interventions European Network for Health Technology Assessment. FINLAY, G. D., ROTHMAN, M. J. & SMITH, R. A. 2014. Measuring the modified early warning score and the Rothman Index: Advantages of utilizing the electronic medical record in an early warning system. Journal of Hospital Medicine, 9, 116-119. FLANAGAN, E.M. 2013. The Implementation of an Early Warning System to a Sub-Acute Unit [Masters dissertation]. Dublin: Royal College of Surgeons in Ireland. FOX, A. & ELLIOTT, N. 2015. Early warning scores: a sign of deterioration in patients and systems. Nursing Management, 22, 26-31.

140 | P a g e

FOY, R., HEMPEL, S., RUBENSTEIN, L., SUTTORP, M., SEELIG, M., SHANMAN, R., & SHEKELLE,

P. G. 2010. Meta-analysis: Effect of interactive communication between collaborating

primary care physicians and specialists. Annals of Internal Medicine, 152, 247-258.

FUNG, K., KHAN, F. & DAWSON, J. 2014. The introduction of an integrated early warning score observation chart-a picture paints a thousand words. Journal Of Patient Safety, 10, 13-19. GEIER, F., POPP, S., GREVE, Y., ACHTERBERG, A., GLÖCKNER, E., ZIEGLER, R., HEPPNER, H. J., MANG, H. & CHRIST, M. 2013. Severity illness scoring systems for early identification and prediction of in-hospital mortality in patients with suspected sepsis presenting to the emergency department. Wiener Klinische Wochenschrift, 125, 508-515. GHANEM-ZOUBI, N. O., VARDI, M., LAOR, A., WEBER, G. & BITTERMAN, H. 2011. Assessment of disease-severity scoring systems for patients with sepsis in general internal medicine departments. Critical Care (London, England), 15, R95-R95. GOLDHILL, D. R., WORTHINGTON, L., MULCAHY, A., TARLING, M. & SUMMER, A. 1999. The patient-at-risk team: identifying and managing seriously ill ward patients. Anaesthesia, 54, 853-860. GOODACRE, S., TURNER, J. & NICHOLL, J. 2006. Prediction of mortality among emergency medical admissions. Emergency Medicine Journal, 23, 372-375. GORDON, C. F. & BECKETT, D. J. 2011. Significant deficiencies in the overnight use of a Standardised Early Warning Scoring system in a teaching hospital. Scottish Medical Journal, 56, 15-18. GRADE Working Group. 2014. Grading the quality of evidence and the strength of

recommendations. Available from http://www.gradeworkinggroup.org/intro.htm

GRIFFITHS, J. R. & KIDNEY, E. M. 2012. Current use of early warning scores in UK emergency

departments. Emergency Medicine Journal, 29, 65-66.

HACKMANN, G., CHEN, M., CHIPARA, O., LU, C., CHEN, Y., KOLLEF, M. & BAILEY, T. C. 2011. Toward a two-tier clinical warning system for hospitalized patients. AMIA ... Annual Symposium Proceedings / AMIA Symposium. AMIA Symposium, 2011, 511-519. HAIG, K. M., SUTTON, S. & WHITTINGTON, J. 2006. SBAR: a shared mental model for improving communication between clinicians. The Joint Commission Journal on Quality and Patient Safety, 32, 167-75. HANDS, C., REID, E., MEREDITH, P., SMITH, G. B., PRYTHERCH, D. R., SCHMIDT, P. E. & FEATHERSTONE, P. I. 2013. Patterns in the recording of vital signs and early warning scores: compliance with a clinical escalation protocol. BMJ Quality & Safety, 22, 719-726.

http://www.ncbi.nlm.nih.gov/pubmed/?term=Goodacre%20S%5BAuthor%5D&cauthor=true&cauthor_uid=16627839

http://www.ncbi.nlm.nih.gov/pubmed/?term=Turner%20J%5BAuthor%5D&cauthor=true&cauthor_uid=16627839

http://www.ncbi.nlm.nih.gov/pubmed/?term=Nicholl%20J%5BAuthor%5D&cauthor=true&cauthor_uid=16627839

http://www.ncbi.nlm.nih.gov/pubmed/?term=Goodacre+and+rems+and+2006

http://www.ncbi.nlm.nih.gov/pubmed/?term=Haig%20KM%5BAuthor%5D&cauthor=true&cauthor_uid=16617948

http://www.ncbi.nlm.nih.gov/pubmed/?term=Sutton%20S%5BAuthor%5D&cauthor=true&cauthor_uid=16617948

http://www.ncbi.nlm.nih.gov/pubmed/?term=Whittington%20J%5BAuthor%5D&cauthor=true&cauthor_uid=16617948

141 | P a g e

HARRIS, P. 2013. Early warning scores in cardiac arrest patients. British Journal of Cardiac Nursing, 8, 432-437. HEALTH SERVICE EXECUTIVE. 2012. National Early Warning Score Observation Sheet. Available from http://www.hse.ie/eng/about/Who/clinical/ natclinprog/acutemedicineprog/earlywarningscore/observation HEALTH SERVICE EXECUTIVE. 2012. Acute Medicine Programme, Irish Association of Directors of Nursing and Midwifery. Facilitators’ guide: education session: healthcare assistants (HCAs) measuring, recording & communicating patients’ vital signs, incorporating the National Early Warning Score (Adult) using the National Patient Observation Chart. Dublin: Health Service Executive. HEALTH SERVICE EXECUTIVE. 2011 Training Manual for The National Early Warning Score and associated Education Programme. Health Service Executive. Dublin, Ireland. http://www.hse.ie/eng/about/Who/clinical/natclinprog/acutemedicineprogramme/earlywarningscore/compass.pdf [accessed 25 February 2015]. HEALTH SERVICE EXECUTIVE. 2011. COMPASS ‘Pointing you in the right direction’. Available from http://www.hse.ie/eng/about/Who/clinical/natclinprog/acutemedicineprogramme/earlywarningscore/compass.pdf HENDRICH, A. L., BENDER, P. S. & NYHUIS, A. 2003. Validation of the Hendrich II Fall Risk Model: a large concurrent case/control study of hospitalized patients. Applied Nursing Research, 16, 9-21. HIGGINS, J. P. T. & GREEN, S. 2011. Cochrane Handbook for Systematic Reviews of

Interventions. Available from http://handbook.cochrane.org/

HIQA. 2015a. Guidance on Budget Impact Analysis of Health Technologies in Ireland. Accessible from https://www.hiqa.ie/system/files/Guidance_on_Budget_Impact_Analysis_of_Health_Technologies_in_Ireland.pdf HIQA. 2014. Guidelines for the Retrieval and Interpretation of Economic Evaluations of

Health Technologies in Ireland. Available from https://www.hiqa.ie/system/files/Guidelines-

Retrieval-and-Interpretation-of-Econ-Lit.pdf

HIQA. 2015b. Health technology assessment of the use of information technology for early

warning and clinical handover systems. Available from

https://www.hiqa.ie/system/files/HTA-of-use-of-information-technology-for-early-warning-

and-clinical-handover-systems.pdf

142 | P a g e

HIQA & NCEC. 2015. National Quality Assurance Criteria for Clinical Guidelines - Version 2.

Available from http://health.gov.ie/wp-content/uploads/2015/07/HIQA-NCEC-National-

Quality-Assurance-Criteria-NCG-v2-April-2015.pdf

HO, L. O., LI, H., SHAHIDAH, N., KOH, Z. X., SULTANA, P. & HOCK ONG, M. E. 2013. Poor performance of the modified early warning score for predicting mortality in critically ill patients presenting to an emergency department. World Journal Of Emergency Medicine, 4, 273-278. HORSWILL, M. S., PREECE, M. H. W., HILL, A., CHRISTOFIDIS, M. J., KARAMATIC, R. M., HEWETT, D. J. & WATSON M. O. 2010. Human factors research regarding observation charts: Research project overview. Available from http://www.safetyandquality.gov.au/wp-content/uploads/2012/01/35986-HumanFactors.pdf HUDDART, S., DICKINSON, M., QUINEY, N., PEDEN, C. J., SWART, M., MCCORMICK, B. & MOHAMMED, M. A. 2015. Use of a pathway quality improvement care bundle to reduce mortality after emergency laparotomy. British Journal of Surgery, 102, 57-66. HUGGAN, P. J., AKRAM, F., ER, B. H. D., CHRISTEN, L. S. J., WEIXIAN, L., LIM, V., HUANG, Y. & MERCHANT, R. A. 2015. Measures of acute physiology, comorbidity and functional status to differentiate illness severity and length of stay among acute general medical admissions: a prospective cohort study. Internal Medicine Journal, 45, 732-740. INSTITUTE FOR CLINICAL SYSTEMS IMPROVEMENT. 2011. Health Care Protocol: Rapid response team. Available from https://www.icsi.org/_asset/8snj28/RRT.pdf INSTITUTE FOR HEALTHCARE IMPROVEMENT. 2016. Early Warning Systems: Scorecards That Save Lives. Available from http://www.ihi.org/resources/pages/improvementstories/earlywarningsystemsscorecardsthatsavelives.aspx JARVIS, S. W., KOVACS, C., BADRIYAH, T., BRIGGS, J., MOHAMMED, M. A., MEREDITH, P., SCHMIDT, P. E., FEATHERSTONE, P. I., PRYTHERCH, D. R. & SMITH, G. B. 2013. Development and validation of a decision tree early warning score based on routine laboratory test results for the discrimination of hospital mortality in emergency medical admissions. Resuscitation, 84, 1494-1499. JARVIS, S., KOVACS, C., BRIGGS, J., MEREDITH, P., SCHMIDT, P. E., FEATHERSTONE, P. I., PRYTHERCH, D. R. & SMITH, G. B. 2015a. Aggregate National Early Warning Score (NEWS) values are more important than high scores for a single vital signs parameter for discriminating the risk of adverse outcomes. Resuscitation, 87, 75-80. JARVIS, S., KOVACS, C., BRIGGS, J., MEREDITH, P., SCHMIDT, P. E., FEATHERSTONE, P. I., PRYTHERCH, D. R. & SMITH, G. B. 2015b. Can binary early warning scores perform as well as standard early warning scores for discriminating a patient's risk of cardiac arrest, death or unanticipated intensive care unit admission? Resuscitation, 93, 46-52.

143 | P a g e

JARVIS, S. W., KOVACS, C., BRIGGS, J., MEREDITH, P., SCHMIDT, P. E., FEATHERSTONE, P. I., PRYTHERCH, D. R. & SMITH, G. B. 2015c. Are observation selection methods important when comparing early warning score performance? Resuscitation, 90, 1-6. JO, S., LEE, J. B., JIN, Y. H., JEONG, T. O., YOON, J. C., JUN, Y. K. & PARK, B. Y. 2013. Modified early warning score with rapid lactate level in critically ill medical patients: the ViEWS-L score. Emergency Medicine Journal: EMJ, 30, 123-129. JONES, B. G. 2013. Developing a Vital Sign Alert System. American Journal of Nursing, 113, 36-46. JONES, D., BELLOMO, R., DEVITA, M.A. 2009. Effectiveness of the medical emergency team: the importance of dose. Crit Care 13:313. JONES, S., MULLALLY, M., INGLEBY, S., BUIST, M., BAILEY, M. & EDDLESTON, J. M. 2011. Bedside electronic capture of clinical observations and automated clinical alerts to improve compliance with an Early Warning Score protocol. Critical Care And Resuscitation: Journal Of The Australasian Academy Of Critical Care Medicine, 13, 83-88. KEEP, J. W., MESSMER, A. S., SLADDEN, R., BURRELL, N., PINATE, R., TUNNICLIFF, M. & GLUCKSMAN, E. 2015. National early warning score at Emergency Department triage may allow earlier identification of patients with severe sepsis and septic shock: a retrospective observational study. Emergency Medicine Journal: EMJ. KELLETT, J. & DEANE, B. 2006. The Simple Clinical Score predicts mortality for 30 days after admission to an acute medical unit. QJM, 99, 771-781. KELLETT, J., DEANE, B. & GLEESON, M. 2008. Derivation and validation of a score based on hypotension, oxygen saturation, low temperature, ECG changes and Loss of independence (HOTEL) that predicts early mortality between 15 min and 24 h after admission to an acute medical unit. Resuscitation, 78, e8. KELLETT, J. & KIM, A. 2012. Validation of an abbreviated Vitalpac Early Warning Score (ViEWS) in 75,419 consecutive admissions to a Canadian regional hospital. Resuscitation, 83, 297-302. KIM, W. Y., SHIN, Y. J., LEE, J. M., HUH, J. W., KOH, Y., LIM, C.-M. & HONG, S. B. 2015. Modified Early Warning Score Changes Prior to Cardiac Arrest in General Wards. PLoS ONE, 10, 1-11. KIRKLAND, L. L., MALINCHOC, M., O'BYRNE, M., BENSON, J. T., KASHIWAGI, D. T., BURTON, M. C., VARKEY, P. & MORGENTHALER, T. I. 2013. A clinical deterioration prediction tool for internal medicine patients. American Journal Of Medical Quality: The Official Journal Of The American College Of Medical Quality, 28, 135-142.

144 | P a g e

KOLIC, I., CRANE, S., MCCARTNEY, S., PERKINS, Z. & TAYLOR, A. 2015. Factors affecting response to National Early Warning Score (NEWS). Resuscitation, 90, 85-90. KYRIACOS, U., JELSMA, J., JAMES, M. & JORDAN, S. 2014a. Monitoring vital signs: development of a modified early warning scoring (MEWS) system for general wards in a developing country. PLoS One, 9, e87073. Kyriacos, U., Jelsma, J. & Jordan, S. 2014. Record review to explore the adequacy of post-operative vital signs monitoring using a local modified early warning score (mews) chart to evaluate outcomes. PLoS One, 9, e87320. KYRIACOS, U., JELSMA, J., JAMES, M. & JORDAN, S. 2015. Early warning scoring systems versus standard observations charts for wards in South Africa: a cluster randomized controlled trial. Trials, 16, 103. KYRIACOS, U., JELSMA, J. & JORDAN, S. 2014b. Record review to explore the adequacy of post-operative vital signs monitoring using a local modified early warning score (mews) chart to evaluate outcomes. PLoS One, 9, e87320. LEONARD, M., GRAHAM, S. & BONACUM, D. 2004. The human factor: the critical importance of effective teamwork and communication in providing safe care. Quality & Safety in Health Care, 13, 185-190. LIAW, S. Y., CHAN, S. W.-C., CHEN, F.-G., HOOI, S. C. & SIAU, C. 2014. Comparison of virtual patient simulation with mannequin-based simulation for improving clinical performances in assessing and managing clinical deterioration: randomized controlled trial. Journal Of Medical Internet Research, 16, e214-e214. LIAW, S. Y., SCHERPBIER, A., KLAININ-YOBAS, P. & RETHANS, J. J. 2011. A review of educational strategies to improve nurses' roles in recognizing and responding to deteriorating patients. International Nursing Review, 58, 296-303. LIAW, S. Y., WONG, L. F., ANG, S. B. L., HO, J. T. Y., SIAU, C., ANG, E. N. K. 2015a. Strengthening the afferent limb of rapid response systems: an educational intervention using web-based learning for early recognition and responding to deteriorating patients. BMJ Quality and Safety, 0, 1-9. LIAW, S. Y., WONG, L. F., CHAN, S. W.-C., HO, J. T. Y., MORDIFFI, S. Z., ANG. S. B. L., GOH, P. S. & ANG, E. N. K. 2015b. Designing and evaluating an interactive multimedia web-based simulation for developing nurses’ competencies in acute nursing care: randomized controlled trial. Journal Of Medical Internet Research,17, e5. LILJEHULT, J. & CHRISTENSEN, T. 2015. Early warning score predicts acute mortality in stroke patients. Acta Neurologica Scandinavica. LINDSEY, P. L. & JENKINS, S. 2013. Nursing Students' Clinical Judgment Regarding Rapid Response: The Influence of a Clinical Simulation Education Intervention. Nursing Forum, 48, 61-70.

http://www.ncbi.nlm.nih.gov/pubmed/24475226









145 | P a g e

LIU, F. Y., QIN, J, WANG, R. X., FAN, X. L., WANG, J., SUN, C. Y., CAO, T., LIANG, X. 2015. A

prospective validation of National Early Warning Score in emergency intensive care unit

patients at Beijing. Hong Kong Journal of Emergency Medicine, 22, 137-144.

LIU, N., KOH, Z. X., CHUA, E. C.-P., TAN, L. M.-L., LIN, Z., MIRZA, B. & ONG, M. E. H. 2014a. Risk scoring for prediction of acute cardiac complications from imbalanced clinical data. IEEE Journal Of Biomedical And Health Informatics, 18, 1894-1902. LIU, N., KOH, Z. X., GOH, J., LIN, Z., HAALAND, B., TING, B. P. & ONG, M. E. H. 2014b. Prediction of adverse cardiac events in emergency department patients with chest pain using machine learning for variable selection. BMC Medical Informatics And Decision Making, 14, 75-75. LIU, N., LIN, Z., CAO, J., KOH, Z., ZHANG, T., HUANG, G. B., SER, W. & ONG M. E. H. 2012. An

intelligent scoring system and its application to cardiac arrest prediction. IEEE Transactions

on Information Technology in Biomedicine, 16, 1324-1331.

LIU, N., LIN, Z., KOH, Z., HUANG, G. B., SER, W., ONG, M. E. H. 2011. Patient outcome

prediction with heart rate variability and vital signs. Journal of Signal Processing Systems, 64,

265-278.

LOBO, R., LYNCH, K. & CASSERLY, L. F. 2015. Cross-sectional audit on the relevance of

Elevated National Early Warning Score in medical patients at a Model 2 hospital in Ireland.

Irish Journal Of Medical Science, 184, 893-898.

Ludkihuize, (2016) re CRD. 2014. Impact of Early Warning Systems on Patient Outcomes. Availabel from https://www.york.ac.uk/crd/publications/effectiveness-matters/early-warning-systems/ LUDKIHUIZE, J. 7/1/2016 2016. RE: Email Correspondence Re: COMET Study. Type to MUPRHY, A. LUDIKHUIZE, J., BORGERT, M., BINNEKADE, J., SUBBE, C., DONGELMANS, D. & GOOSSENS, A. 2014. Standardized measurement of the Modified Early Warning Score results in enhanced implementation of a Rapid Response System: A quasi-experimental study. Resuscitation, 85, 676-682. LUDIKHUIZE, J., BRUNSVELD-REINDERS, A. H., DIJKGRAAF, M. G. W., SMORENBURG, S. M., DE ROOIJ, S. E. J. A., ADAMS, R., DE MAAIJER, P. F., FIKKERS, B. G., TANGKAU, P. & DE JONGE, E. 2015. Outcomes Associated With the Nationwide Introduction of Rapid Response Systems in The Netherlands. Critical Care Medicine, 43, 2544-2551. LUDIKHUIZE, J., DE JONGE, E. & GOOSSENS, A. 2011. Measuring adherence among nurses one year after training in applying the Modified Early Warning Score and Situation-Background-Assessment-Recommendation instruments. Resuscitation, 82, 1428-1433.

146 | P a g e

LUDIKHUIZE, J., SMORENBURG, S. M., DE ROOIJ, S. E. & DE JONGE, E. 2012. Identification of deteriorating patients on general wards; measurement of vital parameters and potential effectiveness of the Modified Early Warning Score. Journal Of Critical Care, 27, 424.e7-13. LYDON, S., BYRNE, D., OFFIAH, G., GLEESON, L. & O'CONNOR, P. 2015. A mixed-methods investigation of health professionals' perceptions of a physiological track and trigger system. BMJ Quality & Safety. MACKINTOSH, N., HUMPHREY, C. & SANDALL, J. 2014. The habitus of 'rescue' and its significance for implementation of rapid response systems in acute health care. Social Science & Medicine (1982), 120, 233-242. MAPP, I. D., DAVIS, L. L. & KROWCHUK, H. 2013. Prevention of Unplanned Intensive Care Unit Admissions and Hospital Mortality by Early Warning Systems. Dimensions of Critical Care Nursing, 32, 300-309. MARTIN, G., DUPRÉ, A., MULLIEZ, A., PRUNEL, F., SLIM, K. & PEZET, D. 2015. Validation of a score for the early diagnosis of anastomotic leakage following elective colorectal surgery. Journal Of Visceral Surgery, 152, 5-10. MASSEY, D., CHABOYER, W. & AITKEN, L. 2014. Nurses' perceptions of accessing a Medical Emergency Team: a qualitative study. Australian Critical Care: Official Journal Of The Confederation Of Australian Critical Care Nurses, 27, 133-138. MATHUKIA, C., FAN, W., VADYAK, K., BIEGE, C. & KRISHNAMURTHY, M. 2015. Modified Early Warning System improves patient safety and clinical outcomes in an academic community hospital. Journal Of Community Hospital Internal Medicine Perspectives, 5, 26716-26716. MCDONNELL, A., TOD, A., BRAY, K., BAINBRIDGE, D., ADSETTS, D. & WALTERS, S. 2013. A before and after study assessing the impact of a new model for recognizing and responding to early signs of deterioration in an acute hospital. Journal of Advanced Nursing, 69, 41-52. MCKAY, H., MITCHELL, I. A., SINN, K., MUGRIDGE, H., LAFFERTY, T., VAN LEUVAN, C., MAMOOTIL, S., ABDEL-LATIF M. E. 2013. Effect of a multifaceted intervention on documentation of vital signs and staff communication regarding deteriorating paediatric patients. Journal of Paediatrics & Child Health, 49, 48-56. MCLOUGHLIN, R. 2015. Implementing the National Early Warning Score for Healthcare Assistants (HCA-NEWS) in an Acute Hospital [Masters dissertation]. Dublin: Royal College of Surgeons in Ireland. MCNEILL, G. & BRYDEN, D. 2013. Do either early warning systems or emergency response teams improve hospital patient survival? A systematic review. Resuscitation, 84, 1652-1667. MERRIEL, A., VAN DER NELSON, H., MERRIEL, S., BENNETT, J., DONALD, F., DRAYCOTT, T. & SIASSAKOS, D. 2015. Identifying Deteriorating Patients Through Multidisciplinary Team

147 | P a g e

Training. American Journal Of Medical Quality: The Official Journal Of The American College Of Medical Quality. MITCHELL, I., MCKAY, H., VAN LEUVAN, C., BERRY, R., MCCUTCHEON, C., AVARD, B., SLATER, N., NEEMAN, T. & LAMBERTH, P. 2010. A prospective controlled trial of the effect of a multi-faceted intervention on early recognition and intervention in deteriorating hospital patients. Resuscitation, 81, 658-666. MOK, W., WANG, W., COOPER, S., ANG, E. N. K. & LIAW, S. Y. 2015. Attitudes towards vital signs monitoring in the detection of clinical deterioration: scale development and survey of ward nurses. International Journal for Quality in Health Care, 27, 207-213. MOON, A., COSGROVE, J. F., LEA, D., FAIRS, A. & CRESSEY, D. M. 2011. An eight year audit before and after the introduction of modified early warning score (MEWS) charts, of patients admitted to a tertiary referral intensive care unit after CPR. Resuscitation, 82, 150-154. MORA, J. C., SCHNEIDER, A., ROBBINS, R., BAILEY, M., BEBEE, B., HSIAO, Y.-F. F., CONSIDINE, J., JONES, D. & BELLOMO, R. 2015. Epidemiology of early Rapid Response Team activation after Emergency Department admission. Australasian Emergency Nursing Journal: AENJ. MORIARTY, J. P., SCHIEBEL, N. E., JOHNSON, M. G., JENSEN, J. B., CAPLES, S. M., MORLAN, B. W., HUDDLESTON, J. M., HUEBNER, M. & NAESSENS, J. M. 2014. Evaluating implementation of a rapid response team: considering alternative outcome measures. International Journal for Quality in Health Care, 26, 49-57. MORRIS, A., OWEN, H. M., JONES, K., HARTIN, J., WELCH, J. & SUBBE, C. P. 2013. Objective patient-related outcomes of rapid-response systems - a pilot study to demonstrate feasibility in two hospitals. Critical Care And Resuscitation: Journal Of The Australasian Academy Of Critical Care Medicine, 15, 33-39. MOSESON, E. M., ZHUO, H., CHU, J., STEIN, J. C., MATTHAY, M. A., KANGELARIS, K. N., LIU, K. D. & CALFEE, C. S. 2014. Intensive care unit scoring systems outperform emergency department scoring systems for mortality prediction in critically ill patients: a prospective cohort study. Journal Of Intensive Care, 2, 40-40. NAIDOO, D. K., RANGIAH, S. & NAIDOO, S. S. 2014. An evaluation of the Triage Early Warning Score in an urban accident and emergency department in KwaZulu-Natal. South African Family Practice, 56, 69-73. Neary, P.M., Regan, M., Joyce, M.J., McAnena, O.J., Callanan, I. 2015,"National early warning score (NEWS) – evaluation in surgery", International Journal of Health Care Quality Assurance, 28 (3),245 – 252. NEW SOUTH WALES CLINICAL EXCELLENCE COMMISSION. 2013. Between the flags keeping patients safe Initiative. And the Recognition and Management of Patients who are Clinically

148 | P a g e

Deteriorating policy directive. Available from http://www.cec.health.nsw.gov.au/programs/between-the-flags NCEC. 2013. National Early Warning Score, National Clinical Guideline No.1. Available from

http://health.gov.ie/wp-content/uploads/2015/01/NEWSFull-ReportAugust2014.pdf

NCEC. 2014. Sepsis Management, National Clinical Guideline No.6. Available form

http://health.gov.ie/wp-content/uploads/2015/06/Sepsis-Mgmt_Full-Report.pdf

NICE. 2007. Acute illness in adults in hospital: recognising and responding to deterioration.

Available from https://www.nice.org.uk/guidance/cg50

NIEGSCH, M., FABRITIUS, M. L. & ANHØJ, J. 2013. Imperfect Implementation of an Early Warning Scoring System in a Danish Teaching Hospital: A Cross-Sectional Study. PLoS ONE, 8, 1-6. NWULU, U., WESTWOOD, D., EDWARDS, D., KELLIHER, F. & COLEMAN, J. J. 2012. Adoption of an electronic observation chart with an integrated early warning scoring system on pilot wards: a descriptive report. Computers, Informatics, Nursing: CIN, 30, 371-379. OGLESBY, K. J., DURHAM, L., WELCH, J. & SUBBE, C. P. 2011. 'Score to Door Time', a benchmarking tool for rapid response systems: a pilot multi-centre service evaluation. Critical Care (London, England), 15, R180-R180. OLSSON, T., TERENT, A. & LIND, L. 2004. Rapid Emergency Medicine score: a new prognostic tool for in-hospital mortality in nonsurgical emergency department patients. Journal of Internal Medicine, 255, 579-587. ONG, M. E. H., LEE NG, C. H., GOH, K., LIU, N., KOH, Z. X., SHAHIDAH, N., ZHANG, T. T., FOOK-CHONG, S. & LIN, Z. 2012. Prediction of cardiac arrest in critically ill patients presenting to the emergency department using a machine learning score incorporating heart rate variability compared with the modified early warning score. Critical Care (London, England), 16, R108-R108. OZEKCIN, L. R., TUITE, P., WILLNER, K. & HRAVNAK, M. 2015. Simulation education: early identification of patient physiologic deterioration by acute care nurses. Clinical Nurse Specialist CNS, 29, 166-173. PARHAM, G. 2012. Recognition and response to the clinically deteriorating patient. Australian Medical Student Journal, 3, 18-22. PATEL, A., HASSAN, S., ULLAH, A., HAMID, T. & KIRK, H. 2015a. Early triaging using the Modified Early Warning Score (MEWS) and dedicated emergency teams leads to improved clinical outcomes in acute emergencies. Clinical Medicine (London, England), 15 Suppl 3, s3-s3.

149 | P a g e

PATEL, A. R., ZADRAVECZ, F. J., YOUNG, R. S., WILLIAMS, M. V., CHURPEK, M. M. & EDELSON, D. P. 2015b. The value of clinical judgment in the detection of clinical deterioration. JAMA Internal Medicine, 175, 456-458. PATEL, M. S., JONES, M. A., JIGGINS, M. & WILLIAMS, S. C. 2011. Does the use of a "track and trigger" warning system reduce mortality in trauma patients? Injury, 42, 1455-1459. PATERSON R., MACLEOD, D. C., THETFORD, D., BEATTIE, A., GRAHAM, C., LAM, S. & BELL, D. 2006. Prediction of in-hospital mortality and length of stay using an early warning scoring system: clinical audit. Clinical Medicine, 6, 281-284. PATTISON, N. & EASTHAM, E. 2012. Critical care outreach referrals: a mixed-method investigative study of outcomes and experiences. Nursing in Critical Care, 17, 71-82. PERIS, A., ZAGLI, G., MACCARRONE, N., BATACCHI, S., CAMMELLI, R., CECCHI, A., PERRETTA, L. & BECHI, P. 2012. The use of Modified Early Warning Score may help anesthesists in postoperative level of care selection in emergency abdominal surgery. Minerva Anestesiologica, 78, 1034-1038. PETERSEN, J. A., MACKEL, R., ANTONSEN, K. & RASMUSSEN, L. S. 2014. Serious adverse events in a hospital using early warning score – What went wrong? Resuscitation, 85, 1699-1703. PRYTHERCH, D. R., SMLTH, G. B., SCHMIDT, P. E., FEATHERSTONE, P. L. 2010. ViEWS - towards a national early warning score for detecting adult inpatient deterioration. Resuscitalion, 81, 932-937. REINI, K., FREDRIKSON, M. & OSCARSSON, A. 2012. The prognostic value of the Modified Early Warning Score in critically ill patients: a prospective, observational study. European Journal of Anaesthesiology (Lippincott Williams & Wilkins), 29, 152-157. RICHARDS, M. 2013. Sepsis management as an NHS clinical priority. Briefing - Professor Sir Mike Richards. ROMERO-BRUFAU, S., HUDDLESTON, J. M., ESCOBAR, G. J. & LIEBOW, M. 2015. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Critical Care (London, England), 19, 285-285. ROMERO-BRUFAU, S., HUDDLESTON, J. M., NAESSENS, J. M., JOHNSON, M. G., HICKMAN, J., MORLAN, B. W., JENSEN, J. B., CAPLES, S. M., ELMER, J. L., SCHMIDT, J. A., MORGENTHALER, T. I. & SANTRACH, P. J. 2014. Widely used track and trigger scores: Are they ready for automation in practice? Resuscitation, 85, 549-552. RONEY, J. K., WHITLEY, B. E., MAPLES, J. C., FUTRELL, L. S., STUNKARD, K. A. & LONG, J. D. 2015. Modified early warning scoring (mews): Evaluating the evidence for tool inclusion of sepsis screening criteria and impact on mortality and failure to rescue. Journal of Clinical Nursing.

150 | P a g e

ROSE, M. A., HANNA, L. A., NUR, S. A. & JOHNSON, C. M. 2015. Utilization of electronic modified early warning score to engage rapid response team early in clinical deterioration. Journal For Nurses In Professional Development, 31, E1-E7. ROTHMAN, M. J., ROTHMAN, S. I. & BEALS, J. 2013. Development and validation of a continuous measure of patient condition using the Electronic Medical Record. Journal of Biomedical Informatics, 46, 837-848. ROYAL COLLEGE OF PHYSICIANS. 2015. National Early Warning Score (NEWS). Available from https://www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-news ROYAL COLLEGE OF PHYSICIANS. 2012. National Early Warning Score (NEWS) Standardising the assessment of acute-illness severity in the NHS. Available from https://www.rcplondon.ac.uk/file/32/download?token=vfwDKQVS SCHMIDT, P. E., MEREDITH, P., PRYTHERCH, D. R., WATSON, D., WATSON, V., KILLEN, R. M., GREENGROSS, P., MOHAMMED, M. A. & SMITH, G. B. 2015. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Quality & Safety, 24, 10-20. SCHEMILT, I., MUGFORD, M., BYFORD, S., DRUMMOND, M., EISENSTEIN, E., KNAPP, M., et al. Chapter 15: Incorporating economics evidence. Higgins JPT GS, (Ed) In: Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration; 2011. SCOTTISH INTERCOLLEGIATE GUIDELINES NETWORK (SIGN). 2014. Care of deteriorating patients. Available from http://www.sign.ac.uk SEYMOUR, C. W., KAHN, J. M., COOKE, C. R., WATKINS, T. R., HECKBERT, S. R., REA, T. D. 2010. Prediction of critical illness during out-of-hospital emergency care. JAMA, 304, 747-754. SHADDEL, F., KHOSLA, V. & BANERJEE, S. 2014. Effects of introducing MEWS on nursing staff in mental health inpatient settings. Progress in Neurology & Psychiatry, 18, 24-27. SHAPIRO, N. I., HOWELL, M. D., TALMOR, D., DONNINO, M., NGO, L. & BATES, D. W. 2007. Mortality in Emergency Department Sepsis (MEDS) score predicts 1-year mortality. Critical Care Medicine, 35, 192-198. SHORR, A. F., MICEK, S. T., JACKSON JR, W. L. & KOLLEF, M. H. 2007. Economic implications of an evidence-based sepsis protocol: Can we improve outcomes and lower costs?*. Critical care medicine, 35, 1257-1262. SHAPIRO, N. I., WOLFE, R. E., MOORE, R. B., SMITH, E., BURDICK, E. & BATES, D. W. 2003. Mortality in Emergency Department Sepsis (MEDS) score: a prospective derived and validated clinical prediction tool. Critical Care Medicine, 31, 670-675.


151 | P a g e

SHUK-NGOR, S., CHI-WAI, O., LAI-YEE, W., CHUNG, J. Y. M. & GRAHAM, C. A. 2015. Is the Modified Early Warning Score able to enhance clinical observation to detect deteriorating patients earlier in an Accident & Emergency Department? Australasian Emergency Nursing Journal, 18, 24-32. SIGN. 2014. Critical appraisal: notes and checklists. Available from http://www.sign.ac.uk/methodology/checklists.html SIMMES, F., SCHOONHOVEN, L., MINTJES, J., ADANG, E. & HOEVEN, J. G. 2014. Financial consequences of the implementation of a rapid response system on a surgical ward. Journal of evaluation in clinical practice, 20, 342-347. SMITH, G. B., PRYTHERCH, D. R., MEREDITH, P., SCHMIDT, P. E. & FEATHERSTONE, P. I. 2013. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation, 84, 465-470. SMITH, M. E. B., CHIOVARO, J. C., O'NEIL, M., KANSAGARA, D., QUINONES, A., FREEMAN, M., MOTU'APUAKA, M. & SLATORE, C. G. 2014. Early Warning System Scores: A Systematic Review. SMITH, T., DEN HARTOG, D., MOERMAN, T., PATKA, P., VAN LIESHOUT, E. M. M. & SCHEP, N. W. L. 2012. Accuracy of an expanded early warning score for patients in general and trauma surgery wards. British Journal of Surgery, 99, 192-197. STARK, A. P., MACIEL, R. C., SHEPPARD, W., SACKS, G. & HINES, O. J. 2015. An Early Warning Score Predicts Risk of Death after In-hospital Cardiopulmonary Arrest in Surgical Patients. American Surgeon, 81, 916-921. SMITH, G. B., PRYTHERCH, D. R., MEREDITH, P. , SCHMIDT, P. E. 2016. Re: cross-sectional audit on the relevance of Elevated National Early Warning Score in medical patients at a Model 2 hospital in Ireland. Ir. J. Med. Sci., 185:267–268. STEWART, J., CARMAN, M., SPEGMAN, A. & SABOL, V. K. 2014. Evaluation of the effect of the modified early warning system on the nurse-led activation of the rapid response system. Journal Of Nursing Care Quality, 29, 223-229. SUBBE, C., KELLETT, J., WHITAKER, C., JISHI, F., WHITE, A., PRICE, S., WARD-JONES, J., HUBBARD, R., EELES, E. & WILLIAMS, L. 2014. A pragmatic triage system to reduce length of stay in medical emergency admission: feasibility study and health economic analysis. European journal of internal medicine, 25, 815-820. SUBBE, C. P., KRUGER, M., RUTHERFORD, P. & GEMMEL, L. 2001. Validation of a modified Early Warning Score in medical admissions. QJM, 94, 521-526.

152 | P a g e

SUBBE, C. P. & WELCH, J. R. 2013. Failure to rescue: using rapid response systems to improve care of the deteriorating patient in hospital. Clinical Risk, 19, 6-11. SUPPIAH, A., MALDE, D., ARAB, T., HAMED, M., ALLGAR, V., MORRIS-STIFF, G. & SMITH, A. 2014. The Modified Early Warning Score (MEWS): an instant physiological prognostic indicator of poor outcome in acute pancreatitis. JOP: Journal Of The Pancreas, 15, 569-576. TARASSENKO, L., CLIFTON, D. A., PINSKY, M. R., HRAVNAK, M. T., WOODS, J. R. & WATKINSON, P. J. 2011. Centile-based early warning scores derived from statistical distributions of vital signs. Resuscitation, 82, 1013-1018. TIRKKONEN, J., OLKKOLA, K. T., HUHTALA, H., TENHUNEN, J. & HOPPU, S. 2014. Medical emergency team activation: performance of conventional dichotomised criteria versus national early warning score. Acta Anaesthesiologica Scandinavica, 58, 411-419. TYSINGER, E. L. 2015. How vital are vital signs? a systematic review of vital sign compliance

and accuracy in nursing. Journal of Science & Medicine, 68 -75.

UMSCHEID, C. A., BETESH, J., VANZANDBERGEN, C., HANISH, A., TAIT, G., MIKKELSEN, M. E., FRENCH, B. & FUCHS, B. D. 2015. Development, implementation, and impact of an automated early warning and response system for sepsis. Journal of Hospital Medicine, 10, 26-31. UNDP. 2015. Human Development Report 2015. Available from http://report.hdr.undp.org/ URBAN, R. W., MUMBA, M., MARTIN, S. D., GLOWICZ, J. & CIPHER, D. J. 2015. Modified Early Warning System as a Predictor for Hospital Admissions and Previous Visits in Emergency Departments. Advanced Emergency Nursing Journal, 37, 281-289. VAN ROOIJEN, C. R., DE RUIJTER, W. & VAN DAM, B. 2013. Evaluation of the threshold value for the Early Warning Score on general wards. The Netherlands Journal Of Medicine, 71, 38-43. WINTERS, B. D., WEAVER, S. J., PFOH, E. R., YANG, T., PHAM, J. C. & DY, S. M. 2013. Rapid-response systems as a patient safety strategy: a systematic review. Ann Intern Med, 158, 417-25. WOOD, S. D., CANDELAND, J. L., DINNING, A., DOW, S., HUNKIN, H., MCHALE, S., MCNEILL, G. & TAYLOR, N. 2015. Our approach to changing the culture of caring for the acutely unwell patient at a large UK teaching hospital: A service improvement focus on Early Warning Scoring tools. Intensive & Critical Care Nursing, 31, 106-115. WORCESTERSHIRE NHS. 2008. Patient at Risk Score (PARS) Clinical Guideline. Available from www.hacw.nhs.uk/EasySiteWeb/GatewayLink.aspx?alId=8521.

153 | P a g e

YODER, J. C., YUEN, T. C., CHURPEK, M. M., ARORA, V. M. & EDELSON, D. P. 2013. A prospective study of nighttime vital sign monitoring frequency and risk of clinical deterioration. JAMA Internal Medicine, 173, 1554-1555. YEALY, D. M., KELLUM, J. A., HUANG, D. T., BARNATO, A. E., WEISSFELD, L. A., PIKE, F., TERNDRUP, T., WANG, H. E., HOU, P. C. & LOVECCHIO, F. 2014. A randomized trial of protocol-based care for early septic shock. The New England Journal of Medicine, 370, 1683-1693. YU, S., LEUNG, S., HEO, M., SOTO, G. J., SHAH, R. T., GUNDA, S. & GONG, M. N. 2014. Comparison of risk prediction scoring systems for ward patients: a retrospective nested case-control study. Critical Care (London, England), 18, R132-R132.

154 | P a g e

Appendix 1: Search Terms - Economic Search

Systematic Literature Search Terms

Clinical Search Terms + Economic Filter

ti/ab 47 "Economics" ti/ab 48 "cost* and benefit*" ti/ab 49 "cost analysis" ti/ab 50 "cost management" ti/ab 51 "cost saving*"

ti/ab 52 "escalation cost*" ti/ab 53 "additional resources" ti/ab 54 "cost effectiveness" ti/ab 55 "education" ti/ab 56 "resources" OR 57 S47 - S56 AND 58 S Clinical Search

Grey Literature Search Terms

Early warning system for health early warning score identification of clinical deterioration track and trigger system health physiological scoring system education economic evidence economic* costs

155 | P a g e

Appendix 2: Tool to Assist Reviewers in Application of Inclusion and Exclusion Criteria

Reference

details

(Author,

year)

Question Yes

No

Not

sure

Comment

here if

excluding

after full text

read

Does the paper focus on one of the key areas

addressed by the research questions?

1. Evaluate the implementation of warning

score/systems or trigger systems

(including escalation protocols,

communication tools and response

approaches) for the detection/timely

identification of physiological

deterioration in terms of the

benefit/harm and level of clinical

validation.

2. Evaluate the implementation of

educational programme for health care

professionals relating to the

implementation of warning

score/systems or trigger systems.

3. Perform an economic evaluation of the

implementation of warning

score/systems or trigger systems.

Is the study/ review pertinent to the acute care

hospital system

Is the sample - adult (non-pregnant) patients

being cared for in acute healthcare settings.

Systematic reviews, meta-analysis, meta-

synthesis, meta-reviews Studies which include

analysis of data prospectively or retrospectively.

The data must either be pre and post adverse

clinical event(s) or pre-post EWS intervention.

However the analysis must help to explicate the

following:

1) Clinical effectiveness (harm/benefit) of Early

Warning Systems

156 | P a g e

2) Clinical validation of Early Warning Systems

In addition studies which evaluate the

effectiveness of educational programmes

preparing health care professionals for the

implementation of Early Warning Systems were

included.

Is the study/review published on or after April

2011?

IF YES TO QUESTIONS ABOVE THEN INCLUDE

157 | P a g e

Appendix 3: Data Extraction Tool

Authors, Date Country Q1. Type of Evidence Review (R) Systematic Review (SR) Meta- Analysis MA i. Aim ii. Dates of review iii. Number and types of

included papers iv. Quality assessment

tools Research study (RS) v. Aim vi. Research

approach/design vii. Details of data

collection viii. Sample details ix. Setting

Q.2: Description(s) of early warning score/ system/intervention Q.3: Features/Components of early warning score/ system including

i. Observation tool and processes

ii. Escalation protocols,

iii. Communication tools

iv. Response approaches

v. Other Q4: EWS system studied in conjunction with other interventions

i. Outreach team/Emergency response team

ii. Observation chart iii. Observation

protocol iv. Educational

programme v. Human factors vi. Other

Q.5: Outcomes assessed & Effects/Impact on outcomes38 What is the impact of using Early Warning Systems on patient health outcomes? i.e. i. Mortality, 30-day mortality, intensive care

unit mortality, unexpected death (without DNR)

ii. Cardiovascular events (cardiac arrest rates, acute coronary syndrome and cardiogenic shock)

iii. Respiratory failure events iv. Measures of sepsis screening,

detection/treatment v. Therapeutic supportive measures e.g. use of

vasopressors, inotropes, delivery of inotropes/ intubation within two hours of ICU admission

vi. Measures of morbidity e.g. Quality of life vii. Measures of early detection, prevention of

deterioration viii. Measures of the period of physiological

instability ix. Measures of patient satisfaction x. Other Q6: What is the impact of using Early Warning Systems on health care professional level outcomes i. Rapid response calls e.g. MET dose ii. Completion of documentation/ recording of

vital signs iii. Other

Q7: What is the impact of using Early Warning Systems on system level factors i. Number of transfers from

wards/ED to intensive care unit, measures of unanticipated admission to intensive care

ii. Number of ventilator days iii. Hospital length of stay iv. Intensive care unit length of

stay v. Safety culture vi. Measures of cost, human

resource use etc. vii. Other Q.8: Details of clinical validation of early warning systems i. Validation of cut-off scores

which correspond with condition severity

ii. Results which further explicate the meaning of the defining characteristics i.e. trigger points of physiological deterioration.

iii. Measures of risk to patient iv. Results which help to

determine the responsiveness (sensitivity, specificity, time to action) of the early warning systems in a clinical setting.

Q.9: Description of Components of early warning score/ system including associated with positive/negative outcomes i. Observation tool and processes ii. Escalation protocols, iii. Communication tools iv. Response approaches v. Other Q.10: Details of implementation solutions

i. Electronic systems, bedside tablets

ii. Other systems Q11. Impact (outcomes assessed & Effects/Impact on outcomes) of implementation solutions (as outlined in Q10) Q.12: Implementation Barriers

i. Patient level ii. HCP level iii. Institutional/

organisational level iv. Policy level

Q.13: Implementation Enablers

i. Patient level ii. HCP level iii. Institutional/

organisational level iv. Policy level

Q14. Level of evidence Q15. Transferability to Irish context Q16 Researcher Comments &/or UCC Authors’ comments

38

Intervention Group=IG; Control Group=CG; Follow up=FUP; S=Statistically significant; NS=Not statistically significant. Findings presented as IG vs CG/FUP unless otherwise stated.

158 | P a g e

Appendix 4a: Tables of Clinical Findings

Table 1. NEWS, ViEWS and MEWS studies Authors, (date) Country Q1. Type of Evidence

Q.2: Description(s) of EWS/intervention Q.3: Features/Components of EWS including Q4: EWS system studied in conjunction with other interventions

Q.5: Outcomes assessed & Effects/Impact Patient at Risk Score on patient health outcomes Q6: Impact of using EWS on health care professional level outcomes

Q7: Impact of using EWS on system level factors Q.8: Details of clinical validation of EWS

Q.9: Components of EWS associated with ±outcomes Q.10: Implementation (Impl.) solutions Q11. Impact of impl. solutions Q.12: Impl. Barriers Q.13: Impl. Enablers


NEWS

Abbott, Vaid, Ip et al. (2015) UK Q1. RS iv. To compare NEWS with an EWS used in the hospital, the (PARS) on the primary outcome – a composite of Critical Care admission or death within 48 h of admission and secondary outcome - hospital length of stay (LOS). v. A prospective, observational cohort study. vi. All adult general medical patients (n=431; n=16 met the primary outcome). The average age was 60.9 (±22.4) years and 46.5% were male. vii. Physiological data and EWSs in bedside charts were collected at admission. viii. 1 hospital

Q2. NEWS developed in conjunction with the Royal College of Physicians of London39,40. PARS was already used within the hospital41 NEWS includes supplemental O2, which is not included in PARS, and weighting of the other parameters differs (see Appendix 4) Q3. i. Each vital sign is graded 0–3 for both PARS and NEWS and added together to give a total score. For PARS, scores of 3 and 5 trigger reviews by the medical team or a CCOT respectively. For NEWS, scores >5 (or 3 in any one parameter) trigger an urgent medical review. A score > 7 triggers a review by a CCOT

Q5. x. NEWS was more strongly associated with critical care admission or death within 48 h than PARS (P=0.056) NEWS ODDS Ratio (OR) 1.54 PARS: OR 1.42 Q6. ii. The electronically calculated PARS score differed from the PAR score recorded in 27% of the patient charts

Q7. iii. Hospital LOS Both NEWS and PARS were poor predictors of hospital LOS

Q14. 2- Q16. Clinical response at a specific EWS score was not investigated, but the authors suggest that patients with a NEWS score >2 are at higher risk of critical care admission or death NEWS is superior to PARS at identifying patients at risk of critical care admission or death within 48 hours.

39

GRIFFITHS, J. R. & KIDNEY, E. M. 2012. Current use of early warning scores in UK emergency departments. Emergency Medicine Journal, 29, 65-66. 40

ROYAL COLLEGE OF PHYSICIANS. 2012. National Early Warning Score (NEWS) Standardising the assessment of acute-illness severity in the NHS. Available from https://www.rcplondon.ac.uk/file/32/download?token=vfwDKQVS 41

WORCESTERSHIRE NHS. 2008. Patient at Risk Score (PARS) Clinical Guideline. Available from www.hacw.nhs.uk/EasySiteWeb/GatewayLink.aspx?alId=8521.

159 | P a g e

Authors, (date) Country Q1. Type of Evidence






or a MET. Alam, Vegting, Houben et al. (2015) The Netherlands Q1. RS v. To explore the performance of the NEWS in an ED with regard to predicting adverse outcomes and to examine the ability of NEWS to predict the need for hospital admission in an ED population. vi. A prospective observational study. vii. NEWS recorded at three time points: on arrival (T0), an hour after arrival (T1) and at transfer to the general ward/ICU (T2). viii. ED patients (n=274) with ESI of 2 and 3 not triaged to the resuscitation room. Median age was 60 years and 49% were male.

Q2 NEWS39,40 Q3. i. The NEW scores were divided into “three aggregates, aggregate 0–4 (low clinical risk), aggregate 5–6 (medium clinical risk) and aggregate 7 or more (high clinical risk)” Q4. vi. Hospital admission, ICU admission, length of stay and 30 day mortality.

Q5. i. 30 day mortality was significantly correlated with NEWS at all measured time points (P<0.05) T0 AUROC = 0.768 (95% CI 0.618, 0.919) T1 AUROC = 0.867 (95% CI 0.769, 0.964) T2 AUROC = 0.767 (95% CI 0.568, 0.916). x. Hospital admissions was significantly correlated with NEWS at all measured time points (P<0.05)

Q7. i. 10 patients admitted to the ICU. ICU admission significantly correlated with NEWS at time points T0, T1 and at T2 (P<0.05). iii LOS was significantly correlated with NEWS, at all measured time points (P<0.05). Median LOS more than doubled for a NEW score >7 compared with a score of 0–4.

Q9. Respiratory rate (RR) was significantly associated with mortality at all measured time points.

Q14. 2+ Q16 NEWS can further risk stratify patients within higher ESI risk categories, for hospital admission, death and need for ICU admission.

Badriyah, Briggs, Meredith et al. (2014) UK Q1. RS iv. To compare two EWS systems; (i) the human generated score NEWS and (ii) DTEWS a score derived algorithmically using decision-tree analysis, to predict 3 outcomes (i) cardiac

Q2. NEWS and DTEWS included the same vital signs (pulse rate, SBP, DBP, breathing rate, temperature, neurological status (AVPU) scale or GCS, SpO2, and record of the inspired gas at SpO2 measurement, date and time of recording. Weighting ranges of DTEWS

Q5 Outcomes assessed Within 24 hours of a given vital sign observation, outcomes of DTEWS and NEWS were similar i. Mortality: NEWS AUROC = 0.894 (95% CI 0.887-0.902) DTEWS AUROC = 0.899 (95% CI 0.982-0.907) ii. Cardiac arrest: NEWS AUROC = 0.722 (95% CI 0.685-0.759) DTEWS AUROC = 0.708 (95% CI 0.669-0.747) x. Any other event:

Q7 i.Unanticipated admission to ICU: NEWS AUROC = 0.857 (95%CI 0.847-0.868) DTEWS AUROC = 0.862 (95CI 0.852-0.872) Q8.i. Decision tree analysis validates the discriminatory function of NEWS to detect the three clinical outcomes measured.

Q8. The detection of ~83% of those who

Q9. Differences in trigger scores between systems was due to differences in weighting attributed to low breathing rates and high systolic blood pressure, because they were infrequent in the dataset used, highlighting the need for clinical insight in the development and interpretation of EWS.

Q14. 3 One comparison study, among medical patients, in one hospital Q15. This study was conducted in a similar health environment to Ireland. Q16. Decision tree analysis is a faster method of deriving a

160 | P a g e







arrest; (ii) unanticipated ICU admission; and (iii) death, within 24 hours of vital sign observation. v. A retrospective analysis vi. A database of vital signs collected (May 2006- June 2008) in a Medical Assessment Unit (MAU) of 1 hospital was used to develop both NEWS and DTEWS. vii. The database included 198,755 vital signs from 35,585 consecutive acute adult (>16 years) admissions viii. One UK hospital MAU

and NEWS differed regarding: Respiration Rate (RR) (lowest; ≤18 and ≤8); SpO2 (highest; 100 and ≥96); any supplemental O2 (score 3 and 2); SBP: lowest ≤89 and ≤90; highest ≥273 and ≥220; number of scores 4 and 5); Pulse Rate (PR: lowest ≤38 and ≤40; highest ≥101 and ≥131; number of scores 5 and 6), respectively, see Appendix 4. Q3. i. Vital signs were collected routinely on PDAs.

NEWS AUROC = 0.873 (95% CI 0.866-0.879) DTEWS AUROC = 0.877 (95% CI 0.870-0.883)

Q6. i. The number of ‘triggers’ were similar for NEWS and DTEWS. However, the ‘trigger point’ at which a response would be initiated was higher for DTEWS than NEWS; 5 and 5, respectively.

will die within 24 hours of a given EWS value requires a response to only 25% of either DTEWS or NEWS values. However, to achieve this, the trigger point for DTEWS must be 5, while that for NEWS must be 4.”

Q10. i. Vital signs were collected routinely on PDAs. Q11. Given the similar efficiencies of the two systems, the resulting workload to detect the same number of outcomes, would be comparable. DTEWS can only reflect on prevalence of vital sign values, but does not consider clinical insight Q13.iii Routine use of Personal PDAs by staff to record vital signs.

EWS the human ‘trial and error’ method and may be employed in the future to develop disease-specific EWS. However, clinical input will be required. DTEWs validated NEWS among medical patients.

Cooksley, Kitlowski & Haji-Michael (2012) UK Q1. RS iv. (i) To assess the effectiveness of MEWS and NEWS in predicting Critical Care Unit (CCU) admission and 30 day mortality in oncology patients; and (ii) to identify the key physiological parameters that predict outcome in this cohort v. Retrospective analysis vi. Data collected by Acute Oncology Nurse Specialists during April 2009 and January 2011 was analysed. MEWS and NEWS calculated.

Q2. 7-item NEWS; Respiratory rate (5 levels, range <8 to >25 breath/min), SpO2 (4 levels, range >96 to <91%), Inspired O2, Temperature (5 levels, range <35 to >390C), Systolic BP (5 levels, range <90 to >220 mmHg), Heart rate (6 levels, range <40 to >130 beats/min), Conscious level (0 Alert, 3 V/P/U). A 7-item MEWS; Respiratory rate (6 levels, range 5-9 to >30 breath/min), Systolic BP (6 levels, range 30-69 to >200 mmHg), Pulse rate (6 levels, range 30-39 to >130 beats/min),

Q5. i. Both EWSs were significant in predicting 30-day

mortality. MEWS score = P=0.004 NEWS score = P=0.0003 Hospital death within 30 days: MEWS score = P≤0.0001 NEWS score = P≤0.0001 Median EWS for patients with 30-day mortality and those alive at 30 days was; MEWS: 5 and 4, respectively. NEWS: 8 and 7, respectively MEWS AUROC = 0.60 NEWS AUROC = 0.62

Key EWS Variables predictive of 30-day mortality Respiratory rate (P=0.0001)

Q7. i. Both EWSs were significant in

predicting CCU admission: MEWS = P=0.037 NEWS = P=0.00046 Median EWS for patients admitted and not admitted to CCU was; MEWS: 5 and 4, respectively. NEWS: 8 and 7, respectively

MEWS AUROC = 0.55 NEWS AUROC = 0.59

Key EWS Variables predictive of CCU admission: Respiratory rate (P=0.0003) Temperature (P=0.033)

Q14. 3 Q15. Track and trigger systems ideally should reflect that some disease groups are more likely to require escalation to CCU. Q16. Both NEWS and MEWS are limited in predicting which patients require CCU admission and 30 day mortality, with the physiological parameters predicting clinical deterioration in this cohort not captured. “The currently used track and trigger systems have poor

161 | P a g e







Control group were patients who discharged without Critical Care admission and alive after 30 days. vii. 840 patients reviewed by the Outreach Team between April 2009 and January 2011 (median age 62.5 (16.4-93.3), 51.2% male). viii. A specialist oncology hospital.

Temperature (6 levels, range 34-34.9 to >390C), Urine output (5 levels, range <10 to >300 mls/hr), AVPU, SpO2 (4 levels, range >96 to <88%).

Temperature (P<0.0001) Other predictive variables Capillary refill time (P=0.003) FiO2 (The Carrico index: ratio of partial pressure arterial oxygen and fraction of inspired oxygen P=0.002) O2 saturation (P<0.0001) Length of time in hospital before trigger (P<0.0001)

Other predictive variables Capillary refill time (P<0.0001) FiO2 (P<0.0001) O2 saturation (P<0.0001) Disease group (P=0.009; patients with haematological cancer were more likely to be admitted than those with lung or gynaecological cancer)

discriminatory value in identifying oncological patients at risk of deterioration. An adapted score more focused upon the key predictive physiological parameters in cancer patients needs to be developed to produce a more effective tool.” (p.1083) However, oncology patients are heterogeneous making this process more difficult.

Capan, Ivy, Rohleder et al. (2015a) USA Q1. RS iv. “To identify optimal patient-centred RRT activation rules using electronic medical records (EMR)-derived Markovian models”. v. Retrospective study with Markov data modelling processes. vi. Infinite-horizon SMDP models were developed to represent the uncertainty in a patient’s health (as measured by NEWS) progression and identify optimal patient-specific, NEWS-based RRT triggers. ”Specifically, for the Markov chain, the patient health states, s ∈ {5, 4, 3, 2, 1, 0}, correspond to NEWS values

Q2. NEW39,40 a total score exceeding 7 suggests RRT activation.

Q7. vi. The total expected resource intensity, including nurse and RRT utilization increases as a patient’s condition deteriorates.

Q9. A highly frail surgical patient (defined using surrogate marker of BSS ≤11 without previous deterioration events would benefit from RRT activation when the NEWS is [1-4] or single extreme value, whereas the threshold is NEWS ≥7 for a moderately frail medical patient. In addition any surgical patient (regardless of BSS at admission) would benefit from RRT activation when the NEWS is [1-4], or with a or single extreme value

Q14. 3 Q16 Limitation: the use of BSS as a surrogate for risk of frailty.

162 | P a g e







{0, 0, [1-4] or a single extreme value, [5-6], ≥7, end of episode}, respectively” pg 109. For the SMDP models, “a decision epoch corresponds to the time period during which a decision is made, i.e., the time a bedside provider team enters a patient’s room during routine hospital rounding. There are two possible actions at each decision epoch: wait, or to initiate RRT” pg 109. vii. n=38,356 adult (≥18 years) general floor medical and surgical patients. During the study period, n= 4833 deterioration events (1941 RRT events, 211 Code45 events, and 2681 unscheduled transfers to the ICU). viii. Mayo Clinic

Capan, Ivy, Wilson et al. (2015b) USA Q1. RS iv. “To develop an SMDP model for the management of a patient’s physiological condition. v. Retrospective study with Markov data modelling processes. vi. A case study of EMR from adult patients in general

Q2. NEWS39,40 The SMDP model allows for stochastically changing health states while determining subpopulation-specific NEWS-based RRT-activation thresholds. The objective is to minimise the total time associated with patient deterioration and stabilisation, including the times associated with clinical distress from the providers

Q6 i. Using the SMDP, the optimum policies for RRT activation at different NEWS scores differ for different subpopulations of patients. RRT activation was optimal for patients in ‘slightly concerning’ or worse health states i.e. NEWS>0 for all subpopulations*, except surgical patients with low risk of deterioration for whom RRT was activated in ‘concerning’ states i.e. NEWS>4. Therefore, two critical sub-populations were identified; (a) for surgical patients with low ROD

Q7. vi. Resource time increases as the patient’s condition deteriorates for all subpopulations; but time to stabilisation and resource time differ depending on the subpopulation.

Q13. Standardized and structured communication between members of the health care providers helps mitigate against communication errors.

Q14. 3 Q16. “EWS-based dynamic and stochastic model can aid data-driven clinical decision making by enhancing the ability to capture changes in patient condition over time I a patient-centred manner.”

163 | P a g e







wards (n=55,385) from Jan 2011 to Dec 2012. Patients were categorised as low, moderate or high risk of deterioration (ROD) during hospitalisation based on the Braden skin score and medical or surgical admission. The stochastic process starts at the beginning of a hospitalised episode, and a decision epioc is a time when a patient’s condition changes. At these epiocs, care providers can act, and the time between decision epiocs includes Time to Stabilisation and FTR. Nurse time is incorporated. vii. 6 patient subpopulations defined by ROD. viii. One hospital

(decision maker’s) perspective, nursing activities, RRT activities.”

who may be healthier and able to recover from deterioration without RRS intervention, and therefore should only activate an RRS at NEWS>4; and (b) escalation should be considered at NEWS>0 for all subpopulations. Comparing the average time to activate RRS on this dataset over the 2 years with the hypothetical time calculated by the SMDP for medical patients with moderate ROD, then the optimal RRT activation was 5.2 hours earlier with the SMDP model. Providers with conservative resource estimates preferred waiting over activating RRT. *The subpopulations investigated were; A: Medical patients with low ROD B: Surgical patients with low ROD C: Medical patients with moderate ROD D: Surgical patients with moderate ROD E: Medical patients with high ROD F: Surgical patients with high ROD

Corfield, Lees, Zealley et al. (2014) UK (Scotland) Q1.RS Iv. To determine whether a single NEWS on ED arrival is a predictor of in-hospital death within 30 days or ICU admission within 2 days, in patients with sepsis. v. Prospective, National audit. vi. Adult patients (≥16 years)

Q2. 6 item NEWS39,40 (plus supplemental O2 (SpO2)); RR, SpO2, Temperature, SBP, PR, Conscious level (AVPU). SpO2 (yes [2]/no [0])

Q5. i. Each rise in NEWS category was associated with an increased risk of mortality when compared to the lowest category For 30-day mortality, the age-adjusted ORs for NEWS categories compared to the baseline category (≤4) 5–6: OR 1.95, 95% CI 1.21 to 3.14 (p=0.01) 7–8: OR 2.26, 95% CI 1.42 to 3.61 (p<0.00) 9–20: OR 5.64, 95% CI 3.70 to 8.60 (p<0.00) AUROC NEWS: 0.71 AUROC age-adjusted NEWS: 0.70

Q7. i. For ICU admission within 2 days the age-adjusted ORs for NEWS categories compared to the baseline category (≤4) 5–6: OR 1.22, 95% CI 0.59 to 2.54 (p=0.59) 7–8: OR 2.01, 95% CI 1.02 to 3.97 (p=0.04) 9–20: OR 5.76, 95% CI 3.22 to 10.31 (p<0.01) AUROC NEWS: 0.67 AUROC age-adjusted NEWS: 0.61

Q14. 2- (i) Missing physiological data for (n=486 patients) prevented the calculation of NEWS. (ii) No data was available for patients who were discharged within 2 days of admission to the ED. (iii) data on ICU admissions after 2 days was unavailable. (iv) only in-hospital mortality was assessed. (v) generalisability to other serious conditions is unknown

164 | P a g e







identified from ED/admission records over 3-months (Mar-May 2009). vii. A national cohort of all adult patients who were admitted for at least 2 days or who died within 2 days were screened for sepsis criteria. Patients with systemic inflammatory response syndrome criteria were included. Complete information was available for 2,003 patients (47% male; median age 72 years). viii. 20 of the 25 EDs nationally participated.

x. Combined outcome (ICU admission and/or mortality). Age-adjusted ORs for NEWS categories compared to the baseline category (≤4) 5–6: OR 1.72, 95% CI 1.14 to 2.60 (p=0.01) 7–8: OR 2.17, 95% CI 1.45 to 3.25 (p<0.00) 9–20: OR 5.78, 95% CI 4.02 to 8.31 (p<0.00) AUROC NEWS: 0.73 AUROC age-adjusted NEWS: 0.70

Q8. iv. Combined outcome: patients admitted to the ICU within 2 days and/or died within 30 days. NEWS of ≥7: Sensitivity = 72% Specificity = 54% PPV =27% NPPV = 90% NEWS of ≥9: Sensitivity = 52% Specificity = 77% PPV = 35% NPPV = 88%

Q15. At a NEWS of ≥7 “an argument can be made for mandating senior ED clinical review for all these patients. There could also be an argument for mandatory review by a critical care outreach team, regardless of ultimate destination.” (p.486) Q16. “An increased NEWS on arrival at ED is associated with higher odds of adverse outcome among patients with sepsis. The use of NEWS could facilitate patient pathways to ensure triage to a high acuity area of the ED and senior clinician involvement at an early stage.” (p.482)

Eccles, Subbe, Hancock et al. (2014) UK Q1. RS iv. To (i) investigate whether patients with conditions associated with chronic hypoxia have consistently high NEWS scores associated with excessive triggers, and (ii) design CREWS to improve specificity for patients with chronic hypoxia. v. A prospective, observational cohort study, with a retrospective

Q2. NEWS39,40 and CREWS42 Q3. NEWS and CREWS differ regarding the scores given to O2 saturation; NEWS: SpO2 (%) 3 ≤91 2 92-93 1 94-95 0 ≥96 CREWS: SpO2 (%) 3 ≤85 2 86-87 1 88-89

Q5. i. 30-day mortality for chronic hypoxia patients NEWS AUROC during stability/at discharge = 0.876 (95%CI 0.788, 0.963) CREWS AUROC during stability/at discharge = 0.913 (95%CI 0.845, 0.981) 30-day mortality for all patients NEWS AUROC during stability/at discharge = 0.829 (95%CI 0.703, 0.955) 30-day mortality for patients without chronic hypoxia NEWS AUROC during stability/at discharge = 0.747 (95%CI 0.516, 0.977)

Q8. Low numbers of deaths in the study (n=23; 11%) prohibited the calculation of relative sensitivities between the scoring systems, however, the authors conclude that their results do not suggest a major decrease in the sensitivity of CREWS compared to NEWS.

Q13. ii. “CREWS may act as a prompt for oxygen prescription and could potentially improve prescription rates and oxygen administration.” (p.111) iii. “The unidirectional scoring of oxygen saturations in CREWS is consistent with NEWS and would allow implementation with minimal alteration to existing documentation and charts.” (p.111)

Q14. 2- Small sample size. Q16. NEWS lacks specificity for patients with chronic hypoxia. “CREWS is a simple variant of NEWS for patients with chronic hypoxaemia that could reduce unnecessary triggers and alarm fatigue, whilst still identifying the sickest patients.” (p.111)

165 | P a g e







application of CREWS to the database for comparison with NEWS.1,2 vi. Prospective data collection of NEWS components recorded and the frequency of thresholds reached from patients’ hospital notes, prescription and observation charts, from during Aug to Oct 2012. NEWS data was collected on admission and during periods of stability prior to discharge. 30-day mortality data was collected 6 weeks after initial data collection. vii. 196 admissions of whom 78 had chronic hypoxia (O2 saturation 88-92%). Mean age was 70 (IQR 19-102) years, 50% were male, 46% had COPD. viii. Respiratory wards (which care for respiratory and general medical patients) in two hospitals in North Wales.

0 ≥90

iii. Mean NEWS score was consistently higher for patients with chronic hypoxia (6 ± 3) than patients without chronic hypoxia at every point measured; On admission: chronic hypoxia: NEWS = 6 ± 3 no chronic hypoxia: NEWS = 3 ± 3 Mean peak NEWS during admission: chronic hypoxia: NEWS = 9 ± 3 no chronic hypoxia: NEWS = 6 ± 3 During stability/at discharge: chronic hypoxia: NEWS = 5 ± 2 no chronic hypoxia: NEWS = 2 ± 2 Q6. The percentage of patients with chronic hypoxia reaching triggering thresholds with NEWS, was higher than the percentage using CREWS at two thresholds (≥5 and ≥6) when patients were stable; Patients scoring ≥5 NEWS (all) = 28% NEWS (no chronic hypoxia) = 14% NEWS (chronic hypoxia) =50% CREWS (chronic hypoxia) = 18% Patients scoring ≥6 NEWS (all) = 18% NEWS (no chronic hypoxia) = 8% NEWS (chronic hypoxia) =32% CREWS (chronic hypoxia) = 14%

Jarvis, Kovacs, Briggs et al. (2015a) UK Q1. RS iv. To (i) determine the risk of serious events within 24

Q2. NEWS16.17

Q4. Escalation of care was recommended at an aggregate NEWS score of ≥6 only.

Q5. For all outcomes an aggregate NEWS score of 5 was associated with significantly higher risk that an aggregate score of 3 (with one vital sign of 3); risk of death and any adverse outcome was significantly higher for a NEWS score of 5 than an aggregate score of 4 or 3 (with one vital sign of 3).

Q7. i. Odds of unanticipated ICU transfer increased (doubled) with each increase of 1 point in the aggregate NEWS scores. Where a single vital sign had a score of 3, the odds increased, but not

Q9. v. “A score of 3 for a single vital sign in NEWS is too low by itself to indicate imminent risk of adverse effect (with the exception of temperature

Q14. 3 One centre, retrospective design. Staff were encouraged to escalate care if ‘worried’, and

166 | P a g e







hours of vital sign measurements, at an aggregate NEWS score of 3, 4, and 5, and at each aggregate score but with a single vital sign score of 3, and (ii) to compare workload at each aggregate score as a result of escalation protocol escalation. v. A retrospective study vi. An electronic database of all adult (≥16 years) patients admitted at or post 21st April 2010 and discharged on or before 23 May 2012, who had at least one overnight stay. vii. 942,887 complete observation sets from 45,678 episodes of care (mean 20.6 observation sets per episode) were collated. 10,000 observation sets were randomly for analysis to minimise any biases due to sicker patients having more frequent observation sets. vii. 64,285 episodes of care of adult patients (≥16 years); mean age 61.8 (SD 20.4) years and 48% male. 2.6% of clinical episodes ended in death. viii. One hospital

An aggregate NEWS of 4 (no vital sign of 3) was associated with higher risk for all outcomes than an aggregate score of 3 (with one vital sign of 3) (P>0.05). i. Odds of death increased (almost doubled) with each increase of 1 point in the aggregate NEWS scores. Where a single vital sign had a score of 3, the odds increased, but not significantly.

NEWS 5 OR =1.00 (0.72, 1.29) NEWS 3 (with a component=3) OR=0.26 (95%CI 0.12, 0.42) NEWS 4 (with a component=3) OR=0.53 (95%CI 0.25, 0.85) NEWS 3 (no component=3) OR=0.20 (95%CI 0.12, 0.28) NEWS 4 (no component=3) OR=0.38 (95%CI 0.22, 0.56). ii. Odds of cardiac arrest increased (approx.. doubled) with each increase of 1 point in the aggregate NEWS scores. Where a single vital sign had a score of 3 within an aggregate score, the odds increased, but not significantly. An individual score of 3 for low temperature (≤350C) was the only single vital sign that significantly increased risk of cardiac arrest above that of an aggregate score of 5. But this is rare therefore loss of consciousness as a single vital sign is a better measure of risk; however risk was not significantly higher that an aggregate score of 5. NEWS 5 OR =1.00 (0.59, 1.44) NEWS 3 (with a component=3) OR=0.24 (95%CI 0.00, 0.55) NEWS 4 (with a component=3) OR=0.66 (95%CI

significantly; NEWS 5 OR =1.00 (0.55, 1.49) NEWS 3 (with a component=3) OR=0.23 (95%CI 0.00, 0.52) NEWS 4 (with a component=3) OR=0.46 (95%CI 0.00, 0.99) NEWS 3 (no component=3) OR=0.22 (95%CI 0.09, 0.38) NEWS 4 (no component=3) OR=0.45 (95%CI 0.13, 0.80)

≤350C). An alternative protocol would be to increase frequency of observation within these patients, but not to escalate only on one vital sign score of 3. This may also apply to other EWS systems. Q13. iii. Vital signs were recorded with handheld electronic equipment and VitalPAC software.

not exclusively based on the NEWS score. This in turn may have led to more interventions for patients with single vital signs of 3. “It is easier to detect and select the correct therapy for a patient with overt single organ derangement than one with multiple subtle physiological abnormalities, and it is not possible to quantify these interventions in the data.” Q16. Within aggregate NEWS scores of 3 or 4 a single vital sign score of 3 increases risk of adverse patient outcome, above the same aggregate scores without a single vital sign achieving a score of 3; but not statistically significantly, and not above an aggregate score of 5. Escalating of care at a score of 3 or 4 (and a vital sign of 3) (the recommended NEWS escalation guideline) increases workload for the bedside nurse and responding doctor, with a modest increase in detection of adverse effects and benefit to patients.” The RCPL guidance should be reviewed. “There may be a case for defining extreme values for each vital sign at which escalation is required,

167 | P a g e







0.17, 1.26) NEWS 3 (no component=3) OR=0.21 (95%CI 0.07, 0.36). NEWS 4 (no component=3) OR=0.43 (95%CI 0.14, 0.74) x. Odds of any adverse outcome increased (approx.. doubled) with each increase of 1 point in the aggregate NEWS scores. Where a single vital sign had a score of 3, the odds increased, but not significantly; NEWS 5 OR =1.00 (0.79, 1.22) NEWS 3 (with a component=3) OR=0.25 (95%CI 0.14, 0.37) NEWS 4 (with a component=3) OR=0.54 (95%CI 0.32, 0.79) NEWS 3 (no component=3) OR=0.20 (95%CI 0.14, 0.27) NEWS 4 (no component=3) OR=0.41 (95%CI 0.27, 0.55) Q6. i. “Escalation care to a doctor when any component of NEWS scores 3 compared to when aggregate NEWS values ≥5, would have increased doctors workload by 40% with only a small increase in detected adverse outcomes from 2.99 to 3.08 per day (a 3% improvement in detection).”

irrespective of the aggregate NEWS score, but they should be more severely deranged that those currently scored 3 in NEWS”.

Keep, Messmer, Sladden et al. (2015) UK iv. To investigate the relationship between NEWS and the diagnosis of septic shock (SS) in the ED. v. A retrospective, observational study.

Q2. NEWS16,17 Q5. iv. The prevalence of sepsis was 9.8% and 5.4% (n=27) has SS An aggregate NEWS score of ≥3 performed best for the identification of patients with SS. For the identification of a patient at risk of SS; NEWS AUROC = 0.89 (95%CI 0.84, 0.94)

Q8. NEWS ≥3 for SS detection had relatively good sensitivity with the least detriment to specificity; NEWS ≥3 Sensitivity=92.6% Specificity=77% PPV=18.7%

Q9. v. This study did not investigate the timing of antibiotics to patients with SS to investigate whether early detection influenced patient outcome.

Q14. 3 Single centre. Low sample size. Retrospective review of data, therefore there is a possibility of incorrect data entry. Q16. Prevalence of SS in the ED is relatively low. NEWS

168 | P a g e







vi. Data was extracted from the ED medical records system and their medical records were reviewed. Baseline characteristics, observation parameters, blood results (international normalised ratio, white blood cells, platelets, creatine, bilirubin and lactate) were collated. Systemic inflammatory syndrome (SIRA), sepsis and SS were defined according to the Surviving Sepsis Campaign guidelines. NEWS was calculated from the initial ED observations. Patients were excluded if they did not have the parameters recorded to calculate NEWS and if they had sepsis but SS could not be determined. vii. 500 consecutive, non-trauma, adult (≥16 years) patients, presenting with Manchester Triage System (MTS) category 1-3, during 21st and 26th July 2013. Median age was 47 (IQR 31-68) years and 48.2% were male. viii. The ED in one hospital (Kings College Hospital)

NPV=99.5% NEWS ≥4 Sensitivity=74.1% Specificity=86.5% PPV=23.8% NPV=98.3%

may be an important intermediate step in the detection of SS in the ED, since screening all patients with the MTS would be time consuming in an environment with a low prevalence of SS. “A NEWS ≥3 at ED triage may be the trigger to systematically screen for septic shock, obtain an early serum lactate and where appropriate start fluid resuscitation and antibiotic therapy.” (p4)

Kolic, Crane, McCartney et al. (2015) UK Q1. RS

Q2. NEWS16,17

Q4. i. Clinical response adequacy

Q5. i. Of the patients who had an adequate clinical response, 8 died (6%) while 6 patients (8.5%) of patients who received an inadequate response

Q13. iii. Increasing diagnostic services out of hours may improve out of hours care.

Q14. 2+ One centre, small sample size, a snapshot of time, in acute medical wards only.

169 | P a g e







iv. “To (i) assess the scoring accuracy and the adequacy of the prescribed clinical responses to NEWS and (ii) assess whether responses were affected by time of day, day of the week and score severity.” v. A prospective, observation al study vi. Data for the first 24 hours of admission during 1st to 15th Oct and 9th to 22nd Dec 2013 was collected prospectively. Baseline characteristics, scores for each vital sign, aggregate NEWS scores, recalculated scores (manually calculated from vital signs recorded on charts) and time to subsequent observation and day and time of day were collated vii. 370 adult patients admitted to an acute medical ward for at least 24 hours. Median age was 77 (range 18-102) years, 50.5% and 49.5% were admitted during the day and night, respectively; 79.7% and 20.3% were admitted during the week, and weekend, respectively. viii. Two acute medical wards in one hospital (The Queen Elizabeth Hospital).

was assessed against the RCP “Standardising the assessment of acute-illness severity in the NHS”2 NEWS score 0 (Category 1) – minimum record observations every 12 hours. NEWS score 1-4 (Category 2) – minimum record observations every 4-6 hours and inform the registered nurse. NEWS score 5 or 6 or one vital sign with a score of 3, (Category 3) – minimum record observations every hour and the registered nurse to immediately inform the medical team, urgent assessments by the clinical responder, and clinical care with monitoring facilities. NEWS score ≥7 (Category 4) – continuous monitoring of observations. The registered nurse to immediately inform the Specialist Registrar or more senior physician, emergency assessment by the critical care clinical team with advanced airway skills and consideration of transfer to a level 2 or 3 clinical environment.

died, giving a relative risk of mortality of 1.35 (95%CI 0.49, 3.74) (Complete outcomes data was only available for 199 of the patient cohort). Q6. i. 74.1% had an appropriate clinical response, 25.9% had an inappropriate response. This did not change significantly with time of day (correct response during day and night were 75.9% and 72.1%, respectively; P=0.404), Patients admitted at the weekend had a worse clinical response (correct response during weekday and weekend were 79.3% and 53.3%, respectively; P<0.0001).; adjusted OR 4.15 (95%CI 2.24, 7.69) The clinical response worsened with increasing NEWS score. Adjusted ORs and P values are compared to baseline NEWS score of 0); NEWS 0; % correct response=92% NEWS 1-4; % correct response =67.5% (P<0.0001) adjusted OR 6.13 (95%CI 3,08, 12.16) NEWS 5-6; % correct response =0% (P<0.0001) adjusted OR 177 (95%CI 20.72, 1510) NEWS 7; % correct response =25% (P<0.0001) adjusted OR 40.64 (95%CI 7.04, 234.7) ii. 18.9% of patients had a incorrectly calculated NEWS score. There was no significant difference in scoring accuracy between time of day (correct score during day and night were 79.7% and 82.5%, respectively; P=0.487), nor between weekday and weekend (correct score during weekday and weekend were 80.3% and 84.0%, respectively; P=0.471). But accuracy decreased significantly with increasing score or worsening physiological derangement. P values are compared to baseline NEWS score of 0);

The authors recommend the use of automated observation calculation across the UK.

Q16. There is a high rate of NEWS scores which are incorrectly calculated, and this has consequences for the adequacy of the clinical response. Clinical response to NEWS scores is significantly worse at weekends, which has implications for standards of care for patients out of hours. There is a trend towards increased mortality for patients who received an incorrect response to a NEWS score. The authors agree with the ‘Seven Day Consultant Present Care” proposed by the Academy of Medical Royal Colleges

170 | P a g e







NEWS 0; % correct score=87.9% NEWS 1-4; % correct score=78.0% (P=0.018) NEWS 5-6; % correct score=69.2% (P=0.072) NEWS 7; % correct score=50% (P=0.008)

Lobo, Lynch & Casserly (2015) Ireland Q1. RS iv. To determine whether a NEWS score of ≥7 in medical patients resulted in a change in clinical management, as a measure of clinical relevance. Also if a modification of NEWS for patients in respiratory failure known as CREWS would have resulted in reduced alarm triggers but no change in patients safety v. A retrospective review of the medical chart of consecutive patients admitted during 1st April 2012 and 14th June 2012. Cross-sectional audit. vii. Medical admissions with one or more NEWS score ≥7 during hospitalisation (n=87).

Q2. NEWS43 and CREW42 Q3. i. NEWS was introduced in March 2012 with ViEWS track and trigger parameters. 43 NEWS score ≥7 triggers ‘‘immediate review by the team registrar or registrar on-call, in addition to informing the team consultant or consultant on-call’’ and ‘‘continuous patient monitoring’’, activating the EMS and planning transfer of the patient to a higher level of care” (p893).

Q5. i. 64.6 % (n=51) of medical patients with a NEWS score ≥7 did not have a change in clinical management; 7.1% (n=2) and 3.9% (n=2) of patients with a change in clinical management or not, respectively, died v.78.6% (n=22) and 94.1 % (n=48) of patients, with a change in clinical management or not, respectively, were discharged home. 35 (94.6 %) of patients with chronic hypoxaemic conditions, were discharged home. x. Change in clinical management among patients with a NEWS ≥7 varied with initial diagnosis; 78.6 % cases with exacerbation of COPD, 80 % cases with exacerbation of CCF and 87.5 % cases with atrial fibrillation had no change in their clinical management. Q6 i. Retrospective application of the CREWS in

Q7. i. 14.3% (n=4) and 2% (n=1) of patients with a change in clinical management or not, respectively, were transferred to an ICU/HDU department Q8. i. Retrospectively applying a CREWS score decreased the number of patients with a trigger score b. NEWS ≥7: n=35 CREWS ≥7: n=11 A NEWS score ≥7 had a PPV of 35.4 % in predicting a change in clinical management. The PPV was higher for patients presenting with atrial fibrillation, 87.5 %.

Q9. i. The two parameters with individual scores of 3 contributing to an aggregate score of ≥7 were O2 supplementation (89.9% of cases) and O2 saturation (31.6 % of cases). 19.7 % of patients were receiving home oxygen therapy which would also give them a score of 3. Patients with chronic hypoxia conditions frequently have scores of 3, even though their condition is stable. Therefore, applying this NEWS trigger score would lead to persistent triggering in these patients.

Q14. 3 Single centre, retrospective study, relatively small sample size, limited generalisability Q16. NEWS was not specific in identifying deterioration in chronic hypoxemic patients primarily because of its respiratory variables. The majority of general medical patients with NEWS scores ≥7 did not have a change in clinical management and were discharged home, it should be noted that a large proportion of patients in this study had underlying chronic disease. This suggests that patients identified at this NEWS trigger score were not acutely deteriorating. The CREWS score may be superior in identifying acutely ill from

42 ECCLES, S. R., SUBBE, C., HANCOCK, D. & THOMSON, N. 2014. CREWS: Improving specificity whilst maintaining sensitivity of the National Early Warning Score in patients

with chronic hypoxaemia. Resuscitation, 85, 109-111. 43

HEALTH SERVICE EXECUTIVE. 2012. National Early Warning Score Observation Sheet. Available from http://www.hse.ie/eng/about/Who/clinical/ natclinprog/acutemedicineprog/earlywarningscore/observation

171 | P a g e







Data on date and time of NEWS score ≥7 and changes in management were extracted. A retrospective CREW score was applied to patients with chronic hypoxaemia (respiratory conditions with a target oxygen saturation of 88–92 %) and a NEWS score ≥7 to determine if it made a difference to the trigger threshold. viii. One acute Model 2 hospital (St. John’s Hospital) with an A&E department from 9 AM to 6 PM 5 days per week and no ICU or HDU. Patients are transferred if they require a higher level of care

patients with chronic hypoxaemic with a NEWS score ≥7 would have reduced the number of reviews by 70.3 %.

chronically ill patients with respiratory disease.

Lydon, Byrne, Offiah et al. (2015) Ireland Q1. RS. iv. “To examine the perceptions of a national Physiological Track and Trigger System (PTTS) among nurses and doctors and to identify variables that impact on intention to comply with protocols.” (p1) v. A mixed-methods study. vi. Semi-structured qualitative interviews based on the theory of planned

Q2. NEWS43

Q3.i. While implementation of NEWS is intended to be standardised across hospitals nationally, specific hospitals have made modifications to the forms.43

iii. ISBAR Q4 iv. Doctors and nurses received formal non-mandatory training in NEWS generally as independent professional groups. An e-

Q5 x. While there was a positive attitude towards PTTS, barriers to its implementation were identified. It sometimes caused tension between doctors and nurses. Nurses had a more positive attitude than doctors and perceived less barriers than doctors.

Q9. The TPB explained only 44% of the variance in the intention to comply with NEWS. Thus, the relationship between intention and behaviours may be more complex than captured by the TPB. Q12. A number of barriers to the implementation of NEWS were identified; ii. (a) The impact of false positives; EWSs have low sensitivity and there is insufficient staffing levels,

Q14. 2- Two centres. A low response rate to the qualitative survey (24%). A wide variety of health professionals were included. Validation of data using a mixed methods design. Q16. Barriers to the implementation of PTTS were identified, which may affect non-compliance with PTTS more so than a negative perception of PTTS by clinical

172 | P a g e







behaviour (TPB) followed by a 27-item web-based quantitative survey developed based on findings from the qualitative interviews. vii. Nurses and doctors participated in the interviews (n=30; interns (n=18; 10 males, 8 females), senior non-consultant doctors (n=2; both male) and nurses (n=10; all female)). 215 nurses and doctors participated in the survey; nurses (n=80), interns (n=29), consultants (n=31), and unknown position (n=17)). viii. Qualitative interviews were performed in one teaching hospital, written surveys were conducted in this hospitals and one other teaching hospital,

learning programme is also available.

especially at night, therefore interns especially encounter many false alarms. High false alarm rates are exasperated by inappropriate use of escalation protocols by nurses and patients with abnormal baseline vital signs not having their NEWS parameters altered by senior doctors. (b). The belief that use of NEWS leads to a failure to use clinical judgement – this was especially a worry among nurses. (c) PTTS had a negative impact on interns’ perceptions of intern-nurse teamwork, possibly due to separate NEWS training and lack of clarity re roles and responsibilities in the RRS. (d) a lack of support from senior doctors, who have a low engagement with NEWS and poor attendance at education and training sessions. Q13. ii. The PTTS had a positive impact on the inter-professional relationships between nurses and senior non-consultant doctors iii. A mechanism to encourage nurses to use their

staff. These barriers are related to sociocultural aspects of introducing a new system into current practice. These sociocultural issues must be addressed in order to improve detection of the clinical deterioration of patients.

173 | P a g e







clinical judgement in their perception of clinical deterioration instead of just a ‘blind adherence’ to the NEWS is required.(p5). Multidisciplinary PTTS training may foster team working and clarify roles, responsibilities and workloads of members and increase support for PTTS implementation by senior doctors, thus improving adherence to protocol.

Petersen, Mackel, Antonsen et al. (2014) Denmark Q1. RS iv. To evaluate the performance of an early warning score system by reviewing all SAEs in a hospital over a 6-month time period. v. An observational study using prospectively collected data relating to adverse events. vi 144 events relating to unexpected death (UD), cardiac arrest (CA) and unanticipated intensive care unit admission(UICU) were used for data analysis. Data collected over a 6-month study period in 2013. vii. Included adult (age ≥ 16)

Q2. An aggregated weighted track and trigger system based on NEWS. Q4.i. An escalation protocol that directs the clinical response was introduced as part of the system. In every patient with a score ≥2 staff must assess airway patency, breathing, and circulation and intervene appropriately according to a pre-defined algorithm.

Q6 i. Nurses escalated care and contacted physicians for events with EWS ≥ 2 and EWS ≥ 3 in 64% and 60% of events of UICU and 58% and 55% for CA events. On call physicians provided adequate care (defined as attended the patient immediately and implemented an appropriate treatment) in 49% of cases of UICU and 29% of cases of the CO when EWS exceeded 5 points. Senior staff was involved according to protocol (with EWS ≥ 9), in 53% and 36% of cases of UICU and CA, respectively. Pg 1699. ii. At least two full sets of EWS were recorded in 87%, 94% and 75% of UICU, CA and UD respectively in the 24 h preceding the event. Patients were monitored according to the escalation protocol in 13%, 31% and 13% of UICU, CA and UD respectively. However higher EWS was significantly associated with a lower probability of being monitored according to the escalation protocol, with the frequency falling from 83% for patients with EWS ≤ 1 to 6% for patients with EWS ≥ 9.

Q7. Poor compliance with the escalation protocol was commonly found when SAEs occurred but level of care provided by physicians was also a problem in a hospital with implemented early warning system pg. 1699. Only in 12 events (8%) was the escalation protocol strictly adhered to; five of these had an EWS < 2 in the 24 h prior to the event. In 132 events (92%) non-adherence to the escalation protocol at one or several levels was noted.

Q 12. Proposed barriers to the use of EWS system (ii, iii taken from discussion , not studied) include i. Higher EWS were

significantly less likely to be monitored according to protocol (in the study),

ii. Failure to recognize deterioration and respond appropriately, lack of education, a feeling among staff that the situation was under control in the ward setting.

iii. Communicative or organizational reasons, inadequate implementation of the RRS, no critical care beds available

Q 14. 2- Limitations of study the low number of events, single-centre setting which makes it difficult to transfer the results to other hospitals, with another case mix of patients. Q16. In a hospital with NEWS implemented poor compliance with protocol and level of care was observed when faced with a deteriorating patient

174 | P a g e







patients admitted to general wards. viii. Departments of surgery and internal medicine, one hospital in Denmark.

Tirkkonen, Olkkola, Huhtala et al. (2014) Finland Q1. RS iv. To evaluate/compare the ability of the hospital’s dichotomised activation criteria and NEWS to discriminate ‘at risk’ patients in hospitalised general ward patients v. Prospective point prevalence trial using prospective data. vi&vii. Data collected on all patients over 18 years in general wards (n=615) using a point prevalence technique by fourth-year medical students between 16:00 and 19:00 h. on 2 dates in September and then in October 2010. viii. Tertiary referral centre in Finland

Q3. Study looked at scoring systems/tools in isolation from afferent limb of a EWS and response team activation protocol.

NEWS- an urgent patient review should occur if the cumulative score ≥5, or if the weighted score for any individual vital sign is 3. If score of ≥7, then MET activation is required.

Dichotomised activation criteria: If one or more of the vital signs met the agreed activation threshold, MET should be activated immediately.

(Prior to study: in 2010, the MET activation frequency was 8.4 calls per 1000 hospital admissions). Note: for the study analysis the ‘worried’ criterion was excluded from the dichotomised criteria and the NEWS criteria.

Q5. i. NEWS score ≥ 5 or if the weighted score for any individual vital sign is 3 was associated with an increased odds of mortality at 30 and 60 days; NEWS score ≥5 30-day mortality: OR 11.8 (95%CI 4.26, 32.6) NEWS score ≥7 30-day mortality: OR 11.4 (95%CI 4.40, 29.6) NEWS score ≥5 60-day mortality OR 5.55 (95%CI 2.91–10.6). NEWS score ≥7 60-day mortality: OR 6.42 (95%CI 2.92, 14.1) x. NEWS score ≥5 or if the weighted score for any individual vital sign is 3 was associated with an increased odds of an adverse event. NEWS score ≥7was associated with an increased odds of adverse effects SAE: I (i.e. one of MET activation, cardiac arrest, emergency ICU admission or death): NEWS score ≥5 OR 14.7 (95%CI 4.32, 50.2). NEWS score ≥7 OR 7.45 (95%CI 2.39, 23.3). SAE: II (ie. cardiac arrest, emergency ICU admission or death): NEWS score ≥5 OR 18.1 (95% CI 4.51, 72.8) NEWS score ≥7 OR 11.5 (95%CI 3.40, 38.6).

Q8i. After adjusting for confounding factors, conventional dichotomised activation criteria were not associated with outcome and discriminated high risk patients poorly. However, NEWS was able to detect high risk ward patients regardless of multiple factors affecting patient outcome. ii. A total of 72 patients (12%) had ‘positive’ dichotomised activation criteria; most common positive criteria was respiratory rate (n=48 patients, 7.8%).

Q14. 2+ Body temperature was not recorded for all patients. Q16. NEWS discriminates high risk patients in a heterogenic general ward population independently of multiple confounding factors. The conventional dichotomised activation criteria were not able to detect high risk patients. (p411)

175 | P a g e







van Rooijen, de Ruijter & van Dam. (2013) The Netherlands Q1. RS iv. To find the “optimal threshold value for EWS on a general ward, and investigate whether it was possible to raise this value from ≥3 without compromising sensitivity” (p 38). v. A prospective observational study. vi. From May 2010 until May 2011, data was collected from all patients in medical and surgical wards. vii. 71,911 EWS values noted (56% on surgical ward, 44% on medical wards) viii. Surgical and medical wards in a Medical Centre, Alkmaar, The Netherlands

Q3. i. Cut-off value was defined as EWS ≥3. Required pre-defined sensitivity (for the study) was defined at 90%. This implies that 90% of all interventions would take place at this cut-off value. In addition, a two-point increase in EWS between two consecutive observations, (possibly indicating Clinical deterioration), was signalled electronically as a trigger. Based on previous EWS scores and after physical examination, an individual’s cut-off value could be re-set if deemed appropriate. ii. If the electronically calculated EWS value exceeded the cut-off value, the program gave a signal to contact the doctor. In the study 6 responses were defined and categorised as interventions (infusion prescription, medication changes, ICU consultation) and other actions (no action, change EWS cut-off value, oxygen supplementation), and interventions were to be registered whenever the EWS threshold was exceeded.

Q5 i. In hospital mortality was 0% for EWSmax=0 and increased almost logarithmically to 1% for EWS=3 and 24% for EWS max ie. ≥ 6. One-year overall mortality was 3%, 12% and 40% for EWSmax=0, EWSmax=3 and EWSmax ≥ 6 respectively; when hospital mortality was excluded this was 3%, 11% and 16% for the respective EWS values.

Q8. Raising EWS cut-off for all patients could lead to unacceptable decreases in sensitivity. By modelling the in the study data- a cut-off value when raised to EWS ≥4 impacted upon the calculated sensitivity changing it to 74%, i.e. below the predefined 90%. Sensitivity decreased even further to 52% if EWS ≥ 5 was used within the model.

Q11. EWS values were calculated automatically from vital signs once they were entered into an electronic patient record.

Q14.2- Q16. Mean EWS values are higher on medical wards (1.4) than on surgical wards (1.2). The cut-off value was reached in 12% (3734) of EWS values registered on medical wards, as opposed to 8% (3279) of all cases on surgical wards. Lower thresholds result in increased workload, at the risk of making staff less cautious.

176 | P a g e







ViEWS Hands, Reid, Meredith et al. (2013) UK Q1. RS iv. To compare the pattern of vital signs and ViEWS data collected from all adult patients admitted to all areas except high care areas including CCUs. Patterns during the 24 h day and different days of the week were investigated. Relationship of observations with the clinical escalation protocol was also investigated. v. A retrospective study vi. Data was extracted (on hourly and daily patterns of vital signs and ViEWS value were extracted for two periods within 24 hours i.e. 08:00-11:59 and 20:00-23:59, with subsequent vital signs recorded in the following 6 h; and time to next observation for vital signs recorded in two periods i.e. 08:00-11:59 and 20:00-23:59) from the hospitals’ vital sign database. Only observation sets with at least one subsequent observation set within 24

Q2. ViEWS51

Q3.i. A commercial system known as VitalPAC which records ViEWS is used throughout the hospital (except in CCU) for routine documentation and charting of vital signs at the bedside, using hand-held PDA devices. This system is integrated into the patient administration system in the hospital. Time of observation is also recorded. The aggregated ViEWS score is automatically calculated. ii. A decisional support is automatically displayed on the PDA following entry of vital sign data and calculation of score. The escalation protocol is based on NICE recommendations44; ViEWS 0-1: 6-12 hourly min interval between observation sets ViEWS 2: 6 hourly min interval between observation sets ViEWS 3-6: 4 hourly min interval between observation

Q6. ii. Overall the proportion of observations with increasing ViEWS scores were; 51.5% (Score 0-1), 18.09% (Score 2), 26.41% (Score 3-6), 2.66% (Score 7-8) and 1.34% (Score ≥9) Identical patterns of documentation were observed on all days studied. Patterns varied with time of day. Documentation at night time was lower than during the day with between 0.93 and 2.87% of vital signs collected hourly between 23:00 and 5:59. This totalled 12.81% of vital signs measured during 23:00 and 5:59 There were large variations in the pattern of documentation, within the 24-hour period with peaks in the morning (6:00-6:69; 13.58%) and evening (21:00-21:59; 8.58%) but little variation in this pattern between days of the week. Patients with higher aggregate ViEWS scores were more likely to have their vital signs measured at night. The percentage of vital signs recorded hourly increased during the day (range 3.35-6.08%) Variation in the percentage of vital signs collected decreased with ViEWS score ≥1. Frequency of observation increased with increasing ViEWS scores, during the day and at night; 23.84% of vital signs measured at night had a ViEWS score ≥9, compared to 10.12-19.97% for other ViEWS values.

Q8. iv. Results show that staff continue to take vital signs according to predetermined patterns that may prefer to use clinical judgment rather than operating by protocol

Q12. ii. The explanation for these findings are not obvious. It is possible that staff prefer to use their clinical judgement and expertise, instead of ViEWS in determining which patients require monitoring. Or the escalation protocol is not achievable given the resources. Another explanation is that vital signs must be fitted between competing patient and clinical activities. Staff may have predetermined hours to make ‘observation rounds’ based on other activities and needs. The peaks may be in preparation for doctors’ rounds. Q13. iii. An integrated ViEWS with handheld PDAs assists vital sign recording and prompting of the escalation protocol.

Q14. 3 Single centre. Large dataset of electronically captured data (thus eliminating possible transcription errors due to data transcription). Q16. There are large variations in observational sampling frequency throughout the day, with identical patterns everyday, and lower frequency at night. Sicker patients have higher frequency of vital sign observations during the day and at night, but less than recommended by the escalation protocol. The pattern of vital sign monitoring is likely to impact on the effectiveness of RRT activation. Therefore, despite clear protocols adherence is low. “There was only partial adherence to the vital signs monitoring protocol. Sicker patients appear more likely to have vital signs measured overnight, but even their observations were often not

44


177 | P a g e







hours was included. vii. 950,043 vital sign datasets were recorded during May 2010 and April 2011. viii. One NHS hospital (Portsmouth Hospitals Trust)

sets ViEWS 7-8: 1 hourly min interval between observation sets ViEWS ≥9: 30min minimum interval between observation sets

Adherence to escalation of monitoring contrasted between day and nighttime; 73.1% of the vital sign observation sets had a subsequent set recorded within 6 hours during 08:00-11:59, compared with 25.32% during 20:00-23:59. This percentage difference was true for all ViEWS scores, including scores ≥9 (08:00-11:59 (86.65%) and 20:00-23:59 (68.78%)). 47.42% of patients with ViEWS=7/8 and 31.22% of patients with a ViEWS ≥9 in the period 20:00-23:59 did not have vital signs recorded in the following 6 hours, indicating that sicker patients with higher VIEWS score got more frequent vital signs but not consistently and less so at night. Time to next observation decreased with increasing ViEWS value, but less than expected by the monitoring protocol. And the Time to next observation was longer at night time for all ViEWS scores 08:00-11:59: ViEWS 3-6 : mean=5.64 h ViEWS 7-8 : mean=4.91 h ViEWS ≥9: mean=4.22 h 20:00-23:59 ViEWS 3-6 : mean=7.88 h ViEWS 7-8 : mean=6.59 h ViEWS ≥9: mean=5.17 h

followed by timely repeat assessments. The observed pattern of monitoring may reflect the impact of competing clinical priorities.”

MEWS Alam, Hobbelink, van Tienhoven et al. (2014)

Q2. The MEWS45 (n=6) and the SEWS46 (n=1) were the

Q5. Results from individual studies are described independently if they are within our search period

Q6. i. A significant decrease in ICU

Q13. The studies that observed a decrease in

Q14. 1- The 7 studies were

45

SUBBE, C. P., KRUGER, M., RUTHERFORD, P. & GEMMEL, L. 2001. Validation of a modified Early Warning Score in medical admissions. QJM, 94, 521-526.

178 | P a g e







The Netherlands Q1. SR iv. “To evaluate the impact of the use of EWS (and its modified forms) with or without an outreach service, on particular patient outcomes: in-hospital mortality, patterns of ICU admission and use, LOS, cardiac arrests and other serious outcomes of adult patients on general wards and in medical admission units (MAUs).” v. All controlled studies which measured the outcomes under investigation, and used a EWS to identify adult (≥16 years) patients at risk vi. Databases PubMed, EMBASE and The Cochrane Library were searched from inception until April 2013. vii. 7 studies encompassing 486,237 patients. These studies were all pre- and post-intervention studies (n= viii. 5 studies were performed in the UK, 1 each from Belgium and Italy.

EWSs evaluated in the included studies. MEWS was developed for use as a bedside evaluation tool for patients in wards and ICUs. Score is calculated from regular assessment of 5 parameters (SPB, pulse rate, RR, temperature and AVPU (A, Alert; V, Verbal; P, Pain; U, Unresponsive) score). Each parameter is given a number from 0 to 3 and a score ≥5 is considered a ‘critical score’. Q4. 3 studies investigated the impact of EWS with a pre-existing and critical care outreach service, 2 studies introduced EWSs and critical care outreach service simultaneously, and 2 focused on the effect EWSs only.

(2011-2015). i. The 6 studies that investigated the effect of the introduction of EWSs on mortality as the endpoint found conflicting results; a significant reduction in in-hospital mortality (n=2); a trend toward decreased mortality (n=2); and no significant differences in in-hospital mortality (n=2) following the introduction of EWSs. ICU mortality was reduced (67% vs 33%), but not significantly (P=0.21; n=1). These studies differed in the study populations investigated. ii. Studies investigating the effect of the introduction of EWSs on cardiopulmonary arrests reported conflicting results (n=2); a decrease in cardiopulmonary arrests (and mortality) among patients who had CPR (n=1), and an increase in cardiopulmonary arrests (n=1) was observed. The latter study contained a more heterogeneous population and a higher number of sick patients compared with the control group. x. SAEs (i.e. number of deaths without an attempt to resuscitation and readmission to ICU within 5 days of ICU discharge) decreased, but not statistically significantly. Q6. ii. Frequency of vital sign documentation increased following EWSs chart implementation (n=2). RR and consciousness level in particular were documented more frequently.

admissions was observed following the introduction of the EWSs in emergency surgical patients (11 to 5%; P=0.0010) (n=1) iii. Differences in the effect of EWSs on LOS was observed. A non-significant trend towards a shorter LOS was observed following the introduction of EWSs (n=1), while in another study median hospital LOS increased significantly for patients admitted to ICU or HDU (n=1). A higher EWSs score was significantly correlated with hospital LOS (n=1) x. A significant increase in HDU admissions was observed following the introduction of the EWSs in emergency surgical patients (14 to 21%; P=0.0008) (n=1)

mortality (n=2) introduced the EWSs chart after an intensive staff training programme. In total three studies emphasised that sufficient training improves patient outcomes.

heterogeneous in EWSs used, presence of a RRT, patient characteristics, sample size and study design. Q16. “Results were mixed but there was a positive trend towards better clinical outcomes following the introduction of the EWS chart, sometimes coupled with an outreach service. The EWS is a simple and easy to use tool at the bedside, which may be of help in recognizing patients with potential for acute deterioration. Coupled with an outreach service, it may be used to timely initiate adequate treatment upon recognition, which may influence the clinical outcomes positively. However, a general conclusion cannot be generated from the lack of use of a single standardised score and the use of different populations.”

Bulut, Cebicci, Sigirli et al. (2014)

Q2. MEWS45 and REMS66is a physiologic scoring system

Q5. i. REMS performed significantly better at predicting

Q7. i. REMS performed significantly better

Q14. 2- Patients with trauma were

46

PATERSON R., MACLEOD, D. C., THETFORD, D., BEATTIE, A., GRAHAM, C., LAM, S. & BELL, D. 2006. Prediction of in-hospital mortality and length of stay using an early warning scoring system: clinical audit. Clinical Medicine, 6, 281-284.

179 | P a g e







Turkey Q1. RS iv. “To compare MEWS and REMS on in-hospital mortality, and as predictor of hospitalisation in general medical and surgical patients admitted to ED.” The primary outcome was the admission of the patient to a ward/an ICU/high dependency unit (HDU) and in-hospital mortality. v. A prospective, multicentre and observational cohort study. vi. a period of 6 months. vii. Prospective collection of 2000 adult patients (≥16 years) in the ED (52% male, mean age 61±19) categorised following triage as ‘red’ (life threatening but treatable injuries requiring rapid medical attention) and ‘yellow ((potentially life-threatening injuries, risk of organ loss, and cases with important rate of morbidity) were included in the study during Oct 2011-April 2012. ‘Green’ (stable/minor injury) patients, those with cardiac arrest and missing information were excluded. viii. General medical and surgical patients admitted to the EDs in 3 education and

designed for the ED based on 5 physiologic parameters (mean arterial pressure, RR, BP, peripheral O2 saturation and GCS score) and age. Each parameter is scored from 0 to 4, except age (0–6 points) to a maximum score of 26.6 Patients are classified as high (>13), intermediate (6–13), and low risk (<6).

In-hospital mortality of patients presenting to ED (P<0.001); REMS AUROC=0.707 (95%CI 0.686-0.727) (P<0.001): MEWS AUROC=AUC 0.630 (95%CI 0.608-0.651) (P<0.001) Mortality risk increased with higher REMS scores; 2.923 (95% CI 0.026 to 4.217) x increased risk in Intermediate (REMS 6– 13) vs low risk (REMS<6) 14.564 (95% CI 4.573 to 46.573) x increased risk in high (REMS>13) vs low risk (REMS<6) x. REMS performed significantly better at predicting hospitalised and discharged patients (P<0.001); REMS AUROC=0.642 (95%CI 0.621-0.663) (P<0.001): MEWS AUROC=AUC 0.568 (95%CI 0.546-0.590) (P<0.001)

predicting admission to ICU/HDU of patients presenting to ED (P<0.001): REMS AUROC=0.589 (95%CI 0.567-0.611) (P<0.001) MEWS AUROC=AUC 0.538 (95%CI 0.516-0.560) (P=0.009)

excluded, limiting generalisability of findings. MEWS and REMS scores were calculated once from parameters recorded upon ED admission. Q16. “The efficiency of REMS was found to be superior to MEWS as a predictor of in-hospital mortality and hospitalisation in medical and surgical patients admitted to ED.”

180 | P a g e







research hospitals

Churpek, Yuen, Winslow, et al. (2015) USA Q1. RS iv. To compare the accuracy of vital signs and MEWS for detecting cardiac arrest in elderly (≥65 years) and non-elderly patients (<65 years). v. Observational, cohort study. vi. Cardiac arrest data was collected prospectively, and vital sign data was collected from electronic health records (Nov 2008-Jan 2013). A MEWS score was calculated for each observation time for each patient. The patients’ first adverse event was included in analysis. vii. 269,999 patient ward admissions, including 424 cardiac arrests; 46% were elderly, and 65% of the cardiac arrests occurred in this age group. 44% and 26% of the elderly and nonelderly patients were male. viii. 5 hospitals (1 urban, 2 suburban, and 2 community, non-teaching hospitals)

Q2. MEWS45 Q3. All hospitals had a nurse-led RRT in place during the study, activated by ‘tachypnea’, ‘tachycardia’, ‘hypotension’ and ‘staff worry’ but specific vital sign thresholds were not specified.

Q5. ii. Elderly patients had a significantly higher cardiac arrest rate (p<0.001); Elderly: 2.2 cardiac arrests/1,000 ward admissions Nonelderly: 1.0 cardiac arrests/1,000 ward admissions Within 4 hours of cardiac arrest all vital signs were significantly different between elderly and non-elderly patients (P<0.001). MEWS score was similar between elderly and nonelderly patients in the whole dataset (median MEWS = 1 (IQR, 1-2). Elderly patients had a significantly lower MEWS score than nonelderly patients 4 hours prior to cardiac arrest (P<0.001); Elderly median MEWS = 2 (IQR, 1-3). Nonelderly median MEWS = 3 (IQR, 2-5). MEWS was significantly more accurate for detecting cardiac arrest in the ward in nonelderly, than elderly patients (P<0.001); Elderly MEWS AUROC: 0.71 (95%CI 0.88, 0.75) Nonelderly MEWS AUROC: 0.85 (95%CI 0.82, 0.88) Older elderly (≥75 years) MEWS AUROC: 0.71 (95%CI 0.66, 0.75) Nonelderly MEWS AUROC: 0.81 (95%CI 0.78, 0.83)

Q7. i. MEWS score was similar between elderly (median MEWS = 2 (IQR, 1-3). and nonelderly (median MEWS = 2 (IQR, 1-4). patients at time of ICU transfer

Q14. 2+ Q16. The accuracy of MEWS to predict cardiac arrest decreases with age. In addition, almost all vital signs more accurately detected cardiac arrest in nonelderly compared to elderly patients. Patient age should be considered when interpreting vital sign data. There is also a need to identify other cardiac arrest predictors (eg. comorbidities, medications) in the elderly to supplement EWSs More accurate methods for risk stratification of elderly patients are necessary to decrease the occurrence of cardiac arrest.

Churpek, Yuen, Huber et al. (2012a)

Q2. MEWS45

Q5. ii. Mean MEWS scores were significantly higher in

Q11. “Although SBP is commonly used in RRT

Q14. 2+ Single centre

181 | P a g e







USA Q1. RS iv.To investigate the effectiveness of MEWS at detecting cardiac arrest in patients on wards. v. A retrospective, longitudinal, nested case control study vi. Cases were consecutive adult patients who experienced a cardiac arrest, between November 2008 and January 2011. Cases were identified using a prospectively collected and verified cardiac arrest quality improvement database. All patients in the same ward at the time (T0) of the case’s cardiac arrest were eligible to be controls. Four controls were selected for each case using a random number generator. vii. 88 patients experienced a cardiac arrest on the ward (cases: mean age 64±16 years and 352 controls (mean age 58±18 18 years; P <0.002). Cases and controls 43% and 49% male, respectively. viii. One academic, tertiary care hospital

A MEWS score was calculated for each patient on admission using the first vital signs recorded and then every 8 h during the 48-h time period prior to T0 using the closest vital signs measured prior to that time point. Q4. An RRT was in place since 2008. It is led by a critical care nurse and respiratory therapist with consultation from a hospitalist physician and/or pharmacist upon request. RRT triggers include “tachypnea,” “tachycardia,” “hypotension,” and “staff worry.” Specific vital sign thresholds or MEWS score are not stated.

cases, with increasing disparity leading up to the event; 24 h prior to cardiac arrest (P<0.001) Cases: 2.3±1.3 Control: 1.5±0.9 48 h prior to cardiac arrest (P<0.005) Control: 1.6±1.0 Cases: 2.1±1.0 In the 48 h preceding cardiac arrest, maximum MEWS was the best predictor: AUROC: 0.77 (95% CI, 0.71-0.82). Other predictors of cardiac arrest: Max RR: AUROC 0.72 (95% CI, 0.65-0.78), Max HR: AUROC 0.68 (95% CI, 0.61-0.74), Max pulse pressure index : AUROC 0.61 (95% CI, 0.54-0.68), Min DBP: AUROC 0.60 (95% CI, 0.53-0.67). Case patients were older (64 ± 16 years vs 58 ± 18 years; P= .002) and more likely to have had a prior ICU admission than control subjects (41% vs 24%; P 5 .001)

activation criteria, incorporation of pulse pressure, pulse pressure index, or DBP in place of SBP into the predictive model may be superior. As pulse pressure is less intuitive than other vital signs and requires a calculation, automated derivation in the electronic medical record may be necessary for this predictor to be most effective.”

Q16. Respiratory rate was the best vital sign predictor of cardiac arrest on the ward. However, the ideal cut-off is unknown, partly because it is often inaccurately measured and poorly documented in hospitalised patients. The MEWS score was significantly different between patients experiencing cardiac arrest and control patients by 48 h prior to the event, however, includes poor predictors of cardiac arrest such as temperature and omits significant predictors such as DBP and pulse pressure index.”

Finlay, Rothman & Smith (2014) USA

Q2. MEWS45 and RI47 RI “is a patient acuity score

Q5. i. RI had superior discrimination of 24 h mortality compared to MEWS

Q8. At a MEWS score of 4, and a corresponding RI score of -16, has similar sensitivity, but RI has twice the

Q9. viii. The superiority performance of RI over

Q14. 3 Single centre study.

182 | P a g e







Q1. RS iv. To compare MEWS with the Rothman Index (RI). v. Retrospective calculation vi. Clinical variable data was extracted from EMR (June 2009-June 2010) vii. MEWS and RI scores were calculated from the EMR of all adult (≥18 years) patients 32,472 patients admitted; with a total of 1,794,910 observations. Of these 617 patients (1.9%) died. viii. 1 regional centre and teaching hospital

based upon summation of excess risk functions that utilize additional data (26 items) from the EMR.47” (p.116) The RI automatically updates when asynchronous vital signs, laboratory test results, Braden Scale, cardiac rhythm and nursing assessments are entered into the EMR. Excess risk is defined as percent absolute increase in 1-year mortality relative to minimum 1-year mortality, for a variable. Excess risk is summed on a linear scale to reflect a patient’s cumulative risk. Laboratory tests are entered when measured, weighting is reduced to 50% after 24 h and they are excluded from the RI after 48 h. RI ranges from -90 to 100, with lower scores indicating increased acuity.

24-hour mortality RI AUROC = 0.93 (95%CI 0.92, 0.93) MEWS AUROC = .82 (95%CI 0.82, 0.33)

likelihood ratio (positive) and reduces false positive alarms by 53%. At a score of 30, RI captures 54% more of the patients who will die within 24 hours: MEWS Score= 4: likelihood ratio (positive)=7.8 likelihood ratio (negative)=0.54 Sensitivity=49.8% Specificity=93.6% PPV=5.2% NPV=99.6% RI Score =-16: likelihood ratio (positive) = 16.9 likelihood ratio (negative)=0.54 Sensitivity=48.9% Specificity=97.1% PPV=10.6% NPV=99.6% RI Score =30: likelihood ratio (positive) = 7.9 likelihood ratio (negative)=0.26 Sensitivity=76.8% Specificity=90.4% PPV=5.3% NPV=99.8%

MEWS may be due to the inclusion of parameters than vital signs. The relative contributions of parameters are; vital signs (35%), nurse assessments (34%), and laboratory test results (31%). Q13. ii. Automatic recalculation of RI the score reduces the burden on healthcare professionals.

Q16. RI outperforms MEWS in identifying mortality within 24 h in hospitalised patients.

Harris P (2013) UK Q1. RS iv. “To describe the use and

Q2. MEWS45 Q3. ii. Response depended on

Q5. i. Mean MEWS score within 24 h prior to cardiac arrest was 2.24. MEWS score 1 = 45% (n=15) of patients

Q13. iii. “Communication tools (e.g. SBAR) may facilitate the referral process by enabling

Q14. 3 Small, non-random sample. One centre.

47

ROTHMAN, M. J., ROTHMAN, S. I. & BEALS, J. 2013. Development and validation of a continuous measure of patient condition using the Electronic Medical Record. Journal of Biomedical Informatics, 46, 837-848.

183 | P a g e







characteristics of the MEWS score in the 24 hours preceding cardiac arrest to establish any pattern and its relationship to the subsequent deterioration of patients.” The objectives were to (i) investigate the frequency with which MEWS score was not calculated in patients with cardiac arrest, (ii) use and characteristics and MEWS and its relationship with patient deterioration and (iii) determine appropriateness of response. v. A retrospective, case note review. vi. Cardiac arrest calls were identified from the hospital switchboard log in a 6 month period. Patient data was abstracted from medical records into a standardised abstraction form. vii. All medical ward in-patients (without a DNAR order) who suffered a cardiac arrest (n=33). Mean age was 78.8 years and 45.5% were male. 87.9% of patients did not survive. viii. One district hospital.

MEWS score; MEWS 1-2: observe and repeat observations and MEWS in 1 h. Treat as necessary MEWS 3-4: Inform senior house officer (day) or clinical site nurse practitioner (night) MEWS >5: Urgent beep to senior house officer/specialist registrar (day) or clinical site nurse practitioner (night) or critical care outreach team.

MEWS score 2 = 21.2% (n=7) of patients MEWS score 3 = 12.1% (n=4) of patients MEWS score 4= 9.1% (n=3) of patients MEWS score 5= 9.1% (n=3) of patients MEWS score 6= 3% (n=1) of patients The majority of cardiac arrests occurred out of hours (69.7%). Q6. i. MEWS scores within 24 hours of the cardiac arrest corresponded to an appropriate response of referral pathways to doctors, but critical care outreach team service is underutilised: MEWS score 1-2 = 10.3% (n=3) of patients had a doctor informed MEWS score 3-4 = 57.2% (n=8) of patients had a doctor informed MEWS score ≥5 = 83.3% (n=5) of patients were referred to a doctor urgently. For 1 patient the critical care outreach team was called. ii. The majority of patients had their MEWS score recorded (n=25; 78.8%) Final observation set prior to cardiac arrest were recorded by staff nurses (n=13; 39.9%) or healthcare assistants (n=20; 60.6%)

nurses to give an accurate, clear and concise telephone referral.” “Implementation advice on how best to use the response strategy is also required – possibly through a joint partnership between the critical care outreach team and the clinical site nurse practitioner on the ward..”

Q16. The majority of observations, including MEWS was recorded by healthcare assistants who may not have the skills of recognising a deteriorating patient, document observations accurately, calculate MEWS correctly or initiate an appropriate response. MEWS score was not above the trigger score of 5 for the majority of patients who suffered a cardiac arrest. Critical care team was underutilised, especially out of hours, despite the presence of a clear response strategy.

Ho, Li, Shahidah et al. (2013) Singapore Q1. RS iv. To validate the use of the

Q2. MEWS45

Q5. i. There were 47 deaths (6.6%) in the MEWS <4 group as compared with 53 (17.0%) deaths in the MEWS ≥4 group (P<0.001)

Q7. i. 267 patients were admitted to HD/ICU (37.4%) in the MEWS <4 group as compared with 86 (27.7%) patients in

Q9. One reason why the MEWS performed poorly in our population may be related to underlying disease

Q14. 3 One centre. Q16. The MEWS score did not

184 | P a g e







MEWS as a predictor of patient mortality and ICU)/ HD admission in an Asian population. v. A retrospective cohort study vi. Socio-demographic and vital sign data was collected during triage in the ED as part of an ongoing study. An aggregate MEWS score was calculated for each patient. 30-day follow up data was obtained from hospital records vii. Critically ill adult patient (≥18 years; n=1,024) attending the ED who required continuous ECG monitoring or had a Patient Acuity Category Scale (PACS) of 1 or 2, during Nov 2006 and Dec 2007. PACS 1 patients are the most critically ill and are attended to immediately. PACS 2 patients are non-ambulant, stable and not at risk of imminent collapse. PACS 3 patients are ambulant and PACS 4 patients are non-emergencies. 713 patients had a MEWS score <4, 311 patients with a MEWS score ≥4 (mean age of 62.3 (15.4) years and 61.4 (18.1) years respectively, with more males than females (ratio of

MEWS Score cut off were poor predictors of mortality MEWS <4 AUROC= 0.68 MEWS <5 AUROC= 0.66

the MEWS ≥4 group. MEWS Score cut offs were poor predictors of ICU/HD transfer MEWS <4 AUROC= 0.49 MEWS <5 AUROC= 0.47 iii. The average LOS for the MEWS <4 group was 6.97 days and for the MEWS ≥4 group was 7.75 days. Q8. i. MEWS <4 and <5 had low sensitivity and specificity for predicting mortality. MEWS <4 Sensitivity=47.0% Specificity=27.9% PPV=6.66% NPV=82.96% MEWS <5 Sensitivity=66.0% Specificity=12.34% PPV=7.53% NPV=77.03% MEWS <4 and <5 had low sensitivity and specificity for predicting ICU/HD transfer. MEWS <4 Sensitivity=74.16% Specificity=33.91% PPV=46.70% NPV=62.70% MEWS <5 Sensitivity=87.75%

condition.

perform well in predicting poor patient outcomes i.e. 30 day mortality or transfer to ICU or HD, for critically ill Asian patients presenting to an ED. Sequential monitoring of MEWS may be more useful than applying specific cut-off values of the MEWS score as a predictor of poor outcome.

185 | P a g e







1.8 in the MEWS <4 group and 1.1 in the MEWS ≥4 group). viii. 1 hospital ED (Singapore General Hospital)

Specificity=16.17% PPV=44.98% NPV=62.84%

Huggan, Akram, Christen et al. (2015) Singapore Q1. RS iv. To explore the usefulness of common assessment tools in predicting outcomes of (i) death or ICU admission and (ii) LOH stay v. A prospective, observational cohort study vi. Data on demographics, diagnosis, comorbid conditions including the CCI, MEWS and modified Barthel Index (MBI; for patients aged ≥65) at presentation to the ED was collected prospectively (within 24 h of admission) by patient record review every day. vii. 398 consecutive admissions prospectively assessed during 25th May to 28th June 2011. Patients were followed until discharge or transfer to ICU/HDU. 16 (4 %) patients died or were transferred to ICU and 99 (25%) stayed for ≥7 days. Of these patients 56% were ≥65 years on admission.

Q2. MEWS45

Q5. x. MEWS aggregate scores ≥5 was significantly associated with death or ICU admission (HR 5.50, 95%CI 1.77, 17.07, P=0.003). MEW scores ≥1 for SBP had a 5-times increased risk of ICU/HDU admission or death (HR 4.78 95%CI 1.74, 13.15, P = 0.002). MEW scores ≥2for RR had a 7.5-times higher risk of ICU/HDU admission or death (HR 7.54 95%CI 2.74, 20.77, P<0.001). There was no independent association between this outcome and the CCI or admission MBI.

Q7. iii. Median LOS was 5 days (IQR 3–7) with 99 (25%) in this group stayed ≥7 days. MEWS aggregate scores ≥5 was not associated with excess LOS (OR 1.26, 95%CI 0.51, 3.14, P=0.616. Excess LOS was associated with a MBI ≤17 (OR 1.93, 95%CI 1.01, 3.7, P=0.048). and altered mental status (AVPU score ≥1 (OR 4.39, 95%CI 1.09, 17.71, P=0.038) at presentation.

Q9. Patients with abnormal SBP or RR should be ‘flagged’ for increased monitoring and assessment. The excess LOS can largely be accounted for by frail elderly patients. Therefore the need of short-and long-term hospital stay patients are different.

Q14. 2- Single centre, small sample size. Q16. Use of MEWS for early detection of deteriorating patients is feasible. An aggregate MEWS score ≥5 was significantly associated with death or ICU admissions among unselected general medical patients. MEWS was not associated with excess LOS; functional status and altered mental status were independent predictors of excess LOS.

186 | P a g e







viii. Two acute medical wards in one tertiary hospital

Kim, Shin, Lee et al. (2015) South Korea Q1. RS iv. “To determine the prevalence and trends of the MEWS prior to in-hospital cardiac arrest on a ward, and to evaluate the association between changes in the MEWS and in-hospital mortality” v. A retrospective analysis of prospectively collected data vi. Data on all in-hospital cardiac arrests was extracted from EMR of the in-hospital cardiac arrest database. MEWS24 was the maximum score calculated 24 hours before, MEWS16 was the maximum score calculated between 24 and 16 hours before, and MEWS8 was the maximum score calculated 8 hours before the in-hospital cardiac arrest. vii. 380 of 501 consecutive adult (>18 years) patients who were monitored, experienced an in-hospital cardiac arrest, and were resuscitated in general wards during Mar 2009 and Feb 2013, were analysed. Patients with incomplete vital sign data to calculate a

Q2. MEWS45

MEWS was calculated for 3 times points, 24, 16 and 8 hour prior to the in-hospital cardiac arrest. MEWS24 was the maximum score calculated 24 hours before, MEWS16 was the maximum score calculated between 24 and 16 hours before, and MEWS8 was the maximum score calculated 8 hours before the in-hospital cardiac arrest.

Q4. i. The Medical Alert Team was introduced in 2009. It is automatically activated if a vital sign becomes abnormal through the electronic medical records monitoring based system that records them on a computer in real time for 24 hours. The team is composed of pulmonary/critical care attending physicians and fellows, junior or senior residents and critical care nurses.

Q5. i. 25.8% of patients survived to hospital discharge. MEWS (adjusted for age and sex) was associated with in-hospital mortality at each time point (P=0.01). MEWS24 OR = 1.14 (95%CI 1.17, 1.70) MEWS16 OR = 1.14 (95%CI 1.17, 1.70) MEWS8 OR = 1.23 (95%CI 1.07, 1.40) An increasing MEWS was not associated with survival. Increasing MEWS OR = 1.24 (95%CI 0.77, 1.97; P=0.38) ii. Median MEWS was 2.0 (1.0-3.0), 2.0 (1.0-3.0), and 3.0 (2.0-5.0) at 24, 16 and 8 hours prior to the in-hospital cardiac arrest. 178 patients had an increasing MEWS, 202 patients had a non-increasing MEWS. MEWS increased significantly between 24 and 8 hours (P<0.01), but not between 16 and 8 hours prior to the in-hospital cardiac arrest. The number of patients in the low (≤2) and intermediate (3-4) risk groups decreased at each time point prior to cardiac arrest, while the number of patients in the high risk (≥5) group increased significantly at each time point (5.6%, 15.2%, 48.9%; P<0.01). However, even 8 hours prior to cardiac arrest43.5% of patients had a low MEWS. Q6. i. Median time of response was 1 (0.5-2.0) min, and median duration of CPR was 19.0 (4.0-23.7) min. 301 patients had a return to spontaneous circulation (79.2%).

Q9. viii. There were no specific characteristics identified which were significantly different between the patient with increasing and non-increasing MEWS. Patients in the non-increasing MEWS group were older, but not significantly than those in the increasing MEWS group.

Q14. 3 Single centre. No control group. Q16. MEWS is a simple, easy to use tool. MEWS is associated with in-hospital mortality at each time point in this patient group. MEWS can also predict cardiac arrest. However, although 46.8% of patients had an increased MEWS Score four hours before a cardiac arrest, 45.2% had a low MEWS up to 8 hours prior to their cardiac arrest. Improvement in MEWS is required. “MEWS alone is not enough to predict in-hospital cardiac arrest.”

187 | P a g e







MEWS score (n=93), those who had a previous cardiac arrest (n=18) and those with a DNAR (n=10) were excluded. Median age was 64 (IQR 53.0-72,0) years, 63% were male viii. One hospital (the Asian Medical Centre).

Parham (2012) Austria Q1. RS iv. To assess MET call initiation and response. Furthermore was MEWS>4 reached before the MET call? v. Retrospective analysis vi. Data (for the 1st MET call) was extracted from medical records and the Resuscitation Educator which records all MET calls 1st Jan to 30th April 2011. Observations within 180 min of the MET call were collected, because 3 h urine output is required to calculate the MEWS. MEWS score was calculated. vii. Adult surgical or medical patients (≥18 years), who had a MET call (n=20; mean age 74.7 years). viii. 1 teaching hospital

Q2. Adult COMPASS MEWS48 was being introduced to assist early deterioration in addition to existing MET call processes. MEWS>4 requires medical review which may prevent a MET call. This is a baseline study from which effect of MEWS implementation can be assessed. Q4. i. MET response system was in place governed by the MET Call Policy. A MET call should be made as soon as a patients meets a MET criteria (based on individual vital signs). All call cases responses must be documented.

Q6. i. 85% (n=17) of MET responses were within 1 min; 15% (n=2) had a delay in the MET response of more than 1 min. 2 met criteria which merited a MET response prior to the observations which resulted in a MET response.

25% (n=5) patients had MEWS>4 within 180 min of the MET call Mean time between MEWS>4 and MET call = 113 min (range 5 to 210 min).

Q12. “Not all observations used in the MEWS calculation were recorded in every observation set. MEWS >4 may have been reached but could not be determined.”

Q14. 3 One centre. Small sample size. Q16.” Identifying and responding to patients with a MEWS >4 may have prevented 25% of MET calls.” Implementation of MEWS may improve MET response to deteriorating patients.

48

HEALTH SERVICE EXECUTIVE. 2011. COMPASS ‘Pointing you in the right direction’. Available from http://www.hse.ie/eng/about/Who/clinical/natclinprog/acutemedicineprogramme/earlywarningscore/compass.pdf

188 | P a g e







Patel, Zadravecz, Young et al. (2015b) USA Q1. RS iv. To compare the objective metrics of MEWS and PAR to identify likelihood of cardiac arrest or transfer to ICU. v. A prospective, blinded validation study vi. During Sept 2011 and Aug 2012 a MEWS was calculated from the vital signs closest to the PAR. Consecutive medical patients were given daily PAR scores prospectively by physicians during a standardised electronic medical record handover. Physicians were blinded to PAR assignment. vii. Of the eligible physicians (n=34 of 51) 28 gave 7244 PAR scores for 3249 individual patients. 51,202 MEWS scores were calculated for these 3249 patients viii. 1 hospital

Q2. MEWS45 and PAR49 Q3. PAR is a subjective 7-point Likert scale assessing physician clinical judgement.

Q5. ii. The combination of PAR and MEWS was more accurate at predicting cardiac arrest within 24 hours. PAR-MEWS AUROC=0.70 (95%CI 0.63, 0.78) PAR AUROC=0.68 (95%CI 0.60, 0.75) MEWS AUROC=0.67 (95%CI 0.61, 0.74) x. The combination of PAR and MEWS was more accurate at predicting the combined outcome of ICU transfer, cardiac arrest and RRT activation within 24 hours. PAR-MEWS AUROC=0.70 (95%CI 0.65, 0.75) PAR AUROC=0.67 (95%CI 0.62, 0.72) MEWS AUROC=0.65 (95%CI 0.61, 0.70) Q6. i. The combination of PAR and MEWS was more accurate at predicting RRT activation within 24 hours. PAR-MEWS AUROC=0.72 (95%CI 0.66, 0.77) PAR AUROC=0.68 (95%CI 0.62, 0.74) MEWS AUROC=0.67 (95%CI 0.62, 0.73) ii. There was a median 84 mins between related PAR and MEWS scores.

Q9. i. PAR and MEWS scores are poorly correlated (P=0.10) which may be due to physicians giving a PAR score based on parameters not measured in MEWS

Q14.2+ Q16. The PAR score which quantifies physician worry, when used in combination with MEWS increases the accuracy of MEWS to predict adverse events within 24 hours.

Roney, Whitley, Maples et al. (2015) USA Q1. SR with narrative description.

Q2. MEWS45 Q5 iv. No MEWS assessment tool combining nursing assessment findings adjusted for systemic inflammatory response syndrome (SIRS) vital sign criteria and laboratory values to aid in the

Q8. ii Findings from the review of literature suggest MEWS tools’ scoring of physiological findings, including vital signs has a positive relationship with earlier detection of clinical

Q14. 1- A large variation in the methodological quality with a lack of blinding, randomisation and control

49

EDELSON, D. P., RETZER, E., WEIDMAN, E. K., WOODRUFF, J., DAVIS, A. M., MINSKY, B. D., MEADOW, W., HOEK, T. L. V. & MELTZER, D. O. 2011. Patient acuity rating: quantifying clinical judgment regarding inpatient stability. Journal Of Hospital Medicine, 6, 475-479.

189 | P a g e







i. To evaluate the evidence reporting outcomes from modified early warning scoring system tools utilisation to prevent failure to rescue in hospitalised adult medical- surgical/telemetry patients. ii. Electronic databases searched included PubMed, MEDLINE, CINAHL, Cochrane Library of systematic reviews and Agency for Healthcare Research and Quality through 2014. Iii n=18 articles evidence ratings included 6% (1) Level I, 44% (8) Level IV, 6% (1) Level V, 33% (6) Level VI and 11% (2) Level VII on “Rating System for the Hierarchy of Evidence”.

identification of both the at-risk and septic patient was identified within the review. No tool was sourced which incorporated all four SIRS components of sepsis screening. x. Limited high-level data and no clinical trials linking use of modified early warning scoring system tool usage to robust outcomes were sourced. Q6. Articles (n=3) described the triggering ability of MEWS tools to bring support for patients at-risk for deterioration based on MEWS scores.

deterioration. groups was reported. Q16. Authors recommend the development of established criteria for validating modified early warning scoring system criteria, organisational-specific reliability testing and multi-site trials.

Pattison & Eastham (2012) UK Q1. RS iv. To review referrals to a CCOT, associated factors around patient management and survival to discharge, and the qualitative exploration of referral characteristics . pg. 71. v. A mixed methods explanatory design vi. Referral episodes (n=407)

Q2. MEWS45 Q3. CCOT called a MEWS score >3.

Q6. Referrals to a critical outreach team (124/407 = 30·5%) mostly made by medical staff. For 23·8% of referrals, there was a delay between the point at which patients deteriorated and the time patients were referred. Notably the average delay was 2·96 h (95% CI 1·97–3·95; SD 9·56). Mean of MEWS at referral: 3·76 (95% CI 3·49, 3·99); at deterioration: 3·96 (95% CI 3·67, 4·18).

Q8 iv. Referral would often be provoked by the culmination of various factors, including blood results, MEWS, and how patients felt. Untimely referrals were associated with lower survival to discharge (χ2p = 0.004) and 3 and 6 month mortality (χ2p = 0.004; P= 0.026) [n = 309). Three- and 6-month mortalities were significantly associated with a higher MEWS at referral (P= 0.022, Z = 2.119; p = 0.010, Z = −2.575).

Q12 Perception by experienced nurses that they used it less as they relied on their own judgement. Ward busyness. Misjudgement by HCPs of their ability to handle patients’ condition. Referral to outreach may threaten trust between ward nurses and doctors who had been managing the situation on the ward. Q13

Q14. 2- Q16. “Mapping outreach episodes of care and patient outcomes can help highlight areas for improvement. This study outlines reasons for referral and how outreach can facilitate patient pathways in critical illness.”

190 | P a g e







for patients (n=318) were evaluated. Nurses (n=7) and doctors (n=2) were interviewed. vii. Patients with cancer referred to the CCOT. viii. Single hospital, UK

Awareness of the significance of early referrals. Non critical, judgemental attitude of outreach team. Outreach- trusted resource. Add weight to nurses’ opinions. Outreach-supportive relationship.

Suppiah, Malde, Arab et al. (2014) UK Q1. RS iv. “To assess accuracy of MEWS and determine an optimal MEWS value in predicting severity in acute pancreatitis (AP)” pg. 569. v. A prospective database of consecutive admissions with AP to a single institution was analysed to determine value of MEWS in identifying severe acute pancreatitis and predicting poor outcome vi. AUROC curves were used for analysis. vii. 142 patients with AP admitted with acute pancreatitis to one institution during the period January to December 2010 viii. Single institution in UK

Q2 MEWS45 Q3 The MEWS chart provides a recommended action depending on the cumulative score: “MEWS = 3 - 4 requires 2-hourly observation plus junior medical review within 30 minutes; MEWS ≥5, or any single parameter scores ≥ 3; or total MEWS increases by ≥3 within 30 minutes requires senior medical opinion together with critical care outreach review” pg. 570.. Q4. Patients were classified as having mild or severe acute pancreatitis as per the Atlanta classification

Q8 i. The optimal highest MEWS per 24 hours period (hMEWS) and mean MEWS per 24 hour period (mMEWS) in predicting severe acute pancreatitis as determined by ROC were 2.5 ((AUC 0.924, 95% CI: 0.849 – 0.998)) and 1.625 (AUC 0.91 4, 95% CI: 0.835–0.993) respectively; with hMEWS ≥3 and mMEWS > 1 utilised in this cohort as MEWS scores are whole numbers pg. 569. On admission the sensitivity, specificity and accuracy, of: hMEWS ≥3 was 95.5%, 90.8%, 92% and for mMWES > 1 was 95.5%, 87.5%, 88.7%. The accuracy of hMEWS ≥3 and mMEWS > 1 increased over the subsequent 72 hours from 92 to 96%, and 89% to 94%.

Q14. 2- Q16 According to the authors: MEWS is suitable for all pancreatitis patients as a routine screening tool on a general surgical ward and can easily be reassessed to reflect changes in clinical course (p 575).

Yoder, Yuen, Churpek et al. (2013) USA

Q2. MEWS45

Q5. x. The median evening MEWS was 2 (IQR, 1-2). The adverse event rate (defined as ICU transfers or

Q7. vii. The frequency of vital sign disruptions was unchanged, with a median of 2 vital sign checks per patient

Q14. 2- Q16. The researchers suggests that the night-time frequency

191 | P a g e







Q1. RS iv. To investigate whether the MEWS could identify low-risk patients who might forgo overnight vital sign monitoring. v. Prospective cohort study vi. Vital signs extracted from the EMR and a MEWS was calculated. The MEWS most closely preceding 11 PM each night was used to stratify patients. The number of night-time (11 PM to 6 AM) disruptions for vital sign monitoring and the occurrence of adverse events, defined as ICU transfers or cardiac arrests in the next 24 hours (11 PM to 11 PM), were compared across all MEWS categories. vii. 54,096 patients included equivalent to 182,828 patient-days on the wards and 1699 adverse events. Data collection from November 4, 2008, and August 31, 2011 viii. Medical wards in an acute hospital.

cardiac arrests within the 24 hour period) increased with higher evening MEWS, from a rate of 5.0 per 1000 patient-days (when the MEWS was ≤1) to 157.3 per 1000 patient-days (when the MEWS was ≥7) (P = 0.003 for trend).

per night and at least 1 disruption from vital sign collection 99.3% of the nights regardless of MEWS category. Almost half of all night-time vital sign disruptions (45.0%) occurred in patients with a MEWS of 1 or less.

of vital sign monitoring for low-risk medical inpatients might be reduced positively impacting on patient sleep and could also have significant health care resource implications. Vital sign data does not incorporate more nuanced markers of clinical status.

Peris, Zagli, Maccarrone et al. (2012) Italy Q1. RS iv. To determine if MEWS calculation can help the

Q2. MEWS (Adapted from Subbe45) Q3. i/ii. Patients with a MEWS of 3 or 4 were transferred to the HDU, whereas a MEWS score

Q5 i. Mortality rate analysis did not differ between the two groups.

Q7i. After MEWS introduction, the HDU

admission increased from 14 % (control) to 21 % (experimental) (P=0.0008; sensitivity 0.4457, 95% CI 0.3725-0.5206; specificity

Q14. 2++ (cases and controls were well matched, low risk of confounding bias and high probability of a causal relationship)

192 | P a g e







anaesthesist select the correct level of care to avoid inappropriate admission to the ICU and to enhance the use of the High Dependency Unit (HDU) after emergency surgical procedures. v. Prospective, before-and-after intervention study. vi. In experimental group, MEWS was calculated by the anaesthesists before the surgical procedure and before discharge from operating room. vii. Control group (n=604, mean age = 52 years) before MEWS (Emergency surgical patients admitted (Jan 2008-Mar 2009), intervention group (n= 478, mean age = 50 years) i.e. after MEWS introduction (emergency surgical patients) (Apr 2009-Jan 2010). viii. Hospital, Italy

of ≥5 was considered criteria for ICU admission.

0.4187, 95%CI 0.3862-0.4517). The predictive values for an appropriate HDU admission (78.6% with 95% CI from 74.7% to 82.3%) After MEWS introduction, number of ICU admissions decreased from 11 % in the control group to 5 % in the MEWS experimental group (P=0.0010; sensitivity 0.7204, 95%CI 0.6178-0.8086; specificity 0.4570, 95%CI 0.4256-0.4887). The predictive values for an avoidable ICU admissions (94.6% with 95% CI from 92.1% to 96.4%).

Q16. Use of a simple and reproducible MEWS may assisted in reducing ICU admissions post-emergency surgery

Shuk-Ngor, Chi-Wai, Lai-Yee et al. (2015) Hong Kong Q1, RS iv. To compare the performances of detecting patient deterioration with and without using the MEWS for a group of patients who are waiting for in-patient beds in a public ED (p 24).

Q2. Scoring system of the MEWS45 Q3. Trigger set at MEWS ≥4, MEWS of 4 classified as intermediate risk- trigger review by senior nurse. MEWS > 4 (critical score).

Q5 x. The primary outcome was a change in patient’s ED management plan instigated by the ED doctor in response to the MEWS critical pathway activation- MEWS > 4. The improvement in the validity of nurses decision making after the introduction of the MEWS system was statistically significant (p<0.00001).

Q7. vii. In the MEWS group, there was approximately 1 episode of activation in every 10 patients but it was 1in 20 patients in the Usual Observation group. Q8. Using MEWS -- 100% sensitivity and a 98.3% specificity in detecting patient deterioration, 100% sensitivity and a 97.8%) specificity in the comparison control group. ii. Overall, RR was found to significantly discriminate between stable patients

Q13. Enhance patient monitoring and recording of respiratory rate.

Q14. 2- Q16. The Clinical Judgment of the nurse- defined as the normal practice by nurses using individual’s nursing knowledge, clinical experience and gut feeling — judging based on strong feelings rather than facts, plus the measurement of 3 vital signs — blood pressure, pulse and

193 | P a g e







v. A prospective observational design vii Total of 545 ED patients- including MEWS group (n=269) and a usual observation control group (n=275) accessed January to March 2013. Mean age of MEWS group 71.6 and control group 70.8 years. viii. A&E Department, Hospital, Hong Kong.

and patients at risk of deterioration in the study.

body temperature or clinical decision-making (p 27). Small numbers equated to small number of adverse events thus limiting the transferability of findings.

Stark, Maciel, Sheppard et al. (2015) USA Q1. RS iv. To define the basis and outcome of in-hospital cardiopulmonary arrest in all surgical patients, identify pre-arrest factors associated with poor prognosis and to investigate the ability of a MEWS to identify patients at higher risk of death.” v. Retrospective study. vi. Patients were identified from EMRs (Jan 2013-Mar 2014). MEWS was calculated using routine nursing assessment vital signs, on admission (n=60), 72- (n=28), 48- (n=32), 24 (n=38) h prior to the code blue and the ‘event day’ (n=51) MEWS scores were available for different number of patients in these time periods

Q2. MEWS45

Q5. In-hospital mortality Maximum MEWS AUROC=0.7827 In multivariate analysis, Max MEWS was a significant predictor of in-hospital death. Maximum MEWS OR = 1.39 95% CI 1.04, 1.85; P=0.025) MEWS score ≥5 OR = 1.39

Q8. In-hospital mortality MEWS score of ≥4 Sensitivity=91% Specificity=48% PPV =71% NPP=80% MEWS score of ≥5 Sensitivity=68% Specificity=68% PPV=74% NPP=61%

Q14. 3 One centre. Limited sample size. Lack of predicts in patient death in surgical patients who have experienced cardiac arrest. MEWS information. Q16. MEWS score calculated on admission “When considering the markedly elevated baseline mortality rate in this cohort, even this relatively modest increase in odds of death is stricking.”

194 | P a g e







because of variation between admission and time of the event vii. All surgical patients, admitted for elective surgery who experienced a ‘code blue’ event (n=62, median age 62 y, 65% male). 56.5% dies in hospital. viii. One hospital (Ronald Regan UCLA Medical Centre)

Urban, Mumba, Martin et al. (2015) USA Q1. RS iv. To investigate if MEWS scores are associated with hospital admissions and recent ED visits. v. A retrospective analysis vi. The 2010 National Hospital Ambulatory Medical Care Survey (NHAMCS) dataset – a national probability sample of ambulatory visits to general, short stay and non-federal hospitals in the US. MEWS was calculated on arrival at the ED with the GCS as a proxy for AVPU (from the NHAMCS vital signs recorded on admission). Whether the patient has been to the ED in the previous 72 h is also recorded. vii. 100 million adult (≥18 years) patient visits to the ED

Q2. MEWS45 Q5. x. Hospital admission from the ED: For every 1 unit increase in MEWS score, patients were 33% more likely to be admitted, even after controlling for demographics (sex, race, ethnicity and age). MEWS score 13=90% chance of hospital admission x. ED readmission MEWS score on admission was not associated with or able to predict an ED visit within 72 h even after controlling for demographics (sex, race, ethnicity and age)

Q12. Vital signs required to calculate an initial MEWS are routinely collected at triage.

Q14. 3 Q16. “Use of MEWS in EDs could be a helpful predictor of the need for hospitalisation and could serve as a focus for early decision making and as a point of comparison for efficacy of intervention in the ED and if the patient is admitted.”

195 | P a g e







for whom a MEWS could be calculated. Mean age was 45.6 years, and 42% were male. 5.7% attended the same ED within 72 h and 16.14% were hospitalised. viii. A “multistaged cluster sample design in 3 stages: 112 geographical primary sampling units, approx. 480 hospitals within these units and patient visits within emergency service areas, from the NHAMCS data.

196 | P a g e

Table 2. Newly developed and improvement of existing EWS systems Authors, (date) Country Q1. Type of Evidence






Alvarez, Clark, Song et al. (2013) USA Q1. RS iv. To (i) derive and validate an automated prediction model based on near real-time electronic medical record (EMR) data to identify patients at high risk of a composite outcome RED outside of ICU, (ii) to compare it with MEWS and clinical judgement-activated institutional RRT, and (ii) to determine if RED events can be detected sooner. v. A retrospective cohort study. vi. All candidate predictor variables data were extracted from the EMR. Data from the previous Daily risk was determined using data from the previous 24 hours. STAT physician orders and medications though to

Q2. MEWS45 Automated prediction model: 14 variables were predictors of RED events in multivariate analysis; i. older age (>54 years), Abnormal vital signs: ii. DBP max <120 mmHg, iii. Saturation of Peripheral Oxygen (SpO2) max ≤86%, Abnormal Laboratory tests iv. Aspartate Aminotransferase (AST) >250U/L v. white blood cell count >11x103 cells/mm3 vi. platelets <100x103 cells/mm3 vii. potassium >5.1 mEq/L Abnormal arterial blood gas (ABG) results viii. PaCO2 (max) ≤22 mmHg

Q5. x. A composite outcome of RED. High Risk Floor assignment was the most predictive individual variable in the model (OR) 5.71 (95%CI 4.34, 7.51) The automated clinical prediction model had good discriminatory for the prediction of RED, and was significantly better than MEWS; Derivation dataset Receiver Operating Characteristic (AUROC) curve 50=0.87 (95%CI 0.85, 0.89) Validation dataset AUROC=0.85 (95%CI 0.82-0.87) MEWS ROC=0.75 (95%CI 0.71-0.78) The Automated clinical prediction model predicted RED significantly earlier that the RRT activation (15.9 vs 5.7 hours prior to the event; P=0.003) Q6. i. RRT was activated for 357 of the study patients. Median number of times a patient at risk was flagged per day was 9 and 2 by the Automated

Q8. The automated clinical prediction model was more sensitive than MEWS; Automated clinical prediction model Sensitivity=51.6% Specificity=94.3% PPV=10% NPV=99.4% MEWS Sensitivity=42.2% Specificity=91.3% PPV=5.6% NPV=99.2%

Q9. Use of more available EMR data in the automated clinical prediction model may explain why it is superior to the simpler MEWS model. Incorporation of physician orders may reflect a physician’s escalating concern about patient’s stability. High Risk Floor assignment may be a proxy for unknown system or process-related factors, e.g. nurse staffing ratios, physician team composition, associated with increased risk. No medication variables were retained in the final model. Q13. iii. Use of additional technologies such as ‘natural language processing’ and ‘adverse drug event

Q14. 2- Used retrospective data from 1 health centre. The Automated prediction model has a moderate false positive rate, findings may be setting-specific Q16. The Automated clinical prediction model outperformed MEWS and human judgement-based RRT, in predicting SAEs. “An automated model harnessing EMR data offers great potential for identifying RED and was superior to MEWS and the clinical judgement-driven RRT.”

50

Receiver Operating Characteristic (ROC) Curves are commonly used to plot the performance of a medical test. The plots are composed of a test outcome with the ‘true

positive rate’ on the ‘y axis’ and ‘false positive rate’ on the ‘x axis’. The area under the ROC curve (AUROC) is a measure of how accurate the diagnostic test is. The higher the score the better the test performance, with the highest possible score = 1.0 (perfect test) and an area under the curve of 0.5 representing a test failure. Reasonable and good discrimination is observed at values of 0.700-0.800 and >0.800, respectively.

197 | P a g e







increase risk of an SAE were also investigated as predictors. Medical wards accounting for the top15% of RED were classified as ‘high risk floors.’ vii. 7,466 adult patients, accounting for 46,974 days, admitted to general medical wards (May 2009-Mar 2010) split 50% each into derivation (n=3,624; mean age 50.5 SD 14.6 years, 56.1% male) and validation (n=3,792; mean age 51 (SD 14.8 years), 54% male) subsamples. Major clinical deterioration occurred in 1.3% and 1.2% of patients in the two subsets, respectively. viii. 1 urban academic hospital

ix. PaCO2 (max) >70mmHg Physician orders x. ABG xi. Electrocardiogram (ECG) xii. Stat Physician Order Summary variable xiii. High Risk Floor Assignment xiv. MEWS Q4. i. The institutional RRT is deployed when the patients has ≥1 of the following: (i) Heart rate (HR) <40/>130 beats/min, (ii) Systolic Blood Pressure (SBP) <90 mmHg, (iii) Respiratory Rate (RR) <8/>30 breaths/min, (iv) partial pressure of oxygen (PaO2) >50%, (v) Oxygen (O2) requirement>50%, and (vi) acute change in mental status

clinical prediction model and RRT, respectively. The automated clinical prediction model was more sensitive , but less specific than the RRT; Automated clinical prediction model: Sensitivity=51.6% Specificity=94.3% Positive Predictive Value (PPV)=10% Negative Predictive Value (NPV)=99.1% RRT: Sensitivity=25.8% Specificity=98.8% PPV=21% NPV=99.4% Automated clinical prediction model predicted an event 5.7 (95% CI 3.1, 8.3) hours before the RRT (15.9±7.7 and 8.4±8.5 hours prior to the RED, respectively).

detection software’ may improve prediction of poor hospital outcomes.

Bian, Xu, Lv (2015) China Q1. RS iv. “To find a scoring system to predict the onset of AHF in patients in the acute heart failure unit. The primary end-point was all-cause mortality either in-hospital or after discharge, the secondary

Q2. SUPER scoring system: SpO2, Urinary volume, Pulse, Emotional state and RR, each scored between 0 and 2 points, see Appendix x. SUPER scoring is divided into low (0-1 point), intermediate (2-3 points), high (4-5 points), and extremely high (6-10

Q5. i. All-cause in-hospital mortality was 19.2% (n=83) ii. 163 patients had AHF 420 times; average of 0.97 times for all patients. SUPER predicted AH onset 3.90±1.94 hours (1-17 hours) earlier. SUPER was significantly better than the MEWS at identifying patients at risk of AHF (P<0.05). There was no statistical significance of adding age to the

Q8. All-cause mortality in-hospital and post-discharge increased significantly with higher SUPER scores (P<0.05). Incidence of AHF by hours increased significantly with higher SUPER scores (below). However there was no statistical correlation between onset and prognosis of AHF.

Q9. Age was not retained in the final model because it is “relatively constant”

Q14. 3 Q16. The SUPER score may be used for predicting the onset of AHF in high risk patients. It may also be used to stratify patients into low-, moderate-, high- and extremely-high risk. It was superior to MEWS in this patient group.

198 | P a g e







endpoint was onset of AHF. v. A retrospective study vi. Within the AHFunit six physiological parameters were measured hourly; HR, RR, SBP, SpO2, urine volume per hour and emotional state. Admission and follow-up data was collected from hospital charts. Patients were follow-up for 6 to 24 months. vii. All adult patients (≥18 years) at risk of AHF triaged to the Chest Pain Centre or the acute heart failure unit between Nov 2011 and June 2014 were eligible (n=433; mean age 64.08±15.67 years; 61% male). All patients had ≥1 risk factor for AHF. viii. 1 hospital (Qilu Hospital)

points) risk. MEWS45

SUPER score SUPER + Age AUROC=0.820 SUPER AUROC=0.811 MEWS AUROC=0.662 Cumulative survival decreased significantly (up to 60 days follow-up) with increasing SUPER score (<0.05).

Low risk (0-1 point): 17.3% Intermediate risk (2-3 points): 61.3% High risk (4-5 points); 84.4% Extremely high risk (6-10 points): 94.0%.

“In patients at high risk of AHF, the SUPER scoring system could predict the onset of AHF 2 to 6 hours earlier. Pre-emptive treatment according to the SUPER score may prevent or delay AHF occurrence to improve quality of life, reduce mortality and waste of medical resources.”

Bleyer, Vidya, Russell et al. (2011) Spain Q1. RS iv. “To examine the association of critically vital signs occurring at any time during the hospitalisation with mortality.” v. Longitudinal analysis of retrospective data. vi. All vital sign measurements from

Q2. MEWS45, ViEWS51 and authors created a ‘Critical vital sign’ scoring system with different weightings for age: a. each vital sign (1 point) and age (60 to <70 (I point), 70 to <80 (2 points), 80 to <90 (3 points)). b. each vital sign (1 point) and age (60 to <70 (I point), >70 (1 points)). c. each vital sign (1 point) and age (<80 (I point), >80 (2

Q5. i. 1 vs 3 simultaneous critically abnormal vital sign, with increasing age, was associated with a 19-fold increase in mortality (all ages: 0.92% and 23.6%: age>70 years, 0.62 and 42%, respectively). AUROC for mortality; ‘Critical vital sign’ a. AUROC = 0.842 ‘Critical vital sign’ b. AUROC = 0.869 ‘Critical vital sign’ c. AUROC = 0.862 VIEWS51 AUROC = 0.862 MEWS AUROC = 0.865

Q8 iv. VIEWS detected more deaths at the same sensitivity as the ‘Critical vital sign’ scoring system.

Q12. iii. Not possible to determine errors in vital sign entry in the electronic system. More precise consciousness level definitions would allow improved data analysis. Q13. iii. Vital signs of all non-ICU and non-intermediate care unit floors are entered

Q14. 3 One hospital. No pre-study training of vital sign entry was undertaken. Q16. Simultaneous presence of ≥3 critically abnormal vital signs any time during hospitalisation was associated with very high in-hospital mortality. These grouping occur more frequently in the first 48 hours.

51

PRYTHERCH, D. R., SMLTH, G. B., SCHMIDT, P. E., FEATHERSTONE, P. L. 2010. ViEWS - towards a national early warning score for detecting adult inpatient deterioration. Resuscitalion, 81, 932-937.

199 | P a g e







hospitalisations over 18 months (2008-09) vii. 1.15 million vital signs from 42,430 admissions of 27,722 patients (51% male, mean age 57.4±17.6 years, mean LOS 6±8.8 days). Vital sign data was obtained from the electronic database. Analysis was restricted to patients with respiratory rate >4 and <60bpm; systolic blood pressure >50 and <250mmHg, temperature >32.2 or <41.10C, pulse >30 or <200bpm. Chart review was conducted in cases where 3 simultaneous critically abnormal vital sign occurred to determine diagnosis and whether the RRT was called. viii. One academic hospital. It is a level 1 trauma centre.

points)). Critical vital signs i.e. levels and ranges associated with a ≥5% chance of mortality were identified as; Systolic blood pressure (<85mmHg) Heart rate (>120 bpm) Temperature (<35 or >38.90C) Oxygen saturation (<91%) Respiratory rate (≤12 or ≥24), and Level of consciousness (any other than ‘Alert’) Q3.ii. “Healthcare professionals were encouraged to call rapid response for patients with the occurrence of any critical vital signs or for patients whom the healthcare team felt were unstable.” The RRT consisted of 2 nurses with ICU experience.

VIEWS (≥11): 1,236 instances with 274 deaths (22.1%) ‘Critical vital sign’ (3 critical vital signs): 998 instances with 232 deaths (22.1%)

Q6. Increased mortality was associated with RRT notification: RRT was alerted within 24 hours for 39.6% of patients without a DNR order. Mortality rate for these patients was 28%. Mortality rate for these patients for whom the RRT was not called was 9.5%. RRT was called for patients with more comorbidities, higher respiratory rates and lower pulse oximetry readings at the time the RRT was called. Average response time was 4.65 hours. Visits to all patients with ≥3 critical vital signs would result in 55 patient visits per month; 13 of these patients would die.

electronically into a computer database. In hospitals with electronic entry of vital sign, immediate electronic notification of RRTs may increase response time and appropriateness

MEWS and VIEWS were validated as being predictive of mortality upon admission and at any point during hospitalization.

Churpek, Yuen, Park et al. (2014a) USA Q1. RS iv. For patients experiencing decline on the ward, ICU transfer is a competing risk for cardiac arrest outcome, as they are triaged to the ICU

Q2. ViEWS51

ViEWS was chosen as the best performing system of 33 tested; Kellett, Kim 201252)

Derived model differed for each outcome: Derived cardiac arrest model: i. Time (hours): time from

Q5. ii. The derived model was significantly more accurate than ViEWS at detecting cardiac arrest (P<0.001), when using patients highest scores. Ever experience cardiac arrest: Derived cardiac arrest model AUROC=0.88 (95%CI 0.84-0.91) ViEWS AUROC=0.78 (95%CI 0.73-0.83)

Q7. ii. The derived model was significantly more accurate than ViEWS at detecting ICU transfer ever and within 24 hours (P<0.001) Ever: Derived cardiac arrest model AUROC=0.77 (95%CI 0.76-0.78)

Q9. “The separation of cardiac arrest and ICU transfer in the model development process is a strength of the study”. Different predictors of each outcome were identified: lower temp predicted cardiac arrest, but not ICU transfer;

Q14. 3 Retro cohort, single centre. Q16. A prediction tool for ward patients was developed and validated using EHR data which simultaneously predicted risk of cardiac arrest and ICU transfer. This model

52

KELLETT J. & KIM A. 2012. Validation of an abbreviated Vitalpac Early Warning Score (ViEWS) in 75,419 consecutive admissions to a Canadian regional hospital. Resuscitation, 83, 297-302.

200 | P a g e







prior to the occurrence of a cardiac arrest. Therefore the aim was “to derive and validate a prediction model for cardiac arrest while treating ICU transfer as a competing risk, using EHR data.” v. Retrospective cohort study vi. All routinely collected vital sign, demographic, location and laboratory data were extracted from the EHR vii. 59,301 hospital admissions (56,649 control (mean age 54±18, 43.2% male), 109 cardiac arrest (mean age 60±16, 50.1% male) and 2,543 ICU transfer (mean age 64±17, 41.3% male) patients) from Nov 2008 to 2011. viii. 1 urban academic centre

first ward vital sign ii. Prior ICU stay iii. HR (beats/min) iv. DBP (mmHg), v. RR (breaths/min) vi. Temperature vii. Supplemental O2 use viii. age (years), ix. Blood urea nitrogen (mg/dL) x. Anion gap (mEq/L) xi. Platelet count (K/uL) xii. White blood cell count (K/uL) Derived ICU transfer model: i. Time (hours): time from first ward vital sign ii. Prior ICU stay iii. HR (beats/min) iv. DBP (mmHg), v. RR (breaths/min) vi. O2 saturation (%) vii. Temperature viii. Mental status (AVPU) viii. Supplemental O2 use ix. age (years), x. Blood urea nitrogen (mg/dL) xi. Anion gap (mEq/L) xii. Haemoglobin (g/dL) xiii. Platelet count (K/uL) xiv. Potassium (mEq/L) xv. White blood cell count (K/uL)

Q4. A nurse-led RRT was in place since 2008 but no

Cardiac arrest within 24 hours: Derived cardiac arrest model AUROC=0.88 (95%CI 0.88-0.89) ViEWS AUROC=0.74 (95%CI 0.72-0.75) Mean derived cardiac arrest model score was higher in patients who suffered a cardiac arrest (48.4) compared to those who did not (40.8) in the 48 hours prior to the event. Q6.i. 564 RRT calls during Nov 2008 to 2011 (average 9.521 calls pre 1,000 admissions). Cardiac arrest patients (8.3%) were significantly more likely to have an RRT call than control patients (0.3%; P<0.001). Use of ViEWS would have resulted in 5,500 more ‘false alarms’ than the derived model

ViEWS AUROC=0.73 (95%CI 0.72-0.74) Within 24 hours: Derived cardiac arrest model AUROC=0.76 (95%CI 0.76-0.76) ViEWS AUROC=0.73 (95%CI 0.72-0.73) Mean derived cardiac arrest model score was higher in patients who were transferred to ICU (44.6) compared to those who were not (40.8) in the 48 hours prior to the event. Q8. Derived cardiac arrest model was more sensitive in the detection of cardiac arrest than ViEWS at the same specificity. Cardiac arrest detection Derived cardiac arrest model (score >53) Sensitivity=65% Specificity=93% ViEWS (score >9) Sensitivity=41% Specificity=93%

hypoxia predicted ICU transfer, but not cardiac arrest. Routinely collected laboratory values added to the model were significant predictors of both outcomes. Q13. Automatic generation of scores by the EHR has the potential to decrease errors in human calculations and launch alerts if externally validated.

was more accurate than ViEWS for both outcomes. This model “could be implemented in the EHR and used in real-time to detect critically ill patients”

201 | P a g e







specific vital sign triggers were used to activate RRTs. Consultation with the attending physician and/or pharmacist is available if requested. There is a separate cardiac arrest team.

Churpek, Yuen, Park (2012b) USA Q1. RS iv. To develop an eCART score using ward vital signs to predict cardiac arrest, and compare its accuracy to MEWS. v. A retrospective cohort study vi. Ward vital signs were extracted from medical records. Information on patients suffering a cardiac arrest was collected prospectively. vii. All hospitalised patients with documented ward vital signs (n=47.427; Nov 2008-Jan 2011). Patients were categorised as (i) suffering a ward cardiac arrest (n=88), (ii) not suffering a ward cardiac arrest but transferred to ICU (n=2,820) and (ii) no cardiac arrest or ICU transfer (n=44,519). viii. 1 academic medical centre

Q2. MEWS22

CART: Maximum and minimum vital sign was used in the CART model derivation. Vital signs in the final CART model were; RR, HR, DBP, and age. (see Appenxix X)

Q3. A nurse-led RRT was in place since 2008 activated by ‘tachypnea’, ‘tachycardia’, ‘hypotension’ and ‘staff worry’ but specific vital sign thresholds were not specified.

Q5. ii. eCART score was significantly more accurate than MEWS for predicting cardiac arrest (P=0.001) eCART AUROC=0.84 MEWS AUROC=0.76 eCART score was statistically higher for CA patients (8±6) compared to controls (4±4) 48 hours prior to the event (P<0.001). The CART score detected cardiac arrest earlier than MEWS (median 48 vs 42 hours prior, P=0.85)

Q6. i. CART score was significantly more accurate than MEWS for predicting ICU transfer (P<0.001) eCART AUROC=0.71 MEWS AUROC=0.67 eCART score was statistically higher for patients transferred to ICU (6±6) compared to controls (4±4), 48 hours prior to the event (P<0.001). Q8. eCART was validated against MEWS for ICU transfer. eCART>17 Specificity=89.9% Sensitivity=53.4% MEWS >4 Specificity=89.9% Sensitivity=47.7% eCART>20 Specificity=91.9% Sensitivity=47.7% MEWS >4 Specificity=89.9% Sensitivity=47.7%

Q9. The addition of DBP instead of SBP was useful in improving the accuracy of CART over MEWS in predicting cardiac arrest. Also inclusion of age means a 70 year old patient will need less vital sign abnormalities than a younger patient to achieve the same CART score. Finally vital signs are weighted according to their predictive ability.

Q14. 2- Single centre Q16. The eCART score is simpler and more accurately detected cardiac arrest and ICU transfer than MEWS. Implementing CART may decrease RRT resource utilisation and lead to improved patient outcomes.

202 | P a g e







The higher sensitivity of CART (>20) over MEWS (>4) would have resulted in 890 less patient calls over the study period, while detecting the same number of cardiac arrests.

Churpek, Yuen, Winslow et al. (2014b) USA Q1 RS iv. To develop and validate an eCART score using commonly collected EHR data v. Observational cohort study vi. All adult patients hospitalized on the wards in 5 hospitals (Nov 2008-Jan 2013). Primary outcome was cardiac arrest, secondary outcomes were ICU transfer, death on the ward, or a combination of the 3 outcomes. vii. 269,999 patient (mean age 60 (SD 20) years; males 40%) admissions were included. The model was developed in the first 60% of the data at each hospital and then validated in the remaining 40%. viii. 5 hospitals

Q2. Predictor variables were vital signs (temperature, heart rate, blood pressure, respiratory rate, oxygen saturation), mental status [AVPU] and laboratory results (white cell count, hemoglobin, platelets, sodium, potassium, chloride, bicarbonate, anion gap, blood urea nitrogen, creatinine, glucose, calcium, total protein, albumin, total bilirubin, aspartate aminotransferase, alanine aminotransferase, and alkaline phosphatase) and age were obtained electronically. (Final model components not stated (16-item from another paper)) Q4. All hospitals had RRTs in place during the study period. These were either nurse- or physician-led. No specific vital sign triggers were used to activate RRTs.

Q5. eCART was more accurate than MEWS in predicting all outcomes (P<0.01): i. Death MEWS AUROC: 0.88 (95%CI 0.88-0.88) CART AUROC: 0.93 (95%CI 0.93-0.93) ii. cardiac arrest MEWS AUROC: 0.71 (95%CI 0.70-0.73) CART AUROC: 0.83 (95%CI 0.82-0.83) x. Combined outcome MEWS AUROC: 0.70 (95%CI 0.70-0.70) CART AUROC: 0.77 (95%CI 0.76-0.77)

Q7. eCART was more accurate than MEWS in predicting ICU transfer i. ICU transfer MEWS AUROC: 0.68 (95%CI 0.68-0.68) CART AUROC: 0.75 (95%CI 0.74-0.75)

Q8. iv. cardiac arrest within 24 hours; at similar specificities, eCART score ≥17 had a higher specificity than MEWS score ≥3. MEWS ≥3 Sensitivity=39% (37-41%) Specificity=90% (90-90%) eCART ≥17 Sensitivity=54% (52-56%) Specificity=90% (90-90%) eCART cut-off of ≥50 would detect; 51% CAs 44% ICU transfers 83% deaths

Q14. 2- The eCART risk score is complex and requires electronic calculation. Some of the ICUU transfers may have been planned. Q16. “We developed an accurate ward risk stratification tool using commonly collected electronic health record variables in a large multi-centre dataset. Further study is needed to determine whether implementation in real-time would improve patient outcomes.”

Christensen, Jensen, Maaløe et al. (2011) Denmark

Q2. BEWS, in development since 2007, calculated from a random sample of vital signs

Q5. i. BEWS score ≥5 was significantly associated with mortality within 48 hours of arrival at the ED:

Q7. i. A BEWS score ≥5 was significantly associated with admission to ICU within

Q12. “Many patients had to be excluded because of insufficient documentation of

Q14. 3 One hospital ED. “Risk of selection bias which would

203 | P a g e







Q1.RS iv. ”To evaluate the ability of BEWS to identify critically ill patients in the ED and to examine the feasibility of using BEWS to activate an multidisciplinary team response.” (p.1) A critically ill patient is defined as one who is admitted to the ICU or dies within 48 hours of arrival at the ED. iv. Retrospective study v. Vital signs from 300 randomly selected ED patients vi. 162 of the 300 ‘red’ randomly selected patients, 49% were male and the mean age was 57 (range 2-98) years vii&viii One hospital ED where patients are categorised as ‘red’ ‘blue’ or ‘white’ following evaluation according to guidelines. Patients categorised as ‘red’ being most in need of immediate treatment. ‘Red’ patients immediately undergo a two-step evaluation; (i) assessment based on “primary criteria” – life threatening signs and symptoms, and if not present (ii) BEWS is calculated based on vital signs.

from patients in the ED. BEWS incorporates (i) respiratory frequency, (ii) pulse, (iii) SBP, (iv) temperature and (v) level of consciousness. Each vital sign is scored from 0-3, and a score ≥5 triggers an emergency call. Patients with a BEWS <5 do not activate an emergency call unless the triage nurse is concerned.

Relative Risk (RR) 20.3 (95% CI 6.9-60.1) compared to BEWS score <5.

x. BEWS score ≥5 was significantly associated with being critically ill: Relative Risk (RR) 6.8 (95% CI 3.3-13.8) compared to BEWS score <5.

48 hours of arrival: RR 4.1 (95% CI 1.5-10.9) compared to BEWS score <5.

Q8. Within 48 hours of admission to ED, at a BEWS ≥5; a. mortality Sensitivity = 83%; Specificity = 83% PPV=16% NPV = 99%

b. ICU admission Sensitivity = 50%; Specificity = 81% PPV=16% = 6% NPV = 98% c. defining critical illness Sensitivity = 63%; Specificity = 82% PPV=16% = 16% NPV = 98%

vital signs” (p.4) This made it impossible to calculate BEWS retrospectively. “The BEWS is a safe tool for prioritising resources to patients in need of rapid and intensive care” (p.4)

lead to an overestimation of the prevalence of critical illness among ‘red’ patients” (p.4) where patients who are more ill have more vital signs measured. Q15. Specificity and NPP of BEWS are high and can be used to both categorise patients in the ED into low and high risk and activate MDT response. Q16. A narrow, arbitrary definition of critically ill patients was used due to the lack of a practical definition. Patients who deteriorated following admission were excluded from the study. The low PPV of BEWS among ‘red’ patients “must be perceived in the light of the low prevalence of critical illness among the ‘red’ patients and our rather narrow definition of critical illness.” (p.4)

204 | P a g e







Dawes, Cheek, Bewick et al. (2014) UK Q1. RS iv. “To determine whether: (a) the Worthing PSS, calculated using VitalPACTM, resulted in a (reduction in mortality and LOS; (b) mortality prediction could be improved with the addition of co-morbidity, biochemical data or both or with the Worthing PSS score recorded at AMU discharge; (c) the mortality prediction of the NEWS is comparable with that of the Worthing PSS; and (d) the severity of illness differed between weekday and weekend admissions.” (p.604) v. Prospective observational study, post-intervention. vi. All physiological patient data on admission, discharge and/or transfer (Feb-July 2010) collated with VitalPACTM software. Patient characteristics, co-morbidity, outcomes, and biochemistry data were taken from the hospital administration and pathology systems.”

Q2. (i) The Worthing PSS, a paper-based EWS developed (using statistical methods) and validated locally from admission physiology for patient hospital mortality in 200553 and introduced in 2008. Worthing PPS has 8 items (scored between 0 and 3): SPB, DBP, HR, RR, oxygen saturation in air (measured with Vital Signs Monitor VS-800); (iii) requirement for supplemental oxygen, Temperature; and Level of consciousness (AVPU (2) NEWS40 (I cannot see the footnote for that one) (includes the provision of supplemental O2 as an additional variable). Q3. i. An electronic clinical data software system, VitalPACTM

was implemented in 2010 which automatically calculated the Worthing PSS and display an alert ii. Electronic alerts

Q5. i. Hospital mortality decreased from 8.3% to 5.2% over 5 years post intervention, but this was not statistically significant after adjustment for the admission Worthing PSS score (P=0.29). Prediction of death within 72 hours: Worthing PSS AUROC (2010): 0.74 (95% CI: 0.69, 0.78) Worthing PSS AUROC (2005): 0.74, 95% CI: 0.71, 0.77) NEWS AUROC: 0.76, (95% CI: 0.72, 0.80). Using the final acute medical unit score, the discriminatory value of the Worthing PSS increased; Worthing PSS AUROC (2010): 0.88 (95% CI: 0.83, 0.94) C-reactive protein (n=1,084 patients) was the only independently predictive of mortality (in addition to the Worthington PSS): AUROC: 0.78, 95% CI: 0.71–0.85. Worthing PSS ≥2 was associated with a mortality of >10%. NEWS ≥4 was associated with a mortality of >10%. No significant difference in the mean Worthing PSS score (1.14 (SD 1.38 (Friday)) and 1.56 (SD 2.05 (Sunday)) or number of deaths between weekdays and weekends (P=0.189).

Q7. Iii Admission Worthington PSS was correlated with hospital LOS. Mean LOS decreased from 4 to 2 days, but this reflected an increase in short stay admissions given the decrease in patients attending with less physiological derangement on admission

Q9 v. Failure of the Worthington PSS may be because of incorrect trigger thresholds used in the system. Q10. i. Introduction of an electronic alerting system did not have a clinical impact on patient outcomes. However, the speed, appropriateness and compliance of the response to alerts was not measured. Q13. This hospital following its merge with St Richards hospital implemented NEWS.

Q14. 2- Relatively small sample size. One centre. Q16. The Worthington PSS and NEWS performed similarly in the prediction of mortality within 72 hours. The predictive performance of the Worthington PSS “was not enhanced by the addition of biochemical variables and co-morbidities.” (p.603) “After the introduction of an electronic alerting PSS, there was no reduction in mortality when adjusting for severity of physiological illness. Furthermore, the predicted mortality reduction in patients with higher admission scores was not seen.” (p.608)

53 DUCKITT, R. W., BUXTON-THOMAS, R., WALKER, J., CHEEK, E., BEWICK, V., VENN, R. & FORNI, L. G. 2007. Worthing physiological scoring system: derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study. British Journal of Anaesthesia, 98, 769-774.

205 | P a g e







vii. Complete data from 3,184 patients. Of these 3020 survived (median age 69 (16-105), 48% male), and 184 died (median age 81 (20-102), 56% male viii. An acute medical unit in 1 hospital (The Worthing site of the Western Sussex Hospitals Trust)

recommended intervention according to a set protocol. All therapeutic management was at the discretion of the attending doctor. Q4. vi. The hospital implemented the electronic clinical data software system VitalPACTM into the Acute Medical Unit (AMU) for a 6-month period (2010) which automatically calculated the Worthing PSS and displayed an alert based on a local protocol.

Escobar, LaGuardia, Turk et al. (2012) USA. Q1. RS iv. To develop a predictive model for unplanned transfer to ICU from medical-surgical wards, or death on the ward for patients who were ‘full code’ (i.e. if patient survived they would have been transferred to ICU) from data available in real-time from EMRs. v. A retrospective case-control study vi. Data on predictors (vital signs, laboratory results, severity of illness scores, longitudinal chronic illness burden score and care

Q2. MEWS45: A MEWS score was calculated from the EMRs of all patients retrospectively. EMR-based model contained 14 variables; 1) directive status (full code /no full code) 2) LAPS (range 0 to 256), standardised and included as LAPS and LAPS squared 3. COPS (range 0 to 701) standardised and included as COPS and COPS squared 4. COPS status: indicator for absent comorbidity data 5. LOS (hours) 6. Time of day 7. Temp (highest in preceding 24 h)

Q7. i. For patients with all diagnoses (development dataset), the EMR-based model performed better than the calculated MEWS for predicting unplanned ICU transfer (or death on ward). In the validation sets, both prediction models performed best in patients with gastrointestinal disease and worse in patients with Congestive heart failure. In both cases the EMR-based model performed better. Derivation dataset All diagnoses EMR-based model AUROC= 0.845 (95%CI 0.826, 0.863) MEWS AUROC= 0.709 (95%CI 0.697, 0.721) Validation dataset

Q9. viii. This model is being implemented in a simulated environment. Q12. ii. Data needed in the model e.g. vital signs are not recorded consistently as the patient deteriorates iii. To use these EMR-based models hospitals must have EMRs and longitudinal data.

Q14. 2- 14 integrated hospitals, large dataset. The model does not incorporate reason for unplanned ICU transfer, nor the fact that the patients should have been admitted to ICU initially. Q16. “EMR-based detection of impending deterioration outside the ICU is feasible in integrated healthcare delivery systems.” This model outperforms MEWS whose score is assigned manually. Future models will investigate discrimination by individual diseases more thoroughly.

206 | P a g e







directives) was extracted from EMRs. Degree of physiological derangement 72 hours before hospitalisation was quantified using the Laboratory-based Acute Physiology Score (LAPS), and comorbidity burden in the preceding 12 months was quantified using the Comorbidity Point Score. Units of analysis were 12-hour patient shifts, defined as an ‘event’ or ‘comparison’ shifts is a patient was transferred to ICU or not, respectively. 10 randomly selected comparison shifts were chosen for each event shift. The cohort was split 50% into the development and validation datasets. vii. Included were 4,036 event shifts and 39,782 comparison shifts from 92,797 adult patients (≥18 years) during Nov 2006 to Dec 2009. viii. An integrative healthcare setting encompassing 14 hospital. All hospitals and clinics have the same IT systems and use the same patient medical record number and care can be tracked between ll hospitals.

8. HR (most recent beats per min; variability in previous 24 h) 9. RR (most recent beats per min; worst or variability in previous 24 h) 10. DPB (most recent in previous 24 h, transformed by subtracting 70 from the actual value and squaring the result) 11. SBP (variability in preceding 24 h) 12. Pulse oximetry (lowest or variability in preceding 24 h) 13. Neurological status (most recent) 14. Laboratory tests (blood urea nitrogen, proxy for measured lactate, haematocrit, total white cell count

Gastro-intestinal diseases EMR-based model AUROC= 0.845 (95%CI 0.783, 0.897) MEWS AUROC=0.792 (95%CI 0.726, 0.857) Congestive heart failure EMR-based model AUROC= 0.683 (95%CI 0.610, 0.755) MEWS AUROC=0.541 (95%CI 0.500, 0.620) Q8. EMR-based models were more than twice as efficient as MEWS; A MEWS trigger score of ≥6 would identify 15% of all ICU transfers, with 34.4 false alarms for each transfer, 52 patients per day would need to be evaluated. A MEWS trigger score of ≥4 would identify 44% of all ICU transfers, with 69 false alarms for each transfer. With the EMR-based model identification of 15% of ICU transfers would result in 14.5 false alarms for each transfer, 22 patients per day would need to be evaluated.

Etter, Takala & Merz (2014) Q2. A simple severity scoring Q5. Q7 Q9. Q14. 2-

207 | P a g e







Switzerland Q1. RS iv. “To review the preceding factors, patient characteristics, process parameters and their correlation to patient outcomes of MET calls since its introduction.” v. Retrospective cohort study vi. Data was extracted from specific database recording information on all MET calls for the 24 h pre-event and during event. vii. All patients assessed by the MET from its introduction (Oct 2009-Dec 2013). Included were 1,317 patients (1,625 MET calls), 69% were male and 19.9% died in-hospital. Mean age of survivors (65.2 (IQR 53.6-75) was lower than non-survivors (68.5 (IQR 58.6-77.1; P<0.001) viii. Dept. of Intensive Care Medicine in 1 hospital

system, called VSS was used based on MET calling criteria: defined as the sum of the occurrence of 6 vital sign abnormalities, each considered 1 point (HR, SBP, RR, SpO2 and GCS; i.e. vital signs were scored if abnormal or outside at a fixed cut off and were not graded according to degree of abnormality peripheral perfusion was also assessed with an abnormal defined as capillary refill times of >3 seconds) Q4. i. MET was introduced in Oct 2009. The MET consists of intensive care specialists and is available 24/7. Calling criteria are objective physiological parameters and a subjective ‘concern’ There is also a cardiac arrest team.

i. All individual vital signs recorded during the MET call had a significant correlation to hospital mortality in univariate analysis. Maximum VSS pre-MET event was a significant predictor of mortality: AUROC = 0.63 (P<0.001) In multivariate analysis, RR and GCS were significantly correlated with in-hospital mortality. RR: OR 1.043 (95%CI 1.019, 1.068; P<0.0001) GCS: OR 0.886 (95%CI 0.820, 0.958; P=0.002) ii. There was a significant decrease in cardiac arrests post MET-implementation: 1.6 to 0.8 per 1000 hospital admissions (2008 and 2013, respectively; P<0.001). Q6. i. There was a significant increase in MET calls from 5.2 to 16.5 per 1000 hospital admissions (P<0.001) post MET implementation ii.14% of MET calls had no vital signs recorded in the 24 h before the MET call.

vi Used existing intensive care resources Q8 I. No grading of severity of vital signs, cut offs at extremes II. RR and GCS strongest predictors of in-hospital mortality 53% had at least one vital sign deranged

v, Implementation of MET did not result in a significant decrease in the delay between the first vital sign abnormality and the MET call. Q12.Barrier ii. There is variability in the assessment and recording of vital signs prior to MET events and delays between vital sign instability and subsequent MET call. These factors could reduce the sensitivity of the triggering system. Q13. iii. Successful use of the MET requires continuous information and education of all healthcare professionals involved in the care of at-risk patients.

Q16. VSS is a significant predictor of mortality in patients assessed by the MET. Increasing MET utilisation coincided with a decrease in cardiac arrest calls.

Jarvis, Kovacs, Badriyah et al. (2013) UK Q1. RS iv. To build a EWS exclusively on routine laboratory tests using DT analysis, for paper-based implementation, towards early identification of in-hospital death.

Q2. LDT-EWS. Q3. i. Male and female LDT-EWS. Biochemical and haematology blood test parameters, with acceptable ranges were included (Hb, WCC, U (0.4-107.1 mmol/L), Alb (10-70 g/L), Cr (8.8-2210 umol/L), Na (100-200

Q5. i. LDT-EWS could discriminate between patients at risk of death and those who were not. The AUROC values differed by validation dataset by gender. For all patients, the min and max AUROCs are given but these differences are not statistically significant; Q9 AUROC= 0.801 (95%CI 0.776, 0.826)

Q9. All laboratory parameters required for calculating LDT-EWS are routinely measured and available within hours of admission. Laboratory tests subsequent to admission are not frequent or complete, taken in a ‘piecemeal’ fashion. Laboratory tests unlike vital signs are subject

Q14.2- Single centre. Treatment strategies were not considered. A large dataset, subject to high quality control measures, over 6 years was used. Repeatability was demonstrated. Q16. “Commonly measured

208 | P a g e







v. A retrospective study vi. An electronic database of common biochemical and haematology blood tests collected soon after admission. vii. Adult (≥16 years) medical patients (n=86,472) during July 2006 and March 2012. With at least 1 overnight stay. Gender-specific LDT-EWS models were constructed using a derivation dataset (Q1; n=3496; 47.7% were male) and 22 discrete validation datasets (Q2-Q23) each 3 months long (n=3428 – 4093). viii. I hospital (Portsmouth Hospitals Trust)

mmol/L) and K (1-15 mmol/L). A 0-3 weighting system for risk bands was developed; 0=risk generated by DT was <mean risk of in-hospital death 1= risk was ≥mean risk and <2 times mean risk of in-hospital death 2= risk was ≥2 mean risk and <3 times mean risk of in-hospital death 3= risk was ≥3 mean risk of in-hospital death

Q16 AUROC= 0.755 (95%CI 0.727, 0.783) Males Q2 AUROC= 0.824 (95%CI 0.792, 0.856) Q16 AUROC= 0.744 (95%CI 0.704, 0.784) Females Q12 AUROC= 0.826 (95%CI 0.796, 0.856) Q10 AUROC= 0.742 (95%CI 0.707, 0.777) Q6 i. A LDT-EWS score of 4 would trigger a response in 40.7% of all laboratory test datasets. 79.7% of all patients having a trigger would subsequently die. Different trigger scores were observed for males and females; Females: A LDT-EWS score of 4 would trigger 35.7% calls with 75.3% of all patients having a trigger subsequently dying. Males: A LDT-EWS score of 5 would trigger 36.7% calls with 75.8% of all patients having a trigger subsequently dying.

to strict quality control measures. Different trigger points for males and females may be appropriate. Q13. iii. Previous studies have suggested that the complex calculations to use laboratory parameters require specific software. However, these authors suggest that the LDT-EWS can be developed into a paper-based system.

laboratory tests collected soon after hospital admission can be represented in a simple, paper-based EWS (LDT-EWS) to discriminate in-hospital mortality” (p1494).

Jones (2013) Virginia, USA Q1, RS iv. To describe the implementation of a nurse-designed, automated system for enhancing patient monitoring on medical–surgical and step-down nursing units. v. . A retrospective over-view of the implementation of a EWS system.

Q2. In 2007, the Virginia based hospital developed a paper-based EWS system, similar to others described in nursing literature, and implemented it on three nursing units however it was observed: that nurses saw “little value in the EWS system and considered it nothing more than a documentation requirement”; the EWS tool was not completed consistently or in a timely

Q6. Nurse satisfaction increased with the use of the VSA system, which eliminated excessive manual data entry required by the earlier manual paper based EWS system

Q13. Personnel to ensure timely entry of vital sign data into electronic system/handheld devices and a functioning wireless network and automated scoring. Color-coded scores on the screen savers provided the charge nurse with a readily accessible tool for continuously monitoring all patients on their unit remotely at the nurses station.

Q14 LOE3 (descriptive paper based on experiences in one centre).

209 | P a g e







manner and was resource intensive (“on a unit with 36 beds filled to capacity, this accounted for a total of 7.2 nursing hours daily, or 2,628 nursing hours annually— the equivalent of one full-time nurse”). In 2009, the EWS system was incorporated into the hospital’s electronic medical record. Subsequently a technology-assisted critical thinking (TACT) committee was created to explore monitoring technologies that could help nurses on these units. A decision was made toto retire the EWS system altogether and design something entirely new. Q3.VSA system seen as an alerting tool: Automated vital sign alert (VSA) system, colour coded VSA chart and a VSA algorithm. VSA chart: Using a simple, color-coded VSA scoring chart -- green to indicate the values within a safe target range and yellow and red to indicate caution and danger, respectively, depending on the degree of deviation. Built upon the idea of “a safe range” of vital sign values. algorithm: promote critical

210 | P a g e







thinking skills and self-confidence.

Kirkland, Malinchoc, O’Byrne et al. (2013) USA Q1. RS iv. “To create and validate a clinical deterioration prediction tool using routinely collected clinical and nursing measurements.” (p135). End point was a serious clinical event within 2 to 12 hours of variable entry. v. A retrospective case-control and cohort chart review study vi. Data on cases were collected from time of arrival until 2 hours prior to an event. Data on controls was collected from time of arrival until hospital discharge. Events were identified from the ICU database, administrative data, Morbidity and Mortality Review Group records, and the RRT committee records. Patient records were reviewed for compliance with inclusion and exclusion

Q3. i. Routinely measured ‘real world’ vital signs were considered; SBP and DBP, mean arterial pressure (MAP), HR, shock index (HR/SBP), RR, temperature, arterial oxygen saturation by pulse oximetry (SaO2), Braden Scale54, and Hendrich II Fall Risk score55. The risk score for the final model was calculated as; Score = – 0.5238 + (0.8639 × Shock index) + (– 0.0998 × Braden) + (0.0814 × RR) + (– 0.0275 × SaO2). Q4. i. RRT was activated by: staff worry; acute and persistent: declining oxygen saturations <90%; change in HR <40 or >130 bpm; change in SBP <90 mm Hg; RR <10 or >28 breaths per minute; change in conscious state including agitated delirium; acute pain chest, or new onset of

Q5. X. The ability of the models to predict an event within 2-12, 12-124 and 24-48 hours was higher in general when serial parameters were analysed instead of a single parameter entry, However for ease of use, single parameters were used in the final model In multivariate regression analysis, significant predictors of clinical deterioration within 2-12 hours were; The Braden Scale OR=0.91 (95%CI 0.84,0.98; P=0.01) RR OR=1.08 (95%CI 1.04-1.13; P< 0.01), SaO2 OR=0.97 (95%CI 0.96-0.99; P< 0.01) Shock index OR = 2.37 (95%CI 1.14-3.98; P< 0.01). In the validation dataset, the predictive ability of the model to identify patient deterioration within 2-12 hours was; Model AUROC=0.71 (95%CI 0.68, 0.74)

Q14. 2+ Single centre. Variability on vital sign monitoring, relatively small number of cases. Q16. This tool created using routinely collected clinical and nursing measurements (Shock Index, RR, SaO2 and the Braden Scale) can serve as a very early warning system for adverse events within 12 hours among hospitalized medical patients.

54

BERGSTROM, N., BRADEN, B. J., LAGUZZA, A. & HOLMAN, V. 1987. The Braden Scale for predicting pressure sore risk. Nursing Research, 36, 205-210. 55

HENDRICH, A. L., BENDER, P. S. & NYHUIS, A. 2003. Validation of the Hendrich II Fall Risk Model: a large concurrent case/control study of hospitalized patients. Applied Nursing Research, 16, 9-21.

211 | P a g e







criteria. vii. Adult patients (≥18 years) admitted to hospitalist care teams were cases (with events) or controls (no events). The derivation group was derived from patients admitted during 2008 (n=1882 of whom 68 were cases (median age 70 (57,82) years and 62% male) and 267 were controls; 3 controls were matched per case matched by age, sex, admission source and diagnosis or symptom. (median age 70 (59,83) years and 62% male)). The validation cohort was all eligible patients admitted during 2009, using the same definitions of cases and controls as the derivation group (n=1946 patients of whom 77 were cases (median age 73 (58,81) years and 43% male) and 1869 were controls (median age 68 (52,80) years and 47% male)). viii. 4 medical units I one hospital

symptoms suggestive of stroke.

Liljehult & Christensen (2015) Denmark Q1. RS iv. To investigate whether an aggregate, weighted track-

Q2. EWS, introduced as a standard tool for monitoring patients in May 201251 It encompasses HR, RR, SBP, Temp, AVPU, and SaO2

Q5. i. The EWSs calculated upon admission and the maximum EWSs were predictive of mortality at 30 days in patients with acute stroke. (P values are difference between survivors and non-survivors). The AUROC for the SSS was higher for the

Q8 i. “Sensitivity was highest for the lowest scores of both admission EWS and max EWS and decreased with rising scores, indicating that the risk of false-negative test results was greatest with the higher

Q9. viii. The authors found correlations between both stroke severity, age, and EWS, but no evidence of confounding on the

Q14. 3 One centre, small sample size. Population is representative of those suffering stroke in Denmark. Retrospective study.

212 | P a g e







and-trigger EWS can be used to identify patients at risk and predict mortality within 30 days in patients with acute stroke. v. A retrospective cohort study. vi. Vital sign data recorded on admission and during the hospital stay were systematically collected to audit the implementation of EWS. Other clinical data was extracted from electronic patient files. vii. Patients (n=274) with acute (ischemic or hemorrhagic) stroke during May to Sept 2012. Patients with subarachnoid hemorrhage were excluded, as diagnosed upon non-contrast CT scan. Stroke severity was assessed using the Scandinavian Stoke Scale (SSS). Mean age was 72.3 (SD 12.7) years and 50% were male. viii. One hospital (Stroke unit of Copenhagen University Hospital)

(scored between 0 and 3) and inspired oxygen (score of 2 for ‘any O2’ and 0 for ‘air’). Patients are stratified based on EWS score; low (EWS 0-1), medium (EWS 2-4), and high (EWS ≥5) risk. Q3. i. and ii “Patients admitted within the first 24 h of symptom onset were monitored with 2-h intervals for the first 24 h and with 4-h intervals for the subsequent 24 h. After the 48 h, all patients were monitored at intervals of at least 12 h according to the following schedule: EWS 0-1: 12-h monitoring intervals; EWS 2: 6-h monitoring intervals; EWS 3-6: 4-h monitoring intervals; and EWS 7-8: 1-h monitoring intervals. Patients with EWS ≥ 9 were monitored at least every 30 min. If EWS exceeded 2 or changed markedly, the physician on call was consulted, and if EWS exceeded 9 or single vital signs changed rapidly, a MET was immediately called.”

prediction of mortality but was not significantly different from EWS(admission P=0.44) or EWS(max P=0.16) EWS on admission AUROC=0.856 (95%CI 0.760, 0.951: P<0.001) Max EWS AUROC=0.949 (95%CI 0.919, 0.980: P<0.001) SSS AUROC=0.901 (95%CI 0.840, 0.961: P<0.001) Mortality at 30 days increased with EWS score on admission, but only those in the high risk group had a significantly high mortality rate than the overall mortality rate (P values); low (EWS 0-1): 2.4% mortality rate (P=0.004) medium (EWS 2-4); 14% mortality rate (P=0.56) high (EWS ≥5); 64% mortality rate (P<0.001) Of the individual vital signs only RR (AUROC 0.673; P 0.005) and AVPU (AUROC 0.721; P < 0.001) were significantly better at distinguishing between survivors and non-survivors than pure chance.

scores. Overall, the sensitivity was best for max EWS. Specificity was low at lower scores and increased with rising scores, indicating that the risk of false-positive test results was greatest with lower scores. Overall specificity was best in admission EWS. PPVs were low at lower scores in both admission EWS and max EWS, but increased with rising scores, whereas NPVs were high at all levels. Adjusted logistic regression models only demonstrated minor confounding effects of stroke severity and age” Admission EWS EWS 1 Sensitivity=79.2% Specificity=80.1% PPV=27.1% NPV=97.6% EWS 4 Sensitivity=50% Specificity=97.3% PPV=63.2% NPV=95.4% Max EWS EWS 1 Sensitivity=100% Specificity=45.3% PPV=14.7% NPV=100% EWS 4 Sensitivity=95.8% Specificity=87.0%

association between EWS and mortality. Therefore they did not adjust for these variables in the analysis of validity. In clinical practice, such an adjustment would not be feasible and it would still be the unadjusted EWS that would guide the clinical decision.

Q16. A EWS from readily available physiological parameters is a simple and valid tool for identifying patients at low, intermediate and high risk of dying after acute stroke. Patients with a EWS≥5 or whose EWS score increases during the admission period indicates closer observation and monitoring and increased risk of death in the later case among patient with acute stroke. This EWS could be used as a tool to select patients who need to be moved to the stroke unit.

213 | P a g e







PPV=41.1% NPV=99.5%

Liu, Koh, Goh et al. (2014b) Singapore Q1. RS iv. To identify the most significant variables to predict MACE in patients with chest pain using ML-based selection, and to compare this score with established EWS scores. MACE is defined as a composite of 4 major cardiac events and included death, cardiac arrest, sustained ventricular tachycardia and hypotension. v. Prospective observational study vi. Data was collected by ECG sensor and a data acquisition device, over 5 mins and 15 heart rate variability parameters were computed. Clinical and demographic data was extracted from ED charts. Random forests was used to select independent variables with the 29 MACE patients and 29 randomly selected patients without MACE. The process was then repeated 500 times. ML was

Q2. MEWS22, TIMI Score56 TIMI outcomes are death, AMI and revascularization within 30 days. Q3. ML score identified 23 variables; 15 heart rate variability variables and 8 clinical signs; GCS, temp, pulse rate, RR, SBP, DBP, O2 saturation and pain score. The top 8 ranked variables were all statistically significant and were; SBP, average of the instantaneous heart rate (aHR), average width of the RR interval (aRR), DBP, triangular index, ratio of LF power to HF power (LF/HF) and HF power norm.

Q5. ii. The ML model with the top thee variables had the highest predictive ability for MACE within 72 hours, than all other models with different number of variables. ML score (3 variable model; SBP, aHR, aRR) AUROC=0.812 (95%CI 0.716, 0.908) ML score (23 variable model) AUROC=0.736 (95%CI 0.630, 0.841) TIMI AUROC=0.637 (95%CI 0.526, 0.747) MEWS AUROC=0.622 (95%CI 0.511, 0.733) A model with GCS, RR, DBP, pain score, STD and avHR had a lower predictive value for MACE within 72 hours than the other models except MEWS. AUROC=0.632 (95%CI 0.564, 0.700)

Q8. i. For prediction of MACE, a ML (3 variables) score of 43 had a; Sensitivity=82.8% Sensitivity=63.4% The ML (23 variables) score of 49 had a; Sensitivity=72.4% Sensitivity=63.0%

Q9. i. SBP, avHR and aRR were the variables which achieved the best prediction of MACE. SBP was the only clinical sign required. 5-min ECG in the ED was feasible and effective in risk prediction of MACE. The optimum number of variables to include in the model is unknown.

Q14. 2- Single centre, relatively small sample, and heterogeneous endpoints. No demographic variables were considered. Q16. A ML devised model could better discriminate patients with MACE within 72 hours of ED admission from those without MACE than TIMI and MEWS. Inclusion of only 3 variables achieved the best prediction scores. “Machine learning-based variable selection seems promising in discovering a few relevant and significant measures as predictors,”

56

ANTMAN E, COHEN M, BERNINK P. J. L. M., MCCABE, C. H., HORACEK, T.,PAPUCHIS, G., MAUNTER, B., CORBALAN, R., RADLEY, D. & BRAUNWALD, E. 2000. The TIMI risk score for unstable angina/non-ST elevation MI—A method for prognostication and therapeutic decision making. JAMA, 284, 835-842.

214 | P a g e







used to build prediction models. The risk score ranges from 0 to 100, with a higher score indicating higher risk. vii. Convenience sample of patients (≥30 years) presenting to the ED with undifferentiated non-traumatic chest pain (n=702), of whom 29 met the primary outcome i.e. MACE within 72 hours of presentation at the ED. Mean age was 61.0 and 60.6 for patients with and without MACE, 62.1% and 66.1% were male, respectively. viii. One tertiary hospital.

Liu, Koh, Chua et al. (2014a) Singapore Q1. RS iv. To develop a novel intelligent scoring system for the early identification of patients at high risk of cardiac arrest within 72 hours, using ECG and vital signs, and to compare it with established EWS. “A hybrid sampling-based ensemble learning strategy is proposed to handle data imbalance” (p1894). Primary outcome was MACE (death, cardiac

Q2. MEWS45, TIMI, and DIST57 Scores Q3. i. Ensemble-Based Scoring System (ESS): 16 HRT, 12 ECG and 8 vital sign parameters were considered for the derivation and validation of the model. An intelligent scoring system was used, which employs ML methods which can improve predictive performance, handle imbalanced data and

Q5. ii. The proposed scoring system ESS was superior at predicting acute cardiac complications within 72 hours than the other scoring systems analysed. ESS AUROC=0.837 (95%CI 0.724, 0.949) DIST AUROC=0.720 (95%CI 0.588, 0.852) MEWS AUROC=0.672 (95%CI 0.537, 0.808) TIMI AUROC=0.621 (95%CI 0.484, 0.757). The ESS predictor parameters which had the best discriminatory ability were; *ECG, HRV, Vital signs AUROC=0.837 (95%CI 0.724, 0.949). ECG, HRV,

Q8 i. ESS had better combination of sensitivity and specificity for the prediction of acute cardiac complications within 72 hours. The low PPV and NPV is due to imbalanced data i.e. there are more patients without complications. ESS had the better discriminatory ability to distinguish patients with and without complications, but it could be enhanced further. DIST also had good discriminatory ability assigning high scores to more patients with complications and low scores to those without. MEWS and DIST had less discriminatory ability.

Q9. i. The ESS predictor models with HRV and vital signs performed least well of the combination of predictors suggesting that the 12-lead ECG is a significant predictor of acute cardiac complications within 72 hours. However, neither the specific ECG parameters nor the vital signs which most contribute to the predictive ability of ESS are unknown. ML techniques are superior

Q14. 2+ Single centre. Potential errors in data collection are possible. Requires validation on another dataset. Q16. In ED patients with chest pain, the12-lead ECG combined with HRV and vital signs were found to strongly associate with acute cardiac complications within 72 h. A novel scoring method ESS has been proposed to integrate multiple sources of predictors for risk stratification, which

57

LIU, N., LIN, Z., CAO, J., KOH, Z., ZHANG, T., HUANG, G. B., SER, W. & ONG ME. 2012. An intelligent scoring system and its application to cardiac arrest prediction. IEEE Transactions on Information Technology in Biomedicine, 16, 1324-1331.

215 | P a g e







arrest, sustained ventricular tachycardia, hypotension). v. An observational, cohort study vi. ECG data was collected by an ECG sensor over 5 mins and 16 HRV parameters were derived, and with a 12-lead ECG and 12 parameters were selected. Vital signs were recorded and the TIMI score was calculated in the ED. A MEWS was calculated from collected data. vii. 564 adult (≥30) patients presenting at the ED with chest pain during Mar 2010 and April 2012. Patient Acuity Category Scale (PACS) 1 (most critically ill, requiring resuscitation) and some PACS 2 (critical but not in imminent danger of collapse) were screened and recruited to the study. Patients were categorised a shaving complications (n=19; mean age 61.1 (SD 12.5), 63.2% male) and no complications (n=545, mean age 60.3 (SD 13.1), 60.3% male). viii. One acute tertiary hospital ED.

enhance system adaptability was used to develop the model. Imbalanced data occurs when the prevalence of the outcome is low in the cohort. An Ensemble Learning-Based Prediction Score which seeks to mimic real life decision making situations where experts opinion are weighted according to experience/knowledge, was used to give a predictive label/risk score.

AUROC=0.812 (95%CI 0.694, 0.930). ECG, Vital signs AUROC=0.815 (95%CI 0.697, 0.932). HRV, Vital signs AUROC=0.759 (95%CI 0.632, 0.886). ^The 12-lead ECG parameters

ESS cutoff score 42.3 (range 0-100) Sensitivity=78.9% Specificity=76.5% PPV=10.5% NPV=99.0% DIST cutoff score 51.8 (range 0-100) Sensitivity=63.2% Specificity=82.9% PPV=11.4% NPV=98.5% MEWS cutoff score 1.0 (range 0-6) Sensitivity=42.1% Specificity=78.5% PPV=6.4% NPV=97.5% TIMI cutoff score 1.0 (range 0-6) Sensitivity=78.9% Specificity=36.7% PPV=4.2% NPV=98.0% The ESS predictor parameters which had the best discriminatory ability were; ECG, HRV, Vital signs Sensitivity=78.9% Specificity=76.5% PPV=10.5% NPV=99.0% ECG, HRV Sensitivity=78.9% Specificity=73.8% PPV=9.5%

to traditional statistical methods for model derivation and may be used to enhance medical decision making. “The ESS method simulates the scenarios in real-world medical settings where more than one opinion is sought before making final decisions. The weights in the ESS algorithm indicate the contribution of their corresponding classifiers and the determination of the weights are derived from a novel hybrid approach. In the ESS algorithm, large weights strengthen the power of prediction while small weights weaken the power, which is a strategy refined from the one suggested in that many individual classifiers in the ensemble could be better than all for decision making. The novelty of the ESS algorithm is its hybrid-sampling-based optimization and its ability in handling imbalanced data.” (p1900-1901).

showed superior performance compared with several existing methods such as TIMI], MEWS and an intelligent scoring method DIST.” (p 1900).

216 | P a g e







NPV=99.0% ECG, Vital signs Sensitivity=78.9% Specificity=74.3% PPV=9.7% NPV=99.0%

Martin, Dupré, Mulliez et al. France (2015) Q1. RS

v. To validate the DULK diagnostic score for anastomotic leakage. vi. Prospective study vii Data collected June 2012and June 2013 from patient charts. viii. 100 patients undergoing elective colorectal surgery

Q2. DULK score Q3. DULK score based upon physiological observations, urinary output, state of restlessness, presence of specific complications, blood results, and nutrition details. A score of ≤3 points requires surveillance and above that specific interventions.

Q8 i. A DULK threshold score >3, offered the best test sensitivity i.e. 91.67% a specificity of 55.68%, a NPV of 98% and a PPV of 22%, and an AUROC curve of 0.83 (p8). Routine use of the DULK score would allow earlier diagnosis of anastomotic leakage. i.e. 3.5 days earlier that is in comparison to routine clinical judgment alone.

Q14. 2-

Q16. Out of the 100 patients, 12% developed a post-operative anastomotic leakage with a specific mortality rate of 16.6% (2 patients).

Mora, Schneider, Robbins et al. (2015) Australia Q1. RS iv. To determine whether patients who will experience physiological deterioration can be identified prior to an RRT call. v. A retrospective, case-controlled study vi. Data on demographics vital signs of patients who trigger an RRT with 24 h of ED admission (cases) and matched controls who did

Q5. i. Cases who triggered an RRT with 24 h had a significantly higher odd of death; OR 4.65 (95%CI 1.86, 11.65) x. There were 154 RRT calls with 24 h of admission. Cases had a significantly higher HR at triage, after 3 h in the ED and at discharge than controls. OR: 1.02 (95%CI 1.02, 1.12) for each beat/min increase in HR prior to ward transfer RR was higher in cases than controls OR: 1.07 95%CI 1.002, 1.030) for each 1 breath/min increase in RR

Q14. 2+ Individual vital signs used. EWS was not investigated. Single centre. Q16. Patients who trigger an RRT call within 24 h are at 4-times higher risk of in-hospital mortality. Patients at higher risk of triggering an RRT activation could be identified through higher RR and HR in the ED.

217 | P a g e







not trigger an RRT response. vii. 154 cases (mean age 68.12 (SD 17.82; 38% male) and 154 controls (mean age 66.81 (SD 19.43; 38% male) were recruited. viii. The ED in one hospital

Ong, Lee Ng, Goh et al. (2012) Singapore Q1. RS iv. To validate a novel ML score, incorporating HRV against MEWS for the risk stratification of critically ill patients in the ED. v. A prospective, non-randomised, observational cohort study vi. ECG tracings (for a minimum of 5 mins in the ED) were measured in real-time, initial vital signs taken during triage, demographics and medical history data were extracted from medical records. vii. 925 adult (≥18 years) patients requiring continuous ECG monitoring with PACS 1 or PACS 2 were recruited during June 2006 and June 2008. Of these 43 (4.6%;

Q2. An ML-based prediction model58

Q5 i. The ML score had better discriminatory ability to predict in-hospital death within 72 h than MEWS; ML score AUROC= 0.741, MEWS AUROC= 0.693 In the low, intermediate and high risk ML score groups, the rate of death within 72 hours increased from 2.3%, to 29.1% and 68.6%, respectively. Median MEWS score was significantly higher for patients experiencing cardiac arrest (4 (IQR 2 to 5) than those who did not (2 (IQR 1 to 4; P<0.001). ii. The ML score had a significantly better discriminatory ability to predict cardiac arrest within 72 h than MEWS (P=0.037) ML score AUROC= 0.781, MEWS AUROC= 0.680 In the low, intermediate and high risk ML score groups, the rate of cardiac arrest within 72 hours increased from 0%, to 1.6% and 13.1%, respectively. Median MEWS score was significantly higher for patients experiencing cardiac arrest (4 (IQR 2 to 5) than those who did not (2 (IQR 1 to 4; P<0.001).

Q8. i. A cutoff ML score ≥ 60 predicted death within 72 h with a; Sensitivity=69.8%, Specificity=73.9% PPV=21.5% NPV=96.0%. A cutoff ML score ≥ 60 predicted cardiac arrest within 72 h with a; Sensitivity=84.1%, Specificity=72.3% PPV=12.5% NPV=98.8%. A cutoff MEWS ≥3 predicted death within 72 h with a; Sensitivity=74.4%, Specificity=55.7% PPV=14.7% NPV=95.5%. A cutoff MEWS ≥ 3 predicted cardiac arrest within 72 h with a; Sensitivity=74.4%, Specificity=54.2%

Q9. The ML score is objective and can be determined in the ED. Depressed HRV parameters were associated with adverse events within 72 hours.

Q14. 2- Single centre. Patients presented with a heterogeneous array of conditions which may have influenced the scores. Patients with non-sinus rhythm were excluded. The ML score is based on one measurement. Q16. ML scores incorporating HRV parameters, age and vital signs were more accurate than the MEWS in predicting cardiac arrest within 72 hours. There is potential to develop bedside devices for risk stratification based on cardiac arrest prediction

58

LIU, N., LIN, Z., KOH, Z., HUANG, G. B., SER, W., ONG, M. E. H. 2011. Patient outcome prediction with heart rate variability and vital signs. Journal of Signal Processing Systems, 64, 265-278.

218 | P a g e







median age 70 (IQR 59-78, 72.1% male) had a cardiac arrest within 72 h and 882 did not (median age 62 (IQR 50-74, 61.5% male). viii. The ED in one hospital (Singapore General Hospital)

PPV=7.4% NPV=97.8%.

Naidoo, Rangiah & Naidoo (2014) South Africa Q1. RS v. To evaluate the use of the TEWS by healthcare workers in an ED. vi. A retrospective observational study vii/viii Study data collected from charts in 2011. N=265 patient records. Mean age (SD) was 41.4 (17.05) years ix. One urban hospital central Ethekwini District, South Africa.

Q2. An adapted version of the MEWS to include mobility and trauma parameters in response to local emergency department termed the Triage Early Warning Score (TEWS). Q3. Physiological indicators – mobility, resting rate, heart rate, systolic BP, temp, AVPU, Trauma.

Q5. Variables were cross-tabulated and a significant association (p-value 0.032) was observed between hospital and in-hospital deaths.

Q8.iv. 53.7% of patients with a TEWS of <7 were discharged and the remaining 46.3% admitted.

Q14. 3 Q16. Authors concluded that an effective triage scoring system ensures the appropriate categorisation of those requiring emergency care i.e. those at risk of clinical deterioration.

Smith, Den Hartog, Moerman et al. (2012) The Netherlands Q1. RS v. To investigate the relationship between the EWS and the occurrence of major adverse events in surgical patients during their stay on a general and trauma surgical ward. vi. Prospective cohort study,

Q2. EWS from Dutch CBO guideline. Q3 i The EWS of patients were determined during the clinical round in the morning, afternoon and evening. The EWS scores were dichotomized into EWS ˂3 versus ≥3 based upon the recommendations of the Dutch CBO guideline.

Q5 i. The cumulative incidence of adverse events during hospitalization was 8·0 %. Patients with an EWS ≥3 were shown to have a significantly higher risk of reaching the combined endpoint (death, reanimation, unexpected ICU admission, emergency operations and severe complications) i.e. EWS ≥3 compared with patients with an EWS ˂3: (OR 12·9, 95%CI 6·4, 25·7) and when adjusted for baseline ASA classification, the odds ratio was 11·3, 95 % CI: 5·5 to 22·9). The AUROC was 0·87 (95 %CI: 0·81 to 0·93). The negative predictive value of an EWS ≥3 was 97%.

Q8. An EWS ≥3 as a positive test result equated to a sensitivity of 74 % and specificity of 82 %. Whilst an EWS≥4 as a positive test result equated to a sensitivity of 54% and specificity of 94%.

Q14. 2+ Consecutive sample of patients- thus enhancing transferability of findings. Q16. An EWS core ≥3 is an independent predictor of major adverse events in patients admitted to a general and trauma surgery war

219 | P a g e







Logistic regression analysis - to assess the relationship between the EWS and the occurrence of death, reanimation, unexpected ICU admission, emergency surgery and severe complications. vii Data collected March- September 2009. viii. Patients (n=572) admitted to a general and trauma surgery ward. Average age of the study population was 50 years. 57.5% ASA grade I. Ix Single Dutch university hospital, Netherlands

ii. Clinical evaluation of the patient’s condition by physician was advised if the EWS was ≥3 Q4. American Society of Anesthesiologists physical status classification (ASA grade).

Umscheld, Betesh, Vanzandbergen et al. (2014) USA Q1. RS iv. To develop, implement and validate an electronic sepsis detection and response system to improve patient outcomes; EWRS: v. Retrospective development and prospective validation, and pre-, post-intervention study vi. All adult patients (≥18 years) derivation cohort to establish trigger thresholds was vii. All adult patients (≥18 years) admitted 1st to 31st Oct

Q2. The EWRS: designed to monitor vital signs and laboratory results in real time. Criteria for severe sepsis was established including; a. Systemic inflammatory response syndrome criteria; Temp <36/>380C; HR >90 bpm RR >20 breaths/min or PaCO2 <32 mmHg. WBC <4000 or >12000 or >10% bands, and b. organ dysfunction criteria: cardiovascular dysfunction based on SBP <100 mmHg and hypoperfusion based on

Q5. i. EWRS score of ≥4 were 10 time more likely to die. Mortality decreased post-intervention but not significantly (8% and 9%, P=0.45) v. Post implementation, there was a significant increase in ordering antibiotics, intravenous fluid boluses and lactate and blood cultures within 3 h of the trigger. x. Composite outcome of (i) transfer to ICU, (ii) RRT call, and/or (iii) death EWRS score of ≥4 = 3.9% of these patients x. Discharge to home increased significantly post-intervention (64% v 58%, P=0.04). x. Sepsis discharge diagnosis increased post-intervention (45% v 39%; P=0.02)

Q7. i. EWRS score of ≥4 were 4 times more likely to be transferred to ICU No significant difference in the number of patients transferred to ICU pre- and post-intervention. The proportion of patients transferred to ICU within 6 h of the alert increased post-intervention (10% vs 7%, P=0.06) iii. No significant difference in hospital LOS pre- and post-intervention. Q8. i. EWRS score of ≥4 for the composite outcome; Derivation cohort:

Q13. The was integrated into a commercial HER.

Q14. 2+ Multi-hospital study Q16. The EWRS can accurately identify non-ICU inpatients at increased risk of deterioration or death and it is feasible to use it in real time to trigger a RRT. Post-implementation early sepsis care improved. “An automated prediction tool identified at-risk patients and prompted a bedside evaluation resulting in more timely sepsis care, improved documentation, and a suggestion of reduced mortality.” (p26).

220 | P a g e







2011 (n=4575; no. encounters 15,567) , were used as the derivation cohort to establish trigger thresholds for EWRS. Mean age was 62 (48.5-70.5) years and 50% were male. Post-implementation ‘Live’ June 6th to Sept 4th 2013. No. of encounters =15,526, mean age 59.7 (46.1-69.6), 50% male. viii. 3 hospitals within the University Pennsylvania Health System.

serum lactate >2.2mmol/L. A risk score was calculated for all patients in the derivation cohort i.e. sum of the criteria above (to a maximum of 6) at one time for the outcomes: (i) transfer to ICU, (ii) RRT call, (iii) death, (iv) composite of i-iii, or (v) sepsis on discharge. Q3. i. Upon EWRS implementation all patients were continuously screened, a ‘text page’ alert was sent to RRT for patients meeting EWRS criteria and nurses were alerted by ‘pop-up’ notification on HER Q4. i. Efferent arm: The RRT consisted of the covering doctor, bedside nurse, and rapid response coordinators. The team had to evaluate the patient within 30 min of the alert

Q6. i. EWRS score of ≥4 were 7 times more likely to experience a RRT Number of alerts decreased post-implementation (3.8% to 3.5%) Post-implementation 99% of coordinator pages and three quarters of nurse notifications were sent successfully.

to screen positive=6% Sensitivity =16% Specificity=97% PPV=24% NPV=94% Likelihood ratio (positive)=5.3 Likelihood ratio (negative)=0.9 Validation cohort: to screen positive=6% Sensitivity =17% Specificity=97% PPV=28% NPV=95% Likelihood ratio (positive)=5.7 Likelihood ratio (negative)=0.9

Tarassenko, Clifton, Pinsky et al. (2011). UK and US Q1. RS

iv. “To develop an early

warning score (EWS) system

Q3. “The alerting system was constructed using the hypothesis that an EWS of 3 (which, in most systems, initiates a review of the patient) should be generated when a vital sign is below the 1st centile or above the 99th

Q7vi. If used in clinical practice based upon data used in the development of the tool (with high risk patients), with four-hourly observations in a 12-h shift, about 1 in 8 at-risk patients would trigger the alerting system during the 12 hour shift. Q8. i.

221 | P a g e







based on the statistical properties of the vital signs in at-risk hospitalised patients” (p1013) through the statistical determination of the ranges of normality, for each vital sign, in the at-risk populations. v. Retrospective analysis of data generated from high-risk hospitalised medical and surgical patients. “Normalised histograms (unit area under the curve) and cumulative distribution functions were plotted for each physiological variable (HR, RR, SpO2 and SBP), for each study and for the 3 datasets combined” (p1014). vi. Data collected using bedside monitors from 2004 and 2008 as part of 3 clinical studies (US and UK data). vii. Dataset = 64,622 hours of vital-sign data, attained from 863 acutely ill inpatients. viii. UK and US hospitals.

centile for that variable (for a double-sided distribution)” pg.1014. Assumption that alerts occur whenever a “score of 3 is assigned to a single variable and a score of ≥4 for the multivariate case” (p1014).

Table (reproduced from publication) - represents the values given to vital signs within the developed EWS. New EWS differs most in respect of Respiratory rate and systolic blood pressure values. Q14. 3 Q16. Authors highlight issues with choosing mortality or ICU admission as the outcome noting that there is no obvious binary outcome for early deterioration. Notably the sample used were mainly elderly patients who were designated as high-risk patients being cared for in areas of intensified monitoring and most probably treatment. The majority of “track and trigger” scores have been developed by using AUROCs using critical events (death, intensive care admission or cardio-pulmonary arrest). Tarassenko et al. (2011) take an alternate approach by analysing the frequency distribution of physiological observations and thereby defining scores purely by the degree of the difference from the statistical mean.

222 | P a g e

Table 3. Multiple comparisons of EWS systems Authors, (date) Country Q1. Type of Evidence






Cattermole, Liow, Graham et al. (2014) Hong Kong Q1. RS Iv. “(1) to validate the Prince of Wales ED Score (PEDS) in comparison with other prognostic scores: MEWS, SCS, REMS, MEES, MEDS (Mortality in the ED Sepsis), Rapid Emergency Medicine Score (REMS), Worthing and NEWS33; (2) to simplify and refine the score, using only variables immediately available in the resuscitation room (including lactate); (3) to validate this new score in the dataset previously used to derive the original PEDS.” (p 804). Primary outcome was ICU admission or death within 7 days of attendance at an ED; secondary outcome was 30 day mortality and hospital LOS. v. A prospective observational study of adult resuscitation-room patients over 3 months” vi. All required clinical,

Q2.”PEDS was derived specifically for patients in an ED resuscitation room to predict death or intensive care unit (ICU) admission.“59

PEDS includes 6 variables: “systolic blood pressure (SBP), GCS, blood glucose, serum bicarbonate (HCO3

–), leukocyte count and a history of metastases. The most important factors were GCS<8, and HCO3

–<22 mmol/L.” PEDS <15 = Low-risk PEDS 15-19 = moderate-risk PEDS >29 = high-risk patients. The Resuscitation Management Score (THERM) was developed from “independently significant variables were used to construct a new scoring tool, based on the ORs derived from logistic regression, and pragmatically according to ease of use. THERM includes 3 items: GCS, HCO3

– and SBP. Scored as;

Q5. x. The AUROC for PEDS decreased from that in the development study (0.91 (95%CI 0.87-0.94)) to the current validation study (0.75 (95% CI 0.69-0.80)) In this validation study PEDS performed adequately but was not superior to other scores. AUROC for each EWS for predicting the composite output of ICU admission or death within 7 days was;

Worthing 0.78 (95% CI 0.72-0.83 MEES: 0.75 (95% CI 0.69-0.80) PEDS: 0.75 (95% CI 0.69-0.80) MEWS: 0.73 (95% CI 0.67-0.79) NEWS: 0.71 (95% CI 0.64-0.76) REMS: 0.70 (95% CI 0.64-0.76) SCS: 0.70 (95% CI 0.64-0.76) MEDS: 0.59 (95% CI 0.52-0.6)

THERM outperformed NEWS in both derivation and validation in patient datasets. AUROC THERM: 0.84 (95% CI 0.786 to 0.884) THERM was significantly better than NEWS, REMS, SCS and MEDS.

Q8. Max THERM score is 37 THERM, high risk cut-off (≤30) Sensitivity = 0.57 (95% CI 0.40-0.73) Specificity = 0.89 (95% CI 0.84-0.93) PPV = 0.50 (95% CI 0.34-0.66) NPP = 0.92 (95% CI 0.87-0.95) THERM, Medium risk cut-off (30.1-35) Sensitivity = 0.89 (95% CI 0.75-0.97) Specificity = 0.65 (95% CI 0.58-0.72) PPV = 0.32 (95% CI 0.23-0.42) NPP = 0.97 (95% CI 0.92-0.99) NEWS, high risk cut-off Sensitivity = 0.65 (95% CI 0.48-0.80) Specificity = 0.71 (95% CI 0.64-0.77) PPV = 0.29 (95% CI 0.20-0.40) NPP = 0.91 (95% CI 0.86-0.95) NEWS, Medium risk cut-off Sensitivity = 0.92 (95% CI 0.78-0.98) Specificity = 0.44 (95% CI 0.37-0.51) PPV = 0.24 (95% CI 0.17-0.31) NPP = 0.97 (95% CI 0.91-0.99) Q12.iii. “Using this composite outcome, may minimise the effect that different ICU admission-policies on the death rate. From the perspective of the ED clinician, any patient who dies or needs ICU is very unwell, and this is the group

Q9. THERM has advantages over NEWS and MEWS which makes it more applicable to the ED; (i) NEWS is entirely physiological not requiring bedside blood tests. (ii) “The scoring of some of the parameters in NEWS also limits its usefulness in the ED. Use of supplemental O2 scores highly, which is reasonable for stable inpatients. But many ambulance and resuscitation-room patients are routinely given oxygen initially, with subsequent titration or removal. To include this as part of the first-look score in the ED would limit its discriminatory function. NEWS does not discriminate between degrees of reduced consciousness; anything below ‘alert’ on the alert, to voice, to pain, or unconscious (AVPU) scale indicates medium risk, regardless of other parameters. Again, in the resuscitation room this

Q14. 2- One hospital ED. Primary outcome was a composite of ICU admission or death within 7 days of attendance at an ED, which are not ‘independent’ outcomes. Q15. “A decrease in performance between derivation and validation is expected, as any derived rule will best reflect the dataset from which it is derived. This is especially the case with smaller studies, and reinforces the need to test a new rule in an independent validation set.” Q16. “PEDS is at least as good as other scores, including NEWS. However, it is unwieldy and relies on results not immediately accessible in the ED. THERM is a new score, derived and validated in an ED setting, using variables readily available, and simple to calculate and stratify. THERM outperforms NEWS and could be used in preference in

59

CATTERMOLE, G. N., MAK, S. K., LIOW, C. H., HO, M. F., HUNG, K. Y., KEUNG, K. M., LI, H. M., CRAHAM, C. A., RAINER, T. H. 2009. Derivation of a prognostic score for identifying critically ill patients in an emergency department resuscitation room. Resuscitation, 80, 1000-1005.

223 | P a g e







demographic and physiological data were collected in real time by dedicated study staff and entered into a standard form. vii. 234 consecutive adult (≥18 years) patients attending the resuscitation room during weekdays during 3 months in 2009. Mean age 65.8 (SD 18.1) years, 41.5% were female. viii. 1 hospital in Hong Kong.

[GCS]+[ HCO3–]. Subtract 4 if

hypotensive (defined as SBP <100mmHg). Max. score=37 Low risk=35.1-37 Medium risk=30.1-35.0 High risk=≤30

that THERM seeks to identify.” would have little discriminatory value. The appropriateness of using NEWS in ED patients (and in the prehospital setting) needs to be questioned.”

critically ill ED patients.”

Churpek, Yuen, Edelson (2013) USA Q1 RS iv. To (i) discuss recently developed and validated risk scores for use on the general in-patient wards, and (ii) to compare newly developed risk scoring systems with systems in a trial database of ward admissions. v. A retrospective study vi. Database of 59,643 hospital admissions. Ward

Q2. Compared the efficacy of multiple risk scoring systems to predict adverse events; Single parameter systems: MERIT and modified MERIT60 Multiple-parameter systems61: Aggregate Weighted Systems VIEWS, MEWS45, SEWS, CART, Worthington physiologic scoring system, and a centile-based system62

Q5. i. AUROCs were highest for mortality. SEWS, VIEWS, and CART were similar in their prediction of mortality (AUROC=0.88 for all). MERIT AUROC = 0.74 (95%CI 0.71-0.76) Modified MERIT = 0.79 (95%CI 0.76-0.81) Multiple parameter (Bleyer) = 0.84 (95%CI 0.82-0.87) Centile-based system = 0.83 (95%CI 0.80-0.86) MEWS = 0.87 (95%CI 0.84-0.89) SEWS = 0.88 (95%CI 0.86-0.90) VIEWS = 0.88 (95%CI 0.86-0.91) CART Score = 0.88 (95%CI 0.86-0.90)

Q7. i. AUROCs were lowest for ICU transfer. CART score was best for predicting ICU transfer MERIT AUROC = 0.64 (95%CI 0.63-0.65) Modified MERIT = 0.69 (95%CI 0.68-0.70) Multiple parameter (Bleyer) = 0.72 (95%CI 0.71-0.73) Centile-based system = 0.71 (95%CI 0.69-0.72) MEWS = 0.74 (95%CI 0.73-0.85) SEWS = 0.75 (95%CI 0.74-0.76) VIEWS = 0.73 (95%CI 0.72-0.75)

Q9. Aggregate weighted scoring systems outperformed the other systems for most outcomes, with the SEWS, MEWS, ViEWS, and CART score being the most accurate for detecting cardiac arrest, mortality, ICU transfer and a composite of the three outcomes. Single-parameter scoring systems had the lowest predictive accuracy but the modified MERIT criteria were more accurate

Q14. 3 Limited to objective vital sign based risk scoring systems for adult patients on general hospital wards. Q16. The authors found a wide range of accuracy across outcomes for a given system and across systems. “Selection of a risk score for a hospital or health-care system should be guided by available variables, calculation method and system resources. Once

60

CRETIKOS, M., CHEN, J., HILLMAN, K., BELLOMO, R., FINFER, S., & FLABOURIS, A. 2007. The objective medical emergency team activation criteria: a case-control study. Resuscitation, 73, 62-72. 61

BLEYER, A. J., VIDYA, S., RUSSELL, G. B., JONES, C. M., SUJATA, L., DAEIHAGH, P. & HIRE, D. 2011. Longitudinal analysis of one million vital signs in patients in an academic medical center. Resuscitation, 82, 1387-1392. 62

TARASSENKO, L., CLIFTON, D. A., PINSKY, M. R., HRAVNAK, M. T., WOODS, J. R. & WATKINSON, P. J. 2011. Centile-based early warning scores derived from statistical distributions of vital signs. Resuscitation, 82, 1013-1018.

224 | P a g e







vital signs were extracted from the electronic health record and the EWS score for each risk scoring system under investigation was calculated in the whole dataset. The highest score prior to the adverse event or during admission for patients who did not experience an adverse event was used in calculations of accuracy. vii. 59,643 medical and surgical ward admissions from Nov 2008 to 2011. Mean age was 55±18 years, 44% were male. viii. 1 urban academic centre

Q4. i. A RRT, separate to the cardiac arrest team, was introduced in 2008. It is critical care nurse led. Respiratory therapy responds to team activations with an attending physician and/or pharmacist available when requested

ii. CART score was best for predicting cardiac arrest MERIT AUROC = 0.63 (95%CI 0.59-0.68) Modified MERIT = 0.69 (95%CI 0.65-0.74) Multiple parameter (Bleyer) = 0.73 (95%CI 0.68-0.78) Centile-based system = 0.70 (95%CI 0.65-0.76) MEWS = 0.76 (95%CI 0.71-0.81) SEWS = 0.76 (95%CI 0.71-0.81) VIEWS = 0.77 (95%CI 0.72-0.82) CART Score = 0.83 (95%CI 0.79-0.86) x. CART score was best for predicting a composite outcome of cardiac arrest, ICU transfer and mortality. MERIT AUROC = 0.64 (95%CI 0.64-0.65) Modified MERIT = 0.70 (95%CI 0.69-0.70) Multiple parameter (Bleyer) = 0.73 (95%CI 0.72-0.74) Centile-based system = 0.72 (95%CI 0.70-0.73) MEWS = 0.75 (95%CI 0.74-0.76) SEWS = 0.76 (95%CI 0.75-0.77) VIEWS = 0.75 (95%CI 0.74-0.76) CART Score = 0.78 (95%CI 0.77-0.79)

CART Score = 0.77 (95%CI 0.76-0.78) Q9. The authors did a prospective validation of CART using only the patients in the dataset not in the original study used to develop CART. AUROC mortality =0.87 AUROC cardiac arrest =0.86 AUROC ICU transfer= 0.76 AUROC composite outcome= 0.77 CART had the highest sensitivity at a specificity of 90% Sensitivity (%) SEWS >3 =55 SEWS >4 =38 SEWS >5 =19 MEWS >3 =67 MEWS >4 =39 MEWS >5 =20 ViEWS >3 =60 ViEWS >4 =41 ViEWS >5 =29 CART >3 =61 CART >4 =49 CART >5 =35 Specificity (%) SEWS >3 =85 SEWS >4 =94 SEWS >5 =97 MEWS >3 =80 MEWS >4 =91 MEWS >5 =96

than the original MERIT criteria for all outcomes.

implemented, ensuring high levels of adherence and tying them to specific levels of interventions, such as activation of a RRT are necessary to allow for the greatest potential to improve patient outcome.”

225 | P a g e







ViEWS >3 =83 ViEWS >4 =91 ViEWS >5 =95 CART >3 =84 CART >4 =90 CART >5 = 95

Dundar, Ergin, Karamercan et al. (2015) Turkey Q1. RS iv. To evaluate the utility of MEWS and ViEWS in predicting both hospitalisation and in-hospital mortality among geriatric patients presenting to the ED with non-traumatic acute surgical or medical diseases. v. A prospective observational study vi. All data collected on admission are recorded in a single patient chart. Data on initial symptoms and physiological parameters was extracted from these charts during 15th Jan and 15th Feb 2014. MEWS and ViEWS were calculated from physiological data recorded. Patients were followed up for 28 days. vii. All geriatric (≥65 years) patients presenting at the ED (n=671). Median age was 75 (IQR 11) years, and 55.9% were male. 48.7% were

Q2. MEWS45and ViEWS51

Q3. i. MEWS highest possible score = 14 ViEWS highest possible score = 21

Q5. i. In-hospital mortality rate of elderly patients presenting to ED was 8.5% In-hospital mortality was predicted by both MEWS and ViEWS, with similar high discriminatory ability (not statistically different). Optimal cut-off scores were 4 and 8 for MEWS and ViEWS, respectively. MEWS AUROC=0.891 (95% CI 0.844, 0.937) ViEWS AUROC=0.900 (95% CI 0.860, 0.941) x. Hospital admission was predicted by both MEWS and ViEWS, with similar discriminatory ability (not statistically different). Optimal cut-off scores were 3 and 6 for MEWS and ViEWS, respectively. MEWS AUROC=0.727 (95% CI 0.689, 0.765) ViEWS AUROC=0.756 (95% CI 0.720, 0.792)

Q8. i. The sensitivity and specificity for hospitalisation was slightly higher for ViEWS than MEWS at similar specificity MEWS score 3 Sensitivity=42% Specificity=89% LR+=3.73 LR-=0.65 ViEWS score 6 Sensitivity=56% Specificity=85% LR+=3.82 LR-=0.51 i. The sensitivity and specificity for in-hospital mortality was slightly higher for ViEWS than MEWS at similar specificity MEWS score 4 Sensitivity=74% Specificity=89% LR+=6.96 LR-=0.29 ViEWS score 8 Sensitivity=84% Specificity=83% LR+=4.92 LR-=0.19

Q9. i. Neither MEWS nor ViEWS could discriminate between geriatric patients who died in the ED from those admitted to the ICU possibly because of the low numbers of deaths in the ED in this cohort (n=4).

Q14. 2- Single centre. Medications and comorbidities and mode of transport to the hospital were not considered. Only first vital signs were used to calculate ViEWS and MEWS, changes were not evaluated. Q16. Both MEWS and ViEWS are easy to use and can predict discharge, hospitalisation and in-hospital mortality among geriatric patients attending the ED with similar performances.

226 | P a g e







discharged, 27.9% and 22.8% were admitted to the ward and ICU respectively. viii. One university hospital

In-hospital mortality rate increase with increasing MEWS score; MEWS 3: 8.0% MEWS 4: 27.9% MEWS 6: 38.9% MEWS 7: 80.0% Risk stratification: ≤4, 4-6, ≥6. A cut-off score of ≥4 is recommended in this patient group In-hospital mortality rate increase with increasing ViEWS score; ViEWS 7: 2.5% ViEWS 8: 18.2% ViEWS 12: 31.6% MEWS 13: 62.5% Risk stratification: ≤8, 8-12, ≥12. A cut-off score of ≥8 is recommended in this patient group

Geier, Popp, Greve et al. (2013) Germany Q1. RS iv. To investigate the diagnostic and prognostic accuracy of the ESI, MEWS and MEDS regarding SSSS. v. A prospective

Q2. ESI63, MEWS (score 0-14, with score ≥5 triggering an alarm)45 and MEDS (max score =27)65. A 5-level ESI was in use since 2009, with a higher score indicating lower treatment urgency.

Q5. i. MEDS had the highest in-hospital 28-day mortality of patients with suspected sepsis (Prognostic accuracy). A high MEWS score is not associated with mortality of this patient group. ESI AUROC=0.617 (95%CI 0.479, 0.755) MEWS AUROC=0.642 (95%CI 0.517, 0.768) MEDS AUROC=0.871 (95%CI 0.796, 0.945)

Q6. There was a low rate of transfer to the ICU when stratified by MEDS score; 7.6%, 11.4% and 33.3% of patients in the low, moderate and high risk MEDS groups were transferred to ICU. The low rate transfer to ICU may be due to a lack of awareness of the high risk of mortality, older patient population, or

Q9. viii. “MEDS score is based on clinical criteria that consider organ dysfunction (e.g. tachypnea or hypoxia, presence of septic shock, platelet count <150,000/mm3, and altered mental status) as well as age,

Q14. 2- Single centre. Small sample size. Not every MEDS category is available at triage management, including number of platelets. Q16. ESI and MEWS do not identify patients with sepsis

63

CHRIST, M., GROSSMANN, F., WINTER, D., BINGISSER, R. & PLATZ, E. 2010. Modern triage in the emergency department. Deutsches Ärzteblatt International, 107, 892-898.

http://www.ncbi.nlm.nih.gov/pubmed/?term=Christ%20M%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

http://www.ncbi.nlm.nih.gov/pubmed/?term=Grossmann%20F%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

http://www.ncbi.nlm.nih.gov/pubmed/?term=Winter%20D%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

http://www.ncbi.nlm.nih.gov/pubmed/?term=Bingisser%20R%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

http://www.ncbi.nlm.nih.gov/pubmed/?term=Platz%20E%5BAuthor%5D&cauthor=true&cauthor_uid=21246025

https://www.google.ie/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&uact=8&ved=0ahUKEwjkxLbwkfDLAhVCuw8KHaRFD5sQFgghMAE&url=https%3A%2F%2Fwww.aerzteblatt.de%2Fint&usg=AFQjCNF-3qyE4jvH9hvdO4DhVVPioLZOFw&sig2=b4qLQEaJP8VSu3iDbmF2sQ&bvm=bv.118443451,d.ZWU

227 | P a g e







observational study. vi. “Data was collected systematically in a registry.” A standard ‘sepsis screening tool’ was used to standardise patient information. Chief complaints were extracted from the ED information system. Charlson Comorbidity Index (max score 37) was calculated. Sepsis diagnosis was achieved using the criteria of the International Guidelines for Management of SSSS. vii. Consecutive (n=151) patients presenting to the ED with suspected sepsis (1st Aug to 30th Sept 2012). Mean age 68.3±18 years, 54.3% male, 45% with SSSS, 27.8% in-hospital mortality with SSSS. viii. One hospital (Nuremberg University Hospital) with two emergency care sites

ESI level 1 patients = acute life threatening or patients require immediate initiation of diagnostics and therapy. Level 2= patients in a high-risk situation whose treatment should start with 10 mins of initial triage assessment Level 3=

CCI: AUROC=0.673 (95%CI 0.558, 0.787) Stratification of 28 day mortality risk by MED score 0-7: 60.3% had low risk 8-12: 25.8% had moderate risk ≥13:13.2% had high risk iv. MEDS had the highest diagnostic accuracy. The others had little diagnostic accuracy; ESI AUROC=0.609 (95%CI 0.518, 0.699) MEWS AUROC=0.641 (95%CI 0.552, 0.730) MEDS AUROC=0.778 (95%CI 0.704, 0.853) Mean MEWS score did not increase to critical cut off of ≥5. Relative MEWS score by sepsis diagnosis was; Patients with SSSS: mean MEWS score =3.9 Patients with uncomplicated sepsis: mean MEWS score =3.28 Patients without SSSS; mean MEWS score =1.81

limited ICU facilities Q8. Diagnostic criteria ESI (≤2) Sensitivity=0.708 Specificity=0.456 PPV=0.543 NPV=0.632 MEWS (≥5) Sensitivity=0.366 Specificity=0.798 PPV=0.619 NPV=0.583 MEDS (≥8) Sensitivity=0.592 Specificity=0.785 PPV=0.712 NPV=0.681 Prognostic criteria ESI (≤2) Sensitivity=0.727 Specificity=0.395 PPV=0.170 NPV=0.895 MEWS (≥5) Sensitivity=0.429 Specificity=0.744 PPV=0.214 NPV=0.889 MEDS (≥8) Sensitivity=0.857 Specificity=0.682

nursing home resident, presence of lower respiratory infection and rapidly terminal comorbid illness “which adds to its diagnostic accuracy in this patient group. This specific data, which is not part of the ESI or MEWS is necessary to detect patients with sepsis who are critically ill. Q12. i. Patients present with heterogeneous complaints making the diagnosis of sepsis difficult for ED healthcare professionals. ii. There was no difference between patients with and without SSSS regarding presenting complaints. Q13. iii. ESI and MEWS could be amended by disease-specific risk stratification tools like MEDS or ESI and MEWS could be extended by including disease=specific parameters instead of using two tools.

with accuracy. ESI, MEWS and CCI have low prognostic accuracy. Systematic use of the MEDS score in the ED could lead to the detection of critically ill patients with sepsis. “The MEDS score provides the basis for a risk adjusted disposition management that follows objective criteria.”

228 | P a g e







PPV=0.305 NPV=0.967 CCI (≥2) Sensitivity=0.818 Specificity=0.465 PPV=0.207 NPV=0.938

Ghanem-Zoubi, Vardi, Laor et al. (2011) Israel Q1. RS iv. “To prospectively compare the prognostic value of 4 scoring systems in patients with sepsis upon admission to general internal medicine departments.” v. Prospective study vi. A computerized database was developed which identified patients with presumed sepsis according to the Systemic Inflammatory Response Syndrome (SIRS) criteria, and was incorporated into the EMR system. Thereafter,

Q2. MEWS45, SCS64: simple clinical score MEDS65: mortality in emergency department sepsis score, REMS66: rapid emergency medicine score

Q5. i. Overall mortality rate = 36.1%. 28 day in-hospital mortality rate = 21.9%. MEWS had the lowest 5-, 10- , 30- and 60-day AUROC prognostic value of the 4 scoring systems. The 28-day mortality was predicted with acceptable values by REMS and SCS. All scoring systems predicted 1-10 day mortality better than 30-day mortality. All 4 scoring systems were appropriate for the detection of early death (1-5 day mortality). Overall in-hospital mortality; MEDS AUROC=0.73 (95%CI 0.70, 0.77) REMS AUROC=0.77 (95%CI 0.73, 0.80) MEWS AUROC=0.69 (95%CI 0.65, 0.73) SCS AUROC=0.77 (95%CI 0.74, 0.80). The authors attempted to create a new score for predicting mortality based on ‘sepsis stages (sepsis,

Q9. This was an old study population, and age was significantly different between survivors and non-survivors. MEWS was the only scoring system not to include age, which may explain its poor performance. Old population with a high mortality rate, not all of which might be attributable to sepsis. Overall mortality is the most appropriate outcome when assessing the impact of sepsis on outcome

Q14. 2- One centre. Q16. While all 4 scoring systems were appropriate for the detection of early death (1-5 day mortality) in this patient group, SCS and REMS were appropriate mortality prediction models for patients with sepsis admitted to general internal medicine departments, for all time points investigated (1-60 day mortality). Note:”28-day in-hospital mortality may fail to capture the true impact of sepsis on subsequent outcomes, and may be too insensitive, failing

64

KELLETT J. & DEANE, B. 2006. The Simple Clinical Score predicts mortality for 30 days after admission to an acute medical unit. QJM, 99, 771-781. 65

SHAPIRO, N. I., WOLFE, R. E., MOORE, R. B., SMITH, E., BURDICK, E. & BATES, D. W. 2003. Mortality in Emergency Department Sepsis (MEDS) score: a prospective derived and validated clinical prediction tool. Critical Care Medicine, 31, 670-675. 66

OLSSON, T., TERENT, A. & LIND, L. 2004. Rapid Emergency Medicine score: a new prognostic tool for in-hospital mortality in nonsurgical emergency department patients. Journal of Internal Medicine, 255, 579-587.

229 | P a g e







physicians were instructed to input the required data necessary for the examined scoring systems via a mandatory questionnaire. Data collected were vital signs (HR, RR, temp, SBP, DBP, SpO2), consciousness (AVPU), breathlessness, new stroke, intoxication or overdose, lower respiratory tract infection, abnormal ECG. vii. All (n=1,072) consecutive adult patients (≥18 years) admitted to a general internal medicine with a presumed diagnosis of sepsis identified from the EMR (Feb 2008 to April 2009). Mean age was 74.7± 6.1 years, Male to female ratio was 1.08:1, 96.2% were admitted through the ED. On admission, 5% and 9.3% had septic shock and severe sepsis, respectively. viii. A community-based university hospital

severe sepsis and septic shock), but it did not predict in-hospital mortality with acceptable accuracy (AUROC=0.65) of this patient group;

to capture important effects on surrogate outcomes, such as the effects of potential therapies.” (p6).

Jarvis, Kovacs, Briggs et al. (2015c) UK Q1. RS iv. To compare the performance of EWSs using three methods of vital sign observation selection (i) all observations, (ii) one

Q2. 35 published EWSs Q5. NEWS performed the best of all 35 EWSs when predicting risk of death within 24 hours. This was true for all three sample methods. Only the EWS of Bakir changed rank significantly depending on the selection method. This was not significant when age was excluded as a variable from the EWS scores. The method of observation selection affected the AUROCs calculated for each EWS system.

Q9. viii. “Age at admission is not useful for discriminating changes in risk during an episode of care, but it is useful for discriminating risk between two episodes of care in which the patients have different ages.”

Q14. 3 Large dataset, one centre, retrospective design. Q16. NEWS has the best discrimination of the 35 EWSs investigate for the prediction of death within 24 hours. “Vital signs and derived EWS

230 | P a g e







randomly chosen observation set per episode, and (iii) one observation set per episode chosen at a random point in time within each episode; to determine whether a lack of independence between data points when sampling patient observations might change the ranking of the EWSs. Age at admission was used for EWSs incorporating age. v. A retrospective study vi. An electronic database of 1,564,143 consecutive vital sign observation sets (mean 21.7 observation sets per episode) during May 2011 and Dec 2012. 10,000 observations were chosen for the random methods of observation selection. vii. 64,285 episodes of care of adult patients (≥16 years); mean age 61.8 (SD 20.4) years and 48% male. 2.6% of clinical episodes ended in death. viii. One hospital

Top three EWS in ‘All observations’ NEWS40 AUROC=0.898 MEWS45 AUROC=0.862 Worthing53 AUROC=0.861 Lowest; Centiles 62AUROC=0.783 Top three EWS in ‘Observations chosen at random’ NEWS AUROC=0.916 MEWS AUROC=0.898 Worthing AUROC=0.885 Lowest; Centiles AUROC=0.763 Top three EWS in ‘Observations chosen at a random point in time’ NEWS AUROC=0.914 MEWS AUROC=0.895 Bakir AUROC=0.883 Lowest; Centiles AUROC=0.775

Bakirs EWS has the greatest weighting for age within the scoring system than any other EWS. Q13. iii. Vital signs were recorded with handheld electronic equipment and VitalPAC software.

values for EWSs that do not include age can be treated as if they were independent.” But observation selection method can change the rank order of EWSs that include age. Age is a more useful discriminator of death when only one observation per episode of care is included in the AUROC calculation, because it reduces the bias of more observations per episode for older patients, when all observations are included.

Jarvis, Kovacs, Briggs et al. (2015b) UK Q1. RS iv. To investigate whether EWS are truncated to a binary score of 0 ‘normal’ or 1 ‘abnormal’ results in a decrease in errors associated

Q2. 36 published EWSs Q3. i. Standard aggregate EWSs were converted to binary score defined of 0 ‘all vital sign parameters are normal’ or 1 ‘abnormal i.e. any parameter with a score of

Q5 i. All aggregate EWSs and binary EWS had an AUROC ≥0.700 for predicting death. Binary EWS had lower discriminatory ability than the standard EWS in general for predicting death, but these differences were not statistically significant. The exception was Bakir’s EWS and CART. Binary NEWS had significantly better discriminatory

Q7 i. All aggregate EWSs and binary EWS had an AUROC ≥0.700 for predicting unplanned ICU admission (except Bakir EWS and CART). Binary EWS had lower discriminatory ability than the standard EWS in general for predicting unplanned ICU admission, but these differences were not

Q9. Bakir’s EWS which weights patient age was outperformed by binary Bakir EWS suggesting that the high weighting score of rage does not accurately represent the increased risk of adverse events with increasing age. Weighting may be too high or

Q14. 3 Large database was used with 18 months of data. Single centre, effects of interventions were not considered Q16. A binary NEWS performs better than other standard

231 | P a g e







with weighting or scoring EWS. Death or ICU admissions within 24 hours were the endpoints investigated. v. A retrospective analysis vi. An electronic database of vital signs were recorded at bedside in real time with electronic equipment. Patient outcomes were extracted from the patient administration system and ICU admission databases. AUROCS were calculated for all observation sets and observation sets of 10,000 (1 per patient) randomly chosen. vii. A vital sign observations from 46,944 adult (≥16 years) patients encompassing 1,564,153 vital signs during May 2011 and Dec 2012 was used. Mean age was 62.5 (SD 20.5) years, 48% were male. viii. A district general hospital

≥1’. ability than all other standard EWS, except the standard NEWS. ii. All aggregate EWSs and binary EWS had an AUROC ≥0.600 for predicting cardiac arrest. Binary EWS had lower discriminatory ability than the standard EWS in general for predicting cardiac arrest, but these differences were not statistically significant. The exception was Bakir’s EWS and CART. Binary NEWS had significantly better discriminatory ability than all other standard EWS, except the standard NEWS. Q6. i. Binary NEWS trigger score ≥3 would detect more adverse outcomes than NEWS score at a trigger score ≥5, but would require a 15% higher triggering rate. The percentage of observation sets that trigger a response is higher for binary (11.8%) than standard (10.2%) NEWS. The number of unique patients that trigger a response daily is higher for binary (n=145 (SD 24)) than standard (n=118 (SD 20)) NEWS.

statistically significant. The exception was Bakir’s EWS and CART. Binary NEWS had significantly better discriminatory ability than all other standard EWS, except the standard NEWS. Q8. The PPV and sensitivity of NEWS is better than binary NEWS; NEWS aggregate score ≥5 Sensitivity=69.7% Specificity=94.2% PPV=11.8% NVP=99.6% Binary NEWS score ≥3 Sensitivity=67.7% Specificity=92.9% PPV=9.6% NVP=99.6%

assigned to the wrong ages. Effect of use of binary EWS on reducing errors was not investigated. Q13. iii. Simplified binary EWSs would not be advantageous in hospitals were EWSs are calculated electronically on PDAs, but would be advantageous where paper-based EWS are used.

EWS, except for the standard NEWS. Therefore, these simplified EWSs may be used to identify patients at increased risk of adverse events. Binary NEWS may result in fewer errors, but a higher workload for RRS.

Jo, Lee, Jin et al. (2012) South Korea Q1. RS iv. To investigate (i) whether the predictive value of ViEWS in unselected critically ill

Q2. ViEWS51, Hypotension, SpO2, Low temperature, ECG change and Loss of independence (HOTEL)67 score, APACHE II, SAPS II and SAPS III scores.

Q5. i. 42.4% (n=64) of participants died. The ViEWS-L score had significantly better predictive value than the ViEWS, HOTEL and APACHE II scores for the four mortality outcomes. The predictive value of ViEWS-L were comparable

Q8 vi Addition of lactate measurement can increase sensitivity of VIEWS in predicting mortality and could equal that of more complex, currently accepted scoring systems in the critically

Q9. i. High levels of blood lactate is common, and may be used in the prediction of mortality among critically ill medical patients with multiple

Q14. 2- Single centre, small sample size, retrospective design, possible bias due to lack of lactate levels for some patients, no control group of

67

KELLETT, J., DEANE, B. & GLEESON, M. 2008. Derivation and validation of a score based on hypotension, oxygen saturation, low temperature, ECG changes and Loss of independence (HOTEL) that predicts early mortality between 15 min and 24 h after admission to an acute medical unit. Resuscitation, 78, e8.

232 | P a g e







patients could be improved by including rapid lactate levels (ViEWS-L), and to (ii) compare ViEWS-L with pre-existing validated ICU risk scoring systems, regarding 1- to 4-week mortality. v. A retrospective, observational study vi. Data was extracted from patient records. Predictive scores were calculated retrospectively from this clinical and laboratory data. vii. Consecutive adult (≥18 years) patients admitted during April 2010 and Mar 2011 to the medical ICU via the ED (n=151), mean age was 65.3±17.2 years and 67.5% were male. viii. An urban, academic, tertiary hospital.

Q3. i. Neither ViEWS nor HOTEL were used in the hospital during the study. UO was not included in the ViEWS score used. ViEWS-L was calculated as; ViEWS+ lactate (mmol/l) according to the regression coefficient. The mean ViEWS-L score was 11.667.3.

with SAPS II and SAPS III for the four mortality outcomes. Hospital mortality (p=0.009) ViEWS-L AUROC=0.802 (95%CI 0.729, 0.875) ViEWS AUROC=0.742 (95%CI 0.661, 0.823; p=0.009) HOTEL AUROC=0.662 (95%CI 0.577, 0.747; p<0.001) APACHE II AUROC=0.689 (95%CI 0.577, 0.747; p=0.024) SAP II AUROC=0.799 (95%CI 0.726, 0.872; p=0.944) SAP III AUROC=0.803 (95%CI 0.729, 0.878; p=0.972) 1-week mortality ViEWS-L AUROC=0.842 (95%CI 0.769, 0.913) ViEWS AUROC=0.707 (95%CI 0.615, 0.800; p<0.001) HOTEL AUROC=0.675 (95%CI 0.580, 0.770; p<0.001) APACHE II AUROC=0.717 (95%CI 0.620, 0.809; p=0.024) SAP II AUROC=0.832 (95%CI 0.760, 0.904; p=0.843) SAP III AUROC=0.815 (95%CI 0.739, 0.892; p=0.568) 2-week mortality ViEWS-L AUROC=0.827 (95%CI 0.755, 0.900) ViEWS AUROC=0.729 (95%CI 0.643, 0.814; p<0.001) HOTEL AUROC=0.669 (95%CI 0.580, 0.757; p<0.001) APACHE II AUROC=0.687 (95%CI 0.592, 0.783; p=0.013) SAP II AUROC=0.805 (95%CI 0.727, 0.884; p=0.660) SAP III AUROC=0.801 (95%CI 0.720, 0.881; p=0.578) 4-week mortality ViEWS-L AUROC=0.803 (95%CI 0.731, 0.876) ViEWS AUROC=0.732 (95%CI 0.650, 0.814; p=0.003) HOTEL AUROC=0.659 (95%CI 0.574, 0.745; p<0.001) APACHE II AUROC=0.671 (95%CI 0.583, 0.760; p=0.010) SAP II AUROC=0.782 (95%CI 0.705, 0.859; p=0.649) SAP III AUROC=0.790 (95%CI 0.712, 0.868; p=0.766)

ill cohort diseases including sepsis, trauma and coronary disease. Appropriate treatment of these patients in the ED may lead to lower scores (i.e. APACHE II, SAP II and SAP III) at ICU admission, thus making these scoring systems less predictive of in-hospital mortality.

patients not admitted to ICU. Q16. The ViEWS-L score out-performed ViEWS, HOTEL, and APACHE II scores and as well as SAP II and SAP III in predicting mortality up to 4 weeks post-admission, of a mixed cohort of unselected critically ill medical patients admitted to the medical ICU through the ED.

233 | P a g e







Mapp, Davis & Krowchuk (2013) Q1. R iv. To examine EWSs and the incorporation of clinical support on their effectiveness in predicting a patient’s potential for deterioration and whether they prevent unplanned ICT admissions and/or death. v. Integrative review vi. Three databases were searched; MEDLINE, Cumulative Index to Nursing and Allied Health Literature (CINAHL) and the Cochrane Collaboration for papers during 2007 and 2012. Search terms used included MEWS, EWS, early Warning Systems, deteriorating patient, patient at risk, shock index, track and trigger systems and failure to rescue. vii. 9 studies which investigated quality improvement and/or comparison of an EWS and a facilities own instrument for detecting patients who are deteriorating, were included. 5 investigated implementation of EWS, with different scoring methods. Study designs varied;

Q2. The EWSs included in the review were MEWS (n=6), however these differed in the vital sign parameters measured, with four of these studies adapting the MEWS for their facility; CART (n=1), Shock Index (SI; n=1) and a EWS modified for their facility (n=2). All of the facilities adapted the original EWSs to improve accuracy in the detection of deteriorating patients; urine output (n=5), age (n=2), O2 saturation (n=4), and feeling/worry (n=2) and other parameters (BMI, blood glucose, chest pain, DBP, increased use of supplemental O2, increase CCC, new focal neurological weakness (n=4) were added to the original EWS. Five of the studies incorporate EMR data. Q3. i. Different trigger scores were used to activate escalation; EWS score ≥2 (n=2); score ≥3 (n=2); ii. One study revised the observational chart with EWS and RRT implementation. iii. One study included SBAR

Q5. i. Post-EWS implementation decreases in-hospital mortality were observed (n=2) ii. Post-EWS implementation variation in the effect of patient cardiac arrest was reported; a 50% decreases in cardiopulmonary arrest were observed (n=1); no change in cardiopulmonary arrest (n=2) Q6. i. Changes in number of RRT calls was reported (n=5. A 50% increase in RRTs was reported (n=3), with concomitant decrease in cardiac arrest scores was decreased (n=1). Decrease in medical team calls was reported by another study, and a 43% decrease was predicted if the MEWS and RRT were implemented hospital-wide (n=1) ii. An increase in confidence in nurse communication with physicians was reported post-implementation because the algorithms within the MEWS and RRT facilitate communication of patient condition (n=4)

Q7. i. A number of studies reported that EWS and RRT implementation predicted unplanned ICU admission (n=3), with some EWSs more predictive than others. One study reported no significant difference in unplanned ICU admissions post implementation. A MEWS modified to contain age and BMI (as well aa HR, RR, SBP, temperature), but not level of consciousness, recorded electronically at the bedside had a similar AUROC to the original MEWS for predicting unplanned ICU admission (n=1): Original MEWS AUROC=0.72 Modified MEWS AUROC=0.722 A SI >0.85 was significantly associated with unplanned ICU transfer (n=1) CART was significantly better than MEWS (both scores captured electronically at bedside) at predicting unplanned ICU transfer. It was more sensitive and specific also (n=1). Cardiac arrest was identified 42 and 48 hours prior to the event by CART and MEWS, respectively.

Q7. i. Accuracy of documentation was increased by interfacing bedside data with EMRs (n=5). This also iii. Algorithms resulted in improved nurse assessment accuracy, assessment skills, critical thinking, ability to recognise changes, decreased in patient care delays, support to nurses from other members of the RRT vi. Staff workload was reported to decrease pot-implementation (n=1)

Q14. 1- All studies were undertaken in single centres and all used retrospective data, with relatively short follow-up times. Q16. No negative consequences of EWS implementation were reported. “EWSs that interface with EMRs and are supplemented with decision aides (algorithms) and clinical support systems produce and effective screening system for early identification of deteriorating patients. This multifaceted approach decreases unplanned ICU admissions and hospital mortality." (p 300-301)

234 | P a g e







retrospective with education (n=1), retrospective case-control study (n=1), retrospective cohort study (n=1), descriptive (n=2), vii. 5 studies implementing EWS were undertaken in the US.

Q4. i. 5 studies included clinical support systems to improve the ability to identify the deteriorating patient. Various algorithms were included to be followed by nurses upon patient deterioration, as dictated by trigger scores.

Moseson, Zhuo, Chu et al. (2014) USA Q1. RS iv. To compare the ability of ICU and ED scoring systems to predict mortality within 60 days. v. A prospective, observational study. vi. During hospital admission, clinical data was collected prospectively. Severity scores were electronically computed and ICU scores were calculated from measures taken in the ED. ED scores were calculated from (i) the first measures in the

Q2. 7 scoring systems were compared; APACHE II, APACHE III, SAPS II, MEWS,

REMS68, PEDS69 and a pre-hospital critical illness prediction score by Seymour70.

Q5. i. There were significant differences in the discriminatory ability of ED and ICU scoring systems (P=0.01), with ICU scores outperformed the ED scoring systems regarding 60-day mortality and in-hospital mortality, ICU scoring systems (60-day mortality). APACHE III AUROC= 0.799 (95%CI 0.707, 0.851) APACHE II AUROC= 0.779 (95%CI 0.728, 0.870) SAPS II AUROC= 0.793 (95%CI 0.722, 0.863) ED scoring systems (no significant differences were observed between the ED scores: P=0.45 re 60-day mortality). Seymour-ED AUROC=0.743 (95%CI 0.674, 0.813) PEDS-ED AUROC=0.709 (95%CI 0.623, 0.794) REMS-ED AUROC=0.700 (95%CI 0.617, 0.782) MEWS-ED AUROC=0.698 (95%CI 0.621, 0.776)

Q9. The findings were the same whether parameters measured in the ED or in the ICU were used to calculate the scores in ED scoring systems. The authors conclude that the superior ability of the ICU scoring systems to predict patients at risk of death is due to their complexity and additional clinical information, in comparison to the ED systems, rather than the time they are calculated. The patient population is also likely to have influenced the findings.

Q14. 2- Single centre. Relatively small sample size. Q16. ICU scoring systems out-performed ED scoring systems in predicting mortality in critically ill patients admitted directly to the ICU from the ED.

68

GOODACRE, S., TURNER, J. & NICHOLL, J. 2006. Prediction of mortality among emergency medical admissions. Emergency Medicine Journal, 23, 372-375. 69

CATTERMOLE, G. N., MAK, S. K., LIOW, C. H., HO, M. F., HUNG, K. Y., KEUNG, K. M., LI, H. M., CRAHAM, C. A., RAINER, T. H. 2009. Derivation of a prognostic score for identifying critically ill patients in an emergency department resuscitation room. Resuscitation, 80, 1000-1005. 70

SEYMOUR, C. W., KAHN, J. M., COOKE, C. R., WATKINS, T. R., HECKBERT, S. R., REA, T. D. 2010. Prediction of critical illness during out-of-hospital emergency care. JAMA, 304, 747-754.

http://www.ncbi.nlm.nih.gov/pubmed/?term=Goodacre%20S%5BAuthor%5D&cauthor=true&cauthor_uid=16627839

http://www.ncbi.nlm.nih.gov/pubmed/?term=Turner%20J%5BAuthor%5D&cauthor=true&cauthor_uid=16627839

http://www.ncbi.nlm.nih.gov/pubmed/?term=Nicholl%20J%5BAuthor%5D&cauthor=true&cauthor_uid=16627839

http://www.ncbi.nlm.nih.gov/pubmed/?term=Goodacre+and+rems+and+2006


235 | P a g e







ED and (ii) measures in the first 24 h in the ICU, vii. 227 critically ill adult (≥18 years) patients who were admitted to ICU directly from the ED. Mean age was 65 (SD 17) years and 51% were male. viii. One academic, tertiary hospital

Seymour-ED was not significantly different from APACHE III (P=0.09). The AUROC increased for all ED scoring systems when parameters measured in the ICU were used, but the differences between the 7 systems remained significant. All scoring systems showed acceptable calibration for this patient group as determined by the Hosmer-Lemeshow goodness of fit.

Both the discriminatory ability (ability to separate patients who do and do not have the outcome of interest) and the calibration (i.e. how well the probability of the outcome predicted by the score agrees with the actual observed risk) should be calculated when assessing the accuracy of prognostic scoring systems.

Smith, Chiovaro, O’Neil et al. (2014) USA Q1. SR i (a) To assess the ability of EWS tools to predict 3 clinical outcomes; cardiac arrest, pulmonary arrest or death within 48 hours of data collection; (b) to evaluate the impact of EWS on in-hospital health outcomes and resources utilisation ii. MEDLINE, CINAHL and the Cochrane Central Register from database inception until May 2014 iii. 21 studies were included; 8 observational studies (6 prospective and 2 case-control) provided primary data on the predictive value of EWS scores in relation to the three clinical outputs. 13 unique models relating to the

Q2. CART (4-item), MEWS: (5, 6, 7,or 12 item), ViEWS (6, 7 item), NEWS (7 item), EWS (6, 7, 16 or Patientrack) and a 7-item clinical marker tool were included. All EWS models contained 5 to 12 items. Included in all EWS were (i) heart rate; (ii) respiratory rate; (iii) systolic blood pressure. Most included (i) level of consciousness/mental status, (ii) temperature, & (iii) urinary output Q4.v. Health outcomes Q4.vi. Resource use

Q5. Results from individual studies included in the systematic review have been described individually. i. Mortality within 48 hours: Results were given for individual studies included) a. Prediction of death: within 48 hours: ViEWS (6 items): AUROC range = 0.890 (0.850 –0.920) to 0.93 (0.91-0.95) for medical and surgical patients (n=1). Prediction of death: within 24 hours: VIEWS (7 item): AUROC range = 0.89 (95%CI 0.820-0.95) to 0.890 (95%CI 0.88-0.89) (n=2) VIEWS (7 item): (AUROC = 0.89 (95%CI 0.89-0.90) (n=1) b. Impact of EWS implementation: (n=7) No significant difference in overall mortality (n=6; 1 RCT, 5 pre-post) Decrease in mortality (n=1; p<0.0001). However, an expansion of critical care service was implemented concurrently. Note: no difference in RCT ii. Cardiac arrest:

Q7: Impact of EWS; i. admissions to ICU; (mixed results) Increase in ICU admissions (n=2); No difference in ICU admissions (n=1); Increase in annual ICU admissions; but decrease post-cardiopulmonary resuscitation (n=1). Decrease in the proportion of clinically unstable patients on wards (n=1) No of clinically unstable patients on ward for ≥6 hours decreased (n=1) Note: no difference in RCT iii. Impact of EWS on the LOS: No difference in LOS in ICU 1-2 years post-implementation (n=2) Decreased in LOS between 47 pre- and 38 days post-implementation (n=1; p<0.001). Increase in LOS between 4 months pre- vs 4 months post-implementation (n=1) Q7 vi. RRT implementation may be high in cost and low in benefit; results from 1 study involving 1 hospital

Q9. v. RRT implementation and transfer to ICU was associated with increased risk of hospital death (n=1). This response may increase costs but not benefits. Q10.i. PDAs used to record vital signs Q11. EWS can predict short- and long-term mortality in patients with extreme vital sign abnormalities, despite use and timeliness of interventions. Q13.iii. RRT use improved EWS scoring consistency

Q14 2++ The one RCT was good quality. Risk of bias in controlled and retrospective study designs, limiting the evidence re impact of EWS on health outcomes and resource utilization Q15. All studies bar one (Uganda) were performed in Westernised countries Q16. EWS tools perform reasonably well in predicting cardiac arrest and death within 48 hours.

236 | P a g e







impact of EWS (1 RCT and 12 pre-post studies). The Quality in Prognosis Studies Assessment tool was used to assess the classify studies reporting the predictive ability of EWS.

a. Prediction: EWS (NEWS; MEWS); CART and a novel electronic EWS) had consistently good predictive performance in cardiac arrest within both 24 and 48 hours of recording vital signs (n=4). AUROC was highest for the electronic EWS (AUROC=0.88) and NEWS (AUROC=0.857) and lowest for VIEWS (AUROC=0.74).

b. Impact of EWS implementation: (n=4) Decrease in number of cardiac arrest calls per admission (n=1; p<0.0001) and at the time of a ‘code blue’ call (n=1; p=0.0024). Increase in cardiac arrest in moderate risk patients (score 3/4), (n=1; p<0.016); no difference in low- (score ≤2) or high-risk (score 5-15). No difference (n=1) Note: no difference in RCT

iii-ix. NR x. Single vital sign prediction: (n=1); Respiratory rate (>35 breaths/min), need for supplemental O2 to 100%/use of a non-breathing mask and heart rate >140 beats/min, were most associated with a life threatening event. Q6. impact on systems i. ≥50% increase in the number of RRT and ICU

liaison team calls (n= 4). ii. Code blue calls decreased (6-33%; n=3) iii. Completion of documentation/recording vital

signs ranged from 53-100% (n=3) No. of observations increased for patients with high EWS scores, especially during the day (n=2)

Q8. i. The proportion of patients dying with a low EWS score (<3) was 0.02%; while the proportion dying with a high score (>11) was 14% (n=1?). At specific EWS scores, trade-offs between sensitivity and specificity were required for prediction of death and cardiac arrest. Sensitivity of ViEWS was ~67% at specificity of 90% for death (n=1). Sensitivity of CART and MEWS were 53% and 48%, respectively at a specificity of 90%, at similar trigger scores (n=1) iii. In 1 study risk of death increased following RRT implementation and transfer to ICU

Romero-Brufau, Huddleston, Naessens et al. (2014) USA

Q2. A variety of triggers using the MEWS, ViEWS, SEWS, GMEWS, NEWS and Worthing

Q8. Positive predictive values ranged from less than 0.01 (Worthing, 3 h) to 0.21 (GMEWS, 36 h). Sensitivity ranged

Q14. 2- Q16. Authors noted that when

237 | P a g e







Q1. RS iv. To evaluate a number of EWS systems (MEWS, SEWS, GMEWS, Worthing, ViEWS and NEWS) and the RRT single parameter activation criteria in use in the institution. Researchers sought to compare their ability to predict the composite outcome of Resuscitation call, RRS activation or unplanned transfer to the ICU, in a time-dependent manner (3, 8, 12, 24 and 36 h after the observation) by determining the sensitivity, specificity and positive predictive values (PPV). Pg. 549. v. Analysis of data generated from a retrospective cohort vi. Data collected in 2011 using a longitudinal database that included patients’ data (vital signs, frailty measures, laboratory test results, demo-graphics, urinary output, etc.) throughout each patient’s hospital stay. Vital signs are manually collected and entered into the electronic medical record by a nurse. vii. Researchers used a large vital signs database (6,948,689 unique time points) from 34,898 unique

scores and RRT criteria were applied to the data using published thresholds to create rule triggers to simulate an alert. Q3 ii In the analysis an event was defined by whether or not the decline in patient condition led to an unplanned transfer to ICU/resuscitation call/RRT call

from 0.07 (GMEWS, 3 h) to 0.75 (ViEWS, 36 h). Thus MEWS had the best specificity, but missed many events; VIEWS detected more events, but identified many false positive alerts. Used in an automated fashion, these would correspond to 1040–215,020 false positive alerts per year. The MEWS, SEWS and the researchers our institution’s RRT criteria demonstrated less of a decay in specificity; however, they did not reach the same degree of sensitivity through time (represented diagrammatically without figures)

the evaluation is performed in a time-sensitive manner, the most widely used weighted track-and-trigger scores do not offer good predictive capabilities for use as criteria for an automated alarm system. For the implementation of an automated alarm system, better criteria need to be developed and validated before implementation. (p 549).

238 | P a g e







consecutive hospitalized patients. viii. 2 hospitals

Romero-Brufau, Huddleston, Escobar et al. (2015) USA Q1. R i. “EWSs try to predict a condition whose prevalence is known to be > 2 % in general care inpatients. Consequently traditional metrics (AUROC, C-statistic, specificity, likelihood ratio) provide incomplete information and can lead to overestimating the benefits of an EWS or underestimating the cost in terms of clinical resources.” (p1)

Q9. Prevalence is of the outcome to be predicted by EWSs is usually low, and as prevalence decreases, so does the PPV. Using metrics that incorporate pre-test probability is preferable. Reports on the performance of EWSs would ideally include information about both goals of the EWS: (i) detecting a high percentage of outcomes, and (ii) issuing few false positive alerts thus clarifying the trade-off in the benefit of the system is the early detection, and the burden or cost of false-positive alerts. To evaluate the benefit ‘sensitivity’ can be used because it provides the percentage of outcomes that the score is able to predict within a specified timeframe. To evaluate the clinical burden, PPV, the NNE or the workup to detection (WTD) ratio, and the estimated rate of alerts can be used. EWSs are really trying to predict instances of physiological deterioration. Surrogate measures of

Q14. 3 Q16. The authors state that “to compare EWSs it is important to report metrics that incorporate the extremely low prevalence and recommend using the PPV, the NNE and/or the estimated rate of alerts combined with sensitivity to evaluate each of the plausible score cut-off values. Including two of these metrics in a graph allows for easy evaluation of practical clinical usefulness both in absolute terms and for comparison of two or more EWSs. Evaluating EWSs in this way demonstrates the balance between the benefit of detecting and treating very sick patients with the associated clinical burden on providers and patients. Clinically, EWSs should not replace clinical judgment and decision-making but should serve as a safety net.”(p5)

239 | P a g e







physiological deterioration include ICU transfers and cardiorespiratory arrests, and calls to the RRT. These proxy outcomes vary locally by hospital and patient population.

Smith, Prytherch, Meredith et al. (2013) UK Q1. RS iv. To test the ability of NEWS to discriminate patients at risk of cardiac arrest, unanticipated ICU admission or death within 24h of a NEWS value and compared its performance to that of 33 other EWSs, using the AUROC curve and a large vital signs database. (p 465) v. Retrospective analysis of prospectively collected data vi. Vital sign data was collected electronically on hand held PDAs vii Vital signs database (n = 198,755 observation sets) collected from 35,585 consecutive, completed acute medical admissions. Data collected from 8 May 2006 and 30 June 2008. The mean (median) ages of the patients were 67.7 (72.6) years (male 65.9 (69.7); female 69.4 (75.5)). xiii Hospital, UK

Q2. The NEWS and 34 other EWS

Q3 i. All extreme values in all physiological parameters in NEWS should score 3 points

Q5 i. The AUROCs for NEWS for death within 24 h, 0.894 (95%CI 0.887–0.902). Similarly, the ranges of AUROCs (95% CI) for the other 33 EWSs were 0.813 (95%CI 0.802–0.824) to 0.858 (95%CI 0.849–0.867) (death). ii. The AUROCs for NEWS for cardiac arrest, within 24 h, was 0.722 (95%CI 0.685–0.759). Similarly, the ranges of AUROCs for the other 33 EWSs were 0.611 (95%CI 0.568–0.654) to 0.710 (95%CI 0.675–0.745) (cardiac arrest). x. The AUROCs (95% CI) for NEWS for any of the outcomes (including cardiac arrest, unanticipated ICU admission, death) within 24 h, were 0 0.873 (0.866–0.879), respectively. Similarly, the ranges of AUROCs (95% CI) for the other 33 EWSs were 0.736 (0.727–0.745) to 0.834* (0.826–0.842) (any outcome).* this EWS system was cited as Paterson R, MacLeod DC, Thetford D, et al. Prediction of in-hospital mortality and length of stay using an early warning scoring system: clinical audit. Clin Med 2006;6:281–4.

Q7 i. The AUROCs for NEWS for unanticipated ICU admission within 24 h, was 0.857 (95%CI 0.847–0.868). Similarly, the ranges of AUROCs for the other 33 EWSs were 0.570 (95%CI 0.553–0.568) to 0.827 (95%CI 0.814–0.840) (unanticipated ICU admission). vii- More efficient systems are associated with decreased workloads i.e. the authors demonstrated the reduction in workload resulting from the use of NEWS instead of the EWS described by Paterson .

Q11. Authors cited that the electronic bedside capture of EWS data, which is increasing throughout the NHS, has been shown to reduce errors in EWS calculation.

Q14. 3 Q16. NEWSs had the best discriminatory ability of 34 EWSs investigated to predict patients at risk of a combined outcome of cardiac arrest, unanticipated ICU admission or death. Also for the individual outcomes of unanticipated ICU admission and death but not individual outcome of cardiac arrest.

240 | P a g e







Yu, Leung, Heo et al. (2014) USA Q1. RS iv. To examine and compare the ability of nine prediction scores to estimate the risk of clinical deterioration. v. A retrospective nested case-control study vi. All clinical variables were collected retrospectively from either an electronic database or the patient’s paper medical records. vii. Non- ICU adult patients on the general hospital wards admitted to two hospitals between 1December 2009 and 31 March 2010 “with infection present on hospital admission, as defined by a validated list of International Classification of Diseases, Ninth Revision (ICD-9) codes indicative of infection” pg. 2. “Cases were patients in this cohort who experienced clinical deterioration, defined as requiring a critical care consult, ICU admission, or death” (p 1). Sample: 328 cases and 328 matched controls. viii. Two hospitals, USA

Q2. The 9 tools were: SOFA, PIRO, ViEWS, Simple Clinical Score SCS, MEDS, MEWS, SAPS II, APACHE II and REMS. Q3. Researchers compared each prediction score’s ability, over the course of 72 hours, to discriminate between cases of deterioration and control patients.

Q7. Q.: At the 0- to 12-hour interval before clinical deterioration, all scores except REMS (AUC 0.67 95%CI 0.62, 0.71) performed with acceptable discrimination (i.e. AUC ≥0.70) and had roughly equivalent AUC. Although SOFA performed the best with an AUROC of 0.78 (95% CI 0.74, 0.81), this was not significantly higher than; PIRO (AUC 0.76 (95% 0.72-0.79), ViEWS AUROC=0.75 (95% 0.71-0.79), SCS AUROC=0.74 (95%CI 0.70-0.78)), MEDS AUROC=0.74 (95%CI 0.70-0.78)), or MEWS AUROC=0.73 (95%CI 0.69-0.77)). However, at the 12- to 72-hour intervals, all scores, with the exception of MEDS (AUROC=0.69 (95%CI 0.63-0.74) at 24 to 48 hours and AUROC=0.71(95%CI 0.64-0.78) at 48-72 hours), no longer performed with acceptable discrimination for mortality (AUROC <0.70). For all models, average scores of cases increased closer to time of clinical deterioration (P <0.05). For the MEWS, SAPS II, APACHE II, and REMS scoring models, this increase was detected as early as 12 to 24 hours before deterioration (P <0.05). For SOFA this increase can be detected even earlier at 24 to 48 hours before clinical deterioration. That is, the average SOFA score of cases during the 24- to 48-hour

Q9. Clinical decision rule which incorporates both the SOFA score and changes in SOFA score. Patients who met the clinical decision rule criteria (earliest available SOFA score ≥3 or and changes (˂3) in SOFA score are almost six times more likely to clinically deteriorate compared with patients who did not (adjusted OR 5.89 (95%CI 3.62 to 9.57). Scores performed better closer to time of clinical deterioration.

Q14. 2- Compared with controls, cases were generally older, more likely to be male, and more likely to be admitted from a nursing home. Q16. “ICU- and ED-based prediction scores can also be used to prognosticate risk of clinical deterioration for non-ICU ward patients. Scoring models that take advantage of score’s change over time may have increased prognostic value over models that use only a single set of physiologic measurements.” (p1). Interpretation is limited to patients with known infections.

241 | P a g e







interval was significantly higher than during the 48- to 72-hour interval (P = 0.01). In contrast, average scores of controls did not increase closer to the index time.

Table 4. Investigation of the effect of chart design on the accuracy and speed of documentation Authors, (date) Country Q1. Type of Evidence






Bunkenborg, Samuelson, Poulsen et al. (2013) Denmark Q1. RS iv. To evaluate the “short and long term effects of systematic inter-professional use of EWS, structured observational charts, and clinical algorithms for bedside action” on unexpected in-hospital death (i.e. sudden, no resuscitation), death after cardiopulmonary resuscitation, and death within 24h of admission for intensive care) death. v. Prospective, non-randomised controlled study

Q2. 3 component clinical intervention: an (i) EWS, (ii) observational chart and (ii) algorithm for bedside action. Q3. i A new monitoring practice i.e. the systematic use of MEWS45(values for SpO2 were also recorded, but with no scoring) obtained and calculated ~8 hourly for terminal patients and more frequently for those with signs of clinical deterioration; Q4 ii. An observation chart had a; colour-coding in green, yellow, orange or red on top

Q5 i. Unexpected patient mortality decreased post-intervention: 25 (1st post-intervention) vs 61 (pre-intervention) per 100 adjusted patient years (P=0.053). Rate ratio: 0.404 (95%CI 0.161-1.012) 17 (2nd post-intervention) vs 61 (pre-intervention) pre 100 adjusted patient years (P=0.013). Rate ratio: 0.271 (95%CI 0.097-0.762) x. The number of SAEs decreased post-intervention (n=21) compared to pre-intervention (n=31).

Q7. i. No difference in the number of ICU admissions were observed (n=17 both pre and post-intervention) which may be attributable to improved patient monitoring and earlier intervention.

Q13. iii. “Recommendations on the vital parameters to assess were strictly perused, the recommended minimum time interval for repeated in-hospital bedside assessments was reduced (from 12 to 8 h), and individual knowledge and skills, as well as interpersonal communication and collaboration, were continuously optimized.”

Q14. 2- Results need to be interpreted in light of the changes in hospital organisation. Results from individual elements of the intervention cannot be elucidated. Q16. “Clinical Intervention comprising systematic monitoring practice, EWS, and observational chart, and an algorithm for bedside management, implemented by inter-professional teaching, training and optimization of communication and collaboration, may significantly reduce unexpected in-hospital

242 | P a g e







before and after clinical intervention implementation” vi. Data was collected over three 4-month periods; (i) pre-intervention (Mar-June 2009); post-intervention ((ii) Sept-Dec 2010) and (III) Mar-June 2011)) vii. All adult patients (≥18 years), without DNRs, with at least a 24h stay during pre-intervention (2009); n=1870 (9,804 in-hospital days of care; mean age 58±19 years; 42% male). Post-intervention recruitment of 2,079 adult patients (mean age 57±20 years; 44% male, 12,584 patient days (2010)) and 2,234 patients (mean age 57±20 years; 41% male, 13,356 patient days (2011)) viii. Medical and surgical wards in one hospital. The number of high intensity monitoring beds increased from 11 to 18 (2009-11) and number of healthcare professional employed increased. There was an organisational expansion of catchment area from 280-460,000 people in 2010. The in-hospital emergency team had been in place for 2 years prior to the start of the study.

of the chart were used for each parameter score and range An algorithm for bedside action; the chart also had a colour-coded algorithm for clinical management (green (MEWS=0: no specific further action) to red (MEWS ≥5 (urgent and appropriate bedside action). Q4. x. Implementation process: (i). Close and continuous collaboration with nursing and medical staff and managers. (ii) Teaching, training and promotion for all staff (iii) Communication and collaboration (iv) Feedback visits

mortality.”

243 | P a g e







Christofidis, Hill, Horswill et al. (2013) Australia Q1. RS iv. “To determine whether experienced health professionals recognise patient deterioration more accurately and efficiently using (a) novel observation charts, designed from a human factors perspective, or (b) chart designs with which they have long-term experience.” (p.657) v. Mixed study design with chart experience and chart type as independent variables. design vi. “Participants were presented with realistic (48 cases, each with 13 consecutive time points) abnormal and normal patient observations recorded on 6 hospital observation charts of varying quality, including the chart the participants were familiar with. Across 48 trials, participants were asked to specify if any of the

Q2. Six charts were evaluated, each included 9 vital signs (RR, O2 delivery, O2 saturation, SBP, DBP, HR, temperature, consciousness and pain). The charts investigated were; 1. Adult Deterioration Detection System (ADDS)71 chart with SBP table 2. ADDS chart without SBP table 3. Multiple parameter track-and-trigger chart 4. Single parameter track-and-trigger charts. 5. No track-and-trigger graphical chart 6. No track-and-trigger numerical chart

Q6. ii. ADDS out-performed the other charts investigated among participants experienced in (i) multi-parameter track-and-trigger and (ii) no track-and-trigger graphical charts. “Compared to the best performing ADDS chart, the multiple track-and-trigger chart yielded ~1.6 times more errors by experienced users. The no track-and-trigger graphical chart yielded ~5.4 times as many errors as participants experienced in a similar chart. These are large effects which could influence appropriate and timely detection of patient deterioration in practice. Percentage errors were lowest on the ADDS chart with SBP table for participants experienced in both multi-parameter and no track-and-trigger chart (10%, 9%, respectively). Highest percentage errors were observed in the no track-and-trigger numerical chart for those in multiple parameter charts (32%) and the no track-and-trigger graphical chart for those experienced in the no track-and-trigger graphic charts (38%). Response time was fastest ADDS chart with SBP table (12 and 11 seconds, respectively) and slowest for the no track-and-trigger numerical chart (18 and 17 seconds, respectively) in participants experienced in both multi-parameter and no track-and-trigger charts. Evidence of the benefits of experience was

Q9. viii. The interdisciplinary staff education programme was hypothesised to be the reason for the outperformance of the multi-parameter track-and-trigger group over the no track-and-trigger graphical chart in speed and accuracy of chart completion, with 89% and 29% of participants in each group completing it, respectively. Q12. ii. Belief that a new chart is inferior to the one it is replacing, or fear that the staff will perform worse because of inexperience with the new chart, leading to poor compliance. Q13. ii. Communicate the results of the current study with healthcare professionals

Q14. 2+ Unknown what specific chart design features are responsible for the benefit observed. Q16. “Superior observation chart design appears to trump familiarity.” (p.665) Participants made significantly fewer errors and responded significantly faster when using a novel, user-friendly, well-designed chart compared with all the other designs, including the charts that they were experienced with in a clinical setting. Implementing the ADDS chart may lead to improved performances.

71

HORSWILL, M. S., PREECE, M. H. W., HILL, A., CHRISTOFIDIS, M. J., KARAMATIC, R. M., HEWETT, D. J. & WATSON M. O. 2010. Human factors research regarding observation charts: Research project overview. Available from http://www.safetyandquality.gov.au/wp-content/uploads/2012/01/35986-HumanFactors.pdf

244 | P a g e







vital sign observations were abnormal or if they were all normal. Participants’ overall error rates (i.e. the proportion of incorrect responses) and response times, the main outcome measures, were calculated for each observational chart.” (p.657) (Sept 2010-April 2011). vii. Experienced healthcare professionals (doctors and nurses; n=101) viii. 1 hospital; The Canberra Hospital

observed, with patients experienced in multi-parameter track –and-trigger charts, faster and more accurate than their counterparts in the no track –and-trigger chart group in both types of chart

Christofidis, Hill, Horswill et al. (2014) Australia Q1. RS iv. “To investigate whether overlapping blood pressure and heart rate graphs improve chart-users’ ability to recognize derangements in these vital signs on hospital observation charts. Many health professionals prefer blood pressure and heart rate graphs to overlap. Due to the use of a visual cue called the ‘Seagull Sign’ to detect physiological abnormalities.” (p.610) The seagull sign equates to a shock index score (i.e. heart rate/SBP) physiologically.

Q4. ii. Four chart designs were investigated: (i) (blood pressure and heart rate graphs were separate or overlapping and (ii) an integrated colour-based track-and-trigger system was present or absent

iv. “Each participant was trained and tested individually in a quiet room. First, they completed a demographic questionnaire. Next, they watched training videos that explained: (a) SBP and heart rate and their normal ranges; (b) track-and-trigger systems; and (c) how to use each chart design.”

Q6. ii. Participants responded faster using separate, not overlapping graphs (P=0.002). Participants also responded faster using track and trigger, verses no track-and-trigger systems (P<0.001) There was no significant main or interactive effect in the error rate of participant group (all P>0.10). However, participants made fewer errors using separate (vs. overlapping) graphs, both on charts with a track-and-trigger system (P < 0.001) and without (P < 0.001). Separate graphs also yielded fewer errors in the presence (vs. absence) of a track-and-trigger system (P < 0.001). ‘Seagull-trained’ participants made fewer errors using separate (v overlapping) graphs (P=0.049); and using separate graphs on designs with (v without) a track-and-trigger system (P=0.04)

Q14. 2+ One centre. Results may not be generalizable to genuine clinical environment. Q16. “Overlapping graphs do not yield the performance advantage that many health professionals assume, either for novices or experienced nurses, even when the Seagull Sign is used.” (p.610) The findings suggest that charts should be designed based on evidence. The current study suggests that blood pressure and heart rate observations are plotted separately, precluding use of the Seagull Sign.

245 | P a g e







v. “A 3x2x2 mixed factorial design, with three independent variables: participant group, graph format (separate vs. overlapping) and alerting system (integrated colour-based track-and-trigger system present vs. absent)”. vi. ‘Seagull-trained’ nurses and novices who were randomly assigned to receive ‘Seagull training’ or not, viewed sequences of SBP, DBP and heart rate observations recorded on 4 chart designs during Jan to May 2011. Participants considered whether 64 patient case observations (spanning 13 consecutive time points) were physiologically normal or abnormal. Speed of completion and errors were recorded. vii. A purposive sample of experienced (n=41).and a convenient sample of novice nurses (n=113) were recruited. viii. One hospital

(p.617) Novices were randomly assigned to one of two ‘Seagull-trained’, or untrained. For ‘Seagull-trained’ novices and all nurses, the training video also explained: (d) the Seagull Sign; and (e) how to find it on charts with overlapping blood pressure/heart rate graphs. Each participant was required to take a 5-item multiple choice examinations following their training and attain a score of 100% in order to be eligible to participate. They resat this examination following the trial.

Findings also support the suggestion that future observation charts should include an effective early EWS, making the Seagull Sign redundant.

Christofidis, Hill, Horswill et al. (2015) Australia Q1. RS

Q2. Three colour based scoring design formats of EWS scoring systems were compared for speed and accuracy of scoring. “Vital

Q5. Fastest responses and fewest errors were observed when individual vital sign rows were not included in the chart ‘no rows’. Speed of response

Q7. Poor performing chart designs resulted in EWS scores that were under- or over-scored by margins (1.0-1.5 units) that could inappropriately influence trigger points.

Q9. Vital sign placement affects the speed and accuracy of chart completion. Faster and more accurate EWS scoring was obtained on

Q14. 2+ A simulation ‘usability study’ with non-health care professionals, therefore uncertain whether results are

246 | P a g e







iv. “To evaluate the effect of EWS system design on the speed and accuracy of scoring” (p.1573). All design formats were based on the ADDS chart. v. “Within-subjects, with scoring system design as the independent variable” (p.1573) vi. Speed of completion and overall score was recorded for 54 time-points for each design format. Data from 18 consecutive time points of 10 vital signs for 9 patient cases was used. vii. Subjects recruited were undergraduate psychology students who were novice chart users (n=47). In previous research the authors observed similar results from healthcare professionals and chart novices, and concluded that no additional value would be had from including “non-naïve novices” e.g. nursing students. Participants receiving training on normal vital sign ranges, track-&-trigger systems and chart usage and were included in the study if they received 100% on an MCQ of this content. Each participant then completed the charts

sign scores were either (i) grouped beneath all the vital sign rows; (2) separated, with each row presented immediately below the corresponding vital sign row; or (3) excluded altogether.” Q3. Ten vital signs were included: RR, O2 delivery, SpO2, SBP, DBP, HR, temperature, 4-hour urine output, consciousness and pain.

‘No’ vs ‘Separate’ rows; 6.35 (95%CI 5.83-6.87) seconds faster response (p<0.001)

‘No’ vs ‘Grouped’ rows; 7.69 (95%CI 7.17-8.20) seconds faster response (p<0.001)

‘Separate’ vs ‘Grouped’ rows; 1.34 (95%CI 0.82-1.86) seconds faster response. The authors report a p<0.001, however, the 95%CI cross 1.0 indicating that the result is not significant.

Response time and target EWS: Positive correlations between response time and target EWS (i.e. 0-8), thus slower scoring times were recorded for more at risk patients

Grouped: r=0.98 (p<0.001)

Separated: r=0.95 (p<0.001)

No rows: r=0.94 (p<0.001) Incorrect scoring: Error rate:

‘No’ vs ‘Separate’ rows; 2.48% (95%CI 0.86-4.11) fewer errors. The authors report a p=0.008, however, the 95%CI cross 1.0 indicating that the result is not significant.

‘No’ vs ‘Grouped’ rows; 2.76% (95%CI 1.01-4.50) fewer errors (p=0.007)

‘Separate’ vs ‘Grouped’ rows no difference (p=1.000).

Under scoring was more frequent in ‘No’ and ‘Separate’ row designs compared to over scoring;

‘No’ rows (affecting 1.70% (95%CI 0.60-2.79) more scores)

‘Separate’ rows (affecting 3.20% (95%CI 1.82-4.56) more scores)

‘Grouped’ rows no difference.

charts formats which did not require the recording of individual vital sign scores. This may be explained by having less visual switches, and visual clutter.

generalizable to the clinical setting with experience chart-users Also findings not generalizable to electronic EWS systems. Q16. “Early-warning scoring systems may be more effective without individual vital sign scoring-rows. Even when charts are designed by multidisciplinary teams of human factors specialists and clinicians, empirical evaluations are essential.” (p.1573)

247 | P a g e







individually next to a simulated patient.

Fung, Khan & Dawson (2014) UK Q1. RS iv. To compare two observation chart designs (i) tradition chart with graphic depiction of observations and (ii) charts with EWS with numerically depicted observations v. An observational study vi. Both chart designs were populated with six clinical scenarios. Participants were given 12 charts each. The time taken to give a diagnosis was recorded (a surrogate for data assimilation) and data interpretation was determined by accuracy of the diagnosis given vii. 100 Healthcare professionals (53 interns, 8 senior house officers, 7 specialist registrars, 6 sisters and 23 registered nurses) viii. One hospital (Basildon and Thurrock University Hospital NHS Foundation Thrust.

Q3. i. A tradition chart with graphic depiction of observations and a chart with EWS with numerically depicted observations developed by the Leading Improvements in Patient Safety Programme (LIPS) within the hospital

Q6 ii. Healthcare professionals completed the graphically-depicted charts 1.6 times faster than the numerically-depicted chart (P<0.0001). Accuracy in recognising the clinical scenarios was more accurate for all 6 scenarios, and significantly so for 3 of the 6 scenarios. Overall accuracy was 90% and 75% respectively for the graphical and numerical charts, respectively (P<0.0001)

Q14. 2+ Q16. Graphical data display is superior to numerical data display in terms of faster and more accurate interpretation of information.

Nwulu, Westwood, Edwards et al. (2012) UK Q1. RS

Q2. SEWS46Error! Bookmark not

defined. Q3.

Q6. i. There were 17% alarms produced from a mean of 419 SEWSs measured daily (a mean of 61, 11 and 2 level 1, 2, and 3 alarms daily).

Q9. Abnormal HR, RR and SpO2 were the vital signs that most often featured in SEWS scores of 6.

Q14. 2- Single centre. Retrospective analysis. Impact on patient outcomes was not evaluated.

248 | P a g e







iv. To describe the implementation of an electronic observation chart within a locally developed Prescribing, Information and Communication System (Nwulu PICO) and evaluate vital sign data capture. v. Retrospective analysis of prospectively captured data. vi. Data from all observation charts was extracted from the PICO system during 1st April and 31st July 2010 for the 6 pilot wards. vii. NR viii. One Hospital

i. The Nwulu PICO system incorporates a decision support system based on clinical protocols and best practice guidelines. An interface on handheld electronic devices has been developed for the entry of physiological observations into flow sheets based on SEWS. A graphic reminder of overdue observations is included and a SEWS score is calculated. A hierarchical set of warnings is incorporated with three levels of alarm and responses; Level 1 ‘yellow’ at a score of 0-3; Level 2 ‘orange’ at a score of 4-5. The SEWS observation chart was implemented during 2009 and 2010, into 6 pilot wards. Q4. i. Email alerts are automatically sent to CCOT for the highest (level 3) SEWS score. CCOT is staffed by ICU-trained nurses and is available to wards weekdays from 8:00 to 16:00. Otherwise the critical care unit is contacted.

ii. The percentage of complete observation sets varied between the 6 wards (69.3% to 92.0%). All 6 vital signs were measured and a SEWS score was calculated for 80.5% of observation sets over the 4 month study period; 12.4% had 5 vital signs (AVPU most often missing) 2.9% had 4 vital signs 0.8% had 3 vital signs 0.9% had 2 vital signs 2.6% had 1 vital signs (AVPU most often measured)

Traditional gaps in observation, e.g. RR were observed in the electronic capture of vital signs. The relative importance of RR in the overall SEWS score decreases at scores of ≥10. It is unrealistic to expect a complete set of vital signs to be measured each time. Vital signs pertinent to the clinical concern of individual patients should be reviewed. Q12. ii. Nurses were initially anxious about using the new electronic chart, fearing increased workload and monitoring. iii. The success of new interventions depends on human interaction with the systems and variable organisational practices.

Q16. Implementation of an electronic observation chart was feasible, although variation in engagement in different wards was observed and could not be explained. This system has the potential to improve detection of and response to deteriorating patients.

249 | P a g e

Table 5. EWS educational initiatives Authors, (date) Country Q1. Type of Evidence






Liaw, Wong, Ang et al. (2015a) and Liaw, Wong, Chanet al. (2015b) 72 Singapore Q1. RS iv. To evaluate the effect of a 3 hour interactive web-based educational programme on nurses’ knowledge and performances in (a) early recognition of changes in vital signs, and responding to clinical deterioration; (b) performing an assessment and intervening using ABCDE (airway, breathing, circulation, disability and expose/examine) and (c) reporting clinical deterioration using ISBAR. v, A randomised controlled study with a pre- and post-test design. vi. Knowledge and performance were evaluated

Q2. Rescuing A Patient In Deteriorating Situations (E-RAPIDS) an interactive web-based educational programme Q4. iv. (a) An animated video demonstrating a patient with cardiopulmonary arrest was used to emphasise the importance of recognising changes in vital signs and reporting quickly. (b). A list of performance tasks re assessing and monitoring the patient was given to the nurses based on the ABCDE mnemonic. The evidence base for these tasks was present in multiple formats – texts, illustrations and audio lung sounds. Nurses were asked to identify appropriate tasks for 5

Q5. vii. Post-intervention, a significantly higher number of nurses in the intervention group monitored RR (48.2% v 25.0%; P<0.05) and pulse rates (74.3% v 37.5%; P<0.01) than in the control group in the simulated environment. Performance in assessing and managing clinical deterioration was significantly higher post-test in the intervention compared to the control group (25.8% v 19.5%; P<0.001). x. Knowledge was significantly higher post-test in the intervention compared to pre-test (21.29% v 18.89%; P<0.001). No difference was observed in the control group post-test compared to pre-test (18.28% v 18.56%; P=0.51). There was a significant difference in knowledge between groups (P<0.001). Q6 ii. Reporting of clinical deterioration was significantly higher post-test in the intervention compared to the control group (12.83% v 10.97%; P<0.001).

Q9. Post-intervention, RR monitoring remained the least measured vital sign during the post-test simulation-based assessment. This may be due to the accessibility of pulse oximetry from the vital sign monitor, or a lack of time during the test. The ABCDE mnemonic improved patient assessment beyond vital sign measurements, by looking for objective patient ques e.g. patient’s colour. Q13. iii. Use of automated respiratory monitoring devices are recommended to assist nurses in RR monitoring.

Q14. 2- Single centre. Results reported are learning 1 week post-intervention, and are only based on mannequin-based assessment. Q16. There was a significant increase in knowledge and performance in assessing, managing and reporting clinical deterioration following participation in a web-based educational programme developed for hospital nurses. Notably the same lead author developed and validated a Rescuing A Patient In Deteriorating Situations (RAPIDS tool) which can be used to

72

RCT reported in two papers: Liaw, S. Y., Wong, L. F., Ang, S. B. L., Ho, J. T. Y., Siau, C., Ang, E. N. K. 2015a. Strengthening the afferent limb of rapid response systems: an educational intervention using web-based learning for early recognition and responding to deteriorating patients. BMJ Quality and Safety, 0, 1-9. Liaw, S. Y., Wong, L. F., Chan, S. W.-C., Ho, J. T. Y., Mordiffi, S. Z., Ang. S. B. L., Goh, P. S. & Ang, E. N. K. 2015b. Designing and evaluating an interactive multimedia web-based simulation for developing nurses’ competencies in acute nursing care: randomized controlled trial. Journal of Medical Internet Research,17, e5.

250 | P a g e







with pre-tests and post-tests, a mannequin-based assessment, a multiple choice questionnaire and a validated performance tool. vii. 67 registered nurses with <5 years nursing experience, and working in general wards were randomly assigned to the intervention or control arm. viii. One hospital

patient scenarios. (c). ISBAR application was demonstrated in E-RAPIDS through animation and multimedia material.

The participants from the interventional group were satisfied with their learning experience and gave positive ratings for the quality of the Web-based simulation. Qualitative commentary related to: relevance to practice, instructional strategies, and fostering problem solving.

Animation, e-simulation, and mnemonics (ie, “ABCDE and ISBAR” make it simple and clear to understand”) were benifical.

assess the performance of learner practitioners when performing simulations of assessment/care of the deteriorating patient (Liaw, Scherpbier, Klainin-Yobas and Rethans , 2011)

Liaw, Chan, Chen et al., (2014)73 Singapore Q1. RS iv. To describe the development of the virtual patient simulation and evaluate its efficacy, on nursing students’ performances in assessing and managing patients with clinical deterioration. vii. n= 57 third-year nursing students in total (n=31 experimental group and n=26 control group). v. A randomized controlled study

Q2. Both groups had participated in a full-scale RAPIDS simulation course eight months earlier. Pre-tests results indicated some deficits in performance of students. A virtual patient simulation, known as e-RAPIDS with five acute medical simulation scenarios (experimental group) versus mannequin-based simulation (control group).

Q6. There was no significant difference between the first and second post-tests for both groups (P=.12). However the mannequin-based simulation (control group) demonstrated a more consistent and sustained improvement at 2.5 months, with little decay over time in clinical performance. The experimental group on a 7-point scale indicated that that they were satisfied with the virtual patient simulation (mean 6.06, SD 0.71), quality of the system (mean 6.01, SD 0.56) and information (mean 6.06, SD 0.50), and perceived net benefits (mean 6.28, SD 0.59) of the program.

Q13. Authors suggested that based upon their results that hands-on simulation coupled with the social interaction underlying collaborative learning experience provided deeper learning compared to multimedia teaching modalities. However to accommodate multiple users at one time, provide learning content for a large group of learners and refresher courses virtual patient simulation provides a more resource neutral solution to updating larger numbers. It allows students to engage in repetitive training using the virtual patient simulation.

Q14. 2-

Kyriacos, Jelsma, James et Q2. MEWS. Q6. Q12. Q14. 2+

73

The intervention tested within this RCT is the same as the one described and tested with registered nurses by Liaw et al. (2015a) and Liaw et al. (2015b) described in the previous row.

251 | P a g e







al. (2015) South Africa Q1. RS. iv. To investigate the (i) impact of a new chart incorporating a MEWS system and a linked training programme on nurses’ response to clinical deterioration, (ii) number of patients with vital signs recorded in the first 8 hours post-surgery and (iii) frequency of vital signs recorded and (iv) nurses’ knowledge v. “A pragmatic, parallel-group, cluster randomised, controlled clinical trial of intervention vs standard care” vi. 114 patient cases (19 from each of the 6 wards) were randomly chosen from 1,427 complete case notes (without DNARs) for review 2 weeks post-trial completion. Patients were blinded to their allocation, nurses were

Q4 i. Cardiac arrest teams were replaced with ward response teams two decades previously and there was no hospital-wide RRTs and no EWS in place on the wards. During training nurses to instructed to respond as follows; MEWS 1: recheck in 30 min and report if no improvement MEWS 2: recheck in 5 min and report if no improvement Single MEWS value of 3: report urgently MEWS ≥3: report urgently ii. MEWS charts and the Cape Town MEWS training programme and manual74,75

were implemented in intervention wards (Mar to July 2010). The interactive training took 2 hours and was voluntary. Control wards delivered standard care

i. Introduction of MEWS was not associated with a significant change in response to clinical deterioration. Overall nurses reported 9 of 221 (4.1%) abnormal vital signs. Intervention ward=unrecorded responses to 121 of 128 (94.5%) MEWS triggers that should have been reported for 50 of 57 (87.7%) patients. Control ward = unrecorded responses to 91 of 93 (97.8%) MEWS triggers that should have been reported for 55 of 57 (96.5%) patients. OR=2.63 (95%CI 0.53, 12.97) ii. 63.2% of patients had vital signs recorded in the first 8 hours post-operatively. All patients recorded blood pressure and heart rate. More patients in intervention than control wards had respiratory rate and all seven vital signs recorded; Respiratory rate Intervention ward=26 of 57 Control ward =2 of 57 OR=24.75 (95%CI 5.5, 111.3) All seven vital signs Intervention ward=5 of 57 Control ward =0 of 57 OR=12.05 (95%CI 0.650, 223.19)

ii. Suboptimal compliance with MEWS was problematic. No nurse observation protocol existed in this hospital prior to this intervention which may have attributed to the low recording in the control arm. “Problems arise when nurses are competent in using technology for the monitoring of vital signs but lack clinical knowledge in interpreting data and intervening appropriately to ensure optimum and safe patient care” Knowledge scores only reached 61.4% and may have been hampered by language difficulties and the inclusion of nurses with no academic qualifications. Q13 iii. This study may have been

One centre. Small sample number. It was conducted in a real life setting. Q16. “A MEWS chart and training programme enhanced recording of all parameters, and nurses’ knowledge, but not nurses’ responses to patients who triggered the MEWS reporting algorithm.” There is no evidence that MEWS improved patient clinical outcomes. Therefore, MEWS did not replace clinical judgement in detecting deteriorating patients.

74

KYRIACOS, U., JELSMA, J., JAMES, M. & JORDAN, S. 2014a. Monitoring vital signs: development of a modified early warning scoring (MEWS) system for general wards in a developing country. PLoS One, 9, e87073. Kyriacos, U., Jelsma, J. & Jordan, S. 2014. Record review to explore the adequacy of post-operative vital signs monitoring using a local modified early warning score (mews) chart to evaluate outcomes. PLoS One, 9, e87320. 75

KYRIACOS, U., JELSMA, J. & JORDAN, S. 2014b. Record review to explore the adequacy of post-operative vital signs monitoring using a local modified early warning score (mews) chart to evaluate outcomes. PLoS One, 9, e87320.







252 | P a g e







not. Data collectors were blinded. Nurse knowledge was assessed by written tests before and after training in intervention wards. Patient vital signs were recoded from existing to MEWS charts and the intervention arm where MEWS charts were blank. vii. 50 of the 122 full-time nurses participated viii. 6 adult surgical wards in one public hospital

(existing observation charts). MEWS charts indicate which scores to record, degree of urgency required and where to enter total score.

iii. Nurse knowledge increased significantly post-intervention (P=0.001) in the intervention group, reaching scores of 61.4%. This knowledge did not translate to improved reporting of triggering patients Pre-intervention mean score: 4 of 23 (19.5%) Post-intervention mean score: 14 of 23 (61.4%) Nurse knowledge did not increase in the control group (P=0.144) Pre-intervention mean score: 9 of 23 (37.2%) Post-intervention mean score: 10 of 23 (41.2%)

strengthened by implementing and evaluating compliance with a standard protocol for frequency of physiological observations of at least every 12 hours for every patient as recommended in the NICE guidelines; a hospital wide training programme for the early recognition and management of deterioration and the redesign of the MEWS chart

Lindsey & Jenkins (2013) USA Q1. RS iv. “To investigate the impact of a novel educational intervention on student nurses’ clinical judgement regarding the management of patients experiencing rapid clinical deterioration.” v, A randomised study with a pre- and post-test design vi. All students completed a clinical simulation lab day including a 90 min Code Blue section. Pre- and post-test understanding of RRS was assessed using an 11-item multiple choice questionnaire (MCQ); their purpose and function (5 items), clinical judgment in activating and participating in

Q4 iv. The RRS education intervention consisted of a 10-min lecture, a written handout of the lecture, a rapid response simulation in which each student had a role, with coaching for the students in conducting assessments, ordering diagnostic tests, performing clinical intervention and maintaining open communication. This was followed by a debriefing to review events, emphasise key information and facilitate questioning.

Q5 x. All students had better scores in the post-tests MCQs. Students in the intervention group had significantly higher post-test scores than in the control group (P<0.001).

Q9. Even though 85% of participants had prior knowledge of or exposure to RRSs, mean pre-test scores were relatively low (57%), suggesting gap in knowledge and clinical judgement despite experience.

Q14. 2- Single centre, convenience sample. External validity may have been threatened by the interaction of the pre-test and the educational intervention. Multiple education interventions were used sequentially, which may have confounded the results. A specific EWS was not used to recognise and monitor clinical deterioration. Q16. “Clinical stimulation is effective in improving student knowledge and clinical judgement, specifically concerning RRSs.” (p61). Outcomes of clinical simulations should

253 | P a g e







rapid response calla (5 items) and the extent to which participants had been exposed to RRSs (1 item). Control and intervention groups completed the pre-test prior to the Code Blue simulation. The control group completed the post-tests MCQ immediately after. All then had RRS education and the intervention group them completed the post-test MCQ. vii. Baccalaureate nursing students (n=79) in the final semester of their degree programme randomised to the intervention (n=40) or control (n=39) group. viii. Midwestern public university.

be empirically examined.

Ludikhuize, de Jonge & Goossens (2011) The Netherlands Q1. RS iv. “To investigate whether nurses trained in the use of MEWS and SBAR are more likely to recognise a deteriorating patient”. v. A quasi-experimental

Q2. MEWS45 and SBAR 76 Implemented in Nov 2009. Q3. i. Both of these tools were implemented on 4 nursing wards with the framework of the COMET (Cost and Outcomes analysis of Medical Emergency Teams) study. Nurses

Q6 i. More trained (77%) than untrained nurses (58%) would respond correctly by reviewing the patient immediately (P=0.056). Trained nurses requested respiratory rate significantly more often than untrained nurses (53% and 25%, respectively; P=0.025). There was no difference in the number of other vital signs within MEWS measured.

Q12. Feedback sessions with participants identified some barriers;

i. a) Measuring MEWS is voluntary; b) a lack of involvement by physicians who were not properly informed about the protocol, and therefore were

Q14. 2- Single centre. Nurses did not have the opportunity to assess the patient. Q16. Nurses who are trained in MEWS and SBAR can identify a patient who is deteriorating and react

76

HAIG, K. M., SUTTON, S. & WHITTINGTON, J. 2006. SBAR: a shared mental model for improving communication between clinicians. The Joint Commission Journal on

Quality and Patient Safety, 32, 167-75.

http://www.ncbi.nlm.nih.gov/pubmed/?term=Haig%20KM%5BAuthor%5D&cauthor=true&cauthor_uid=16617948

http://www.ncbi.nlm.nih.gov/pubmed/?term=Sutton%20S%5BAuthor%5D&cauthor=true&cauthor_uid=16617948

http://www.ncbi.nlm.nih.gov/pubmed/?term=Whittington%20J%5BAuthor%5D&cauthor=true&cauthor_uid=16617948

254 | P a g e







study vi. Assessments and responses of nurses to a deteriorating patient chart were observed. vii. Nurses who were trained (n=47) and not trained (n=48 were presented with a nursing chart of a fictitious deteriorating patient case ad as to respond as they would normally, during Sept 2010 (1 year post-implementation). Median age was 28 years, 80% were female. viii. One hospital (the Academic Medical Centre in Amsterdam). Trained nurses were from 3 wards (2 surgical and 1 medical) and untrained nurses were from 3 other wards (2 surgical and 1 medical) and

received education using a simulated patient case. Q4. iv. Within one month of implementation of MEWS and SBAR nurses received interactive training sessions each lasting 1 hour, in small groups (n=15). Nurses were trained to measure MEWS as ‘clinically indicated’ i.e. where one vital sign was abnormal (score ≥1), the entire aggregate score had to be calculated, and to escalate (inform a clinician according to SBAR methodology) at an aggregate MEWS score ≥3. Other interventions including posters, feedback sessions, face-to-face conversations, small posters in each nursing chart, were also used to enhance training

67% and 12% of the trained and untrained nurses contacted the physician immediately (P=0.059). There was no difference in the proportion of escalations by ‘worried’ between the nurse groups. ii. Of the trained nurses, 11% (n=4) calculated MEWS score correctly. Of these 1 nurse followed (2%) protocol correctly by contacting the clinician. SBAR was used by 4% (n=1) of trained nurses; measured parameters were relayed to the clinician in 60% of phone calls. Respiratory rate was relayed significantly more frequently by trained (83%) than untrained (40%) nurses, if it had been measured (P=0.117)

unfamiliar with the clinical significance of MEWS/SBAR, c) established hospital culture hampered immediate physician notification. Q13. iii. a) continuous electronic measurement of vital signs, b) making MEWS measurements mandatory, c) encourage physicians to use MEWS, d) mandatory bedside patient evaluation by physician rather than telephone consultation.

more appropriately than nurses who are not trained. However, these tools are rarely used one year post-implementation despite rigorous implementation of MEWS and SBAR methodology. Communication remains suboptimal. Training works but there was little change in behaviour, and there is “room for improvement.” Future research on implementation strategies is required.

Merriel, van der Nelson, Merriel et al. (2015) UK Q1. RS iv. “To establish whether a short multidisciplinary training intervention can improve recognition of the deteriorating patient using an aggregated physiological parameter scoring system (EWS).” (p1)

Q2. Multidisciplinary (Nursing, medical, and allied nursing staff) training (one hour training using real-life scenarios, simple tools and structured debriefing relating to EWS and SBAR. Training was delivered in an area nearby to the wards to create a realistic setting for participants. Q3. Recording if key

Q6. ii. After the educational programme;

(a) Staff were more likely to calculate EWS scores correctly, compared to pre-intervention (68.02% vs 55.12%; risk ratio = 1.24, 95% CI 1.07, 1.44).

(b) Observations were more likely to be performed at the correct frequency, compared to pre-intervention (78.57% vs 68.09%; risk ratio = 1.20, 95% CI 1.09, 1.32)

Q9 vi. “Teams may have worked more cohesively after training, as suggested by medical review of unwell patients taking place more quickly after training” (p5) viii. The taking of observations as per EWS guideline improved from 46.10 to 58.57% following implementation of the educational intervention (risk

Q14.2+ Attempts to minimise seasonal and selection bias were made. Longer follow-up time. Q16. “Multidisciplinary training, according to core principles, can lead to more accurate identification of deteriorating patients, with implications for

255 | P a g e







v. Observational before-after study vi. Documentation of EWS use was collected from a pragmatic number of patient notes (n=250) in the 6 months pre- and 6 months post-implementation to minimize bias from potential seasonal variation in outcomes. Charts were chosen by medical records staff. Data on number of RRT calls and time of the call were also collected. vii Participants in the 22 training sessions included nursing staff (n=83) and junior doctors (n=19). 78% of eligible nursing staff and 83% of eligible junior doctors participated. Baseline data were obtained from 282 patients in the pre-implementation and 210 patients in the post-implementation period on 3 separate surgical wards. viii. National Health Service teaching hospital in the United Kingdom

physiological variables: temp. pulse, RR, BP, SpO2).

ratio = 1.33; 95% CI 1.13, 1.57).

subsequent care and outcome.” (p1)

Ozekcin, Tuite, Willner et al. (2015) USA Q1. RS iv. To improve nurses’ ability recognise and assess

Q3. iii. SBAR Q4, iv. The aim of the educational programme was to increase

Q5. v. Time to application of the first correct critical intervention was faster, decreasing from 37% to 25% into the scenario time from scenario 1 to 2. x. The mean knowledge score increased

Q14. 2- Single centre, convenience sample. Immediate evaluation of knowledge and response.

256 | P a g e







deteriorating patients, treat and escalate care v. An intervention study with a pre- and post-test and observational design vi. A 14-item MCQ was used pre- and post-test regarding signs of clinical deterioration and how to communicate escalation was measured for each participant. Confidence was measured on a Likert scale. Improvement of care following simulation was evaluated by (a) shorter time to recognition of unstable state (b) faster escalation to care and (c) consistent use of SBAR for reporting communicating the patient’s condition. The project took place over 4 weeks, with 10 simulation sessions, with 3 to 5 nurses per simulation. vii. 35 registered nurses who worked on the ward for at least 6 months participated. viii. The cardiac surgical universal care unit in one hospital

recognition of cues and clinical triggers for potential clinical deterioration in patient assessment and to empower nurses to communicate concern about clinical deterioration through the use of a specific escalation protocol (to experienced nurses from critical care), and report concern using SBAR. The educational programme was in 2 phases an e-learning module and simulation scenarios using a PDSA (Plan, Do, Study, Act) cycle framework. The simulation element included 2 instability scenarios conducted in groups. An intervening debrief was incorporated between scenarios. The programmes were evaluated by comparing pre- and post-test knowledge.

significantly post-intervention (84.6% SD 10%) compared to the pre-test score (56.5% SD 17%) P<0.0001). Participants were significantly more confident in recognising clinical deterioration (P=0.001), responding to an unstable patients (P<0.0001), and felt empowered to coordinate immediate responders (P=0.0001) and to use SBAR (P=0.04). Years of experience did not correlate with knowledge pre- or post-test. Q6. i. Time to escalation to care was faster, decreasing from 66% to 61% into the scenario time from scenario 1 to 2. iii. SBAR was consistently used in communication of clinical deterioration. But qualitative analysis of responses revealed a wide range of confidence/or not among nurses regarding escalating to more senior people.

Q16. “Use of e-learning, simulation and debriefing, with SBAR communication for an escalation-of-care protocol, can improve instability recognition and communication, resulting in improved knowledge and decreased time to critical actions.” (p.166)

Rose, Hanna, Nur et al. (2015) USA Q1iv. The aim was to investigate whether use of an electronic MEWS (eMEWS) as a clinical decision. tool would improve eMEWs

Q2. eMEWS and the Capsule Vital Signs Now (CVSN), a portable medical device that provides wireless transmission of collected vital signs into the Meditech documentation system

Q6. i. A decrease in RRT calls was observed post-implementation (17/90 days) compared to pre-implementation (23/90 days; 100% survival) A decrease in CB calls was observed post-implementation (1/90 days) compared to pre-implementation (6/90 days; 1 patient died).

Q7. i. The number of unplanned ICU admissions increased post-implementation (64%) compared to pre-implementation (43%)

Q12. ii. Interviews with staff revealed that (i) staff did not know what staff were in the RRT; (ii) there were insufficient and malfunctioning handheld CVSN units

Q14. 2- Cohort study, no control group, no data in relation to patient dependency or number of calls per population Q16. There was an

257 | P a g e







documentation would result in more appropriate the faster activation of RRT by nurses. v, A pre-, post-intervention analysis of a quality improvement project vi. The effectiveness of staff education on eMEWS was assessed by examining eMEWS scores and RRT and Code Blue (CB) data in two 90-day phases: pre- and post-intervention. Data on (a) number of RRT calls, (b) number of CB calls, (c) reason for the RRT/CB call, (d) the eMEWS score triggering the call, (e) outcome of the RRT/CB call, (f) the patient location at the time of the call, and (g) the patient location after the call, was extracted. Staff knowledge was assessed by a specific survey assessing (1) knowledge/ experience of eMEWS/RRT before initial eMEWS/CVSN implementation; (2) awareness of eMEWS/CVNS and RRT currently used. vii. 108 core staff members participated, mean age (42.12 SD 11.96) years. These were registered nurses, licensed practical nurse, certified nurse assistant,

Q4. i. A RRT was implemented 1 year prior to this study iv. Staff were educated and supported by the CVSN project trainers and super users at the time of rollout. Staff were educated on eMEWS only through materials located on the facility’s intranet. Knowledge was low, therefore this educational intervention was investigated; The multi-modal Educational intervention; A 3-min presentation one-on-one or small group education on the purpose and use of eMEWS. strategies to rescue the patient, significance of MEWS as a clinical decision support tool, instructions for documentation of eMEWS in the CVSN system and manually, with special emphasis on the documentation of loss of consciousness; a review of the process of activating the RRT, and emphasis on the importance of the trending data display at the bedside, timely documentation at the bedside, and increased communication among clinical caregivers based on eMEWS information.

The mean eMEWS score at RRT activation increased post-implementation (2.3 ± 1.79; range 0-6) compared to pre-implementation (3.2 ± 1.79; range 1-6) ii. The number of undocumented eMEWS scores decreased post-implementation (0/17 RRT events; 0%) compared to pre-implementation (11/23 RRT events; 49%)

Q13. iii. Responses implemented as in response to barriers; (i) Nursing supervisors were asked to identify members of the RRT to core staff at the beginning of an event; (ii) more CVSN units were ordered and processes to address malfunctioning CVSN units was addressed.

improvement in the use of the eMEWS score as a clinical decision making tool to engage the RRT by core staff and improved patient rescue strategies following this educational intervention, documentation was also improved. Changes incorporated and recommended as a result of these findings are; (a) eMEWS training will be incorporated into the training of all new employees, (b) frequent brief eMEWS educational items will become part of monthly staff meetings, (c) core staff will be included in monthly reviews of RRT and CB events and (d) a bidirectional level of communication will be established regarding problem identification and solving, doing and reflecting, and uniform buy-in from all stakeholders establishing the ultimate goal of eliminating failure to rescue (p5,6)

258 | P a g e







nurse technologist, and respiratory therapist. viii. Three non-critical units in one community hospital

Educational materials with key messages including the eMEWS algorithm score, trigger points, and instructions on the physical process of engaging the RRT were also given to the staff on preprinted 4x6 cards.

Shaddel, Khosla & Banerjee (2014) UK Q1. RS iv. To assess the level of confidence of mental health nurses, and their ability to make clinical decisions, before and after introduction of the MEWS tool with associated 15 minutes of training on MEWS. v. A pre and post-MEWS training intervention survey vi. Survey vii.n=19 nurses working in either an inpatient learning disability unit or psychiatric unit. viii. Inpatient psychiatric settings UK hospital.

Q2 Introduction of MEWS and MEWS training

Q6 iii The mean before and after level of nurses’ confidence in their clinical judgement was 3.73, and 4.63 respectively (Z= 3.81, P=0.0001).

Q14. 2- Q16 Use of MEWS can increase the level of confidence of nursing staff working in mental health inpatient wards and their ability to recognise and manage physically deteriorating patients.

259 | P a g e

Table 6. Implementation of EWS and RRTs.







Bunkenborg, Samuelson, Poulsen et al. (2013) Denmark Q1. RS iv. To evaluate the “short and long term effects of systematic inter-professional use of EWS, structured observational charts, and clinical algorithms for bedside action” on unexpected in-hospital death (i.e. sudden, no resuscitation), death after cardiopulmonary resuscitation, and death within 24h of admission for intensive care) death. v. Prospective, non-randomised controlled study before and after clinical intervention implementation” vi. Data was collected over three 4-month periods; (i) pre-intervention (Mar-June 2009); post-intervention ((ii)

Q2. 3 compon ent clinical intervention: an (i) EWS, (ii) observational chart and (ii) algorithm for bedside action. Q3. i A new monitoring practice i.e. the systematic use of MEWS77 (values for O2 saturation were also recorded, but with no scoring) obtained and calculated ~8 hourly for terminal patients and more frequently for those with signs of clinical deterioration; Q4 ii. An observation chart had a; colour-coding in green, yellow, orange or red on top of the chart were used for each parameter score and range An algorithm for bedside

Q5 i. Unexpected patient mortality decreased post-intervention: 25 (1st post-intervention) vs 61 (pre-intervention) per 100 adjusted patient years (P=0.053). Rate ratio: 0.404 (95%CI 0.161-1.012) 17 (2nd post-intervention) vs 61 (pre-intervention) pre 100 adjusted patient years (P=0.013). Rate ratio: 0.271 (95%CI 0.097-0.762) x. The number of SAEs decreased post-intervention (n=21) compared to pre-intervention (n=31).

Q7. i. No difference in the number of ICU admissions were observed (n=17 both pre and post-intervention) which may be attributable to improved patient monitoring and earlier intervention.

Q13. iii. “Recommendations on the vital parameters to assess were strictly perused, the recommended minimum time interval for repeated in-hospital bedside assessments was reduced (from 12 to 8 h), and individual knowledge and skills, as well as interpersonal communication and collaboration, were continuously optimized.”

Q14. 2- Results need to be interpreted in light of the changes in hospital organisation. Results from individual elements of the intervention cannot be elucidated. Q16. “Clinical Intervention comprising systematic monitoring practice, EWS, and observational chart, and an algorithm for bedside management, implemented by inter-professional teaching, training and optimization of communication and collaboration, may significantly reduce unexpected in-hospital mortality.”

77

SUBBE, C. P., KRUGER, M., RUTHERFORD, P. & GEMMEL, L. 2001. Validation of a modified Early Warning Score in medical admissions. QJM, 94, 521-526.

260 | P a g e







Sept-Dec 2010) and (III) Mar-June 2011)) vii. All adult patients (≥18 years), without DNRs, with at least a 24h stay during pre-intervention (2009); n=1870 (9,804 in-hospital days of care; mean age 58±19 years; 42% male). Post-intervention recruitment of 2,079 adult patients (mean age 57±20 years; 44% male, 12,584 patient days (2010)) and 2,234 patients (mean age 57±20 years; 41% male, 13,356 patient days (2011)) viii. Medical and surgical wards in one hospital. The number of high intensity monitoring beds increased from 11 to 18 (2009-11) and number of healthcare professional employed increased. There was an organisational expansion of catchment area from 280-460,000 people in 2010. The in-hospital emergency team had been in place for 2 years prior to the start of the study.

action; the chart also had a colour-coded algorithm for clinical management (green (MEWS=0: no specific further action) to red (MEWS ≥5 (urgent and appropriate bedside action). Q4. x. Implementation process: (i). Close and continuous collaboration with nursing and medical staff and managers. (ii) Teaching, training and promotion for all staff (iii) Communication and collaboration (iv) Feedback visits

Clifton, Clifton, Sandu et al. (2015) UK Q1. RS iv. “To understand factors associated with errors using an established paper-based

Q2. A 5-item EWS (HR (bpm), BR (respirations per minute), temperature (0C), SBP (mmHG), and SpO2 (%) which had been standard practice in the ward for ≥1 year before the study began. An

Q6. ii. 85.2% of observational sets were complete for the 5 vital signs. 77.9% had the 5 vital signs and weightings. 65.5% were complete with correct aggregate score. Temperature was most often missing (in 11.4% observational sets).

Q9. viii. Errors in EWSs as documented suggest that clinicians are using information additional to the vital signs or the EWS system wihc does not completely

Q14. 2- A single surgical ward, with a site-specific EWS chart and training, and staff were aware they were participating in a study.

261 | P a g e







EWS system; (i) the types of error, (ii) where they are most likely to occur, and (iii) whether ‘errors’ can predict subsequent changes in patient vital signs.” (p.2) v. A retrospective analysis of a prospectively collected EWS database. vi. A database established during the Computer ALerting Monitoring System 2 (CALMS-2) study was used. This database consisted of paper-based EWS observation sets collected from consecutive, eligible, postsurgical patients. Paper-based EWSs were electronically transcribed at the end of each working day independently by two trained ward staff, into two databases. Data transcribed were (i) vital signs recorded, (ii) weights assigned to each vital sign, and (iii) aggregate scores recorded by ward staff. vii. 200 postsurgical (upper-gastrointestinal) patients, (16,795 observation sets). Median age was 64 years (IQR 15 years) viii. A specialist surgical ward (between 20 and 28 beds) in one large UK teaching hospital.

aggregate EWS score of ≥4, or a single vital sign of 3 was deemed to have an ‘alert.’ Q4. iv. Ward staff received training in the use of the EWS when it was introduced or when they first started work on the ward using a structured training package developed by the hospital’s Recognition of Acutely Ill and Deteriorating Patients committee. Annual updates occurred as part of the resuscitation training.

A number of different errors were observed: a. 33.1% of observations sets that should have led to an alert had no score recorded b. 18.4% of observations sets that caused alert, did so incorrectly c. incomplete observations sets should have led to an alert more often than complete observation sets (15.5% vs 7.6%, P<0.001) d. incomplete observations sets less often had appropriate alerting scores recorded than complete observation sets (61.3% vs 70.0%, P<0.001) e. observations sets that should not have led to an alert, but which had an alerting score recorded was more common in incomplete than complete observation sets (3.2% vs 0.9%, P<0.001) f. Mis-scoring was more common when leaving a sequence of ≥3 consecutive observation sets with aggregate scores of 0 (55.3%) than within the sequence (3.0%, p<0.001). g. Percentage of errors were higher in observation set sequences where EWS values were ≥1 than EWS values of 0 (27.8% v 3%) 16.9% of complete observation sets had errors in the aggregate score, assignment of weights, or both. Error rates were not increased at night, even though staffing levels were lower.

encapsulate patient risk. If correctly predictive aggregate scores are taken to be the optimum, assessment of patient status, the effect of ‘error’ rates reduce the proportion of complete observation sets that should have an alerting score but did not from 30 to 8%. Assessments of EWS performance should report proportions of alerts missed or generated in error, instead of the overall error rate. Q10. i. The negative influence of preceding stable aggregate EWS score sequences on the ability of clinical staff to identify the first signs of patient instability could be dealt with electronically by allowing clinicians to see the previous observation sets only when all the vital signs had been entered. “This approach may not be necessary if assignment and summation of weights were automated.” (p.5) Assigning a weight to clinician concern within the EWS may improve the accuracy of the EWS.

Q16. Decreasing the number of incomplete observation sets would improvement paper-based EWS performance and patient care. “Missed alerts are particularly common in incomplete observation sets and when a patient first becomes unstable. Observation sets that ‘incorrectly’ alert or ‘incorrectly’ do not alert are highly predictive of the next observation set, suggesting that clinical staff ‘outperform EWS’ by detect deterioration and improvement in advance of the EWS system by using information not currently encoded within it. Work is needed to understand how best to capture this information and incorporate it into future EWS.” (p.1) Since ‘clinician concern’ is not part of the aggregate EWS, “there is a danger that the ‘power of the score’ will subjugate clinicians concern and the wiliness of a RRT to respond to patients with a ‘normal’ EWS score.” (p.5)

262 | P a g e







Q12. iii. Despite lower staffing levels at night, there was no difference in error rate from daytime.

Considine, Rawet, & Currey (2015) Australia Q1. RS iv. “To evaluate the effect of the staged implementation of a RRS on reporting of clinical deterioration in ED patients.” (p.219) v. A pre-post, retrospective cross sectional study. vi. Data was collected by medical record audit. vii. A random selection of adult (≥18 years) patients attending the ED with shortness of breath, chest pain or abdominal pain, as the most commonly presenting complaints in the ED. 150 patients were randomly selected in each of 4 years: pre-implementation (2009) and post-implementation (2010, 11, 12). Median age (n=600) was 55 years (IQR=36-72) and 47.3% were male. viii. One 300-bed district hospital with between 63,426 and 66,013 ED

Q2. ED Clinical Instability Criteria (ED CIC) implemented in 2009. The ED CIC has 10 items: Breathing ((i) obstruction/threatened airway; (ii) SpO2 <90%; (iii) RR <10 or >30 breaths/min; (iv) arterial blood gases pH<7.20); Circulation ((v) HH <50 or >120 beats/min; (vi) SBP <90 or >200 mmHg; (vii) urine output <20 mL/h or <100mL/6h); Disability (viii) GCS score >2 points or sudden decrease in consciousness) (ix) repeated or prolonged seizures and (x) Worried? (Patients may have a sudden deterioration in their medical condition requiring urgent medical review but not meet the above criteria. If patients fulfilled any of these criteria an escalation protocol was triggered. Q3. The escalation of care protocol was implemented in 2009.

Q5. i. In-hospital mortality was significantly higher among patients who deteriorated in the ED.

vii. Frequency of documented clinical deterioration fulfilling the ED CIC criteria was 14.8% (318 episodes in 89 patients). There was no significant difference in the frequency of clinical deterioration between 2009 and 2012. Unreported clinical deterioration decreased from pre- to post-implementation; this was clinically significant but not statistically significantly (P=0.141); 2009-10: 17.9% decrease 2010-11: 13.5% decrease 2011-12: 1.3% decrease Overall decrease 32.7% (2009-12; P=0.198). RR >30 breaths/min, HR >120 beats/min and SBP <90 mmHg were most commonly unreported. Between 2009 and 2012 unreported RR and HR decreased by 57.6% and 15.9%, respectively. SBP decreased between 2009 and 2011 (45.7%) and increased during 2011 and 2012 (10%). Patients who were significantly more likely to deteriorate (p<0.001) if they arrived by ambulance, were triaged to Australasian Triage Scale (ATS) categories 1 or 2, and had a mean 2.8 hour longer median stay compared to patients who did not

Q9. v. Each element of the of the ED RRS had an independent effect on the 32.7% decrease in unreported clinical deterioration; (i) 17.9% reduction following introduction of the ED CIC and escalation of care protocol, (ii) 13.5% reduction following implementation of the single parameter track-and–trigger chart. The further 1.3% decrease in 2012 may be explained by ‘diffusion of innovation’ i.e. communication about the innovation which results in changes in the structure or function of a system.

Q14. 2- Q16. “A staged ED specific RRS decreased the frequency of unreported clinical deterioration. Each specific element of the RRS had an independent but cumulative effect in reducing unreported clinical deterioration. However, the effectiveness of the RRS plateaued when dissemination and implementation were complete and information regarding the RRS reverted to diffusion. Ongoing strategies to ensure clinical engagement with the aims, structure and function of RRSs are needed if ongoing improvements in reporting of clinical deterioration are to occur over time.” (p.225)

263 | P a g e







attendances per year. ii. ED clinicians were required to alert the emergency physician in charge if a patient breached any of the ED CIC criteria, who was required to review the patient within 5 minutes of the notification. Q4. ii. Observation chart changed to a ‘single parameter track-and-trigger chart’ (implemented in 2010). This chart required graphical recording of RR, HR and BP. Space to record the ED CIC physiological parameters were colour coded in red as a visual trigger that escalation was required.

deteriorate. Patients who deteriorated in the ED were also significantly more likely to be admitted than those who did not (31.9%; P<0.001).

Considine, Charlesworth, Currey et al. (2014) Australia Q1. RS iv. To determine prevalence of emergency MET responses for clinical deterioration within 24 hours of emergency admission, and investigate the effect on patient outcome of MET within or beyond 24 hours v. A retrospective descriptive exploratory study vi. Cardiac arrest, MET and ED databases were audited

Q4. i. The MET consists of a consultant intensivist or ICU registrar, a specialist ICU nurse, and medical registrar and medical staff from the patient’s treating team. MET is activated for the following; Difficulty breathing RR<8 or >30 breaths/min O2 saturation <90%, despite O2 administration at 6L/min via a simple mask HR <50 beats/min or >130 beats/min SBP <90mmHg

Q5 x. ED LOS was not significantly different for patients whose MET response was within 24 hours of emergency admission and those whose MET response was beyond 24 hours of admission. Q6. i. 819 patients in the hospital required 1480 emergency responses; 1203 (81.3%) of which were MET activations Most of these patients (n=587) were admitted through the ED; these patients constituted 819 (55.3%) of all MET responses. Patients whose first MET call was within 24 hours

Q7 i. One in eight patient required ICU admission following MET activation. Patients whose MET response was within 24 hours of emergency admission (7.6%) were significantly less likely to be admitted to ICU following MET response compared to those whose MET response was beyond 24 hours of admission (13.9%; P=0.039). iii. Patients whose MET response was within 24 hours of emergency admission were significantly more likely to have a shorter hospital LOS (7 days) compared to those whose MET response was

Q14. 3 Single centre. No established EWS linked with the MET. Q16. MET responses to emergency admissions through the ED constitute more than half of the MET calls in the hospital. More than one-quarter of MET activations for emergency admissions occur within 24 hours.

264 | P a g e







for data on eligible patients admitted during 2012. The Australasian Triage Scale (ATS) was extracted. vii. Adult (≥18 years) medical or surgical patients admitted via the ED, and who required MET response for clinical deterioration (n=819 patients requiring MET, n=587 admitted via ED. Of these ED patients median age was 79 (IQR 65-86) and 50.9% were male). viii. The ED in one hospital (Box Hill Hospital).

New or unrelenting chest pain Acute change in conscious state, or seizure Clinician concern The ATS ranges from 1 (immediate assessment and treatment) to 5 (treatment within 120 mins)

were significantly more likely to have a ATS category 1 (5.4%) compared to those whose MET response was beyond 24 hours of admission (1.2%; P=0.005). Median time to 1st MET response was 59 hours for 28.4% of patients. 1st MET response within 24 hours for clinical deterioration occurred within 24 hours of emergency admission for 28.4% (n=167) patients Patients whose MET response was within 24 hours of emergency admission were significantly less likely to have recurrent emergency responses during their hospital stay (9.7%) than those whose MET response was beyond 24 hours of admission (34.0%; P<0.001)..

beyond 24 hours of admission (11 days; P=0.039).

De Meester, Das, Hellemans et al. (2013a) Belgium Q1. RS iv. To investigate the effect of a standardised nurse observation protocol implementing MEWS and a colour graphic observation chart” on observation frequency and prevalence of SAEs i.e. number of deaths without a DNR order and readmission to ICU, in the 5 days post-ICU discharge. v. An observational data analysis pre- post-intervention study. vi. Data on type and

Q2. MEWS implemented as a standard nurse observation protocol in Nov 2009. Q3. Patients were observed at ICU discharge, admission to the ward, 4 hours post-ward admission and every 12 hours thereafter. The observation protocol was adjusted based on MEWS score; Observation frequency was increased to every 30 mins and the patients physician notified if; (i) MEWS score

Q5. i. Number of unexplained deaths decreased from 4 to 0 patients during pre- and post-intervention, respectively. This was not statistically significant. x. There was an absolute reduction of 2.2% (95%CI -0.4%, 4.67%) in SAEs 5-days post-ICU discharge, from 5.7% pre-intervention to 3.5% post-intervention. This was not statistically significant. The mean MEWS was significantly higher in patients who experienced an SAE, in the shift during which an SAE occurred, and in the preceding three shifts, compared to the score of patients who did not experience an SAE (P<0.001). MEWS ≥4 had a predictive value for SAEs in the 5 days after ICU discharge. These patients had a 2 times greater risk of an SAE

Q8. A MEWS score of 2 during the shift in which the SAE occurred, or a MEWS score of 3 for the preceding one, two or three shifts had the highest sensitivity and specificity; Shift of the SAE, MEWS 2; AUROC=0.796 (95%CI 0.631,0.961) Sensitivity=69.2% Specificity=84.6% One shift before the SAE, MEWS 3; AUROC=0.751 (95%CI 0.573,0.930) Sensitivity=61.5% Specificity=92.3% Two shifts before the SAE, MEWS 3; AUROC=0.774 (95%CI 0.597,0.950)

Q10. A mean observation frequency of at least two-times in 24 hours was not achieved, during the 5 days post-ICU discharge, with a very low frequency of observations at night. Risk stratification of patients at the beginning of night shifts may help to identify those in need of more frequent observations.

Q14. 2+ Q16. Introducing a standard nurse observation protocol with a MEWS score calculated after ICU discharge increased the observation frequency, particularly in the non-monitoring areas, and decreased the number of SAEs. The effect was obtained by implementing the afferent limb of the RRS without introducing a RRT apart from the existing cardiac arrest team. MEWSs scores of 4 had a predictive value for SAEs in

265 | P a g e







frequency of vital signs recorded and MEWS calculated was extracted retrospectively from patient records for 5-days post-ICU discharge to medical or surgical wards in the pre-intervention period (Nov 2008-Feb 2009 and June 2009-Oct 2009). Post-intervention (Nov 2009-June 2010) data was screened prospectively. vii. Adult patients (≥16 years) without a DNR order pre- (n=530) and post-intervention (n=509). (Cardiac surgery patients were excluded because of a pre-existing ward protocol for observation and communication. The majority of patients were admitted to hospital for surgery (65%). viii. 14 medical and surgical hospital wards in one tertiary referral hospital. 39 of the 573 beds provided intensive care.

increased by 2 points between two measurements, (ii) one vital sign had a score of 3, (iii) MEWS score = 4; (iv) nurse did not feel safe with patient’s condition Q4. A pre-existing cardiac arrest team.

OR 2.09 (95%CI 1.29, 3.40) Q6.

i. ii. Overall compliance was low; the observation protocol was observed in 53% of the patients. Mean patient observation frequency during 5-days post ICU discharge increased significantly post-intervention compared to pre-intervention (P=0.005),

ii. Pre-intervention; 0.9993 (95%CI 0.9637-1.0350) iii. Post-intervention; 1.0732 (95%CI 1.0362-1.1101).

Mean patient observation frequency was significantly lower at night than during the day (P<0.001). There was no significant difference in mean patient observation frequency at night post-intervention compared to pre-intervention (P=0.065). Mean patient observation frequency increased significantly post-intervention compared to pre-intervention (P<0.001).

Sensitivity=61.5% Specificity=88.5% One Shift before the SAE, MEWS 3; AUROC=0.695 (95%CI 0.494,0.897) Sensitivity=53.8% Specificity=88.5% A MEWS score of 3 at ICU discharge was a poor predictor of an SAE within 5 days. AUROC=0.602 Sensitivity=40.0% Specificity=76.0% SAPS of 3 at ICU admission predicted an SAE within 5 days as follows. AUROC=0.703 Sensitivity=61.0% Specificity=74.0%

patients post-ICU discharge, up to three shifts prior to the SAE. But the sensitivity of MEWS is relatively low, therefore clinical judgement is crucial. Intensive care specialists can use MEWS or SAPS 3 to assess patients at risk of an SAE post-ICU discharge.

De Meester, Verspuy, Monsieurs et al. (2013b) Belgium Q1. RS iv. “To determine the effect of SBAR (Situation, Background, Assessment, Recommendation)

Q2. MEWS and coloured graphical observation chart was in place from Nov 2009. SBAR was introduced in 2011, with a structured education and instruction on the use of SBAR for the nurses.

Q5. i. Number of unexpected deaths decreased significantly post-intervention; relative risk ratio (RRR) = -227% (95%CI -793,-20) (P<0.001). Pre-intervention: 16 (0.99/1000 admissions) Post-intervention: 5 (0.34/1000 admissions)

Q7. i. Number of unplanned ICU transfers increased significantly post-intervention; RRR=50% (95%CI 30, 64) (P=0.001). Pre-intervention: 51 (13.1/1000 admissions) Post-intervention: 105 (13.1/1000

Q9. vi. SBAR helped nurses who may previously have been reluctant to call a doctor because of uncertainty or a fear of ‘looking stupid’. Q13.

Q14. 2+ Single centre. Q16. Introduction of SBAR increased perception of effective communication and Collaboration in nurses. Using SBAR items in patient records,

266 | P a g e







communication78 in deteriorating patients on the perception of effective communication and collaboration between nurses and physicians and on the incidence of SAEs (unexpected death, unplanned ICU admission and cardiac arrest team calls) in adult hospital wards.” (p.1193) v. A pre-, post-intervention study. vi. (i) data on SBAR items was extracted from patient records up to 48 hours prior to the SAE, and (ii) a nurse questionnaire, including the Communication, collaboration and Critical Thinking Quality Patient Outcomes Survey Tool (CCCT) was undertaken to measure nurse-physician communication and collaboration. vii. SBAR items were checked in 207 SAEs occurrences among 37,239 adult (≥16 years) admissions without a DNAR order, in 210,074 in-patient days. 81 SAEs occurs pre-intervention (July 2010-

Is this Q2?

There was no significant difference in mortality pre- and post-intervention. Q6. i. No significant difference in cardiac arrest team calls pre- and post-intervention. ii. Frequency of SBAR item notations increased significantly post-intervention compared to pre intervention (56% and 32%, respectively; P<0.005). For SAE’s the percentage of records with all 4 SBAR elements increased significantly post-intervention compared to pre intervention (35% and 4%, respectively; P<0.001). iii. Nurses total score on the CCCT Tool increased significantly post-intervention, as did all the subscales; (i) collaboration, (ii) communication with physicians and (iii) overall perception of communication.

admissions) iii. No significant difference in hospital LOS pre- and post-intervention.

iii. “To help nurses in the use of SBAR they were educated in critical thinking in order to become more confident in the assessment of a patient’s condition and in the formulation of a recommendation for treatment to a doctor.” (p.1195) Next step is to educate and instruct doctors on the use of SBAR.

nurses were more prepared prior to calling a physician and more able to make recommendations based on thorough assessment. A shift towards earlier detection, trigger and response was observed with a decrease in unexpected deaths and an increase in unplanned ICU transfers, which is potentially attributable to the introduction of SBAR.

78

LEONARD, M., GRAHAM, S. & BONACUM, D. 2004. The human factor: the critical importance of effective teamwork and communication in providing safe care. Quality & Safety in Health Care, 13, 185-190.

267 | P a g e







April 2011) and 126 post-intervention (May 2011-Marxh 2012). 425 nurses involved in the direct care of medical and surgical patients completed a questionnaire pre- (n=245; response rate=72%), and post-intervention (n=180; response rate=53%). viii. Antwerp University Hospital, a 16 ward hospital

Drower, McKeany, Jogia et al. (2013) New Zealand Q1. RS. iv. To evaluate the introduction of a EWS system (patient observational chart with escalation protocol) on the incidence in-hospital adult (≥16 years) cardiac arrest. v. A before-after retrospective evaluation study during two 12 month periods between March 2009 and March 2011. vi. Incidence of cardiac arrests pre- and post-intervention was obtained from the hospital Resuscitation Audit Forms. Cardiac arrests that occurred in the ED, ICU or operating theatres were excluded. vii. 44,184 adult hospital

Q2. The 8-item EWS tool was developed within the hospital; level of consciousness (AVPU); RR, O2 flow rate, O2 saturation, HR, SBP, Temp and 4 hour urine output. Q3.ii. A cardiac arrest team with a core staff of an ICU registrar. A senior ICU nurse, a senior cardiac care unit nurse, is activated through telecommunication system operators that respond to all cardiac arrests. Q4. EWS and the escalation protocol were integrated into the Adult Deterioration Detection system (ADDS) observational chart modified locally – orientation, parameters that attracted a score and location of the

Q5. i. The rate of cardiac arrests per admission

decreased by 38% post-intervention implementation (P=0.005); Pre-intervention: 4.67 per 1000 admissions Post-intervention: 2.91 per 1000 admissions The number of cardiac arrest responses decreased post-intervention implementation; Pre-intervention: 8.5 arrests per month Post-intervention: 5.5 arrests per month

Q6. i. There was a non-significant increase in the

number of emergency calls post-intervention implementation; Pre-intervention: 7.5 per month Post-intervention: 9.1 per month

Q9. Of the 621 arrest calls made in the 24 month study period, only 168 (27%) were cardiac arrests as defined by the study protocol. The others were (i) medical emergency calls (n=199; 32%) or (ii) calls without a completed audit form or an audit form with insufficient information (n=254; 41%).

Q14. 2- One hospital. Non-randomised study design. Introducing a number of changes simultaneously means the individual effect of EWS cannot be assessed. Large amounts of missing data were problematic. Q16. Introducing an EWS system in the form of an ADDS chart in addition to an existing cardiac arrest team response appears to have decreased the number of in-hospital cardiac arrest responses during the implementation period, without significantly increasing the number of medical emergency calls, in a tertiary hospital.

268 | P a g e







admissions; 21,806 between 2009 and 2010; 22,378 between 2010 and 2011. viii. A 600-bed hospital

EWS score-dependent escalation protocols.

Edelson, Retzer, Weidman et al. (2011) USA Q1. RS iv. “To develop and test a judgement-based scale for conveying the risk of clinical deterioration.” v. A prospective, observational study. vi. PAR was completed a survey once per shift, every 2-4 days. Surveys were completed at the end of their shifts to minimise the chance of them changing their clinical plan based on the PAR score. vii. Physicians (internal medicine interns, residents, attending physicians) and midlevel practitioners (nurse practitioners, physician assistants) (total n=40) were asked to complete a PAR score for all their patents (n=1,663), resulting in 6,034 individual scores on 3,419 patient-days (Jan to June 2008). viii. Nine adult medical wards in 1 hospital (The University of Chicago Medical Centre)

Q2. PAR is a 7-point Likert score from 1 (extremely unlikely) to 7 (extremely likely) representing the likelihood that a patient will experience a cardiac arrest or be transferred to ICU, within the next 24 hours.

Q5. x. Average PAR was 2.9±1.4, with moderate inter-rater reliability (weighted kappa 0.32-0.43). 74 of 3419 patient days resulted in cardiac arrest or patient transfer to ICU (2.6%). PAR could predict cardiac arrest or patient transfer to ICU within 24 hours; PAR (all clinicians) AUROC = 0.82 (95%CI 0.77,0.87) The ability of PAR to predict cardiac arrest or transfer to ICU within 24 hours differed significantly depending on the user (P=0.01). PAR (residents) AUROC = 0.69 (95%CI 0.59, 0.78) PAR (attendings) AUROC = 0.85 (95%CI 0.78,0.90)

Q8. A PAR score cut-offs had the following characteristics for predicting cardiac arrest or patient transfer to ICU within 24 hours; PAR≥4 Sensitivity=82,4% Specificity=68.3% PAR≥5 Sensitivity=84.6% Specificity=62.2%

Q14. 2- Pilot study in one centre, not blinded. Q16. Clinical judgement of patient stability can be reliably quantified with PAR. Implementation of PAR could improve the communication regarding at risk patients between healthcare professionals during handoffs.

269 | P a g e







Hackmann, Chen, Chipara et al. (2011) USA Q1. RS. iv. “To investigate implementation of a two-tier EWS to identify the signs of clinical deterioration and provide early warning of serious clinical events i.e. transfer to ICU. “ v. A proof of concept study involving (i) retrospective data analysis and (ii) clinical trial. vi. A EMR dataset of demographic and clinical data, including vital sign data, and whether the patients was transferred to ICU, 28,927 hospital visits from 19,116 distinct patients (July 2007 to Jan 2010). Logistic regression was used to predict patient outcome i.e. ICU transfer or not, for a single 24 h period for each patients not admitted to ICU and 26 h for those who were, from this data. Real-time simulation was performed on data from 1,204 patients (n=1,284 hospital visits) during Oct 2010 and Dec 2010. vii. NR viii. One hospital

Q2. EWS Q3. i. Two-tier system: (1st) automatic identification of patients at risk of clinical deterioration using EWS from existing EMR databases calculated using machine learning algorithms, and (2nd) the real-time detection of clinical event based on real-time vital sign data collected from on-body WSN technology attached to those high-risk patients. Data is sent to the EMR and EWS scores are assigned to patients in real time using ‘machine-learning techniques’ to analyse the data. ii. Real-Time Event Detection System (RES): When an clinical deterioration event is identified, the RES automatically notifies nurses through the hospital’s paging system to provide early intervention

Q7. i. 398 variables were used to predict patient outcome i.e. transfer to ICU. The model had an AUROC = 0.8834. Using a ‘real-time simulator’ of the model, the predictive value for transfer to ICU was; AUROC = 0.7293

Q8. Predict ICU transfer; Retrospective data: Specificity= 0.9500 Sensitivity=0.4877 PPV= 0.3138 NPV= 0.9753 Accuracy= 0.9292 Real time simulator data: Specificity= 0.9492 Sensitivity=0.4127 PPV= 0.2955 NPV= 0.9691 Accuracy= 0.9229

Q14. 2+ One centre. Q16. This integrated approach for EWS monitoring and response is feasible.

270 | P a g e







Jones, Mullally, Ingleby et al. (2011) UK Q1. RS iv. To determine whether automated clinical alerts using the Patientrack intervention, increases compliance with an EWS protocol and improve patient outcomes. v. An historical controlled study vi. 3 phases: (i) Baseline data capture (47 days from Nov –Dec 2007) (ii) Implementation of electronic observation capture and EWS calculation. Doctors were alerted by traditional systems, (38 days from Aug –Sept 2008) (iii) Electronic capture of data (as in phase ii) with automated electronic alerts to appropriate clinical team member (47 days from Nov –Dec 2007). vii. All consecutive patients admitted during each phase (Total n=1,481; 13,668 observation sets between baseline and intervention. (baseline 705 patients and 7820 observations median age 70 (52-80), 52% male); alert 776 patients and 5848 observations; median age 65

Q2. 5-item EWS HR, Systolic BP, RR, Temperature, AVPU. Q3. Patientrack, implemented in 2000, is on a central web server manages and monitors data and EWS scores and sends alerts and reissues reminders for non-response and non-attendance until the alert is managed. An alert protocol is in place whereby for patients with an (i) EWS 3-5 the nurse in charge is alerted, or (ii) EWS ≥3 x 5 within 24 h or EWS ≥6 a senior doctor is contacted directly and must attend within 30 mins. Q4. vi. Hospital LOS, compliance with the EWS protocol, cardiac arrest incidence, critical care utilisation (CCU) and hospital mortality.

Q5. i. No significant difference in mortality was observed between baseline and alert (9.7%-7.6%: P=0.19) ii. No significant difference in cardiac arrest incidence was observed between baseline and alert (0.4%-0%; P=0.21) Q6. i. Clinical attendance to patients with EWS 3, 4 or 5 increased from 29% at baseline to 78% with automatic alerts (P<0.001) Clinical attendance to patient with EWS >5, increased from 67% at baseline to 96% with electronic alerts (P<0.001) ii. EWS score accuracy improved from 81-100% with electronic calculation

Q7. i. The number of CCU bed/hospital bed-days decreased between baseline and alert (P=0.04)

ii. There was a decrease in LOS between baseline and alert phase: 9.7 v 6.9 days (P<0.001)

Q8. Enablers: Bedside observations were taken manually and entered into a PDA, which was connected wirelessly allowing ‘whole-of-wards’ view. Use of alerting logic in the electronic system enables the matching of clinical competency to the individual’s clinical condition as measured by their EWS. This improves attendance and decreases time lags. Additional acute illness training was also implemented for nurses on the wards, but the direct effect of this initiative could not be measured in this study.

Q14. 2- Historical data is subject to confounders, e.g. seasonal influence and internal adjustments to the hospital pathways. Q16. “Bedside entry of electronic clinical observations and matching of aggregated EWS score to automated alerting logic using the Patientrack system significantly improves timely clinical attendance to acutely ill adult medical patients with EWS score ≥3. An associated reduction in critical care use was also reported.”

271 | P a g e







(49-77), 53% male) viii. The MAU and one general medical ward of one hospital.

Gordon & Beckett (2011) UK (Scotland) Q1. RS iv. To review the completeness and correctness of SEWS documentation for patients with a triggering score of 4, thus causing clinical concern and requiring medical review. v. A prospective audit study vi. SEWS charts were analysed each morning after a night shift for 14 days completeness of vital sign documentation and correctness of vital sign weighting and SEWS score vii. In-hospital patients with a SEWS ≥4, disordered physiological parameters or concern. Ward staff knew the audit was being undertaken. Patients in wards (n=156) and the Combined Assessment Unit (n=25) were included. viii. One hospital (The Royal Infirmary Edinburgh).

Q3. SEWSError! Bookmark not defined.

SBP, HR, O2 saturation, temp, RR, UO, AVPU)

Q3. i. A SEWS trigger score ≥4 triggers a medical review within 20 mins. Q4. i. Hospital wards have access to the Hospital At Night Team. The Combined Assessment Unit is covered by a team of doctors and nurses. One medical registrar covers the wards and Combined Assessment Unit. iv. An ongoing SEWS educational programme for both nurses and clinicians is in place in the Combined Assessment Unit.

Q6. ii. Errors in SEWS chart documentation were common. No chart recorded UO, indicating that 100% of charts were incomplete, when UO was omitted, vital sign observations missing in 64% of charts There is variability in which individual vital signs are recorded or not. Mistakes in score correction were also observed: SEWS aggregate score not calculated: 55% SEWS aggregate score incorrectly calculated: 21% Frequency of incomplete observation on incomplete charts on Ward (n=77): UO recorded: 0% RR recorded:49% Temp recorded: 55% Neurological status recorded: 68% HR recorded: 97% SBP recorded: 97% O2 saturation recorded: 96% Frequency of errors on in incorrect charts in the wards (n=121); SEWS score not calculated on incorrect charts: 55% SEWS score incorrectly calculated on incorrect charts: 21% Frequency of incomplete observation on incomplete charts in the Combined Assessment Unit: (n=3) UO recorded: 0% RR recorded:66%

Q9. v. Incomplete charts and incorrect or absent SEWS aggregate scores overnight means that the trigger is not being activated at night, even for these patients with a SEWS score ≥4. viii. SEWS is used significantly better (P<0.01) in the Combined Assessment Unit which has an ongoing educational programme for SEWS for both nurses and clinicians Q12. ii. UO may have been preferentially recorded on separate fluid balance charts, instead of on the SEWS chart. The fact that it is ignored on the SEWS chart must be addressed. iii. The difference in SEWS documentation between the Combined Assessment Unit and the wards may reflect a different culture between the two areas. Q13. iii. Documentation of

Q14. 2+ Single centre. Q16. SEWS is not being used optimally overnight. “Basic observations are often incomplete, and the SEWS chart poorly understood and acted upon.” (p 15) There is inconsistency in the use of SEWS between hospital areas; it is most correctly used in the Combined Assessment Unit, but less well on the wards. Complex track and trigger scoring systems are difficult to use and prone to errors.

272 | P a g e







Temp recorded: 66% Neurological status recorded: 66% HR recorded: 100% SBP recorded: 100% O2 saturation recorded: 100% Frequency of errors on in incorrect charts in the Combined Assessment Unit (n=8): SEWS score not calculated on incorrect charts: 50% SEWS score incorrectly calculated on incorrect charts: 25%

observations for patients with a SEWS ≥4 overnight should take priority over uninterrupted sleep on the wards. Targeted educational programmes in wards are required to ensure efficacy of SEWS, with audits to assess effectiveness; clear monitoring plans; and specifically trained ‘observationists’ may improve efficacy.

Ludikhuize, Brunsveld-Reinders, Dijkgraaf et al. (2015) The Netherlands Q1. RS iv. A RRS was implemented over 24 months (i) MEWS and SBAR over 7 months and an RRT over the next 17 months. The aim was “to describe the effect of implementation of a RRS on the composite endpoint of cardiopulmonary arrest, unplanned ICU admission, or death.” (p2544). v. A pragmatic, prospective multicentre, before and after trial. vi. The pre-implementation period was the 5 months prior to implementation during which baseline data was collected prospectively

Q2. MEWS Q3. i. Implementation of RRS was mandated by the Dutch government in 2008. This paper is from the Costs and Outcomes analysis of Medical Emergency Teams (COMET) Study. MEWS and SBAR were implemented over 7 months When one MEWS vital sign parameter was above the normal range, or when it considered necessary by the treating physician or nurse, it was mandatory to determine the MEWS score. At MEWS score ≥3 the physician on that ward was informed with communication structured using the SBAR tool.

Q5 i. There was a significant reduction in in-hospital mortality per 1000 admitted patients post-implementation compared to pre-implementation (adjusted OR=0.802 (95%CI 0.644, 1.000); p=0.05) ii. There was a significant reduction in cardiopulmonary arrest per 1000 admitted patients post-implementation compared to pre-implementation (adjusted OR=0.607 (95%CI 0.393, 0.937); p=0.018) x. There was a significant reduction in the composite endpoint (cardiopulmonary arrest, unplanned ICU admission, or death) per 1000 admitted patients post-implementation compared to pre-implementation (adjusted OR=0.847 (95%CI 0.725, 0.989); p=0.036). Note: The composite endpoint was chosen because of the low anticipated numbers to reach the individual endpoints. Q6

Q7 i. There was a non-significant decrease in the number of unplanned ICU admission per 1000 admitted patients post-implementation compared to pre-implementation (adjusted OR=0.878 (95%CI 0.755, 1.021); p=0.092)

Q9. i. An effective RRS may result in a decrease in unplanned ICU admission rates because of early detection and intervention on the wards, alternatively earlier detection may lead to an increase in unplanned ICU admission rates for treatment. Therefore, ICU admission rates may underestimate the beneficial effect of RRSs. Adjusting for potential confounders (age, gender, individual hospital and urgency of admissions) improved the internal validity of the pre- post-intervention study. The sequential implementation of the RRS

Q14. 2+ The largest trial assessing the effectiveness of an RRS. Q16. Nationwide implementation of a RRS was associated with a 15% adjusted risk reduction in the composite endpoint of cardiopulmonary arrest, unplanned ICU admission, or death.

273 | P a g e







(April to Aug 2009). Effects of the RRS was measured in the last 5 months of the 24 month implementation period (July to Nov 2011). Admission data was provided by hospital information departments and outcome data was extracted from clinical report forms. vii. All adult (≥18 years) patients admitted to the study wards (n=166,569, representing 1,031,172 hospital admission days), Mean age of patients did not change between pre- (62.2 (18) years) and post-implementation (62.3 (18) years). Male patients constituted 49.2% and 50.1% of patients pre- and post-implementation. viii. 12 hospitals (2 surgical and 2 non-surgical wards in each hospital)

Physician had to assess the patient within 30 mins, treats or activates RRT, if treatment does not work or the physician does not arrive with 30 mins, RRT is activated iii. SBAR Q4 i. The RRT was implemented during the 17 months post-implementation of the MEWS and SBAR. The RRT consisted of an ICU nurse and physician trained in Fundamental Critical Care Support at least. . iv. All physicians and nurses working on a COMET ward were trained in the specific instruments i.e. MEWS and SBAR using standardized toolkits (pocket cards and posters, provided by the primary investigators). Deviation from the MEWS threshold was allowed in specific circumstances based on patient characteristics for instance in a patient with chronic hypoxemia, but should be clearly mentioned by the physician within the patient chart.

i. The rate of RRT activation increase between implementation and final RRT phase (last 5 months) from 6.8 to 7.3 per 1000 patients admitted. RRT activation by a physician was most frequently; RRT activation by a nurse decreased from 15% in the RRT implementation phase to 9% in the final RRT implementation phase. This was accompanied by an increase in resident RRT activation.

did not result in a consistent and gradual decrease in patients experiencing each adverse event. This suggests that MEWS/SBAR on their own is less effective at reducing adverse effects. Q13. iii. “A more mandatory nature of implementation and measurement of outcomes would assist in the continual optimization and research into RRS” (p2550)

Ludikhuize, Smorenburg, de Q2. MEWS45 Q5. Q9. Q14. 2-

274 | P a g e







Rooij et al. (2012) The Netherlands Q1. RS iv. To describe current practice in the measurement and documentation of vital signs, and to investigate whether MEWS would be useful in the detection of deteriorating patients on wards, who had an adverse event. v. A retrospective, observational study. vi. Vital signs in the 48 hours before the adverse event were extracted from medical and nursing charts. MEWS scores were calculated retrospectively for each time one or more vital sign was measured and recorded. vii. 204 medical and surgical patients with a severe adverse event including cardiopulmonary arrest, unplanned admission to ICU, emergency surgery, emergency surgery or unexpected death during 2007 were included. The median age was 67 (IQR 57-76) and 61% were male. viii. A university hospital

Q3. A trigger score of ≥3 should result in an attending physician being notified

X. The median MEWS score prior to an adverse event was 2 (IQR 1.3). Q6. ii. In the 48 hours prior to an adverse event 2,688 vital sign measurements were taken. A median of 3 parameters were taken per measurement. Recording of all vital signs was rare, but were less frequent for patients with a MEWS ≥3. Pulse, BP and temp were the most common combination of vital signs measured, being recorded in 19% of all measurements; Pulse: Measured in 72% of all cases; 91% when MEWS ≥3. SBP: Measured in 73% of all cases; 91% when MEWS ≥3. RR: Measured in 23% of all cases; 47% when MEWS ≥3. Temp: Measured in 49% of all cases; 58% when MEWS ≥3. Consciousness: Measured in 7% of all cases; 9% when MEWS ≥3. Worried: Measured in 12% of all cases; 9% when MEWS ≥3. Urine output: Measured in 17% of all cases; 30% when MEWS ≥3. Peripheral saturation: Measured in 43% of all cases; 67% when MEWS ≥3. 81% of patients had a MEWS of ≥3 at least once in the 48 h before the adverse event. The percentage of MEWS ≥3 increased when vital signs were measured closer to the adverse event. Median time a MEWS ≥3 was recorded before the adverse event was 13 (IQR 4.8, 27.1) hours

i. Urine output and level of consciousness are the vital signs which are rarely measured, while pulse rate and SBP are most frequently measured. RR ‘the neglected sign’ was only documented in less than half of patients with a MEWS ≥3. Some vital signs may not have been recorded because of lack of need e.g. consciousness in a well oriented patient.

Single centre, observational study, retrospective MEWS score calculation. Q16. There is a lack of measurement and documentation of vital signs in patients 48 hours prior to an adverse event. 81% of patients who experienced an adverse event could have been identified prior to the event using MEWS score. Implications of track and trigger systems should include qualitative (including clinical judgement) as well as quantitative data.

Ludikhuize, Borgert, Binnekade et al. (2014)

Q2. MEWS45

Q5. ii. There were 5 cardiopulmonary arrests during the

Q7. i. There were 61 unplanned ICU

Q9. i. Failure of MEWS calculation

Q14. 2- Single centre. Started

275 | P a g e







The Netherlands Q1. RS iv. To investigate whether using a ‘protocol’ defining the measurement of MEWS three times daily improved the degree of implementation of the raid response system. Outcomes were compliance with protocol, degree of delay in physician notification and RRT activation in patients with MEWS ≥3. v. A quasi-experimental study; wards were randomised to protocolised vital signs vs vital signs as clincally indicated. In both arms the RRS algorithm was the same and activated at a score of 3 vi. All vital signs of participants were recorded at the end of the month from nursing charts. Data on all adverse events (cardiopulmonary arrests and unplanned ICU admissions) and RRT activations were also extracted. MEWS score was also calculated retrospectively based on actual set of vital signs measured. vii. Patients admitted to medical or surgical wards for at least one overnight stay

Q3. i. An RRS was implemented on 18 hospital wards. An RRS protocol was implemented in June, and was officially in use from 1st Sept 2011. The RRS specifies that the critical MEWS score is ≥3. ii. At MEWS ≥3 the nurse should notify the patient’s physician, who should attend within 30 min, assess and indicate treatment. The physician could activate the RRT then or if there is no improvement after the first intervention, the RRT is notified. 10 wards were randomised to the protocolised vital signs a minumum of 3 times/day and 8 to the control arm (vital signs as clinically indicated) after stratification for medical and surgical wards Q4. i. The RRT consists of an ICU physician, nurse who attends the patient within 10 mins and it operates 24/7. iv. Nurse training started in June 2011 with 3 nurses per ward receiving training. These nurses then trained their colleagues. Physicians had training sessions separately during hand-over

3 months of the study x. Rate of adverse events (cardiopulmonary arrests or unplanned ICU admissions) decreased on the protocol and control wards during Sept and Nov; Protocol wards: 13.4 (Sept) and 8.5/1000 hospital admissions (Nov) Control wards: 9.1 (Sept) and 6.5/1000 hospital admissions (Nov) Q6. i. Delays in identifying deteriorating patients was observed in 49% (n=28/57) and 50 % (n=2/4) in the protocol and control arm respectively, but the delays were shorter on the protocol wards (20 and 44 h, respectively (P=0.79). There was a significantly higher number of RRT calls in the protocol (n=62/84) compared to the control wards (n=22/84; P<0.003) Protocol wards: 11.8 (Sept) and 19.6/1000 hospital admissions (Nov) Control wards: 8.0 (Sept) and 6.5/1000 hospital admissions (Nov) ii. 68% compliance with measuring MEWS 3-times per day was observed in the protocol wards. Calculation of MEWS was significantly higher in wards randomised to the protocol compared to control wards (P<0.001); Number of MEWS measured Protocol ward: 70% (2513/3585) MEWS calculations; median number of measurements per day =3 (IQR 2,3). Control ward: 2% (65/3013) MEWS calculations; median number of measurements per day =2 (IQR

admissions during the 3 months of the study The number of ICU admissions from protocol wards decreased during the study, and on the control wards to a less extent Protocol wards: 67% (n=10/15; Sept) and 26% (n=6/23; Nov) Control wards: 57% (n=4/7; Sept) and 50% (n=3/6; Nov)

in the control wards may be due to incomplete routine full set of vital sign sets, especially in those where RR is missing, and/or lack of knowledge regarding recognition of abnormal vital signs, however, reasons are unknown. The increased utilisation of RRT, better compliance with MEWS and a decrease in patient adverse events suggests that there may be a dose-response effect between RRT utilisation and decreased patient adverse events- Q13. iii. The multi parameter system introduced resulted in more comprehensive vital sign measurements

collecting data immediately upon implementation with a short follow-up. Some bias information may be present in nursing charts. Unselected patient group Q16. Vital signs and MEWS determination three times daily, results in better detection of physical abnormalities, significantly more frequent call outs and more reliable activation of RRT. A trend towards a decrease in adverse effects was also observed, especially in the protocol wards where MEWS was calculated regularly.

276 | P a g e







during 1st Sept and 31st Nov 2011 were randomised to have a full set of vital signs including MEWS measured at least 3-times daily ‘protocol’ (n=373 patients; 3585 measurements) or to /MEWS measurement when clinically indicated ‘control’ i.e. when a MEWS-sub score was ≥1 during regular vital sign measurement (n=432 patients; 3013. measurements). Mean age was 56.7 (SD 17.7) years and 49% were male. viii. One university hospital

meetings. Pocket cards, posters staff emails and a local website advertising the RRS algorithm were also used.

1, 2). There were significantly less missing vital signs, significantly less measurements with 3 or more errors, and significantly more measurements with no errors, in the protocol wards compared with the control ward (all P<0.001).

Mackintosh, Humphrey & Sandall (2014) UK Q1. RS iv. “To explore the social and institutional processes associated with the practice of rescue and implications for the implementation and effectiveness of RRSs within acute health care.” (p233) v. An ethnographic case study vi. Data was collected during 180 hours of observation, staff interviews and documentary review (protocols and audit data) between Jan and Dec 2009. Bourdieu’s concept of field,

Q3.i. Westward had an established RRS with a paper-based EWS Eastward has more than one EWS within its acute service. It was piloting an electronic intelligent assessment technology (IAT) on 2 medical wards where vital signs were recorded by handheld devices and aggregate scores were electronically calculated and prompts for subsequent actions were given, e.g. repeat observations or calls for help. Q4.i. Westwards has a CCOT staffed by critical care nurses and physiotherapists.

Q5. x. “Three themes illustrated the nature of rescue work within the field and collective rules which guided associated occupational distinction practices; (1) the ‘dirty work’ of vital sign recording and its distinction from diagnostic (higher order) interpretative work; (2) the moral order of legitimacy claims for additional help and (3) professional deference and the selective managerial control of rescue work.” (p233)

Q8. iii. ‘Dirty work, the routinisation of vital sign monitoring’; All healthcare participants’ emphasised that observing vital signs is core to detecting changes in physiological condition. But vital sign monitoring is seen as a ‘basic’ function, and with increasing nursing tasks, this is delegated more to HCAs. More complex interpretive work to assess deteriorating patients remains in the realm of nurses. EWSs legitimises this division of labour. But distinctions between observation and interpretive work is blurred as, e.g. HCAs know the patients better. ‘Calling for help’ the significance of boundaries; EWS was used to facilitate the nurse/HCA – medical boundary. But viewed differently by the different

Q12. iii. Inter-professional differences in the adherence with EWS is socially authorised. Q13. iii. Organisations need to address power hierarchy between medial teams to reduce delays in response to deteriorating patients.

Q14. 3 Two centres Q16. This study “adds a nuanced understanding of patient safety on the front line, challenging notions of the ‘quick fix’ safety solution.” (p233). Responsibility for rescue is distributed across the whole organisation and is not limited to the frontline staff.

277 | P a g e







capital and habitus were applied to their substantive codes; routine work, identification of a problem, asking for help, responding and structural influences. vii. Clinical staff (n=35) doctors, nurses, healthcare assistants (HCAs) and managers) participated in interviews, viii. Two hospitals (study IDs - Westward and Eastward purposely selected because they have different RRSs.

Eastward has no CCOT. professions. EWS scores above a trigger score enabled nurses and HCAs to call for help. But ‘worry’ and other cues outside those measured by EWSs were taken less seriously. EWS was viewed by medical staff as a tool to aid nurses recognition and referral of a potentially deteriorating patient to the medical team, who then assessed the clinical significance of the situation. EWSs assist junior doctors with hand overs and enhances their level of competence. However, they learnt that they had to limit errors in escalating to senior doctors. The CCOT assisted junior staff to manage deteriorating patients without calling a senior doctor. Hierarchical relationships between specialist and generalist teams contributes to delays in response. Auditing processed: the visibility of rescue work. In both hospitals auditing focused primarily on nursing activities (monitoring and calling for help) while medical tasks (e,g, ordering diagnostic tests and chasing results) were not examined routinely. This perpetuates distinctions and blame between health professionals. At Westward auditing was undertaken by doctors and compliance with the monitoring and escalation process was reported at weekly senior nurse meetings and displayed within ward corridors. High levels of compliance helped individuals to gain individual

278 | P a g e







status. At Eastward implementing the IAT enabled audits of adherence to protocol (documentation and responses. However, self-monitoring became less frequent throughout the study period, and breaches became normalised and explained by nurses being ‘busy’.

Massey, Chaboyer & Aitken (2014) Australia Q1. RS v. The aim of this study was to explore nurses’ experiences and perceptions of using and activating a MET. vi. An interpretive qualitative approach. vii. Semi-structured interviews between March 2011 and August 2011 viii. Ward nurses (n=15) ix. Hospital in Southeast Queensland, Australia

Q3. iv. Noted as context that METs have been developed and implemented with the aim of improving recognition of and response to deteriorating patients but often not activated or used effectively by nursing staff (p133).

Q12. Reasons for delays in activating the MET included, not knowing if it was the right thing to do, not wanting o appear an idiot, fear of getting into trouble, fear of reprisals or punishments. This meant MET was often not used as an early intervention strategy but instead used when patient had suffered an event like cardiac arrest. Misinterpretation and lack of understanding of the role of the MET was a barrier. Conferring with the nurse in charge before MET activation reinforcing the understanding that a culture of seeking support to validate clinical decisions exits in some institutions.

Q14.3 Q16. Four qualitative themes included: (1) sensing clinical deterioration; (2) resisting and hesitating; (3) pushing the button; and (4) support and leadership. Concluded that “Recognising, and managing the deteriorating patient is complex, challenging, and multifaceted” (p137).

Mathukia, Fan, Vadyak et al. (2015) USA Q1. RS. iv. To describe the experience and impact of

Q2. MEWS Q3. i. MEWS was implemented in a pen-and-paper format in June 2013.

Q5. i. The overall mortality decreased post-implementation compared to pre-implementation. 2011: 2.3% 2012: 2.0% 2013: 1.5%

Q5. i. The percentage of RRT patients transferred to higher level care decreased post-implementation compared to pre-implementation. 2011: 63%

Q9. v. RRT response may have resulted in patients being moved to a hospice, which would have impacted on the mortality rate, but which was

Q14. 2- Single centre. Q16. Institution-wide implementation of MEWS had resulted in improved patient

279 | P a g e







implementing MEWS in a protocolised way. v. A retrospective and prospective observational study vi. Data on the number of RRTs, Code Clues, and result of each RRT and Code Blue per 100 patient-days (100 PD) was collected from all non-ICU wards monthly during Jan 2010 and June 2014. vii. NR viii. One hospital (Easton Hospital)

Q4. ii. A chart was developed with the scoring system and the consequent escalation responses iv. Education of all registered nurses and medical staff was conducted on 3 pilot medical surgical units. This was extended following the pilot study vi. Monthly reviews were conducted to monitor adherence.

2014: 1.2% Non-significant trend towards better survival of non-ICU ‘Code Blues’ since MEWS implementation 2011: 65% 2012: 43% 2013: 65% 2014: 71% ii. The percentage of non-ICU ‘Code Blues’ has decreased since MEWS implementation (P<0.01) 2011: 0.05 per 100 PD 2012: 0.05 per 100 PD 2013: 0.02 per 100 PD 2014: 0.02 per 100 PD The percentage of RRT calls that progressed to ‘Code Blues’ decreased significantly since MEWS implementation (P<0.05) 2011: 4.17 per 100 PD 2012: 7.14 per 100 PD 2013: 2.27 per 100 PD 2014: 0.00 per 100 PD Q6 i. There was a significant increase in RRT calls post-implementation, compared to pre-implementation (P<0.01) 2011: 0.24 per 100 PD 2012: 0.25 per 100 PD 2013: 0.38 per 100 PD 2014: 0.48 per 100 PD

2012: 68% 2013: 64% 2014: 50%

not captured. vi. Communication between healthcare professionals was improved due to the quantifiable evaluation of patient condition due to MEWS. However, MEWS cannot replace critical thinking skills. viii. It is difficult to quantify the exact contribution of MEWS to overall improved patient care and decreased mortality because other quality improvement interventions were also ongoing. Q12. ii. High rate of false alarm is a disadvantage, and may be improved with the move to an electronic form of MEWS. Q13. ii. High levels of adherence to MEWS is necessary to ensure effectiveness iii. MEWS has been incorporated into the EMR, thus decreasing risk of errors and improving standardisation. O2 saturation has been added, thus changing MEWS to SEWS

outcomes with an increase in the use of RRTs, a decrease in cardiopulmonary arrests, less progression from RRT to Code Blue and a decrease in mortality rate.

McNeill& Bryden (2013) UK

Q2. Single parameter scoring systems (n=2) and aggregate

Q5 i. There is no evidence that the implementation of a

Q7. i. Introduction of AWSS was associated

Q9. Different approaches have

Q14. 1+ The level of available evidence

280 | P a g e







Q1. SR i. To assess whether EWSs or emergency response teams improve hospital survival compared to usual care. Also do EWSs or RRT (MET, CCOTs and multidisciplinary teams [MDT]) effect unplanned ICU admission, ICU mortality, LOS in ICU, hospital LOS or cardiac arrest rates ii. The Ovid Medline, EMBASE, CINAHL, Web of Science, Cochrane library and NHS databases and non-catalogued resources were searched in September 2012 for papers published between 1996 and Feb 2012. assessing the efficacy of EWS and RRT within clinical trials and comparative studies in adult patients against predefined outcome measures iii. 43 studies were identified; observational before and after studies (n=30), observational after time (n=2), retrospective observational (n=1), multicentre cohort comparison studies (n=1), multicentre interrupted time series analysis (n=1), and a prospective ward randomised trial (n=1) study

weighted scoring systems AWSS (n=4) were investigated. Q4. i. Medical emergency teams (MET; n=20) and multidisciplinary outreach teams or RRTs (n=22) were included. RRTs are not clinician led, making them different from MET teams.

single parameter EWS triggering system alone has a positive effect on hospital survival; no change (n=1) or a reduction were observed (n=2). Introduction of AWSS was associated with improved hospital survival/reduction in hospital mortality (n=2), or no change (n=1) MET teams introduction was associated with decrease in mortality/improved survival (n=9) or no difference (n=4) MDT outreach team introduction was associated with a decrease in mortality/improved survival (n=3) or no difference (n=3) ii. There is weak evidence that the implementation of a single parameter EWS triggering system alone has a positive effect on cardiac arrest rates (n=2). The effect of the introduction of AWSS was variable; no difference (n=1) and a reduction (n=1) in cardiac arrest rates was observed. MET teams introduction was associated with decrease in cardiopulmonary arrest (n=7) or no difference (n=5) Q6 ii. Introduction of AWSS was associated with more complete documentation of DNACPR orders (n=1). MET teams introduction was associated with increase documentation of DNACPR orders (n=1)

with decreased ICU admission (n=2) or no change (n=1) MET teams introduction was associated with decreased ICU admission (n=4) or no difference (n=2) MDT outreach team introduction was associated with a decrease in ICU admission (n=2) or no difference (n=2); and a decrease in ICU readmission (n=2). iii. Introduction of AWSS was associated with reduced LOS (n=1) MET teams introduction was not associated with reduced LOS (n=1). MDT outreach team introduction was associated with a possible increase in LOS (n=1).

been taken in the UK, US and Australia. Single parameter EWS may aid the identification of deteriorating patients. AWSS implementation improves hospital survival and decreases unplanned ICU admission and cardiac arrest. Effect on ICU and hospital LOS is uncertain MET teams were found to improve hospital survival, decrease unplanned ICU admission and cardiac arrests, but the effect of MET on hospital LOS and ICU mortality is unclear. Ongoing review of the activation process and education programmes is required. Outreach teams are effective at reducing ICU readmission and reducing hospital mortality. There is less evidence for the effectiveness of nurse-led MDTs. Outreach teams focusing on the education and support of ward staff should be developed. Education is essential for the success of the EWS and RRS

for assessing efficacy of EWS and RRS is low quality. Matching of control groups within studies is poor. Q16. For RRSs to be effective a ‘whole system’ approach should be adopted and aggregate weighted scoring systems are more effective than single parameter systems. RRSs are most effective for patients with predictable clinical decline than among patients who are post-operative or acute haematology patients.

281 | P a g e







intervention. All interventions were most effective during routine working hours.

Moon, Cosgrove, Lea et al. (2011) UK Q1. RS iv. “To determine whether cardiac arrest calls, the proportion of adult patients admitted to ICU after CPR and their associated mortalities were reduced in the 4 year period after the introduction of a 24/7 CCOS and MEWS Charts.” (p150) v. A retrospective audit analysis of prospectively collected data of a pre-, post-implementation study. vi. Adult admissions, ICU admission rates, cardiac arrest calls, admission to ICU following CPR and demographic data was collected pre- (2002-05) and post-implementation (2006-09) vii. There were 213,117 adult admission pre and 235,516 post-implementation. The ages (68 v 69 years; P=0.09)

Q2. NUTH NHS MEWS79 Q4. i. A CCOS was established in 2001, and from 2005 it became a 24/7 service. iv. Educational programmes were provided regarding early recognition of physiological deterioration and use of the MEWS charts. vi. An ICU was opened in 2003 and in 2005 all acute medical admissions were referred to another hospital.

Q5. i. There was significant 7.1% decrease in in-hospital mortality decreased post- (697/year) compared to pre-implementation (750/year; P<0.0001). Deaths per adult admission decreased significantly post- (1.2%) compared to pre-implementation (1.4%; P<0.0001). There was a significant decrease in in-hospital mortality following ICU admission post-implementation (42% v 52%, P=0.05). Q6. i. The number of cardiac arrest calls relative to total adult hospital admissions decreased significantly post- (n=584; 0.2%) compared to pre-implementation (n=767; 0.4%; P<0.0001)

Q7 i. There was aa 24.5% increase in annual admission to ICU post-implementation. The number of patients admitted to ICU having undergone CPR in hospital decreased significantly post- (2%) compared to pre-implementation (3%; P=0.004)

Q14. 2- Retrospective analysis, and the two cohorts were admitted at different times. The timing of adverse effects was not captured and the influence of DNACPR forms was not considered. Q16. Adverse patient outcomes (mortality, cardiac arrests and transfer to ICU following CPR) decreased significantly in the four years post implementation of MEWS charts with CCOS. The introduction of MEWS and CCOS ‘should be considered a positive influence although other factors may have also had an impact on outcomes and it is therefore difficult to quantify specific effects.” (p154)

79

GOLDHILL, D. R., WORTHINGTON, L., MULCAHY, A., TARLING, M. & SUMMER, A. 1999. The patient-at-risk team: identifying and managing seriously ill ward patients. Anaesthesia, 54, 853-860.

282 | P a g e







and severity of illness of patients admitted to ICU (APACHE II 18 v 21:P=0.12) did not differ significantly pre- and post-implementation. viii. One hospital (Freeman Hospital)

Mok, Wang, Cooper et al. (2015) Singapore Q1. RS v. To explore nurses’ attitudes towards vital signs monitoring in the detection of clinical deterioration in general wards. vi Survey vii Researcher designed

survey --Cronbach’s alpha

was 0.71 for the 16-item V-scale viii. General ward nurses (n=614) ix. Tertiary acute care hospital, Singapore

Q6 ii 56.9% of nurses perceived blood pressure changes as the first indicator of deterioration, and 46% perceived that an altered respiratory rate was the least important indicator. Most nurses (59.8%) relied on percentage oxygen saturations (SpO2) to evaluate respiratory dysfunction, and 27.4% noted that they make fast estimates of the patients’’ respiratory rate. iii 89.0% were confident in their reporting of deteriorating vital signs such that the team would review the patient. Nurses’ attitudes were significantly influenced by whether they had a degree qualification (p<0.001) and by greater duration of working on a general ward i.e. for >5 years (p<0.05) or working on a ward with a specialty (p <0.01). .

Q12. Regarding documentation- nurses perceived that the monitoring of vital signs was time consuming (21.0%) and overwhelming (35.3%). Q.13 Perceived sense of confidence among nurses to report deterioration in vital signs.

Q14. 2- Q16. According to the authors nurses had limited understanding of the key indicators of deterioration.

Moriarty, Schiebel, Johnson et al. USA (2014) Q1. RS v. To determine the effect of RRT implementation on failure to rescue (FTR). vi. Longitudinal study

Q2 RRTs are the efferent arm of a rapid response systems and can be described as a designated group of healthcare providers with critical care expertise who can assemble quickly to deliver care at the bedside of deteriorating patients (p49).

Q5 i. Overall hospital mortality in the pre RRS implementation period was 1.5% compared with 1.6% in the full post-implementation period (P = 0.299). No significant decreases were observed pre- and post-implementation for cardiopulmonary resuscitation events. iv. There was a non-significant decrease in sepsis FTR between pre- and full post-implementation periods (P = 0.064).

Q7i. The unplanned ICU transfer rate increased significantly from 13.7 transfers per 1000 floor days to 15.2 transfers per 1000 floor days (P < 0.001). vi. Activation of RRT average between 50/1000 and 70/ 1000 discharges. About 60% of RRT calls are among medical patients.

Q13 A culture change to ensure acceptance of RRT along with education and increased resources noted by authors in discussion.

Q14. 2- Q16. Initial results post implementation of RRT showed no difference. However, results in the second-year post-implementation of RRT revealed a decrease in the FTR measure as well as an increase

283 | P a g e







Vii Modified version of the AHRQ FTR measure used to collect data, which identifies hospital mortalities among medical and surgical patients with specified in-hospital complications. Viii Data were gathered retrospectively from administrative databases, of all inpatients discharged between 1 September 2005 and 31 December 2010 ix. Two academic hospitals in Midwest, USA.

In this study it constituted: a critical care nurse, critical care fellow and respiratory therapist. The health care provider at the bedside may activate the rapid response system based on “concern” or physiologically based “criteria”.

vii. Reduction in the FTR rate was associated with a substantial increase in the number of RRT calls.

Children accounted for 2.5% of RRT and resuscitation events.

in the unplanned ICU transfer rate occurring corresponding to an increase in the number of RRT calls per month. Effect of RRT was not visable in terms of patient level data until 18 months post intervention. .

Morris, Own, Jones et al. 2013 UK Q1. RS iv. “To establish and test the feasibility of measurement of a comprehensive set of mutually exclusive outcomes in the 7 days after referral of patients to a RRT, to facilitate audit and aid analysis of failure-to-rescue events.” v. Observational cohort study. A prospective pilot. vi. Data on patient outcomes following an RRT were collected using a standardised performa, over 10 weeks (Jan to Mar 2010). Progress was noted on days 1,3, and 7 following RRT trigger through patient note reviewed, except for patients

Q2. MEWS. A score of ≥3 suggested referral to the RRT (single parameter track-and-trigger system). Q4. The RRT consisted of: (i) critical care outreach nurses working with the CCU between 7.30 and 21.00, Monday to Friday and a night team consists of advanced nurse practitioners in the district hospital. (ii) The RRT consists of 9 critical care outreach nurses led by a nurse consultant 24 hours per day in the University hospital. A matrix of mutually exclusive patient outcomes was developed by expert opinion.

Q5. x. positive and negative patient outcomes 1-7 days post RRT trigger Day 1 post-RRT (n=146) 75% of patients had a positive outcome 69% of patients with MEWS ≥5 had positive outcomes Negative outcomes: 15.8% (n=23) on wards still triggering MEWS after 24 h. 0.7% (n=1) had a cardiopulmonary arrest. Day 3 post-RRT (n=86) 90% (n=90%) of patients had a positive outcome Negative outcomes: 10.5% (n=9) on wards still triggering MEWS after 24 Day 7 post-RRT (n=67) 88% (n=59%) of patients had a positive outcome Negative outcomes: 1.5% (n=1) had a cardiopulmonary arrest. 10.5% (n=9) on wards still triggering MEWS after 24

Q7. Day 1 post-RRT 21.7% (n=31) patients transferred to CCU Of these 15.7% (n=23 ) had a delayed CCU transfer (negative outcome)

Q9. “A more recent joint audit showed positive outcomes for 85% of 158 patients with a MEWS trigger score of ≥5.” “Better results were achieved in the general hospital during the day likely due to the structure of the RRT.” Better results were observed in the University Hospital.” Q12. Iii. There is no system for benchmarking patients with an RRS in hospital. Q13. iii. PDAs could also be used to record RRT responses.

Q14. 2- The Multidisciplinary Audit and Evaluation of outcomes of rapid response (MAELOR) tool was developed in one centre Q16. The MAELOR tool can be used to classify RRT episodes with readily available information, identify and target areas performing sub-optimally and can facilitate comparison of RRTs

284 | P a g e







who were discharged, died, had a DNAR following RRT trigger or once admitted to CCU. vii. All consecutive patients (n=146) seen by the RRT. Mean age was 67 (SD 20) years and 53% were male. viii. 2 hospitals; a district general and a university hospital.

Positive outcomes: (i) timely ICU admission (i.e. <4 h); (ii) alive on ward and no longer triggering, (iii) died with terminal care pathway and had DNAR; (iv) Alive with DNAR and documented treatment limitations, (v) other (new unrelated RRT trigger, chronic condition leading to continuous trigger, discharged). Negative outcomes: (i) delayed ICU admission (i.e. >4 h); (ii) still triggering, (iii) cardiopulmonary arrest; (iv) Outcome unknown or lost to follow-up

Percentage of positive outcomes differed between medical and surgical wards (63-77%). Outcomes were significantly better when RRT was triggered during the day compared to th night shift )P<0.05).

McDonnell, Tod, Bray et al. (2013) UK Q1. RS v. To evaluate the impact of a complex intervention (an intervention which included training, new observation charts and a new track and trigger system) for the detection and management of deteriorating patients on knowledge and confidence of nursing staff in an acute hospital. vi. A before and after mixed methods study vii. A survey which examined knowledge and confidence

Q3. The new model involved changes including: • modification of the existing track and trigger (T&T) to include new parameters e.g. oxygen saturation • use of a new T&T graded response algorithm • modification of the observation chart to include the T&T • patients could be ‘stepped up’ to the PAR chart (if they ‘triggered’ on the algorithm on the standard chart) and visa versa. • training (30-45 minutes) delivered by a nurse

Q6. Knowledge and confidence to recognize and manage deteriorating patients increased statistically. Post-intervention the total numbers of concerns expressed was reduced from 4.3 (SD 2.6) out of 10 to 3.7(SD 2.3), a reduction of 0.6 points (95% CI -0.91 to -0.26, P 0.0001). The importance of intuition also came through strongly in the qualitative data for experienced nurses

Q9. Observation chart with T&T highlighted as supporting recognition of the deteriorating patient. Q13. T&T provides an objective mechanism of assessing patients, increasing self-awareness (it just rings more alarm bells) and ‘highlighting the problems and patients at risk of deterioration . Increasing confidence of nursing staff to manage deterioration.

Q14. 2- Q16. Following the intervention, knowledge, and confidence to recognize and manage deteriorating patients increased; the number of concerns were reduced. Scores were higher for registered than support workers and healthcare assistants before and after the intervention. Interviews confirmed these findings and provided detail on how nurses felt the new system had improved practice. Conclusion. The new model

285 | P a g e







in recognition and management of deteriorating patients (n = 213). The final number of paired responses was 66% (213/322). 15 nursing staff participated in qualitative interviews. Data collected in 2009. viii. A district hospital in England

specialist. had a positive impact on the self-assessed knowledge and confidence of registered nurses, support workers, and healthcare assistants. Similar initiatives should take into account the clinical context and tailor training packages accordingly.

Niegsch, Fabritius & Anhøj (2013) Denmark Q1. RS. iv. To assess the degree to which the Ward Observational Charts (WOC) guidelines were adhered to post-implementation v. A 7-day (Mon-Sun) prospective, observational, randomised, cross-sectional, point prevalence study of WOC guideline compliance in hospital patients. vi. All hospital beds were randomised to a day of the week, and each day between 16:00 and 21:00 all WOC vital sign parameters were recorded and the total MEWS score calculated by the investigator for all beds randomised to that day. A structured questionnaire was used to interview the ward nurse when a patient had an

Q2. MEWS Q3. i. WOC guidelines were introduced between 2007 and 2009. WOC is an observational chart on with MEWS is calculated. The chart has a graphical layout and is colour coded, with escalation protocol incorporated. All vital signs had to be recorded and a total MEWS score had to be calculated and registered, correct adjustment of observation frequency undertaken and contact to physician or MEW had to be documented to comply with WOC guidelines. ii. In 2010 guidelines stated that MEWS had to be recorded three times in the first 24 hours. If MEWS is 0, it was then recorded once in 24 hours; MEWS=1, observe 3 times per day; MEWS=2,

Q5. x. 58% (n=77) of patients were observed and managed correctly according to guidelines. Of the 55% (n=73) patients with an abnormal MEWS calculated by the investigator, staff were aware of 60% (n=44) of cases. Q6. i. 38% (n=50) of patients had an abnormal MEWS recorded; of these 38% (n=19) had were correctly escalated by nurses, while 62% were not. ii. 77% (n=101) of patients had a MEWS score calculated at least once within 24 hours. However of these 12% (n=12) did not have all vital signs required; and 12% (n=12) had all vital signs recorded, but no aggregate score recorded. Single vital signs were recorded between 80% (GCS) and 95% (Temperature) for all patients. Of the 31 patients who did not have an appropriate escalation, nurses reported that this was due to; the vital signs were normal for the patient (n=9), patient was known to have abnormal vital signs (n=5), patient appeared well despite abnormal vital signs (n=2), patients were observed after initiation of treatment (n=4), nurses were closely observing

Q9. Implementation of WOC guidelines has resulted in the systematic observation of hospitalised patients, including RR at a level similar to the other vital signs. The recording of GCS with the least frequency may be due to staff being unfamiliar with it. Q12. ii. High compliance with all elements of the guidelines are necessary for them to be effective. iii. Current education, especially regarding the documentation of actions and escalations may be deficient. Q13 iii. The WOC education for all staff requires redesign incorporating e-learning and

Q14. 2+ Single centre. Weekday effect was accounted for by collecting data over 7 days. Date of data collection was not recorded, so years post-implementation could not be assessed. Q16. Long-term WOC guideline implementation has not been completed satisfactorily. The main component which is lacking is the documentation of the actions resulting from an abnormal MEWS.

286 | P a g e







abnormal MEWS to assess whether s/he was aware of the patient’s condition. After 48 hours, each patients WOC MEWS score was compared with that calculated by the investigator, and the level and frequency of MEWS within the intervening 48 hours was documented. vii. 132 patients were included. viii. Twelve medical wards in one hospital (Naestved Hospital).

observe 3 times per day, and inform ward doctor; MEWS=3-4, acute supervision by attending doctor, observe in 1 hour; MEWS ≥5, acute supervision by attending doctor, observe in 30 mins; Deterioration with change of MEWS ≥3, call MET. Q4. iv. Mandatory education in the form of a 2 hour lecture, including instructions on WOC and MET was introduced for all staff.

the patient (n=2), or the nurse would check the patient later (n=1), another observational chart was in use (n=2), the patient was terminal (n=1), or the WOC was discontinued (n=1).

simulation. Use of an electronic EWS may improve standardisation, calculation documentation and escalation.

Oglesby, Durham, Welch et al. (2011) UK Q1. RS v. To produce and test a tool to analyse the efficiency of intensive care admission processes across 6 countries. Such a tool would facilitate centres to recognize the features causing delays. vi. A pilot multicentre service evaluation vii The time-period between either a physiological trigger or callout of RRT) to admission to intensive care unit (Door) was logged as ‘Score to Door Time’. viii. 177 ICU admissions ix. ICUs (n=17) from the United Kingdom, Ireland,

Q2. ViEWS scores were calculated retrospectively from the bedside observations. ViEWS was not implemented in any centre at the time of the study. Q3.i Data was collected from a range of RRT, MET and CCO teams in a range of international healthcare systems. Q4 vi. Severity of illness on admission to ICU using the APACHE II .

Q7 vii Score to Door Time (STDT) for 177 admissions was a median of 4:1 hours (IQR) 1:49 to 9:10). Time from physiological trigger to activation of a RRS was a median 0:47 hours (IQR 0:00 to 2:15). STDT was a median 4:32 hours (IQR 2:24 to 10:03, n = 142) for UK patients and a median of 1:41 hours (IQR 00:47 to 3:15, n = 35) for non-UK patients (P < 0.0001). the time was longer outside of outside normal working hours. Time period from physiological trigger to call-out of RRT was a median 0:47 hours (IQR 0:00 to 2:15, n = 120). Time period from call-out to ICU admission was a median of 2:45 hours (IQR 1:19 to 6:32). A total of 127 (71%) admissions were deemed to have been delayed. Pg. 1.

Q12 (see answer to Q7) Delayed critical admissions are due, at least in part to restricted availability of critical care beds particularly and an associated complex referral systems RRS were introduced to help prevent unnecessary ICU admissions but they cannot function in isolation.

Q14. 2- Q16. “Score to Door Time seemed to be largely independent of illness severity and, when combined with qualitative feedback from centres, suggests that admission delays could be due to organisational issues, rather than patient factors. Score to Door Time could act as a suitable benchmarking tool for Rapid Response Systems and helps to delineate avoidable organisational delays in the care of patients at risk of catastrophic deterioration.” (p1).

287 | P a g e







Denmark, United States of America and Australia.

Causes for delay included initial improvements after treatments on the ward (n=4), diagnostic or therapeutic procedures prior to ICU admission (n = 12), a need to stabilise the patient prior to transfer (n = 9) and a cardiopulmonary arrest. Pg. 4. For patients with STDT of >4 hours (n = 94), there were only 14 cases (14.9%) where a clinical reason for delay could be established. For 40 (42.6%) of those patients organisational problems were cited, including waiting for senior reviews (n = 11), the lack of an available ICU bed (n = 14), and insufficient human resources. Linear regression analysis yielded three significant (P < 0.05) predictors of STDT: centre location, patient age and APACHE II score. Correlation between ViEWS and STDT was weak (Spearman’s r = -0.058, P < 0.02) suggesting that a greater severity of illness at the time of RRS contact did not lead to timelier admission. (p 4).

Patel, Jones, Jiggins et al. (2011) UK Q1. RS iv. To investigate the effect of the implementation of MEWS with a critical Care Outreach service on mortality in a trauma unit. v. A retrospective, pre- and post-implementation study vi. Diagnosis, primary procedures and death information on all emergency

Q2. MEWS

Q3.i. MEWS was implemented in all trauma ward and orthopaedic wards in 2005. At a trigger score of 4, nurses sought the advice of senior medical staff, and/or referral to the CCOT if necessary. Q4.i. The CCOT consists of specialist ICU nurses.

Q5 i. There were 889 deaths, significantly more females (77%) than males (33%: p<0.001), despite fewer female admissions. The annual in-hospital mortality rate was 0.4% during the study period. The in-hospital mortality rate decreased for all patients, males and females post-implementation compared to pre-implementation, but not statistically significantly. Decreased mortality rate was: All patients: 0.9% (95%CI 0.53, 1.31; P=0.092) Males: 0.4% (95%CI 0.003, 0.81; P=0.214)

Q9. viii. There has been an increase in ward rounds by specialists in elderly medicine in orthopaedic hospitals and the extra hip operations at weekends, to meet national standards. These changes may have also contributed to the observed trends in mortality. This cohort of patients were elderly with comorbidities

Q14. 3 One centre. Retrospective study. The quality of documentation and trigger response were not commented on within the paper. Pre-operative comorbidities were not assessed. Q16. Different NHS Trusts have implemented different versions of the track and trigger systems, many of

288 | P a g e







trauma inpatient admissions and deaths during Jan 2002 and Dec 2009 (3 years pre- and 4 years post-implementation) was extracted. vii. 32,149 patients were admitted (55% male); viii. University Hospitals of Leicester trauma unit

Females: 1.5% (95%CI 0.81, 2.21; P=0.108)

and polypharmacy which may mask their clinical condition. Furthermore, elderly patients are not often suitable cases for ICU so track and trigger systems may be of limited value in this cohort, and this may explain the lack of statistical significance in observed mortality trends.

which have not been validated. In this instance there was a non-statistically significant decrease in mortality post-implementation of MEWS with critical Care Outreach service. MEWS may not be sensitive enough to identify physiological deterioration in the elderly orthopaedic patients Cost-effectiveness of the CCOT requires investigation.

Patel, Hassan, Ullah, Hamid, Kirk (2015) UK Q1 RS iv. To analyse hospital medical records about cardiac arrest and MET calls, over a period of 12 months between March 2013 and February 2014. v. A retrospective, post-implementation study vii Comparison of data March 2013 and February 2014

Q2 MET team call out Q7. On the introduction of the MET in 2008, 257 cardiac arrest calls and 30 MET calls were made. Conversely, between March 2013 and February 2014 the number of MET calls had increased to 932, whilst cardiac arrest calls had significantly reduced to 119 (P=<0.0001). All calls were attended within 15 minutes. Of the total activated MET calls, only 20 patients developed cardiac arrests. The MET on-call team resulted in a 54 % reduction in the number of cardiac arrest calls.

Q 14. 3 (limited description , 1 page)

Reini, Fredrikson & Oscarsson (2012) Sweden Q1. RS Iv. To assess the prognostic ability of MEWS at ICU admission (MEWSin) and at ICU discharge (MEWSout) in predicting outcome after

Q2. MEWS: SBP, RR, HR, Temperature, level of consciousness (Confused, A, V, P, no response) Q3. Trigger tool: For stable patients MEWS is documented daily. Patients

Q5. i. ICU mortality

MEWSin >6 vs <6; 24 vs 3.4% (P<0.001) OR 5.58 (95%CI 2.39-20.56) MEWSin AUROC: 0.80 (95%CI 0.72-0.88) SOFA AUROC: 0.91 (95%CI 0.86-0.97) SAPS III AUROC: 0.89 (95%CI 0.83-0.94)

Q7. iv. LOS in ICU

MEWS >6 vs <6: OR 2.30 (95%CI 1.40-3.76)

vii. MEWSout score did not predict ICU readmission.

MEWSout >6 vs <6: OR 0.98 (95%CI 0.69-1.37)

Q11. MEWS facilitated rapid communication between nursing staff and physicians.

Q14. 2- 1 centre. Patient selection criteria was broad and heterogeneous Q16. A 6-item MEWS is a useful predictor of ICU mortality, 30-day mortality and LOS in ICU.

289 | P a g e







clinical care ii. Prospective, observational study. vi. A MEWS was measured on ICU admission on patients breathing spontaneously and then hourly on these patients until discharge. Socio-demographic data, ICU LOS, diagnosis, readmission and mortality data were extracted from a computer-based ICU registry. vii. All adult patients (≥16 years) admitted to the ICU (n=518; mean age 57.6 years; 59% male) during Oct 2008 and Dec 2009. 354 patients had spontaneous ventilation on admission, 164 patients had tracheal intubation prior to ICU admission. On discharge, 419 patients were breathing spontaneously. The remaining patients had 164 patients had tracheal intubation prior to ICU admission viii. One tertiary care general ICU in a University Hospital

with: MEWS=1, are monitored every 8-12h; MEWS=2, are monitored every 4-8h; MEWS=3-4, attending physician is contacted and patient is monitored every 1-2 h; MEWS≥5 triggers the summoning of the 24h Critical Care Outreach Service (CCOS). If in need of critical care, the patient is then transferred to ICU. Q4. Mortality in ICU, 30-day mortality, LOS, readmission to ICU.

MEWSin >6 and >24 h stay in ICU vs <6; OR 4.31 (95%CI 2.31-8.06) 30-day mortality MEWS >6: OR 4.31 (95%CI 2.31-8.06)

Q8. ICU mortality

MEWSin >6: Sensitivity= 62% Specificity= 85%

It is less predictive of ICU patient outcome than the more specific ICU tools (SOFA and SAPS III), but it is a more simple tool, not requiring laboratory results.

Subbe & Welch (2013) UK Q1.R i. Clinical deterioration is complex and protocols have been developed for the early detection of deterioration. The aim was to review use of

Q3. i. RRS consist of the ‘afferent arm’ i.e. detection of clinical deterioration on the ward, and the ‘efferent arm’ i.e. the clinical response to deterioration. These

Q5. ii. It has not been definitively shown than the RRS itself is the causative factor in the reduction in cardiopulmonary arrests identified (n=1), or whether the institutional culture is the more salient reason. Implementation of a Track and Trigger EWS was associated with >50% reduction in cardiopulmonary

Q9. i. There is a search for ways to improve RRS performance. The significance of timing of escalation is unknown. Objective measures of RRS performance to drive improvements are lacking.

Q16. 3 “Failure to rescue patients with signs and symptoms of impending catastrophic illness is a complex process. Reliable recording of vital signs, recognition of abnormalities, communication of concerns

290 | P a g e







RRS in hospitals to improve care to deteriorating hospitalised patients and prevent ‘failure to rescue’

pathways depend on the 5 R’s; (a) Recording of physiological observations at bedside. (b) Recognition of degree of abnormality and the clinical implication (c) Reporting (to a health care professional with the skills to identify and treat (d) Response by treating appropriately (e) Repetition of previous 4 steps as a feedback loop to identify failure to improve. The reliability of each element in the chain is necessary if the system is to be a success.

arrests (n=1). v. There is variation in response to deterioration in different health systems, and often in different locations and times within a health system. These differences include the specialists responding to a trigger at any one time and standards are difficult to establish. ‘Care bundles’ for different diseases including sepsis, stroke and heart failure, where component processes are documented, can help towards standardisation of response or treatments. Q6. ii. A full set of vital signs includes RR, HR, SBP, Consciousness and Temp. There are a number of problems with vital sign measurement; (a) This is often done by junior or inexperienced staff. (b) RR, the most important vital sign, is often not recorded because it must be recorded manually over 1 full minute. (c) Experienced staff can identify signs of deterioration even when vital signs are within normal ranges (d) Abnormalities can be viewed as less serious among over worked staff EWSs and educational interventions can improve vital sign monitoring and recording by comparing vital signs against reference ranges. The best EWSs classify up to 90% of patients as at risk or not correctly. They increase reliability of monitoring because all of the vital signs are required to calculate the aggregate score. Standardisation of a EWS scoring system may simplify training of nurses and other healthcare professionals and drive comparative research.

Q10.i. Use of PDAs and tablets with Wi-Fi capacity can improve many points on the chain, especially recording of vital signs (n=1). Improvement in patient outcomes including reduced mortality (n=1) and LOS (n=1) and fewer patients being admitted to ICU following cardiopulmonary arrest (n=1) have been reported as a result.

and a timely response can dramatically reduce adverse events and improve outcomes. Standardisation and automation will further improve prospects for patients at risk.”

291 | P a g e







iii. Reporting of abnormalities is frequently the weakest link and is often by telephone and across discipline and experience. Use of tools like SBAR may improve this process. Repeating the loop of the 4 ‘R’s’ is essential to avoid failure to rescue.

Huddart, Dickinson, Quiney et al. (2015) UK Q1. RS iv. 1 in 1100 people in the UK undergo emergency laparotomy annually, and 30-day mortality rate is 14.9%. The aim was to compare risk-adjusted 30-day mortality after emergency laparotomy pre- and post-implementation of the ELPQuiC bundle, v. (a) Assessment of current practice and (b) Pre-post-intervention study vi, Data was collected day to day by an assigned person in each hospital and anonymised data was entered into the Electronic Database for Global Education. Baseline data (existing databases and retrospective data collection)

Q2. Evidence-based ELPQuiC (including EWS) Key recommendations of ELPQuiC Q3 i. All emergency admissions have a EWS score assessed on presentation, with; ii. Graded escalation policies to senior staff and ICU referral (NICE Guidelines). Broad-spectrum antibiotics given to patients with suspicion of peritoneal soiling/sepsis diagnosis. When a decision is made to perform a laparotomy, the patient is given the next emergency theatre place (or within 6 h of decision). Start resuscitation using goal-directed techniques as soon as possible (or within 6 h of admission,

Q5. i. There was a significant 3.5% (95%CI -1.4, 8.4) decreased in overall crude 30-day mortality post-implementation (P=0.152); Pre-implementation: 14.0% (95%CI 10.1, 18.0%) Post-implementation: 10.5% (95%CI 7.6, 13.5%) Using Cumulative sum (CUSUM) plots80 adjusted for individual patients’ predicted risk of 30 day mortality, there was an overall increase of 5.97 patient lives saved per 100 patients treated post-intervention (P<0.001). Overall adjusted risk of death decreased from 15.6% (95%CI 12.5, 18.9%) to 9.6% (95%CI 7.4, 11.8%) (P=0.003). v. Use of goal-directed fluid therapy increased significantly. Involvement of senior surgeons and anaesthetists in patient care increased significantly in one hospital Q6. ii. Completion of EWS increased in all four hospitals post-MEWS implementation.

Q7. iii. No decrease in LOS was observed post-implementation. vi. Improvements were observed within existing resources, without resulting in an increase in LOS. (However, costs associated with implementation of MEWS or costs of senior surgical, anaesthetic input as well as restructured use of the oprating room was not discussed

Q9. iv. Use of goal-directed fluid therapy and ICU admission are the elements which may have the biggest impact on mortality assuming these were driven by EWS Q13. ELPQuiC was Implemented using Quality improvement methodology.

Q14.2+ Lack of statistical significance in decreased mortality one hospital may be due to low patient numbers and therefore low power. However implementation in four hospitals provided external validity for the findings. No control hospital for comparison. Q16. “Introduction of a 5-component evidence-based care bundle, which included EWS use augmented by senior clinical input, sepsis and fluid algorithms led to a significant reduction in P-POSSUM (Physiological and Operative Severity Score for the enUmeration of Mortality and Morbidity) risk-adjusted 30-day mortality.” (P102)

80

CUSUM plots are used to show the cumulative difference between expected risk of death (defined by case mix-adjusted P-POSSUM score) and observed outcome (0 alive, 1 died). An increasing CUSUM reflects the saving of lives, a decreasing CUSUM reflects loss of lives and a stable CUSUM is neutral.

292 | P a g e







was submitted 3 months pre-implementation on consecutive patients. vii. In the 3 months prior to implementation there were 299 consecutive patients (mean age 65.6 (SD 15.8) years, 46.2% male); in the 8 months post-implementation there were 427 consecutive patients (mean age 65.8 (SD 16.5) years, 47.3% male); viii. ELPQuiC was implemented in 4 National Health Service hospitals, with each site having discretion re quality improvement strategies of implementation.

Admit all patients to ICU post-emergency surgery. iii. Methods of implementation included poster, email, education programmes, regular presentation of project data and case studies, development of specific mechanisms to ensure prompt sepsis management, radiological investigations and theatre prioritisation.

Schmidt, Meredith, Prytherch, et al. (2015) UK Q1.RS iv. To determine if the introduction of EPSS would reduce hospital mortality. v. A retrospective, observational, pre-post intervention design. vi Data collected before, during and after the implementation of a hospital-wide EPSS vii NR viii. Two large acute general hospitals in England

Q4 vi A measure of EPSS implementation

Q5. Monthly observed mortality of 56 diagnosis groups used to calculate expected mortality in post intervention phase. During implementation of EPSS across site 1-QAH, crude mortality81 fell from 7.75% (2168/27 959) at baseline (2004) to 6.42% (1904/29 676) after implementation (2010) (p<0.0001), with an estimated 397 fewer deaths. Similarly, at UHC (site 2), crude mortality fell from 7.57% (1648/21 771) at baseline (2006) to 6.15% (1614/26 241) (2010) (p<0.0001) at UHC (estimated 372 fewer deaths) pg. 12.

Q6. 98% of all vital sign datasets recorded were complete after implementation.

Q10 The EPSS, which uses wireless handheld computing devices, replaced a paper-based vital sign charting and clinical escalation system. The EPSS—VitalPAC—is a software package that: a). Prompts nurses to record (on handheld computing devices) a complete set of vital signs at the patient’s bedside at appropriate intervals.

b). Automates the conversion of vital sign readings to weighted early warning scoring system to calculate

Q 14 2+ Q 15 limited until EHR is implemented nationally Q16 Authors note an association between an EPSS specifically designed to increase the reliability of the collection, calculation of EWS, documentation and display of vital signs in hospital and an associated significant reduction in hospital mortality. The results being mirrored in two hospitals support this claim.

81

Mortality in the 56 diagnosis groups was reported as opposed to all-cause mortality.

293 | P a g e







the patient’s EWS c). Provides instant bedside decision support to the staff, on a graded basis according to the EWS value. d).messages are delivered using standard colour-coded screen messages. e). Can be viewed remotely.

Q13. iii According to the authors, achieving surveillance through EPSS for the whole patient journey appears to have been extremely important in reducing mortality.

Limitation: EPSS could be associated with a Hawthorne effect. EPSS Implementation Process followed the essential components of the five-ring ‘Chain of Prevention as follows: 1. Education 2. Monitoring and 3. recognition

Integration of the EPSS in the user’s workflow,

Use of simple input screens, asking for data only when it was required and enabling tracking of the responses to the decision support

Requires the entry of a full vital signs dataset at each routine observation

Uses a graded response strategy recommended by NICE82 to automatically determine the timing of the next vital signs measurement

The EPSS warns of attempts to enter out-of-range data 4. Call for help, 5. Response

Rapid delivery of decision support

82


294 | P a g e







Instantaneously available across the whole hospital via wireless 6. Systems thinking

Anticipation of the user’s needs, ensuring user acceptability by incorporating user feedback

Stewart, Carman, Spegman et al. (2014) USA Q1. RS iv. To investigated the impact of MEWS on (i) the decision-making process to trigger the RR system, and (ii) cardiopulmonary arrests. v. Mixed methods (i) a retrospective medical record review pre- and post-implementation of the MEWS and (ii) nurse-led focus groups. vi. (i) Medical records review of all in-hospital adult patients who had an RRS activation, 12 months pre- and 12 months post-MEWS introduction.(ii) 5 focus groups were held with a total of 11 RNs, all female who work day and evening shifts.. vii. Pre-intervention there were 39 RRS, mean age 67

Q2. MEWS, introduced in 2011 and fully implemented in 2012. MEWS is integrated into the electronic medical record system. Q3. ii. A MEWS score of ≥3 recorded by nursing assistants must be reported to a registered nurse.

Q5. Clinically but not statistically significant outcomes were observed. ii. Cardiopulmonary arrests decreased post-MEWS introduction intervention compared to pre-MEWS introduction (P=0.878): post-intervention: n=11 pre-intervention: n=14 Q6. i. RRS activation increased post-MEWS introduction compared to pre-MEWS introduction (P=0.288): post-intervention: n=55 pre-intervention: n=39 x. During focus groups nurses described themselves as “knowledgeable/very knowledgeable in their ability to recognise patients experiencing clinical deterioration, and as moderately/very experienced in caring for patients with signs of clinical deterioration.” 3 themes regarding use of MEWS by nurses were identified in the focus groups: (i). decision making: MEWS scores assist with triage of multi-patient

Q9. viii. Nurses suggestions to improve current MEWS: Customise ‘normal’ vital signs to account for individual patient variances; Q12. Barriers identified by focus group participants: Need to manually enter vital sign data into the electronic medical record. No mechanism to alert nurses to missing or inaccurate data.

Q14. 2- One hospital. Small numbers, short follow-up time. Q16. Nurses do not rely exclusively on the MEWS score to trigger interventions. Instead they perform further patient assessments and use clinical judgement to decide whether to trigger an RRS. The MEWS score is easily understood by all HCPs and facilitates interdisciplinary communication and is useful in the description of unit acuity. Upward trends in RRS activation and downward trends in cardiopulmonary arrests are positive outcomes and suggest clinical significance.

295 | P a g e







±15.9 years, 57% male. Post-intervention: 55 RRS, mean age 65±16.5 years, 39% male. Higher mean CCI score in the post-intervention than the pre-intervention group (2.04 and 1.41, respectively). None of these differences were statistically significant. viii. 3 non-cardiac monitoring medical-surgical units within 1 hospital

assignments and as an alert to a patients changing condition. MEWS scores plus patient assessment to gather more information (and intervention within scope of practice) is used to trigger RRS, not MEWS alone. Neither scoring nor weighting is given to ‘worry/concern’ in the aggregate score. (ii) Communication “MEWS score is a valuable tool to enhance interdisciplinary communication with physicians, nursing assistants and nursing administrators”, especially to express a sense of urgency to physicians and obtaining information from nursing assistants. “MEWS score was used to evaluate current unit acuity, determine staffing needs, and prepare for the possibility that a patient may require transfer.” (iii) administrative support Support from nursing administrators and nurse-to-nurse support, gave participants confidence to activate the RRS without fear of ridicule.

Winters, Weaver, Pfoh et al. (2013) USA Q1. SR i. To evaluate the effectiveness and implementation of RRSs in acute care settings. ii. PubMed, PsycINFO, CINAHL, and the Cochrane Central Register of Controlled Trials were searched from 1st Jan 2000 to 30th Oct 2012. iii. 43 articles were included; 26 effectiveness and 17 implementation studies.

Q4 iv. Educational programmes for staff often accompany RRS implementation, but these vary in content and design with simulation education being uncommon, and cognitive aids being common

Q5. i. Conflicting effects of RRS on mortality were reported. RRS is associated with reduced but not statistically significant rates of mortality, (n=1 SR); RR 0.96 (95%CI 0.84, 1.09) RRS had a positive effect on total hospital mortality (n=18 of 23 studies; 7 of these were significant.) ii. RRS had a favourable effect on cardiorespiratory arrest in the majority of studies. RRS is associated with reduced rates of cardiorespiratory arrest outside ICU (n=1 SR); RR 0.66 (95%CI 0.54, 0.80). RRS had a positive effect on cardiorespiratory arrest (n=19 of 20 studies).

Q7. vi. Costs of RRS were not evaluated

Q9. MET activation was 35 times more likely based on ‘worry’ than vital signs (n=1) Potential harms from introducing RRSs include “‘deskilling’ ward staff, inappropriate patient care for other patients, staff conflict, and diversions of ICU staff from usual care in the ICU and communication errors by introducing additional providers” (p491). Q12 and 13.

Q14. 1++ Moderate strength evidence was found that RRSs have a positive effect on patient outcomes. Q16. RRS have a positive effect on cardiopulmonary arrest rates, but total hospital mortality is not significantly reduced. Cultural systems problems need to be addressed if RRSs are to be effective in

296 | P a g e







Q6 i. Implementation of RRSs varied widely depending on local need and resources. The proportion of patients with a delayed RRS activation decreased significantly over time (40.3% v 22.0%; P<0.001) (n=1) RRS team structure effected its utilisation; the splitting of teams into two with different activation and processes resulted in significantly increased RRS utilisation (n=1). Benefit of RRS may depend on the patient population; medical benefited whereas surgical patients did not (n=1).

ii. Barriers and facilitators that affected nurses activation of RRS; “adequate education regarding the purpose and function of RRS, clinical expertise, support by medical and nursing staff, nurses’ familiarity with and advocacy for the patient and nurses’ workload.” (n=1) (p422). A change in culture from one of blame to a supportive one, with knowledge of activation criteria, communication and teamwork are facilitators of RRS. Optimum team composition remains to be elucidated

increasing patient deterioration and reducing preventable deaths.

Wood, Candleand, Dinning, et al. (2015) UK Q1. RS v. To increase compliance with completion of an adult EWS and paediatric EWS tool (PEWS) and improve timely escalation and to improve the care of the deteriorating ward patient vi. Three phased project described in paper Phase 1: staff engagement and defining the ward culture Phase 2: five multifaceted high impact interventions,

Q2. EWS Q3. Varied responses for each threshold EWS or clinical concern culminating in EWS ≥ 6 (the mandated escalation level to consultants)

Q6. i. Improvement in standards of nursing escalation of greater than 100%, from 22%, (Quarter 1) to 57% (Quarter 4). Standards in medical escalation improved by 18%, from 31% (Quarter 1) to 37% (Quarter 4) ii. Improvement in performance of 4-hourly observations for the first 24 hours of admission —with a relative improvement of greater than 50% from 65%(Quarter 1) to 96% (Quarter 4)

Q9. The mean EWS prompting admission to critical care was 7 for adults. Overall there was evidence of Consultant involvement in only 51% of cases of adult patients. In the sickest adult patients, observations often improved following initial medical intervention and that early review within working hours may prevent deterioration and need for escalation out of hours service. Q.10: Implementation (Impl.) solutions

Q14. 2-

297 | P a g e







tailored based upon user feedback Phase 3: continual fortnightly case note audit

Monthly audit results communicated verbally and via e-mail to each ward in addition to placing them on performance boards. Staff dedicated to supporting the implementation and tracking of the EWS. Q.12: Scale of service improvement. Access to doctors. Lack of equipment. Higher workload or fewer nursing staff. Influence of hierarchy. Negative responses when escalating Q.13: Engagement and support of key stakeholders. Obtaining staff engagement and involving clinical champions at the beginning and throughout the process contributed to the sustained improvement especially amongst nursing staff. Identifying and defining ward culture. A no-blame approach whilst maintaining transparency in order to maximise learning from case reviews.

298 | P a g e

299 | P a g e

Appendix 4b: Components of the Individual Early Warning Scoring Systems

EWS (Clifton et al. 2015)

Score 3 2 1 0 1 2 3

Respiratory rate (breaths/min)

≤8 9-18 19-24 25-29 ≥30

SpO2 (%) ≤91 ≥92

Heart Rate (beats/min)

≤40 41-50 51-100 101-110 111-129 >130

Systolic blood pressure (mmHg)

≤90 91-100 100-199 ≥220

Temperature (0C) ≤35 35.1,37.9 ≥38.0

EWS (van Rooijen et al. 2013)

Score 3 2 1 0 1 2 3


<9 9-14 15-20 21-30 >30

Pulse Rate (beats/min)

51-100 101-110 111-130 >130


<70 70-80 81-100 101-200 >200

Temperature (0C) <35.1 35.1,36.5 36.6,37.5 >37.5

Consciousness Alert (A) Voice (V), Pain (P) Unresponsive (U)

EWS for acute stroke patients (Liljehult & Christensen 2015)

Score 3 2 1 0 1 2 3


≤8 9-11 12-20 21-24 ≥25

SpO2 (%) ≤91 92-93 94-95 ≤96


≤40 41-50 51-90 91-110 111-130 ≥131


≤90 91-100 101-110 111-219 ≥220

Temperature (0C) ≤35 35.1,36.0 36.1,38.0 38.1-39.0 ≥39.1

Inspired O2 Any O2 Air

CNS response A V, P, U

300 | P a g e

EWS (Jones et al. 2011)

Score 3 2 1 0 1 2 3


<8 9-14 15-20 21-29 >30


≤40 41-50 51-100 101-110 111-130 >130


<70 71-80 81-100 101-199 >200

Temperature (0C) <35 35.1,36.0 36.1,37.9 38.0,38.9 >39.0

CNS response Alert (A) Voice (V), Pain (P), Unresponsive (U)

EWS (Petersen et al. 2014)

Score 3 2 1 0 1 2 3


≤9 9-11 12-20 21-24 >24

SpO2 (%) <92 92-93 94-95 >95


<41 41-50 51-90 91-110 111-130 >130


<91 91-100 101-110 111-219 ≥220

Temperature (0C) <35.1 35.1,36.0 36.1,38.0 38.1,39.0 >39.0

Neurological A V,P,U

NEWS in Ireland (NCEC 2013)

Score 3 2 1 0 1 2 3


≤8 9-11 12-20 21-24 ≥25

SpO2 (%) ≤91 92-93 94-95 ≥96


≤40 41-50 51-90 91-110 111-130 ≥131


≤90 91-100 101-110 111-249 ≥250

Temperature (0C) ≤35.0 35.1,36.0 36.1,38.0 38.1,39.0 ≥39.1


Any supplemental O2 No Yes

301 | P a g e

NEWS (Griffiths & Kidney 2012)

Score 3 2 1 0 1 2 3


≤8 9-11 12-20 21-24 ≥25

SpO2 (%) ≤91 92-93 94-95 ≥96


≤40 41-50 51-90 91-110 111-130 >131


≤90 91-100 101-110 111-219 ≥220

Temperature (0C) ≤35 35.1,36.0 36.1,38.0 38.1,38.5 38.1,39.0 ≥39.1


Any supplemental O2 Yes No

NEWS (Abbott et al. 2015)

Score 3 2 1 0 1 2 3


≤9 9-11 12-20 21-24 ≥25

SpO2 (%) <92 92-93 94-95 ≥96


<41 41-50 51-90 91-110 111-130 >130


<91 91-100 101-110 111-219 >219

Temperature (0C) ≤35 35.1,36.0 36.1,38.0 38.1,39.0 >39.0


Any supplemental O2 Yes No

NEWS (Badriyah et al. 2014)

Score 3 2 1 0 1 2 3


≤8 9-11 12-20 21-24 ≥25

SpO2 (%) ≤91 92-93 94-95 ≥96


≤40 41-50 51-90 91-110 111-130 ≥131


≤90 91-100 101-110 111-219 ≥220

Temperature (0C) ≤35 35.1,36.0 36.1,38.0 38.1,39.0 ≥39.0



302 | P a g e

NEWS (Cooksley et al. 2012)

Score 3 2 1 0 1 2 3


≤8 9-11 12-20 21-24 ≥25

SpO2 (%) >96 94-95 92-93 <91


<40 40-49 50-89 90-109 110-129 >130


≤90 90-99 100-109 110-219 ≥220

Temperature (0C) ≤35 35.1,35.9 36.0,37.9 38.0,38.9 ≥39.0

Neurological A V/P/U

Any supplemental O2

O2 No O2

NEWS (Keep et al. 2015; Kolic et al. 2015)

Score 3 2 1 0 1 2 3


≤8 9-11 12-20 21-24 ≥25

SpO2 (%) ≤91 92-93 94-95 ≥96


≤40 41-50 51-90 91-110 111-130 ≥131


≤90 91-100 101-110 111-219 ≥220

Temperature (0C) ≤35 35.1,36.0 36.1,38.0 38.1,39.0 ≥39.1


Any supplemental O2

Yes No

NEWS (Jarvis et al. 2015a; Kolic et al. 2015)

Score 3 2 1 0 1 2 3


≤8 9-11 12-20 21-24 ≥25

SpO2 (%) ≤91 92-93 94-95 ≥96


≤40 41-50 51-90 91-110 111-130 >131


≤90 91-100 101-110 111-219 ≥220

Temperature (0C) ≤35 35.1,36.0 36.1,38.0 38.1,38.5 38.1,39.0 ≥39.1


Any supplemental O2

Yes No

303 | P a g e

Binary NEWS (Jarvis et al. 2015b)

Score 1 0 1

Respiratory rate (breaths/min) <12 12-20 >20

SpO2 (%) <96 ≥96

Heart Rate (beats/min) <51 51-90 >90

Systolic blood pressure (mmHg) <111 111-219 >219

Temperature (0C) <36.1 36.1,38.0 >38.0



BEWS (Christensen et al. 2011)

Score 3 2 1 0 1 2 3


≤8 9-14 15-20 21-30 >30


≤40 41-50 51-100 101-110 111-130 >130


≤70 71-80 81-100 101-199 >199

Temperature (0C) ≤35 35.1,36.0 36.1,38.0 38.1,39.0 >39.0

Neurological A V P U

PARS (Abbott et al. 2015)

Score 3 2 1 0 1 2 3


<10 10-19 20-29 30-39 >40

SpO2 (%) <85 85-89 90-94 >95


<40 40-49 50-59 100-114 115-129 >130


<70 70-79 80-89 100-179 >180

Temperature (0C) ≤35 35.1,35.9 36.0,37.4 37.5,38.4 >38.5

Neurological Confused A V P U

Urine Output (ml/kg/h)

Nil <0.5 Dialysis 0.5-3 >3

Worthing PSS (Dawes et al. 2014)

Score 0 1 2 3


≤19 20-21 ≥22

Pulse ≤101 ≥102


≥100 ≤99

Temperature (0C) ≥35.3 <35.3

O2 saturation in air 96-100 94 to <96 92 to <94 <92

Neurological A Other

304 | P a g e

MEWS (Subbe et al. 2001) used by Urban et al. (2015)

Score 3 2 1 0 1 2 3


≤8 9-14 15-20 21-29 ≥30


0-39 ≤40-50 51-59 60-100 101-110 111-129 >130


0-69 70-80 81-100 101-149 150-169 170-179 ≥180

Temperature (0C) ≤35 35.1,1-38 38.1,39.5 ≥39.6

Neurological A (GCS-15) V (GCS-14) P (GCS-13-9) U (GCS-8-0)

MEWS (Cooksley et al. 2012)

Score 3 2 1 0 1 2 3


5-9 10-13 14-19 20-24 25-29 ≥30

SpO2 (%) >96 95-92 91-88 <88


30-39 40-49 50-99 100-119 120-129 >130


30-69 70-79 80-109 110-159 160-199 ≥200

Temperature (0C) 34.0,34.9

35.0,35.9 36.0,37.9 38.0,38.9 >39.0


Urine output (mls/hr)

<10 10-29 30-200 201-300 >300

MEWS (Harris 2013)

Score 3 2 1 0 1 2 3


<9 9-14 15-20 21-29 >30


<30 30-40 40-100 100-110 110-130 >130


<60 60-80 80-90 90-160 160-170 170-200 >200

Temperature (0C) <34.5 34.5,38.5 >38.5


Urine output over 6 h (ml)

No urine

<120 <360 >900

305 | P a g e

MEWS (Churpek et al. 2013)

Score 3 2 1 0 1 2 3


<9 9-14 15-20 21-29 >30


<40 41-50 51-100 101-110 111-129 >129


<70 71-80 81-100 101-199 >199

Temperature (0C) <35 35,38.4 >38.4


MEWS (Ludikhuize et al. 2011; Ludikhuize et al. 2015)

Score 3 2 1 0 1 2 3


<9 9-14 15-20 21-29 >30


<40 40-50 51-100 101-110 111-130 >130


<70 70-80 81-100 101-200 >200

Temperature (0C) <35.1 35.1,36.5 36.6,37.5 >37.5


Worried about patient’s condition: 1 point Urine production below 75 ml during precious 4 h: 1 point Saturation below 90% despite adequate O2 therapy: 3 points

MEWS (Kim et al. 2015; Stark et al. 2015)83

Score 3 2 1 0 1 2 3


<9 9-14 15-20 21-29 >30


<40 41-50 51-100 101-110 111-129 >130


<70 71-80 81-100 101-199

Temperature (0C) <35 35, 38.4 >38.5


83

Kim et al. (2015) defined lowest RR as ≤8 (score 2)

306 | P a g e

MEWS (Dundar et al. 2015)

Score 3 2 1 0 1 2 3


<9 9-14 15-20 21-29 ≥30


<40 41-50 51-100 101-110 111-129 ≥130


<70 71-80 81-100 101-199 ≥200

Temperature (0C) <35 35, 38.4 >38.5


MEWS (Ho et al. 2013)

Score 3 2 1 0 1 2 3


<9 9-14 15-20 21-29 >30


≤40 41-50 51-100 101-110 111-129 >130


≤70 71-80 81-100 101-199 ≥200

Temperature (0C) <35 35, 38.5 >38.5


MEWS (Goldhill et al. 1999 in Moon et al. 2011)

Score 3 2 1 0 1 2 3


<8 8-20 21-30 >30


<40 40-50 51-100 101-110 111-130 >130


<70 71-80 81-100 101-180 181-220 201-220 >220

Temperature (0C)

<34 34.0, 35.0 35.1, 37.5

37.6,38.5 38.6,40.0 >40.0

Neurological Confused/agitated A V P U

307 | P a g e

MEWS (Patel et al. 2011)

Score 3 2 1 0 1 2 3


≤8 9-14 15-20 21-29 ≥30


≤40 40-50 51-100 101-110 111-130 ≥130

Temperature (0C) <35 35.1, 36 36.1, 37.9 38, 38.5 ≥38.6

Sedation score 0-1 2 3 4

Urine catheterised (ml)

Nil <0.5 ml/kg/hr for 2 h

<0.5 ml/kg/hr for 1 h

>3.0 ml/kg/hr for 2 h

Urine non-catheterised (ml)

p/u in last 12 h:No

p/u in last 12 h:Yes


See Table below

Patient’s normal systolic blood pressure

Cu

rrent systo

lic blo

od

pressu

re

200 190 180 170 160 150 140 130 120 110 100 90 80

200 0 0 0 1 1 1 2 2 3 3 4 5 5

190 0 0 0 0 1 1 1 2 2 3 3 4 5

180 0 0 0 0 0 0 1 1 2 2 3 3 4

170 1 1 0 0 0 0 1 1 2 2 3 3 4

160 1 1 1 0 0 0 0 0 1 1 2 2 3

150 1 1 1 1 0 0 0 0 0 1 1 2 2

140 2 2 1 1 1 1 0 0 0 0 1 1 2

130 2 2 2 1 1 1 0 0 0 0 0 1 1

120 2 2 2 2 1 1 0 0 0 0 0 0 1

110 3 3 2 2 2 2 1 0 0 0 0 0 0

100 3 3 3 3 2 2 1 1 0 0 0 0 0

90 4 4 3 3 3 3 2 2 1 0 0 0 0

80 4 4 4 4 3 3 3 2 2 1 1 0 0

70 4 4 4 4 4 4 3 3 2 2 2 1 0

60 4 4 4 4 4 4 4 4 3 3 3 2 1

50 5 5 5 5 5 5 5 5 4 4 4 3 2

40 6 6 6 6 6 6 6 6 5 5 5 4 3

308 | P a g e

COMPASS MEWS (Parham 2012)

Score 3 2 1 0 1 2 3


<8 9-20 21-30 31-35 >36


<40 40-50 51-99 100-110 111-130 >130

Temperature (0C) <34 34.1, 35

35.1, 36 36.1, 37.9 38, 38.5 >38.6

Sedation score 0-1 2 3 4

Urine for 4 h or urine for 24 h (ml)

<80 or <480

80-119 or 480-714

120-899 or 720-4800

>800 or >4800


See Table below

Usual knows systolic blood pressure

Cu

rrent systo

lic blo

od

pressu

re

190 180 170 160 150 140 130 120 110 100 90 80

200s 0 0 1 1 2 2 2 3 3 4 5 5

190s 0 0 0 1 1 1 2 2 3 3 4 4

180s 0 0 0 0 0 1 1 2 2 3 3 4

170s 1 0 0 0 0 1 1 2 2 3 3 3

160s 1 1 0 0 0 0 0 1 1 2 2 2

150s 1 1 1 0 0 0 0 0 1 1 2 2

140s 2 1 1 1 0 0 0 0 0 1 1 1

130s 2 2 1 1 0 0 0 0 0 0 0 1

120s 2 2 2 1 1 0 0 0 0 0 0 0

110s 3 2 2 2 1 1 0 0 0 0 0 0

100s 3 3 3 2 2 2 1 1 0 0 0 0

90s 4 3 3 3 2 2 2 2 1 1 0 0

80s 4 4 4 4 4 4 4 4 4 4 1 0

70s 4 4 4 4 4 4 4 4 4 4 4 4

MEWS (Reini et al. 2012)

Score 3 2 1 0 1 2 3


<9 9-14 15-20 21-29 ≥30


≤40 41-50 51-100 101-110 111-129 ≥130


≤70 71-80 81-100 101-199 ≥200

Temperature (0C) <35 35.1,36.0 36.1, 38.0 38.1,38.5 >38.5


309 | P a g e

ViEWS (Prytherch et al. 2010 in Dundar et al. 2015)

Score 3 2 1 0 1 2 3


≤8 9-11 12-20 21-24 ≥25

Heart Rate (beats/min) ≤40 41-50 51-90 91-110 111-130 ≥131


≤90 91-100 101-110 111-249 ≥250

O2 saturation ≤91 92-93 94-95 ≥96

Temperature (0C) ≤35.0 35.1, 36.0 36.1, 38.0 38.1, 39.0 ≥39.1


Receiving supplemental O2 therapy

Air Any O2

ViEWS) (Prytherch et al. 2010 in Churpek et al. 2013)

Score 3 2 1 0 1 2 3


<9 9-110 12-20 21-24 >24



≤90 91-100 101-110 111-249 ≥250

O2 saturation <92 92-93 94-95 96-100

Temperature (0C) <35.1 35.1, 36.0 36.1, 38.0 38.1, 39.0 >39.0



No Any O2

ViEWS (Prytherch et al. 2010 in Jo et al. 2013)

Score 3 2 1 0 1 2 3


≤8 9-11 11-20 21-24 ≥25



≤90 91-100 101-110 111-249 ≥250

O2 saturation ≤84 85-89 90-94 ≥95

Temperature (0C) ≤35.0 35.1, 36.0 36.1, 38.0 38.1, 39.0 ≥39.1



Air Any O2

310 | P a g e

SEWS (Paterson et al. 2006 in Churpek et al. 2013)

Score 3 2 1 0 1 2 3


<9 9-20 21-30 31-35 >35

Heart Rate (beats/min) <30 30-39 40-49 50-99 100-109 110-129 >129


≤70 70-79 80-99 100-199 >199

O2 saturation <85 85-89 90-92 93-100

Temperature (0C) <34.0 34.0.34.9 35.0, 35.9

36.0, 37.9

38.0,38.9 >38.9


DTEWS (Badriyah et al. 2014)

Score 3 2 1 0 1 2 3


≤18 19-20 21-24 ≥25

SpO2 (%) ≤89 90-92 93-94 95-99 100


≤38 39-46 47-89 90-100 >101


≤89 90-116 117-272 ≥273

Temperature (0C) ≤35.8 35.9,36.0 36.1,36.4 36.5,37.1 37.2,37.9 ≥38


Any supplemental O2

No Yes

311 | P a g e

LDT-EW (Jarvis et al. 2013)

Score 3 2 1 0 1 2 3

Males

Haemoglobin (Hb) ≤11.1 11.2-12.8 ≥12.9

White Cell Count (WCC) ≤9.3 9.4-16.6 ≥16.7

Serum Urea (U) ≤9.4 9.5-13.7 ≥13.8

Serum Creatinine (Cr) ≤114 115-179 ≥180

Serum Sodium (Na) 133-140 ≥141

Serum potassium (K) ≤3.7 3.8-4.4 4.5-4.7 ≥4.8

Serum albumin (Alb) ≤30 31-34 ≥35

Females

Hb ≤12.0 12.1-14.8 ≥14.9

WCC ≤12.6 12.7-14.8 ≥14.9

U ≤8.4 8.5-13.8 ≥13.9

Cr ≤91 92-157 ≥158

Na ≤134 135-140 ≥141

K ≤3.3 3.4-4.5 ≥4.6

Alb ≤28 29-34 ≥35

SUPER (Bian et al. 2015)

Score 0 1 2

SpO2 (%) 99-100 95-98 ≤94

Urine volume (ml/h) >50 30-50 ≤30

Pulse (beats/min) <90 90-140 >140

Emotion* 0 −/- - +

Respiratory rate (breaths/min) ≤20 20-30 ≥30

*Emotion indicates restlessness, excitement, agitation or overstimulation, delirium (+), normal or

sedation state (0), depression, apathy, unresponsive, lethargy (−), drowsiness, coma (- -)

CART (Churpek et al. 2012a in Churpek et al. 2013)

Score 0 4 6 8 9 12 13 15 22


<21 21-23 24-25 26-29 >29


<110 110-139 >139

Diastolic blood pressure (mmHg)

>49 40-49 35-39 <35

Age <55 55-59 >69

312 | P a g e

Longitudinal analysis of one million vital signs in patients in an academic medical center revealed

that the simultaneous presence of ≥3 critically abnormal vital signs any time during hospitalisation

was associated with higher in-hospital mortality as outlined within this Table (Bleyer et al. 2011)

Vital sign 5% mortality 10% mortality 20% mortality

Systolic blood pressure (mmHg): low

80 to <85 65 to <70 55 to <60

Diastolic blood pressure (mmHg): low

20 to <30

Diastolic blood pressure (mmHg): high

120 to <130

Mean arterial pressure (mmHg): low

40 to <50

Heart rate (beats/min): high

120 to <130 140 to <150 150 to <160

Temperature (°C): low 34.4 to <35 33.9 to <34.4

Temperature (°C): high 38.9 to <39.4 39.4 to <40

Respiratory rate (breaths/min): high

24 to 28 28 to 32 36 to <40

Respiratory rate (breaths/min): low

10 to 12 4 to 8

Oxygen saturation (%) 90 to <91 81 to <82

Level of consciousness Not alert Sedated No response

GCS 14 13

*Blank cells indicate that no vital sign range achieved corresponding level of mortality

MEDS* (Shapiro et al. 2007 in Geier et al. 2013)

Variable Points Comment

Terminal Illness 6 Rapidly fatal illness such as metastatic cancer with perceived 30-day mortality

Age > 65 years 3

Tachypnea or hypoxia 3 RR > 20 breaths/min, requiring O2 by mask, O2 saturation < 90%

Shock 3 SBP < 90 after appropriate IVF bolus

Thrombocytopenia 3 <150,000 cells/mm3

Bandemia 3 >5%

Nursing home resident 2

Lower respiratory tract infection 2

Altered mental status 2 By history or examination

*max score=27

313 | P a g e

REMS (Bulut et al. 2014)

Score 0 1 2 3 4 5 6

Age <45 45-54 55-64 65-74 >74

Heart rate (beats/min) 70-109 55-69 40-54 <40


12-24 10-11 6-9 34-49 <6

25-34 <49

Mean arterial pressure (mmHg)

70-109 50-69 130-159 <49

110-129 >159

GCS >13 11-13 8-10 5-7 <5

O2 saturation >89 86-89 75-85 <75

THERM* (Cattermole et al. 2014)

THERM score was defined as: [GCS] + [HCO3−]. Subtract 4 if hypotensive

HCO3− (to a maximum of 22mmol)

Hypotension (SBP<100 mmHg)

High-risk: THERM ≤30

Medium-risk: THERM 30.1–35

Low-risk: THERM 35.1–37

*max score=37

VSS (Etter et al. 2014)

Each of the six vital sign abnormalities (heart rate, systolic blood pressure, respiratory rate, oxygen

saturation, GCS, peripheral perfusion [capillary refill time of > 3 seconds is considered abnormal]) is

considered as one VSS point.

VSS=total sum of all VSS points at one point in time

Score

Respiratory rate (breaths/min) 1

Heart Rate (beats/min) 1

Systolic blood pressure (mmHg) 1

SpO2 1

GCS 1

capillary refill times of >3 seconds 1

314 | P a g e

Appendix 5a: BMJ Quality Checklist

Extract Study design

1. The research question is stated.

2. The economic importance of the research question is stated.

3. The viewpoint(s) of the analysis are clearly stated and justified.

4. The rationale for choosing alternative programmes or interventions compared is stated.

5. The alternatives being compared are clearly described.

6. The form of economic evaluation used is stated.

7. The choice of form of economic evaluation is justified in relation to the questions addressed.

Data collection

8. The source(s) of effectiveness estimates used are stated.

9. Details of the design and results of effectiveness study are given (if based on a single study). 10. Details of the methods of synthesis or meta-analysis of estimates are given (if based on a synthesis of a number of effectiveness studies).

11. The primary outcome measure(s) for the economic evaluation are clearly stated.

12. Methods to value benefits are stated.

13. Details of the subjects from whom valuations were obtained were given.

14. Productivity changes (if included) are reported separately.

15. The relevance of productivity changes to the study question is discussed.

16. Quantities of resource use are reported separately from their unit costs.

17. Methods for the estimation of quantities and unit costs are described.

18. Currency and price data are recorded.

19. Details of currency of price adjustments for inflation or currency conversion are given.

20. Details of any model used are given.

21. The choice of model used and the key parameters on which it is based are justified.

Analysis and interpretation of results

22. Time horizon of costs and benefits is stated.

23. The discount rate(s) is stated.

24. The choice of discount rate(s) is justified.

25. An explanation is given if costs and benefits are not discounted.

26. Details of statistical tests and confidence intervals are given for stochastic data.

27. The approach to sensitivity analysis is given.

28. The choice of variables for sensitivity analysis is justified.

29. The ranges over which the variables are varied are justified.

30. Relevant alternatives are compared.

31. Incremental analysis is reported.

32. Major outcomes are presented in a disaggregated as well as aggregated form.

33. The answer to the study question is given.

34. Conclusions follow from the data reported.

35. Conclusions are accompanied by the appropriate caveats

315 | P a g e

Appendix 5b: EunetHTA Toolkit Economic Evaluations- Transferability

To assess transferability 27. How generalisable and relevant are the results, and validity of the data and model to the relevant jurisdictions and populations? 28. a) Are there any differences in the following parameters? I. Perspective II. Preferences III. Relative costs IV. Indirect costs V. Discount rate VI. Technological context VII. Personnel characteristics VIII. Epidemiological context (including genetic variants) IX. Factors which influence incidence and prevalence X. Demographic context XI. Life expectancy XII. Reproduction XIII. Pre- and post intervention care XIV. Integration of technology in health care system XV. Incentives b) If differences exist, how likely is it that each factor would impact the results? In which direction? Of what magnitude? c) Taken together, how would they impact the results and of what magnitude? d) Given these potential differences, how would the conclusions likely change in the target setting? Are you able to quantify this in any manner? 29. Does the economic evaluation violate your national/regional guidelines for health economic evaluation?

316 | P a g e

Appendix 6: Extraction Table – Economics Review

Study Intervention Design (number of studies)

Condition(s) or population targeted

Type of EWS The type of economic evaluation

Outcome Measurement

HIQA (2015) Electronic NEWS Systematic Economic Literature Review (n=3) & BIA

Patients in a level 1 Irish Hospital (excluding maternity and paediatrics)

NEWS HTA N/A

NCEC (2013) NEWS Systematic Economic Literature Review (n=2) & BIA

Acute medical patients

NEWS BIA ICU LOS

NCEC (2014) NEWS as part of Sepsis Management (SM) framework

Systematic Economic Literature Review (n = 16) & BIA

SM-Adult, paediatric and maternity patients

NEWS + SM protocol

BIA ICU LOS, mortality, post sepsis syndrome

Simmes et al. (2014)

Implementation of a RRS Before and after study Surgical patients ≥72 hours post major general surgery

Rapid Response System

Cost description ICU LOS

Subbe et al. (2014)

Implementation of a computer-assisted triage system

Before and after study Acute medical unit patients

Advanced computer assisted triage system

Cost description Hospital LOS and costs

317 | P a g e

Analysis Details

Study (1)Setting-country or jurisdiction (2) Perspective (3) Time Horizon

(4) Included costs (cost type, cost categories) and resource items

(5) Data source costs and resource use

(6) Data source outcomes and benefits

(7)Methods of measuring or valuing outcomes and benefits

(8) Discounting (rate and reference year)

(9) Currency and currency conversions

(10) Analysis of sensitivity and uncertainty

HIQA (2015) (1) Ireland (2) public provider (3) 5 years

Software, hardware, integration fees, and implementation: project staff, education, and clinical leadership

UK pilot study; UK suppliers

Literature (Jones et al. 2011), Irish LOS data

N/A N/A EUR (adjusted for CPI and PPI for British pounds sterling to Euros)

CI estimated

NCEC (2013) (1) Ireland (2) public provider (3) 1 year

Initial phase: staff and non-staff costs. On-going intervention costs: non-staff, staff and savings

Literature and local expert opinion

Literature Reduced LOS in ICU estimated in monetary terms but interpreted as Efficiency savings

N/A Not detailed

None

NCEC (2014) (1) Ireland (2) public provider (3) 1 year

Initial phase: education and technology. On-going intervention: technology, staff

Literature, HIPE data and local costing information

NHS Briefing Prof Sir Mike Richards (Richards, 2013)

€4,500 per patient (improved ICU outcomes, reduced mortality and post sepsis syndrome)

N/A Euros None

Simmes et al. (2014)

(1)Netherlands (2) public provider (3) 3 years (1 year pre and 2

Implementation and maintenance costs: time taken to construct implementation plan, extra materials ICU & ward; RRS

Previous study by same authors. Prices for personnel and ICU costs retrieved from the Dutch guideline for cost

Hospital data – mean RRS per patient &

Mean RRS costs per patient day in monetary terms and

N/A 2009 prices using Dutch CPI (€)

None

318 | P a g e

years post implementation)

coordination ward, RRS coordination ward. Training costs: development, material, overheads, teachers, nursing training time and nursing time spent on observation vital signs

analyses in health care unplanned ICU days

Unplanned ICU days

Subbe et al. (2014) (1) District General Hospital- North Wales, UK.

(2)public provider

(3) 1 year (6 month pre-intervention + 6 month post intervention

Navigator salary; mean cost per patient reduction and overall cost reduction

Hospital Hospital – LOS and cost of care

Reduction in LOS pre and post intervention and cost of care pre and post intervention

N/A Sterling

(2011/2012)

None

319 | P a g e

Result Details

Study Costs and resource use Implementation Costs

Electronic costs (implementation)

Escalation Costs

Additional resources - work load

Education & Materials Costs

Electronic costs (education)

Outcomes / Costs averted

HIQA (2015)

Total cost for type one (including implementation costs) over five years is €1.0 million and type two is €1.3 million for each site. Nationally, this amounts to €40.1 million for type 1 and €51.4 million for type 2 over 5 years.

Project management and staff training Type 1: €227,453

Type 2: €119,200

License fees, hardware & maintenance Type 1: €767,117 Type 2: €1,189,762

Unknown Unknown n/a €23,310 Not estimated

NCEC (2013)

Initial phase: staff €7.47million and non-staff €18,000 (trainees and trainers)

(NEWS charts, taking additional measurements and charting scores considered negligible)

n/a Unknown but likely to increase

Not estimated

Initially €18,000 (manuals, CDs, charts etc.)

On-going: €425,000 per annum

n/a €4.2 million (reduction in ICU bed days, cardiac respiratory arrests)

NCEC (2014)

Total Costs: €1,936,567 (€1.4 million incurred in initial set up and €0.5 annual costs)

Total Savings: €12 million per annum

Estimated Sepsis Costs per annum €125 million

Education included in device costs; simulation training cost unknown. Technology: handheld PC readers, integrated interface with LAB €1.415 million. Local coordinator

(n/a n/a n/a Initially: education included in device costs; simulation training cost unknown. Ongoing training = negligible

n/a €12m per annum (reduction in lab tests and improved ICU outcome, mortality, post sepsis syndrome)

320 | P a g e

€410,000 (6 WTE)

Simmes 2014

Mean RRS costs were €26.87 per patient-day: implementation €0.33 (1%), training €0.90 (3%), nursing time spent on extended observation of vital signs €2.20 (8%), MET consults €0.57 (2%) and increased number of unplanned ICU days after RRS implementation €22.87 (85%). In the scenario analysis mean costs per patient-day were €10.18.

Implementation RSS/patient day: €0.33 (1% of mean RRS cost) – included the construction of an implementation plan and extra materials for the ward and ICU.

n/a n/a Coordination – 1 x nurse hour per week - €1,568 and continuation 20 nurse hours per year and 10 doctor hours per year – €2,050 Total coordination and continuation cost of RRS = €3,618 p/a

Training - items included in costs: development; materials; overheads; teachers and nurse training time – total €27, 291 or €0.90 per patient-day (3% of mean RRS cost per patient-day).

n/a No change in LOS. Unanticipated ICU admissions increased post-implementation (2.5%-4.2%). No change in ICU LOS.

321 | P a g e

Subbe 2014

Salary of Navigator -£25,484 (training & intervention period), Navigator cost per patient: £36

n/a n/a n/a n/a n/a n/a Mean LOS Reduction 1.85 days for patients with a very low risk of death.

Translates to -£482/very low risk patient. Overall cost reduction= £250,158 Based on the salary of the “Navigator” this represented a return on investment of £12 pounds return for every £1 pound invested