QI Skills Lab Reliability 101

Scott Wexelblatt and Sandy Fuller OPQC Fall Learning Session

September 28, 2015

Objectives

• Summarize key concepts of Reliability • Demonstrate understanding of Reliability

Science through hands-on learning • Identify one next step to apply Reliability

Science to team goals for OPQC

Goals for High Reliability Health Care

Organization • No needless deaths • No needless pain • No helplessness • No unwanted waiting • No waste

Don Berwick, Institute for Healthcare Improvement

What is “Reliability”?

• The measurable capability of an object to perform its intended function in the required time under specified conditions. (Handbook of Reliability Engineering, Igor Ushakov editor)

• The probability of a product performing without failure a specified function under given conditions for a specified period of time. (Quality Control Handbook, Joseph Juran editor)

• The extent of failure-free operation over time. (David Garvin)

Definition of Reliability for Health Care

The capability of a process, procedure or health service to perform its intended function in the required time under existing conditions.

Quantifying “Reliability”

• “Reliability” = Number of actions that achieve the intended result ÷ Total number of actions taken

• Defect rate = 1 - “Reliability”

• It is convenient to use defect rate as an index, expressed as an order of magnitude

– 10-1 means that 1 time in 10, the action fails to achieve its intended result

– 10-2 means that 1 time in 100, the action fails to achieve its intended result

Starting Definitions of Reliability

RELIABILITY

Reliability = Failure Rate =

The Math number with intended outcome total processes or procedures

number without intended outcome total processes or procedures

9 10

1 10

95 100

5 100

Levels RELIABILITY

Reliability Failure Rate ~ 9 Level 1 (10-1) ~ 1 10 10 ~95 Level 2 (10-2) ~ 5 100 100

~995 Level 3 (10-3) ~ 5 1000 1000

Health Care Process Reliabilities

(Un)Reliability Outcome/Process

10-1

Beta blockers for MI >3 Glycosylated hemoglobin tests/2 yrs

10-2 Polypharmacy in the elderly Medication injuries Deaths in risky surgery

10-3 Neonatal mortality General surgery deaths

10-4 Deaths in routine anesthesia

10-5 Deaths from major radiotherapy machine failures

10-6 Deaths from seismic non-compliance

For further reading, see: McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. New England Journal of Medicine. 2003;348.

Reliability and Evidence Based Care

Observation: Almost all studies that investigate the reliability of the application of clinical evidence conclude that it would be classified as a Chaotic process or a 10-1 process.

Design for Reliability

Level 1. Intent, vigilance and hard work Level 2. Design informed by reliability science and research in human factors Level 3. Design of high reliability organizations (Weick)

Level 1 Concepts: Intent, Vigilance and Hard Work

(designing basic failure prevention)

• Standardize using orders sets, Pathways, Protocols • Working harder next time • Feedback of information on compliance • Awareness and training

Factors Affecting Human Vigilance (Examples)

• Fatigue • Environmental Conditions • Task Design • Psychological Conditions • Competing Demands

Level 2 concepts Human Factors and Reliability Science

(designing sophisticated failure prevention, basic failure identification and mitigation)

• Decision aids and reminders built into the system • Desired action the default(based on evidence) • Redundancy • Scheduling • Real-time Identification of Failures • Standardization of process

Examples of Level 2 Change Concepts

• Standardization of Process: All attendings conduct rounds the same, Trauma team.

• Standard Scheduling Rules: Call Center • Decision Aid Pop-ups: “Remember to give Flu

Shot”, “Did you order a drug level?” • Default to the appropriate option: Patients get

asthma education whether physician orders or not.

Additional Level 2 Examples

• Redundancy: Two people check narcotics, Order read back for verbal orders, second person verifies charge capture at the end of clinic

• Checklists: Direct Admit Checklist, Handoff Checklist.

• Scheduling: An area is scheduled to be cleaned every morning, does not need to be requested.

• Real-Time ID of Failures: Identify and Mitigate Evidence-based Recommendations.

Level 3 Concepts (Sophisticated Behavioral Designs)

• Take advantage of habits and patterns • Make the system visible • Clear and unambiguous communication • Mindfulness – Weick and Sutcliffe: “High Reliability Organizations”

DESIGN FOR RELIABILITY

• Identify desired level of reliability at the outset.

• Identify key process steps and failure modes.

• Redesign and test using components predicted to achieve the desired level of reliability.

Process Name _________________________________ FA

ILU

RE

MO

DE

S

INTE

RV

EN

TIO

NS

C

UR

RE

NT

P

RO

CE

SS

Example – Making a Perfect Cup of Coffee Every time

Pour water in

the machine

Add coffee Turn on the coffeemaker

Pour coffee into mug Add milk Add sugar

Too much water Too little water

Too much coffee Forgot filter

Forget to plug in coffeemaker Coffee maker

does not heat up water

Wrong mug Mug very cold

Too much milk Not enough milk

Too much sugar

Not enough sugar

Checklist - Level 2 (102)

Decision Aid - Level 2 (10-2)

Awareness and training - Level 1

(10-1) Desired Action

Default Kit - Level 2 (10-2)

Decision Aid - Level 2 (10-2)

Checklist - Level 2 (10-2)

Preoccupation with failure level 3

(10-3)

Standardization of essential tasks -

Level 2 (10-2)

Real time identification of

failures - Level 2 (10-2)

Desired Action Default Kit - Level

2 (10-2)

Desired Action Default Kit - Level

2 (10-2)

Group Exercise • Break into 4 groups • Think of one process for either NAS or Progesterone • Identify the high level steps • For each step

– consider what could go wrong – prioritize the ones with likely failures – identify at least 1 or 2 interventions you could test to

improve the process • Determine level of Reliability for each intervention • What could you do to increase the level of reliability? • What tools might you use?