systematic learning from mistakes: achievements and challenges andy sutherland, nhs information...

Systematic learning from mistakes: achievements and challenges

Andy Sutherland, NHS Information Centre for health and social care

“All men make mistakes, but only wise men learn from their mistakes”

(Winston Churchill)

“Learn from the mistakes of others – you can never live long enough to make them all yourself”

(John Luther)

Summary

• Background and approach• Incidents – handling and analysis• Publication process• Improvement cycle• Present position• Reflections and next steps

Background

• No consistent approach to handling• Internal panic• External and internal blaming• Perception of repeating errors• Low customer confidence

Approach

• A system for handling things that go wrong• A system for learning from them

Adverse incidents process

• Notification• Confirmation• Evaluation• Handling plan• Handling

or

AAHAA…

• AlertInternally to Director and Head of Profession

• AssessImpact, options, considerations

• Handling planWhat to do, who to tell, when;

• Authorisation• Action

The first ‘A’ does not stand for ‘Action’!

And then

• Review• Learning• Implementing changes

…by means of

• Review meetings• Analysing root causes and drawing out

lessons• Openly available documentation

– Library of incidents – Library of root causes and lessons

• Regular learning fora (therapy and action)• alerts

Analysis

• 352 incidents March 2008 to May 2011

• Categorised by potential damage to NHS IC• 33 high • 199 medium• 120 near miss – eg trapped internally

Root cause analysis scoring

• None• Evidence of thought, but not cause• A cause, but not a root cause• A reasonable root cause analysis

Good and bad root causes

• “The cause of the problem was most likely due to the template being copied from another table.”

• “High level of risk identified but not effectively managed”

• “Not having a system… that was proven to meet clear and specific requirements.”

Lessons learned scoring

• No evidence• Evidence of thought• Lessons described• Evidence of lessons implemented

Lessons learned – good and bad

• “Additional checks to be implemented on the final report”

• “Processes will become ever more robust now that the work has been brought within the IC”

• “All web entries should have clear review dates attached …process for reviews…”

Incidents by month, March 08 – May 11

0

4

8

12

16

20

Incidents by month - Root cause found?

0

4

8

12

16

20

root cause

cause

thought

none

Incidents by month – lessons learned?

0

4

8

12

16

20

implemented

described

thought

no

Number of incidents by department - lessons learned?

0

20

40

60

80

100

a b c d e f g h i j k l m n o p

implemented

described

thought

none

Source of incidents

publication

data

pq

other

Reflection

• Incidents being reported• Handling improved (better feedback)• Root causes and lessons learned patchy• Little evidence of learning across organisation• Scope for action on publications

Publication process

• Systematic approach• Guidance on each stage• Clear responsibilities• Clear records

A process to improve!

Publication process

Production

Publication

ReviewingCompleted Publication

PlanningProcess Initiation

Input Guidance and

Templates

Output Documents,

Approvals and Records

Planning

Process Initiation

ProductionPublication Mandate

Mandate Approval

Publication Mandate

Brief

Publication Brief

Approval

Publication Brief

Plan

Plan Approval

Publication Plan

Template

Create the Team

Approval: Roles and

Responsibilities defined.

Confirm agreement to policies and procedures

Guidance on creating the publication

team

Design and Development

Design Approval including

customers and

stakeholders

Design and Development

guidance

Production

Planning PublicationData

Preparation

Data Preparation

approval

Approved set of data

Data Preparation

guidance

Analysis

Data analysis approval

Data Analysis process

Protocols for checking the analysis

Prepare draft

publication

Final draft approval

Style Guide

Preparing the draft

publication guidance

Publication

Production ReviewingPre-publication

Pre-Publication approval

Pre-Publication guidance

Guide to press release

production

Printing and distribution guidance

Approval and record of

confirmation of proof reading

Printing and Distribution

Review

Publication

Opportunities for improvement

guidance

Publication review approval

Publication review with

users

Completed publication

Mandate Approval

Responsibilities

Chief Executive

EDG (Directors)

Head of Profession

Programme Head

Programme Manager

Section Head

Quality Programme Manager

BriefDesign and Dev

Briefing and Press Rel.Review

PlanCreate the TeamDesign and Dev

Process/Pub Rev

BriefDesign and Dev

Process/Pub Review

Design and DevData Preparation

AnalysisFinal Draft approval

BriefingPress releasePrepublication

Printing

Press Release

Records…Template

Links to incidents

• The Planning stage includes review of lessons learned across the NHS IC

• The Production stage incorporates lessons (eg extra checks) from incidents

• The Review stage includes drawing out lessons learned from incidents during production…

…and feeds back into planning

Improvement cycle

• Incidents lead to lessons• Lessons lead to

Alerts

Improved processes• Publication process holds improved

processes and ensures they are implemented• Improved processes lead to

Fewer incidents

Example

• Breach of the Code of Practice – pre-release access list issued late

• Root cause: excessive willingness to accommodate late changes

• Lesson: set cut off time and freeze• Implemented and promulgated through

process• No further incidents

Present position

• Better handlingReduced panic

Involvement of Directors

Engagement of external stakeholders

Better feedback

• High level of reporting?Few unreported incidents coming to light

Salutary examples of complications from not reporting

• Evidence of lessons learned

But…

…

• Improvement still needed on root causes• Some good but some bad practice• Learning needs to be promulgated across the

organisation

Reflections

• Organisational change is hard• It takes time• It is necessary to

Make it easy for people to do the right thing

Avoid blame

but

Keep up the pressure

• Be open: a mistake made feels bad; a mistake learned from feels good

Next steps

• Stronger management emphasis on drawing out root causes and lessons - KPIs

• Developing experts to help with this• Continuing support – learning fora• More regular ‘alerts’• Benchmarking

“A man’s errors are his portals of discovery”

(James Joyce)

“This is also true for organisations”

(Andy Sutherland)