Quality Improvement Methodology – Next Steps

Purpose of this Session

Consider the components of a learning system in your own improvement activity :

4. Sequential testing of new theories

6. Planning for spread at scale

What stage are you at in relation to these aspects of the improvement journey?

What do you need to do next to ensure that your tests are scaled up and spread correctly?

The Improvement Guide, API

Aim

Measures

Changes

Testing & Implementation

Cycles of Tests Build Confidence

AP D

S

AP

D S

APD

SA P

DS

Proposals, theories, hunches, intuition

Changes that will result in improvement

Learning from data

• 1 child• 1 day• 1 family• 1 setting

Move to 3,5,7…. as confidence grows

Start small

Years

Quarters

Months

Weeks

Days

Hours

Minutes

Drop down next “two levels” to plan test cycle!

You can only learn as quickly as you test!

Components of a Learning System

1. System level measures

2. Explicit theory or rationale for system changes

3. Segmentation of the population

4. Learn by testing changes sequentially

5. Use informative cases: “Act for the individual learn for the population”

6. Learning during scale-up and spread with a production plan to go to scale

7. Periodic review

8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

Sequential Testing & Scale Up

Hunch/TheoryA structured handover

will ensure accurate information sharing between prof teams

Data/learning/adapting

Cycle 1: Test developed handover form 1 HV/1 SW 1 family (Mon)

Cycle 2: Test with 3 family handovers/ same HV (Wed)

Cycle 3: Test with 5 more families/ same HV (Mon/Tu)

Cycle 4:Test with all handovers-1 HV/1week

AimAchieve improved communication

process for HV and SW handovers using standardised format

Cycle 5: other team HVs/SWs- 1 day

AP D

S

AP

D S

APD

SA P

DS

Cycle 6: Test all handovers-1 week

Implement

Learning Through Sequential Testingat your tables discussion time

Where are you currently?

Can you describe an example of the following?

Multiple tests with different people/under different conditions?

Learning/data captured to describe your testing journey?

Testing v Implementation

Testing – Trying and adapting existing knowledge on small scale. Learning what works in your system. Multiple tests in a variety of conditions…………………

Implementation – Making the change a part of the day-to-day operation of the system. Permanent change

Would the change persist even if its champion were to leave the organisation?

Avoid implementation until confident that processes are robust

How Many Tests?

Components of a Learning System

1. System level measures

2. Explicit theory or rationale for system changes

3. Segmentation of the population

4. Learn by testing changes sequentially

5. Use informative cases: “Act for the individual learn for the population”

6. Learning during scale-up and spread with a production plan to go to scale

7. Periodic review

8. People to manage and oversee the learning system

From Tom Nolan PhD, IHI

Better Ideas•Develop the case•Describe the ideas

Set-up•Target population•Adopter audiences•Successful sites•Key partners•Initial spread plan

Knowledge Management

Measurement and Feedback

Communication (awareness & technical)

Social System•Key messages•Communities•Technical support•Transition issues

Leadership•Topic is a key strategic initiative•Goals and incentives aligned•Executive sponsor assigned•Day-to-day managers identified

A Framework for Spread

Institute for Healthcare Improvement

Things to ConsiderChecklists For Spread

•Leadership, Better Ideas & Set Up

•General Communication & knowledge transfereveloping Measurement, Feedback and Knowledge Management Systems

Take a strategic approach to scaling up

Level of testing

Strategy

1

5

25

125

250

Increasing uptake of Healthy Start Vitamins

Level of testing

Strategy

1 1 EY practitioner gives vits to 1 mum

5

25

125

250 Free vitamins in Asda with every pregnancy kit

Are You Ready for Scale up & Spread?Table discussion time

In the context of your current improvement activity:

Have you been testing a theory so that it could be considered ready for implementation in the area you are working? and/or…

Do you have a strategy for moving to scale up and spread to other sites/teams?

The Seven Spreadly SinsStep 1 Start with large pilots

Step 2 Find one person willing to do it all

Step 3 Expect vigilance and hard work to solve the problem

Step 4 If a pilot works then spread the pilot unchanged

Step 5 Require the person and team who drove the pilot to be responsible for system-wide spread

Step 6 Look at process and outcome measures on a quarterly basis

Step 7 Early on expect marked improvement in outcomes without attention to process reliability

Institute for Healthcare Improvement

To be clear

DO NOT DO

THIS AT

HOME!!!

Sustaining Improvement

Having the correct measures to provide assurance that new processes are reliable

Measuring compliance or satisfaction through regular and random sampling of the population

Understanding the variation that exists in your data

What measures?

Outcome measures – directly relates to the overall aim what is the result? how is the system performing?

Process measures – are the processes that contribute to the aim performing as planned?

Balancing measures – assessing from different dimensions unanticipated consequences, other factors influencing the outcome

If we don’t understand the variation that lives in our data, we will be tempted to…

• Deny the data as it doesn’t fit with our view of reality• See trends where there are none• Try to explain natural variation as special events• Blame and give credit to people for things over which

they have no control• Distort the process that produced the data• Kill the messenger!

V RA I A

T

IO

N

Common Cause Variation

Define measures that will measure the impact of the

Improvement work over time

They will guide your progress through and beyond testing to implementation and monitoring for continuous improvement.

Different ways of measuring e.g.,

• Percent compliance with process

• A count of correct attempts/number of attendances

• Verbal feedback /surveys

Measurement of Improvement

Month Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov

Measure 83 80 81 84 83 85 68 87 89 92 91

HC Data Guide

Why a run chart and not just a graph or table?

Month

Median = 84%

Run Chart

Testing with all

Families

Prompt sticker

tested in case referral

Testing screening tool with 1 family

Testing screening tool with 5

families

Prompt sticker used on all referrals Testing screening

tool with 3 Families

Source: The Data Guide by L. Provost and S. Murray, Austin, Texas, February, 2007:

Non-Random Rules For Run Charts

A Shift: 6 or more

A Trend:5 or more

An astronomical

data point

Too many or too few

runs

0

5

10

15

20

25

9/1

/08

10/1

/08

11/1

/08

12/1

/08

1/1

/09

2/1

/09

3/1

/09

4/1

/09

5/1

/09

6/1

/09

7/1

/09

8/1

/09

9/1

/09

10/1

/09

11/1

/09

12/1

/09

1/1

/10

2/1

/10

3/1

/10

4/1

/10

5/1

/10

Nu

mb

er

Month

What is this data telling you?

Data 11 1 3 4 5 9 10 4 11 7 10 10 7 6 22 4 2 2 1 3 4

0

5

10

15

20

259/1

/08

10/1

/08

11/1

/08

12/1

/08

1/1

/09

2/1

/09

3/1

/09

4/1

/09

5/1

/09

6/1

/09

7/1

/09

8/1

/09

9/1

/09

10/1

/09

11/1

/09

12/1

/09

1/1

/10

2/1

/10

3/1

/10

4/1

/10

5/1

/10

Nu

mb

er

Month

What is this data telling you?

Median 1 1 2 2 3 3 4 4 4 4 5 6 7 7 9 10 10 10 11 11 22

Median = 5

Shift

Shift

Trend

AstronomicalPoint

Measurement Principles• Develop aims before measuring

• Design measures around aims ‘How Good By When’

• Be clear on your operational definitions

• Establish a reliable baseline

• Track progress over time using annotated run charts• Teams need measures to give them feedback that the changes

they are making are resulting in improvement• Need to understand common cause and special cause variation to

ensure we don’t over/under react to situations

Appreciation of a System• Complex system of interaction between people, procedures and equipment• Success depends on integration, not performance of individual parts

Understanding Variation• Variation is to be expected – everything we measure varies We make decisions based on interpretation• Data over time – data story of what has been happening

Theory of Knowledge•Change is prediction of improvement based on knowledge of the system•Learning from theory, experience Operational definitions are the basis for improvement with PDSA cycles for learning

Psychology•Interaction of people with systems•Motivation & will of individuals & teams•Situation awareness/decision making•Managing stress and fatigue•Helps planning for change management

Improvement

Spread to Total System

(Additional units, sites, organisations)

Single-unit prototype: segments

System Targeted for Implementation(Defined by Aim)

What’s the Scope of Change?

As you move from pilot testing to

implementation to spread, your population of

interest will need to be adjusted.

Thank You