Slide 1

Quality Improvement Methodology Next StepsPurpose of this SessionConsider the components of a learning system in your own improvement activity :

4. Sequential testing of new theories 6. Planning for spread at scale

What stage are you at in relation to these aspects of the improvement journey? What do you need to do next to ensure that your tests are scaled up and spread correctly?We are going to look more closely at a couple of components of the learning system to help you think about where you are currently at and what you could do next in order to take your current work to this stage.2

The Improvement Guide, APIAimMeasuresChangesTesting & Implementation

33Hopefully you have opted for this session because you are feeling comfortable with the MFI and how it can be or is being applied in practice to take forward improvement activity in your CPP or within your own practice.

Reinforce the 3 questions and PDSA cycle to plan and guide improvement activityCycles of Tests Build Confidence

Proposals, theories, hunches, intuitionChanges that will result in improvementLearning from dataNorthwest Improvement Initiative R. Scoville & I.H.I.31You should also be familiar with the concept of multiple sequential cycles to test your theory by gradually building your learning through trying out a theory in different conditions with different people before it is ready to be implemented into everyday practice. Working our way up to implementation is a matter of building belief in the effectiveness of a change. First tests are small scale, under optimal conditions: changes that fail here are discarded quickly.If they succeed, we explore them in more detail: by replication under other circumstances, with different practitioners, in different locales, etc. If the change looks robust, it will gather the approval it will need for implementation. Along the way, were building confidence in the change. It also gives us the opportunity to make adaptations as we test so that the end product is fit for purpose across all the areas or population you need it to be.1 child1 day1 family1 setting

Move to 3,5,7. as confidence grows

Start small As you know beginning with a very small test population allows you to test something out without making a system change that you are not convinced is going to add value.

But it is the activity of scaling up the testing so that you grow confidence in the new process or activity through learning and adapting as you go that really builds the momentum for making improvement happen across the wider population.

5YearsQuartersMonthsWeeksDaysHoursMinutesDrop down next two levels to plan test cycle!You can only learn as quickly as you test! Northwest Improvement Initiative R. Scoville & I.H.I.And if you have been involved in carrying out tests - you will know that you only learn as quickly as you test.And the more regular the PDSA tests are carried out the quicker you become knowledgeable about the practical application of the theory you are working on.

Components of a Learning System System level measuresExplicit theory or rationale for system changesSegmentation of the populationLearn by testing changes sequentiallyUse informative cases: Act for the individual learn for the population Learning during scale-up and spread with a production plan to go to scalePeriodic reviewPeople to manage and oversee the learning system

From Tom Nolan PhD, IHI

7This is a key component of a learning system, which engages a number of people at the point of service delivery in improvement activity allowing the opportunity to influence and shape how the theory can best be applied in practice.7Sequential Testing & Scale UpHunch/TheoryA structured handover will ensure accurate information sharing between prof teamsData/learning/adaptingCycle 1: Test developed handover form 1 HV/1 SW 1 family (Mon)Cycle 2: Test with 3 family handovers/ same HV (Wed) Cycle 3: Test with 5 more families/ same HV (Mon/Tu) Cycle 4:Test with all handovers-1 HV/1weekAimAchieve improved communication process for HV and SW handovers using standardised format Cycle 5: other team HVs/SWs- 1 day

Cycle 6: Test all handovers-1 week Implement Sustain in practice and spread to other areas8Learning Through Sequential TestingWhere are you currently?Can you describe an example of the following?

Multiple tests with different people/under different conditions?Learning/data captured to describe your testing journey?

Thinking about your current activity consider these questionsTake 5 mins to discuss with your neighbour If you dont have a worked example at the moment can you describe a potentially high impact theory that you may be able to influence in the future?

9 Testing v ImplementationTesting Trying and adapting existing knowledge on small scale. Learning what works in your system. Multiple tests in a variety of conditions

Implementation Making the change a part of the day-to-day operation of the system. Permanent change Would the change persist even if its champion were to leave the organisation?

Avoid implementation until confident that processes are robust Reminder that this will require process measurement to inform and assure of the reliability and sustainability of the change as it is imbedded in practice.10How ManyTests?

11Components of a Learning System System level measuresExplicit theory or rationale for system changesSegmentation of the populationLearn by testing changes sequentiallyUse informative cases: Act for the individual learn for the population Learning during scale-up and spread with a production plan to go to scalePeriodic reviewPeople to manage and oversee the learning system

From Tom Nolan PhD, IHI

12The next crucial step when you have carried out multiple tests and are beginning to scale up the testing is to consider a plan that will enable you to take this work to scale across the wider community or population.12Better IdeasDevelop the caseDescribe the ideasSet-upTarget populationAdopter audiencesSuccessful sitesKey partnersInitial spread planKnowledge ManagementMeasurement and FeedbackCommunication (awareness & technical)Social SystemKey messagesCommunitiesTechnical supportTransition issuesLeadershipTopic is a key strategic initiativeGoals and incentives alignedExecutive sponsor assignedDay-to-day managers identifiedA Framework for SpreadInstitute for Healthcare Improvement13

Things to ConsiderChecklists For Spread

Leadership, Better Ideas & Set Up

General Communication & knowledge transfer

Developing Measurement, Feedback and Knowledge Management Systems

Are You Ready for Scale up & Spread?In the context of your current improvement activity:

Have you been testing a theory so that it could be considered ready for implementation in the area you are working? and/orDo you have a strategy for moving to scale up and spread to other sites/teams?

Breakout discussion at tables

Have a think about where you have got to on this journey. At what stage is your current activity and improvement work?

Continuing to test not ready to scale up?Scaled up in own team/office but not yet considered spread to wider system?Scaled up and spread across the system? If so, how did you do this.? Do you have data to support your improvement?

15The Seven Spreadly SinsStep 1Start with large pilots

Step 2Find one person willing to do it all

Step 3 Expect vigilance and hard work to solve the problem

Step 4 If a pilot works then spread the pilot unchanged

Step 5 Require the person and team who drove the pilot to be responsible for system-wide spread

Step 6 Look at process and outcome measures on a quarterly basis

Step 7 Early on expect marked improvement in outcomes without attention to process reliabilityInstitute for Healthcare ImprovementIf you follow these steps your spread effort will fail16Sustaining Improvement Having the correct measures to provide assurance that new processes are reliable

Measuring compliance or satisfaction through regular and random sampling of the population

Understanding the variation that exists in your dataIn relation to measurement dont over react to individual data points seek to understand the influencing factors that impact on the data. This all comes back to understanding what the normal or common cause variation is in the system..17What measures?Outcome measures directly relates to the overall aim what is the result? how is the system performing?

Process measures are the processes that contribute to the aim performing as planned?

Balancing measures assessing from different dimensions unanticipated consequences, other factors influencing the outcomeEg. Workstream outcome measure for developmental milestonesProcess measures for activites that are taken forward by nursery teams and HVs the new processes tested and in place to measure compliance with the process. Balancing does this scaled up activity have an unintended consequence on the team involved eg the length time it takes to do an activity or to complete a proforma, does this prevent other necessary work from taking place?

18If we dont understand the variation that lives in our data, we will be tempted toDeny the data as it doesnt fit with our view of realitySee trends where there are noneTry to explain natural variation as special eventsBlame and give credit to people for things over which they have no controlDistort the process that produced the dataKill the messenger!VRAIATIONCommon Cause Variation

Define measures that will measure the impact of theImprovement work over time

They will guide your progress through and beyond testing to implementation and monitoring for continuous improvement. Different ways of measuring e.g.,Percent compliance with processA count of correct attempts/number of attendances Verbal feedback /surveys

Measurement of ImprovementMeasures monitor and guide your progress through an improvement journey.

And you will need to define the measures that are going to be able to demonstrate the efforts of your improvement work

A number of different approaches can be taken;

It could be percentage compliance with somethingA count of correct attempts at doing somethingOr verbal feedback21MonthJanFebMarAprMayJunJulAugSepOctNovMeasure8380818483856887899291HC Data Guide Why a run chart and not just a graph or table?MonthMedian = 84%Run ChartTesting with all FamiliesPrompt sticker tested in case referral Testing screening tool with 1 familyTesting screening tool with 5 familiesPrompt sticker used on all referrals Testing screening tool with 3 Families2222Source: The Data Guide by L. Provost and S. Murray, Austin, Texas, February, 2007: Non-Random Rules For Run Charts

A Shift: 6 or more

A Trend:5 or more

An astronomical data point

Too many or too few runs23

MonthWhat is this data telling you? Data 11 1 3 4 5 9 10 4 11 7 10 10 7 6 22 4 2 2 1 3 4

MonthWhat is this data telling you? Median 1 1 2 2 3 3 4 4 4 4 5 6 7 7 9 10 10 10 11 11 22 Median = 5ShiftShiftTrendAstronomicalPoint25

Measurement PrinciplesDevelop aims before measuring

Design measures around aims How Good By When

Be clear on your operational definitions

Establish a reliable baseline

Track progress over time using annotated run charts

Teams need measures to give them feedback that the changes they are making are resulting in improvement

Need to understand common cause and special cause variation to ensure we dont over/under react to situations

Appreciation of a System Complex system of interaction between people, procedures and equipment Success depends on integration, not performance of individual partsUnderstanding Variation Variation is to be expected everything we measure varies We make decisions based on interpretation Data over time data story of what has been happeningTheory of KnowledgeChange is prediction of improvement based on knowledge of the systemLearning from theory, experienceOperational definitions are the basis for improvement with PDSA cycles for learningPsychologyInteraction of people with systemsMotivation & will of individuals & teamsSituation awareness/decision makingManaging stress and fatigueHelps planning for change management

ImprovementAims & Values27You really need to understand your whole system so that the messiness of life doesnt catch you out.

What might systems tell us most systems result from a complex system of interaction between people procedures and equipment. Success depends on integration, not the performance of individual parts.What might psychology tell us how people interact with each other and the system. Helps with planning for change management. Motivation and will of individuals and teamsWhat might variation tell us - everything we measure varies and we make decisions based on interpretation of this variation. Eg was the improved performance this week the result of the change we made or just luck. Examining data over time.concept introduced by Shewhart which allows us to learn from the data story of what has been happeningWhat might theory of knowledge tell us change is a prediction of improvement based on knowledge of a system. If the changes do not result in an improvement as predicted we need to identify what happened and use the understanding and learning from this to help refine the theory. This is the basis for a learning cycle and this is built into the PDSA cycle.

Spread to Total System(Additional units, sites, organisations)Single-unit prototype: segmentsSystem Targeted for Implementation(Defined by Aim)

Whats the Scope of Change?As you move from pilot testing to implementation to spread, your population of interest will need to be adjusted.Improvement Science in Action2828Planning for spread is crucial even at an early stage in your improvement work, it is not too soon to be thinking about the scope of this change in your system and wider population.

Understand the difference between testing and implementation and dont be tempted to spread too quickly to the wider system without a strategy to support it this will give you the best chance of achieving success for the long term.

Thank You29Chart1111345910411710107622422134

NumberSubgroup

wksDatabase01CCT_660671ChartTypeRun_ChartSourceDataAddress40299FirstRowHeadingsExtendSubgroupSize0RateBase0DataInColumnsForceLowerLimitZeroForceUpperLimitOneIndex0Values1Phase 1121AutoIndexMonthSubgroupMedian1Sep-08115.02Oct-0815.03Nov-0835.04Dec-0845.05Jan-0955.06Feb-0995.07Mar-09105.08Apr-0945.09May-09115.010Jun-0975.011Jul-09105.012Aug-09105.013Sep-0975.014Oct-0965.015Nov-09225.016Dec-0945.017Jan-1025.018Feb-1025.019Mar-1015.020Apr-1035.021May-1045.0

Sheet1MonthNumber of InfectionsSep-0811Oct-081Nov-083Dec-084Jan-095Feb-099Mar-0910Apr-094May-0911Jun-097Jul-0910Aug-0910Sep-097Oct-096Nov-0922Dec-094Jan-102Feb-102Mar-101Apr-103May-10411134591041171010762242213411 1 3 4 5 9 10 4 11 7 10 10 7 6 22 4 2 2 1 3 4

Sheet1111345910411710107622422134

Number of InfectionsSubgroup

Sheet2

Sheet3

Chart1111345910411710107622422134

NumberSubgroup

wksDatabase01CCT_660671ChartTypeRun_ChartSourceDataAddress40299FirstRowHeadingsExtendSubgroupSize0RateBase0DataInColumnsForceLowerLimitZeroForceUpperLimitOneIndex0Values1Phase 1121AutoIndexMonthSubgroupMedian1Sep-08115.02Oct-0815.03Nov-0835.04Dec-0845.05Jan-0955.06Feb-0995.07Mar-09105.08Apr-0945.09May-09115.010Jun-0975.011Jul-09105.012Aug-09105.013Sep-0975.014Oct-0965.015Nov-09225.016Dec-0945.017Jan-1025.018Feb-1025.019Mar-1015.020Apr-1035.021May-1045.0

Sheet1MonthNumber of InfectionsSep-0811Oct-081Nov-083Dec-084Jan-095Feb-099Mar-0910Apr-094May-0911Jun-097Jul-0910Aug-0910Sep-097Oct-096Nov-0922Dec-094Jan-102Feb-102Mar-101Apr-103May-10411134591041171010762242213411 1 3 4 5 9 10 4 11 7 10 10 7 6 22 4 2 2 1 3 4

Sheet1111345910411710107622422134

Number of InfectionsSubgroup

Sheet2

Sheet3