reliability theory concepts: (how to be a coach) delivering uniformly excellent and highly reliable...
TRANSCRIPT
![Page 1: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/1.jpg)
Reliability Theory Concepts: (How to be a Coach)
Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System
Roger Resar MDSenior Fellow Institute for Healthcare
Improvement
Sept 2012Sept 2012
![Page 2: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/2.jpg)
© 2000 Institute for Healthcare Improvement
Session Objective
Learn how to evaluate improvement work for reliable design concepts
![Page 3: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/3.jpg)
Highlights of Reliable Process Design
(See Word Document Handout for Details)
![Page 4: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/4.jpg)
Key Questions To Analyze Testing and Implementation
Key Question Your EvaluationIs the connection between goals and process clear?
Is the design strategy primarily vigilance and hard work?
Has some degree of segmentation been used to test the design?
Is standard work with testing been part of the design?
Is a design methodology being used?
Are small tests of change being used in a rapid cycle?
Is data collection rapid enough?
![Page 5: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/5.jpg)
© 2000 Institute for Healthcare Improvement
Testing :Connection Between Goals and
Process
Key Question: Is the connection between goals
and process clear?
![Page 6: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/6.jpg)
© 2000 Institute for Healthcare Improvement
Prime Importance
Designing and testing a process or a sequenced set of processes which fails to connect to the outcome goal makes no sense
![Page 7: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/7.jpg)
© 2000 Institute for Healthcare Improvement
Outcome Goals Key Drivers Processes
Decrease Falls Reliable risk assessment of patients for falls
Red socks for at risk patients
![Page 8: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/8.jpg)
© 2000 Institute for Healthcare Improvement
Outcome Goals
Key Drivers Processes
Decrease Falls
Toileting in at risk patients
Every 2 hour toileting rounds on at risk patients
![Page 9: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/9.jpg)
Key Questions To Analyze Testing and Implementation
Key Question Your EvaluationIs the connection between goals and process clear?
Is the design strategy primarily vigilance and hard work?
Has some degree of segmentation been used to test the design?
Is standard work with testing been part of the design?
Is a design methodology being used?
Are small tests of change being used in a rapid cycle?
Is data collection rapid enough?
![Page 10: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/10.jpg)
© 2000 Institute for Healthcare Improvement
Implementation and SpreadVigilance and Hard Work as a
Design Strategy
Key Question: Am I guilty of hard work and vigilance
as my only design strategy?
![Page 11: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/11.jpg)
Improvement Concepts Associated with less than 95% Performance (Primarily can be described as intent, vigilance, and hard work)
Common equipment, standard order sheets, multiple choice protocols, and written policies/procedures
Personal check lists
Feedback of information on compliance
Suggestions of working harder next time
Awareness and training
![Page 12: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/12.jpg)
Improvement Concepts Associated with 95% Performance (Uses human factors and reliability science to design sophisticated failure prevention, failure identification, and mitigation)
Decision aids and reminders built into the system
Desired action the default (based on scientific evidence)
Redundant processes utilized
Scheduling used in design development
Habits and patterns know and taken advantage of in the design
Standardization of process based on clear specification and articulation is the norm
![Page 13: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/13.jpg)
© 2000 Institute for Healthcare Improvement
Hard Work and Vigilance
Is a commendable trait, but not a good design strategy
Is necessary but not sufficient to attain and sustain 95% performance
Feels comfortable because we have been trained to expect hard work and vigilance in our colleagues
![Page 14: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/14.jpg)
I would be So Bold as to Say:
If human factor concepts do not make up at least 25% of the improvement effort on a given project, chances are the project is
stalled or will never achieve sustained 10-2 performance
![Page 15: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/15.jpg)
How to measure the human factor component
• Determine the number of tests of change involving human factors testing
• Study the finished design
• Look at the defects generated and the approach to solving them
![Page 16: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/16.jpg)
Key Questions To Analyze Testing and Implementation
Key Question Your EvaluationIs the connection between goals and process clear?
Is the design strategy primarily vigilance and hard work?
Has some degree of segmentation been used to test the design?
Is standard work with testing been part of the design?
Is a design methodology being used?
Are small tests of change being used in a rapid cycle?
Is data collection rapid enough?
![Page 17: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/17.jpg)
© 2000 Institute for Healthcare Improvement
Testing and SpreadSegmentation and testing designs
Key Question: Have we taken advantage of
segmentation?
![Page 18: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/18.jpg)
© 2000 Institute for Healthcare Improvement
Why Segmentation is Helpful
Allows for the control of some variables
Defines the boundaries around which sequential expectations for success can be found
More likely to test the validity of the design rather than deal with barriers
Fosters a deeper understanding of the design complexity required for the project
Forces understanding of the differences between segments as design strategies
Allows the formation of more predictable timelines
![Page 19: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/19.jpg)
Finding your first segment• The segment must represent a reasonable volume
• The segment should have clear cut defined boundaries
• The segment should have willing participants so the barrier of agreeing is not a problem
• The segment should allow for key articulated variables or barriers to be neutralized
• The first segment should establish a design theme
![Page 20: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/20.jpg)
20
Identification of Other Segments
• The total number of segments for a topic should not exceed 4-5
• Segments should follow some theme in design (route of admission, type of physician, etc)
• Segments should differ by a distinct design feature
• The initial division of segments can be adjusted as the design is developed
• The segments should cover the population involved in the topic
![Page 21: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/21.jpg)
Team Results IHI Reliability Collaborative
![Page 22: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/22.jpg)
Key Questions To Analyze Testing and Implementation
Key Question Your EvaluationIs the connection between goals and process clear?
Is the design strategy primarily vigilance and hard work?
Has some degree of segmentation been used to test the design?
Is standard work with testing been part of the design?
Is a design methodology being used?
Are small tests of change being used in a rapid cycle?
Is data collection rapid enough?
![Page 23: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/23.jpg)
Testing and ImplementationStandardization versus Standard
Work
Key Question: Is the team committed to really setting up
standard work?
![Page 24: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/24.jpg)
Standard Work
• Standard work allows for the training of new employees and the testing of current employees
• If 5 front line process users can describe the process you have a good chance to achieve 95% performance and to sustain the performance over time
![Page 25: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/25.jpg)
The Pneumovax Example
• Commonly “Give Pneumovax if indicated”
• Poorly defines a process
• Default is too commonly not to give the Pneumovax
• No testing of competency or training of new employees can occur
![Page 26: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/26.jpg)
New Standardization Concepts
• Standardize to provide the appropriate infrastructure (the how, what, where, who and when)
• The “what” we are standardizing is based on medical evidence
• The “how” does not need medical evidence but rather on systems knowledge
• Initial standardized protocols with small time investment by experts tested at a very small scale
• Changes to the protocol in the initial stages should be required and encouraged
• Defects are studied and used to redesign the process
![Page 27: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/27.jpg)
Key Learning Point
A single standardized process within the acceptable science is superior to allowing multiple processes while we decide which is the best because it allows testing for competency and training new employees
![Page 28: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/28.jpg)
Key Questions To Analyze Testing and Implementation
Key Question Your EvaluationIs the connection between goals and process clear?
Is the design strategy primarily vigilance and hard work?
Has some degree of segmentation been used to test the design?
Is standard work with testing been part of the design?
Is a design methodology being used?
Are small tests of change being used in a rapid cycle?
Is data collection rapid enough?
![Page 29: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/29.jpg)
The Three Step Design for Reliability
Design Techniques Steps1-Identify the process to standardize
2-Segment the population to test the
design for anomalies
Prevent initial failure by standardizing the
process to achieve 80% (step 1)
1-Utilize a robust HF concept to make visible failures from step 1 after step 1 has achieved 80% reliability
2-Once the failure is identified, apply an action to mitigate the failure
Identify failures in step 1 and apply an action to achieve 80% for these failures (step 2)
1-Identify common failures
2-Develop a method to measure and study failures
3-Utilize knowledge of common failures to redesign either step 1 or step 2
In either step 1 and/or step 2 detect the failures, and use the knowledge from analysis of the failures to redesign (step 3)
![Page 30: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/30.jpg)
Methodology High Points
• Perfection is the enemy of reliable design (step one 80%; step two 80%)
• People who use the design do the designing
• Constant testing based on what defects were observed
• Segmentation tests the design
![Page 31: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/31.jpg)
Key Questions To Analyze Testing and Implementation
Key Question Your EvaluationIs the connection between goals and process clear?
Is the design strategy primarily vigilance and hard work?
Has some degree of segmentation been used to test the design?
Is standard work with testing been part of the design?
Is a design methodology being used?
Are small tests of change being used in a rapid cycle?
Is data collection rapid enough?
![Page 32: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/32.jpg)
© 2000 Institute for Healthcare Improvement
TestingRapid Cycle Change Methodology
Key Question: How frequent are the cycles of
testing?
![Page 33: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/33.jpg)
© 2000 Institute for Healthcare Improvement
How to Evaluate
Are testing cycles being used on an acceptable basis?
Are huddles occurring after the tests for quick redesign?
Are records being kept of the tests?
Do all team members have test responsibilities?
![Page 34: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/34.jpg)
Critical Failure Mode Essentials
• A measurement of critical failure modes needs to be part of the initial design strategy
• Assesses the defects that occur from the current design
• Should be prioritized in terms of overall affect on the reliability of the process change
• Should be used to redesign the process
![Page 35: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/35.jpg)
Key Questions To Analyze Testing and Implementation
Key Question Your EvaluationIs the connection between goals and process clear?
Is the design strategy primarily vigilance and hard work?
Has some degree of segmentation been used to test the design?
Is standard work with testing been part of the design?
Is a design methodology being used?
Are small tests of change being used in a rapid cycle?
Is data collection rapid enough?
![Page 36: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/36.jpg)
Testing and ImplementationData Collection and
Measurement
Key Question: Are small samples being collected by the team as the project is ramped up and placed
on a run chart?
![Page 37: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/37.jpg)
Measurement
• Small samples over time should be use to determine if the process is improving
• Data should be collected by the team with strict attention to the agreed upon tempo
• Data can initially be collected for segments
• Process measurements should be the primary team measures
• Outcome measures are needed but do not need to be collected by the team
• Outcome aims can be set at 0 or 100%, but your process aims should be 95% (10-2)
![Page 38: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/38.jpg)
5 charts/day run chart
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
day 1 day 2 day 3 day 4 day 5 day 6
![Page 39: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/39.jpg)
5 charts/day run chart
![Page 40: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/40.jpg)
© 2000 Institute for Healthcare Improvement
Testing Implementation and SpreadHitting the Wall
Key Question: What should we do when the team just seems not to be making
headway?
![Page 41: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/41.jpg)
© 2000 Institute for Healthcare Improvement
1-Use the 7 Question Analysis for testing and implementation 2-Check the rules of engagement
3-Use the spread analysis
![Page 42: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/42.jpg)
What Teams Should Expect From Leadership
• Clearly describe the organizational outcome goals (VAP, CLI, Mortality etc)
• Understand the relationship between the processes the teams are working on and the outcome goals of the organization
• Set process expectations for the teams all elements of the ventilator bundle will be done 95% of the time on eligible patients)
• Demand data to show how reliable the process has become
• Setting reasonable timelines
• If outcomes have not improved and process reliability is high provide resources to determine the “correctness of performance” of the processes
![Page 43: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/43.jpg)
What Leaders Should Expect of Teams to Reliably Achieve the Organizations Safety Goals
• Expect the Initial focus of work should be on “getting the process right” with a known connection to an outcome
• Expect the team to take a set of processes to an agreed upon level of reliability within a specified timeline
• Expect the teams to use good design principles in improvement work not just hard work and vigilance
• Expect teams to develop good designs by using rapid cycle small tests of change
• Expect adequate process structure to sustain the work
![Page 44: Reliability Theory Concepts: (How to be a Coach) Delivering Uniformly Excellent and Highly Reliable Interventions Across a Healthcare System Roger Resar](https://reader030.vdocuments.site/reader030/viewer/2022032806/56649efd5503460f94c10d60/html5/thumbnails/44.jpg)
Key Questions To Analyze Spread
Key Question Your EvaluationHave you repeated the small test cycles as you spread from the initial site
Is the process of spread dependent on one person
Has some degree of segmentation been used to spread
Has customization been allowed or encouraged
Is the same team who developed the pilot now responsible for spread
Have you shifted your focus from process reliability to outcomes too early