interpreting safety culture survey results and action planning june 17, 2011 katherine jones, pt,...
TRANSCRIPT
Interpreting Safety Culture Survey Results and Action Planning
June 17, 2011
Katherine Jones, PT, PhD
Anne Skinner, RHIA
HSOPS
Acronyms
2
AHRQ = Agency for Healthcare Research and Quality
HRO = High Reliability Organization
HSOPS = Hospital Survey on Patient Safety Culture
HSOPSObjectives1. Define “culture of patient safety” (safety culture)
2. Identify four components of safety culture
3. Use tools and reports from survey results to:
1. Identify change over time associated with patient safety interventions and benchmark results to the national database
2. Identify variation in safety culture by work area and job title in HSOPS results
3. Compare beliefs and behaviors within HSOPS dimensions to identify practices needed to support safety culture
4. Describe key practices that support safety culture
5. Recognize potential for response shift bias among when evaluating impact of patient safety interventions
6. Recognize role of leadership in engineering culture change
7. Develop an action plan to engineer key practices that support safety culture
HSOPS
“The biggest challenge to moving toward a safer health system is changing the culture from one of blaming individuals for errors to one in which errors are treated not as personal failures, but as opportunities to improve the system and prevent harm.” IOM (2001). Crossing the Quality Chasm: A New Health System for the 21st Century, p. 79
The Problem and Challenge…
“The problem is not bad people; the problem is that the system needs to be made safer . . .”
IOM (2000). To Err is Human: Building a Safer Health System
5
Healthcare SystemStructures & Processes
OrganizationalStructures & Processes
Individual ProviderStructures & Processes
Quality at Point of CareInterpersonal
CareTechnical
Care
Chain of Impact at the Point of Care
The quality, safety and value of care can be no better than the structures and processes used by providers in direct contact with the patient. Culture is a lens through which organizations support providers at the point of care.
Nelson et al. (2002) Joint Commission Journal on Quality Improvement, 28, 472-493.
Swuste P. (2008). Human Factors and Ergonomics in Manufacturing, 18, 438-453.
Beliefs -- Culture – Behaviors
HSOPS
What did you measure with HSOPS? Enduring, shared, LEARNED* beliefs and behaviors
that reflect an organization’s willingness to learn from errors**
Four beliefs present in a safe, informed culture*** Our processes are designed to prevent failure
We are committed to detect and learn from error
We have a just culture that disciplines based on risk
People who work in teams make fewer errors
**Wiegmann. A synthesis of safety culture and safety climate research; 2002. http://www.humanfactors.uiuc.edu/Reports&PapersPDFs/TechReport/02-03.pdf
***Institute of Medicine. Patient safety: Achieving a new standard of care. Washington, DC: The National Academies Press; 2004.
*Schein, E. Organizational Culture and Leadership. 4th ed. San Francisco, CA: John Wiley & Sons; 2010.
HSOPS
Beliefs Assessed with HSOPS Our processes are designed to prevent failure
“Our procedures and systems are good at preventing errors from happening.”—national db 62% - 82%*
We are committed to detect and learn from error “When a mistake is made, but is caught and corrected before affecting the patient, how often
is this reported?”— national db 44% - 67%* “Mistakes have led to positive changes here.”— national db 54% - 74%*
We have a just culture—discipline is based upon risk taking “Staff worry that mistakes they make are kept in their personnel file.”R— national db 25% -
47%*
People who work in teams make fewer errors “People support one another in this department.” – national db79% - 92% “When one area in this department gets really busy, others help out.”— national db 59% -
78%*
*10th%ile and 90th%ile for 1032 hospitals reporting to AHRQ 2011 national comparative database
HSOPS
Three Levels of Culture
Schein, E.H. Organizational Leadership and Culture 4th ed. San Francisco:John Wiley & Sons; 2010, p.24, 27. 8
“…in many organizations, values reflect desired behavior but are not reflected in observed behavior.”
“…in many organizations, values reflect desired behavior but are not reflected in observed behavior.”
HSOPS
Identify areas of culture in need of improvement
Increase awareness of patient safety concepts
Evaluate effectiveness of patient safety interventions over time
Conduct internal and external benchmarking,
Meet regulatory requirements
Identify discrepancies between beliefs and observed behaviors within subcultures and microcultures
Goals of Culture Assessment…why did you measure safety culture?
9
Nieva, Sorra. (2003). Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care, 12(Suppl II), ii17-ii23.
HSOPS
Regulatory Requirement
Conduct HSOPS to meet Joint Commission Leadership Standards (Standard LD.03.01.01)
http://www.jointcommission.org/NR/rdonlyres/D53206E8-D42B-416B-B887-491B6D5AA163/0/HAP_LD.pdf
Leaders regularly evaluate the culture of safety and quality using valid and reliable tools
Leaders prioritize and implement changes identified by the evaluation
HSOPS
Four Components of Safety Culture
INDIVIDUALS FEEL VALUED
INDIVIDUALS ARE TREATED WITH RESPECT
INDIVIDUALS FEEL VALUED
INDIVIDUALS ARE TREATED WITH RESPECT
11
A culture of safety is informed. It never forgets to be afraid…
Reason, J. (1997). Managing the Risks of Organizational Accidents. Hampshire, England: Ashgate Publishing Limited.
Battles et al. (2006). Sensemaking of patient safety risks and hazards. HSR, 41(4 Pt 2), 1555-1575.
SENSEMAKING
TRUST
= Flexible
HSOPS
How to Become an HRO: Engage in Continuous Improvement
HSOPS
Measure Beliefs and Behaviors with HSOPS Developed by AHRQ to provide healthcare organizations with a valid tool
to assess safety culture http://www.ahrq.gov/qual/hospculture/
42 items categorized in 12 dimensions
2 dimensions are outcome measures at dept/unit level
7 dimensions measure culture at dept/unit level
3 dimensions measure culture at hospital level
2 additional items are outcome measures at dept/unit level
Number of Events Reported
Patient Safety Grade
HSOPSHSOPS
Original AHRQ Survey available http://www.ahrq.gov/qual/patientsafetyculture/
AHRQ Comparative Database for HSOPS 2011 Comparative Database for Benchmarking
1032 hospitals; 472,397 respondents
Stratis will submit your results to database
Report comparing your hospital to national data
Trending hospitals asked to describe interventions
14
HSOPSUNMC Rural HSOPS
Available at www.unmc.edu/rural/patient-safety
Developed by UNMC as part of AHRQ Partnerships in Implementing Patient Safety Grant 05 -07
Collapses work areas and position to reflect CAH environment
Allows sorting by Work Area/Position if > 5 respondents
Creates valid benchmark data for CAHs
Allows valid tracking of safety culture over time within a CAH to evaluate patient safety interventions
10 additional items added by UNMC to evaluate TeamSTEPPS
15
16
Original AHRQ HSOPS
Rural-Adapted AHRQ HSOPS
32%
3.3%
Original AHRQ HSOPS
Rural-Adapted AHRQ HSOPS
21%
3.2%
Reason’s Components HSOPS Dimensions or Outcome Measures
Reporting Culture - a safe organization is dependent on the willingness of front-line workers to report their errors and near-misses
•Frequency of Events Reported (O)
•Number of Events Reported (O)
Just Culture - management will support and reward reporting; discipline occurs based on risk-taking
•Nonpunitive Response to Error (U)
O = Outcome measureU = Measured at level of unit/departmentH = Measured at level of hospital
Reason’s Components HSOPS Dimensions or Outcome Measures
Flexible Culture - authority patterns relax when safety information is exchanged because those with authority respect the knowledge of front-line workers
•Teamwork w/in Units (U)
•Staffing (U)
•Communication Openness (U)
•Teamwork ax Units (H)
•Hospital Handoffs (H)
Learning Culture - organization will analyze reported information and then implement appropriate change
•Hospital Mgt Support (H)
•Manager Actions (U)
•Feedback & Communication (U)
•Organizational Learning (U)
•Overall Perceptions (O)
•Patient Safety Grade (O)
HSOPSYour ResultsResource PurposeReports from Excel Tool
ANALYSIS - Contains raw data Generates spreadsheet to upload for national database Instructions for interpretationDemographics of respondentsContains dimension and item level results in the aggregate, by department, position, direct patient care, action planning sheet
Benchmark Tool COMMUNICATIONCompare aggregate results to peer group (external benchmark)Compare aggregate results over time Compare results by work area and job title to the aggregate
Item Level Over Time
COMPARISONS AND COMMUNICATIONCompare item level results over time and to peer groupIncludes responses to teamwork questions
Comments Coded by Theme
CONTEXTOpen ended comments coded by culture-related themes Provides respondents’ direct feedback
Action Plan PLAN - Work sheet to anchor action plan in history, mission and strategic goals; identify practices needed to support safe culture
HSOPS
Principle-drive NOT event-driven
Planned approach NOT piecemeal
Proactive NOT reactive
Understand latent conditions Anticipate the next error
Focus on performance/behavior
Action Planning: What is needed
Reason, J. (1997). Managing the Risks of Organizational Accidents. Hampshire, England: Ashgate Publishing Limited.
Schein, E.H. Organizational Leadership and Culture 4th ed. San Francisco:John Wiley & Sons; 2010.
HSOPS
Interpreting Results to Develop an Action Plan Anchor plan in history, mission, strategic goals
Understand response rate (> 60% best)…are results generalizable?
Identify organization-wide areas In need of improvement Improved due to specific interventions
Wrap your mind around reverse worded items
Identify gaps between beliefs and behaviors within 4 components
HSOPS
Interpreting Results to Develop an Action Plan Identify variation in microcultures by work area/job
title Relate open-ended comments to results
Recognize potential for response shift bias in repeat reassessments
Consider how management uses information
Explicit plan to strengthen 4 components within depts by implementing specific practices that close the gap between beliefs and behaviors
Communicate results and plan
24
25
HSOPS
ORGANIZATION WIDE AREAS IN NEED OF IMPROVEMENT
Lowest Scores
Handoffs and Transitions (35%)
Teamwork Ax Depts (40%)
Nonpunitive Response (45%)
Significant Changes
Feedback & Communication about Error (+13%)
Teamwork W/in Units (+10%)
Overall Perceptions (+7%)
Teamwork Ax Units (-5%
Handoffs & Transitions (-5%)
HSOPS
REVERSE WORDED ITEMSGAPS BETWEEN BELIEFS & BEHAVIORS
Percent Positive 2011 HSOPS Database
(n=1032 Hospitals)
http://www.ahrq.gov/qual/hospsurvey11/hospsurv111.pdf
Gaps Between Beliefs & BehaviorsPercent Positive 2011 HSOPS Database (n=1032 Hospitals)Reporting Culture
http://www.ahrq.gov/qual/hospsurvey11/hospsurv111.pdf
Just Culture
Teamwork Culture
Teamwork Culture
http://www.ahrq.gov/qual/hospsurvey11/hospsurv111.pdf
Learning Culture
Why are microcultures different?• Quantity, relevance, timeliness of information available
differs due to leadership• Methods of information sharing differ
– Personal– Through standard channels– Teams do whatever it takes to get the right information to the
right people at the right time
• These methods reveal what is important to leaders– My personal power and glory (pathologic)– Maintenance of positions, rules, turf (bureaucratic)– Mission of organization (generative)
Westrum, R. A typology of organizational cultures. Quality and Safety in Healthcare 2004;13:22-27.
33
Action Planning: A Reporting culture is engineered by implementing practices
Practices/Tools
Reporting Form
Near miss log
Chart audit
Secret Shopper
Safety Briefings
Leadership WalkRoundsTM
Bulletin board/ suggestion box/telephone hotline
Successful reporting systems (Leape, 2002)
Nonpunitive
Confidential
Independent
Expert analysis
Timely
Systems-oriented
Responsive
HSOPS
Reporting Action Plan & AimsWe need to strengthen our REPORTING CULTURE because:
Just 1/3 of all respondents agreed that “near misses” are frequently reported.
Comment: “There is a strong belief by some staff that errors are recorded and held against staff….”
We can do this by:
(1) educating all employees about the role of near miss reporting—learning about risks and hazards in systems without harming patients; ( 2) implementing use of a “near miss” reporting log in all departments; (3) including discussion of near misses at departmental briefs (including sift change), huddles, and debriefs; and (4) including discussions of near misses in regular Leadership WalkRounds.
HSOPS
Action Planning: A Just culture is engineered by implementing practices
Practices/Tools Understanding human error (Reason 2003, 2006)
Active errors (sharp end) Latent errors
Just Culture and behavior (Marx, 2001) Conduct: human error, negligence, reckless, intentional rule
violation Disciplinary decision-making: outcome-based, rule-based, risk-
based Unsafe Acts Algorithm Disruptive Behavior Policy/Standards
Known medicalcondition?
NO NO NO YES
NOYES
YES
YES
YESNO
YES
YES NO
Culpable Gray Area Blameless
NOYES
YESNO
Adapted from James Reason. (1997). Managing the Risks of Organizational Accidents.
Execute Just Culture . . . UNSAFE ACTS ALGORITHM
Were the actions as intended?
Evidence of illness or substance use?
Knowingly violated safe procedures?
Pass substitutiontest? (Could someone else have done the same thing)?
History of unsafe acts?
Were the consequencesas intended?
Were proceduresavailable, workable, intelligible, correct and routinely used?
Deficiencies in training, selection, or inexperienced?
Substance abusewithout mitigation
Sabotage, malevolent damage
Substance usewith mitigation
Possible recklessviolation
System inducedviolation
Possible negligentbehavior
System inducederror
Blameless error, corrective training, counseling indicated
Blameless error
NO
HSOPS
Just Culture Action Plan & AimsWe need to strengthen our JUST CULTURE because:
35% of all respondents DISagreed with the reverse-worded statement, “Staff worry that mistakes they make are kept in their personnel file.”
Comment: “There is a strong belief by some staff that errors are recorded and held against staff….”
We can do this by:
(1) educating all staff about the impact of human error on patient safety and the role of just and fair culture in patient safety program; (2) implementing the use of the Unsafe Acts Algorithm by all managers to transparently determine individual vs. system accountability in adverse events.
Action Planning A Flexible culture is engineered by implementing practices
Team Strategies & Tools to Enhance Performance & Patient Safety
http://teamstepps.ahrq.gov
“Could definitely use more TeamSTEPPS training. Some questions difficult to answer.”
“TeamSTEPPS has brought some very positive changes in the hospital…we do Huddle each morning before the hospital Huddle..”
“TeamSTEPPS training has changed the way I think about my job, and the communication processes in my department.”
“TeamSTEPPS and Service Excellence is working. Did create chaos for a short time.”
“I don't feel very comfortable with the TeamSTEPPS program. It’s a great program; we just haven't practiced using it enough to make us comfortable with all the strategies or tools.”
HSOPS
Flexible Action Plan & AimsWe improved/need to strengthen our FLEXIBLE (Teamwork-Oriented) CULTURE :
91% of acute/skilled respondents agree that they support one another; 71% help each other out when it gets busy
84% of all respondents agree they will speak up but only 53% will do so to those with more authority
23% of all respondents DISagreed with the reverse-worded statement, “Problems often occur in the exchange of information across hospital units.”
We can continue to improve by: Ensuring consistent use of briefs, huddles, debriefs and seeking/offering task assistance within departments; use of the Two Challenge Rule and CUS to make it psychologically safe for staff to speak up to those with more authority; and use of structured communication during hand-offs and transitions (SBAR, I PASS the BATON) across hospital departments.
Action Planning: Reporting, Just, and Flexible practices support Learning
Ultimately, the willingness of workers to report depends on their belief that the organization will analyze reported information and then implement appropriate change—organizational practices support a learning culture.
Practices/Tools Individual RCA
Aggregate RCA
FMEA
Safety Briefings
Leadership WalkRoundsTM
Close the loop with reporting…feedback
HSOPS
Learning Action Plan & AimsWe improved/need to strengthen our LEARNING CULTURE
61% of all respondents agree they are given feedback about changes put into place based upon event reports
70% agree that “Mistakes have led to positive changes here.”
We can continue to improve by :(1) including front line staff in retrospective (root cause analysis) and prospective (failure mode and effect analysis) organizational learning, (2) conducting Leadership WalkRounds focused on proactive discussion of risks and hazards, (3) use of briefs, huddles, and debriefs in all departments to integrate organizational learning into daily work.
HSOPSResponse Shift Bias
Definition: tendency for an individual to overestimate their knowledge, skills, and behaviors in a pretest because their understanding of a concept is limited prior to the program intervention.
We have patient safety problems in this department. (73% before TS “shift” to 67% after)
(R)Problems often occur in the exchange of information across hospital departments. (45% before TS “shift” to 36% after)
44
45
SENSEMAKING
TRUST
Conclusion: HSOPS Guides Implementation of an Infrastructure for Patient Safety
Interaction between effective practices results in sensemaking
Sensemaking requires data, which is interpreted within the context of the experiences of those in direct contact with patients*
Sensemaking can not occur without data, trust and teamwork
Leaders drive sensemaking *Battles et al. (2006). Sensemaking of patient safety risks and hazards. HSR, 41(4 Pt 2), 1555-1575.
HSOPS
Shift Towards a Culture of Safety
HSOPSLessons Learned
Behaviors support an informed safe culture Measure safety culture using appropriate survey and
effective data collection methods
Create an infrastructure that supports reporting
Adhere to principles of just culture
Implement team training to support a flexible culture
Learn from error in the context of daily work (Safety Briefings and Leadership WalkRounds)
Teams must systematically learn from events using individual RCA and aggregate RCA to learn from multiple non-harmful errors
47
HSOPS
Leaders manage culture or it manages them….Create a compelling positive vision
Define the change goal as solving a performance problem…not “changing culture”
Provide formal training in groups
Ensure new behaviors lead to success, satisfaction Provide opportunities for practice, coaching, feedback
Provide positive role models
Provide support groups for learning problems
Create structures consistent with new way of thinking/working/behaving
Schein, E.H. Organizational Leadership and Culture 4th ed. San Francisco: John Wiley & Sons; 2010.
Lessons Learned
HSOPS
Diffusion of Innovations…
“Getting a new idea adopted, even when it has obvious advantages, is difficult. Many innovations require a lengthy period of many years from the time when they become available to the time when they are widely adopted.” – Rogers in Diffusion of Innovations, p. 1
49
HSOPS
The Responsibility of Leadership
“Our systems are too complex to expect merely extraordinary people to perform perfectly 100% of the time. We as leaders have a responsibility to put in place systems to support safe practice.”
James Conway,
former VP and COO Dana Farber Cancer Institute
HSOPS
Contact Information
Katherine Jones, PT, PhD
Anne Skinner
Web site where tools are posted
www.unmc.edu/rural/patient-safety