federation of state physician health program 2012 annual meeting fsphp april 23-26, 2012williams bw,...

1

Federation of State Physician Health Program

2012 Annual Meeting

FSPHP April 23-26, 2012Williams BW, Swiggart WH, Ghulyan MA, Vickers, K, Williams MV

Williams BW, Swiggart WH, Ghulyan MA, Vickers, K, Williams MV 2FSPHP April 23-26, 2012

THE DISRUPTIVE PHYSICIAN: THE DEVELOPMENT AND USE OF A 360 INSTRUMENT AS A MONITORING TOOL

Betsy White Williams PhD MPHAssistant Professor Director of Outcomes and ResearchOffice of Continuing Medical EducationRush University Medical CenterClinical Program DirectorProfessional Renewal CenterLawrence, KS

William H. Swiggart, MS, LPC/MHSPAssistant in MedicineVanderbilt Department of MedicineCo-DirectorCenter for Professional HealthVanderbilt University School of MedicineNashville, TN

Marine V. Ghulyan, MA

Research Analyst

The Center for Professional Health

Vanderbilt University School of Medicine

Nashville, TN

Kayci Vickers

Research Assistant

Professional Renewal Center

Lawrence, KS

Michael V. Williams, Ph.D.

Wales Behavioral Assessment

Principal

Wales Behavioral Assessment

Lawrence , KS

Williams BW, Swiggart WH, Ghulyan MA, Vickers, K, Williams MV 3

Learning Objectives

Understand the results of 360 degree assessment in monitoring of intervention effectiveness.

Understanding the likely changes in the results from 360 degree evaluations over time.

Understanding the interpretation of outliers in utilizing a 360 degree evaluation to determine intervention efficacy

FSPHP April 23-26, 2012


Presentation - Context

Physicians identified as disruptive practitioners are increasing being referred to Physician Health Programs. Heretofore it has been difficult to monitor the progress of these physicians post intervention.

This presentation discusses the development of a 360 survey instrument. While we were interested in evaluating and comparing

the data between cases and a comparison sample, we were particularly interested in the use of the 360 for

both identification and monitoring. The 360° survey was recently developed based on

input from experts and a review of the literature.



3 Core Competency Survey (3CC)

It is not enough to have good motives; others respond to our behavior.

Physicians are often not given essential feedback about their behavior.

The Three Core Competency Survey (3CC) is designed to provide feedback from those we work with.



Disruptive behavior

“Intimidating and disruptive behaviors can foster medical errors, contribute to poor patient satisfaction and to preventable adverse outcomes, increase the cost of care, and cause qualified clinicians, administrators and managers to seek new positions in more professional environments.”

Issue 40: Behaviors that undermine a culture of safety | Joint Commissionhttp://www.jointcommission.org/assets/1/18/SEA_40.PDF



Aggressive

Anger Outbursts

Profane/Disrespectful Language

Throwing Objects

Demeaning Behavior

Physical Aggression

Sexual Comments or Harassment

Racial/Ethnic Jokes

PassiveAggressive

Derogatory comments about institution, hospital, group, etc.

Refusing to do tasks

Passive

Chronically late

Not responding to call

Inappropriate or inadequate chart notes

Spectrum of Disruptive Behaviors



Disruptive behavior

“The term “disruptive behavior” is changed in the standardsThe term “disruptive behavior” in two elements

of performance (LD.03.01.01, EPs 4 and 5) has been revised to “behavior or behaviors that undermine a culture of safety.” ”

Joint Commission online November 11, 2011 http://www.jointcommission.org/assets/1/18/

jconline_Nov_9_11.pdf



Disruptive Behavior Leads to Communication Problems…Communication Problems Lead To Adverse Events1

Communication breakdown factored in OR errors

50% of the time2

Communication mishaps were associated with

30% of adverse events in OBGYN3

Communication failures contributed to 91% of

adverse events involving residents4

Gerald B. Hickson, MDJames W. Pichert, PhD

Center for Patient & Professional AdvocacyVanderbilt University School of Medicine

1. Dayton et al, J Qual & Patient Saf 2007; 33:34-44. 3. White et al, Obstet Gynecol 2005; 105(5 Pt1):1031-1038.

2. Gewande et al, Surgery 2003; 133: 614-621. 4. Lingard et al, Qual Saf Health Care 2004; 13: 330-334


Failure to Address Disruptive Conduct Leads To

Team members may adopt disruptive person’s negative mood/anger (Dimberg & Ohman, 1996)

Lessened trust among team members can lead to lessened task performance (always monitoring disruptive person)... effects quality and patient safety (Lewicki & Bunker, 1995; Wageman, 2000)

Felps, W et al. 2006. How, when, and why bad apples spoil the barrel: negative group members and dysfunctional groups. Research and Organizational Behavior, Volume 27, 175-222.


Disruptive Behavior Creates fear confusion or uncertainty vengeance vs. those who

oppose/oppress them hurt ego/pride grief (denial, anger,

bargaining) apathy burnout unhealthy peer pressures

ignorance (expectations, behavior standards, rules, protocols, chain of command, standards of care)

distrust of leaders dropout: early retirement

or relocation errors disruptive behavior

begets disruptive behavior

Vanderbilt University and Medical Center Policy #HR-027


“The Perfect Storm”

PhysicianHospital/Clinic

The external system The internal system


Two systems interact

Good skills

Poor skills

Functional & nurturing

Dysfunctional

http://www.heartsurgery-tampa.com/images/surgeon.jpg


Etiologies

Institutional Factors Scapegoats System Reinforces Behavior Individual Pathology may over-shadow institutional

pathology


Etiologies

Personal Factors Individual pathology Life Stressors Lack of knowledge and skills


2004 AAMC Council of Deans

“Physicians are often poorly socialized and enter medical school with inadequate social skills for practice.”

“There is a growing body of literature documenting residency programs do notprepare resident physicians adequately for the practice of medicine.”


16FSPHP April 23-26, 2012

Potential Resources for Healthy Coping

Courses Coaches,

counselors Comprehensive

Evaluation 360° Evaluations Risk Managers Physician

Wellness Treatment

Centers

Office of General Counsel

State BME Professional Societies QI Officers EAP Others State Physician Health

Program


CME Remediation of “disruptive behavior”

Development of insight

Development of Skills

Development of implementation strategy

Feedback and monitoring



18FSPHP April 23-26, 2012

General Trends

At 3 months, significant improvements in 20 of the 22 physicians Increased motivating behaviors and motivating

impact Decreased disruptive behaviors and disruptive

impact Changes in behavior reported by “others”

Samenow CP, Swiggart W, Blackford J, Fishel T, Dodd D, Neufeld R, Spickard A. A CME Course Aimed at Addressing Disruptive Behavior. Physician Executive; 34 (1) Jan/Feb 2008: 32-40.


360 Survey to Provide Feedback and Monitor Behavior

BASED on CORE COMPETENCY AREASInterpersonal and Communication SkillsProfessionalismSystem based practice



Survey Development

Review of 300+ publications of the evidence focused articles on physician professional behavior;

Abstraction of assessment items with a expert based Delphi process to yield candidates for the final scale.



Survey Development

Interpersonal and Communication SkillsUses verbal communication to provide appropriate feedback to others.

Makes others feel comfortable approaching to ask questions or make suggestions.

Communicates effectively with patients.

© Williams, Swiggart, and WilliamsFSPHP April 23-26, 2012


Survey Development

Professionalism Willingly performs all tasks, functions, or responsibilities that are typically expected of him/her.

Responds promptly to telephone and pages.

Reports timely to hospital/clinical duties.

Responds quickly and appropriately to administrative communications.



Survey Development

System-based practice Creates a sense of teamwork and

valued contribution by team members.

His/her behavior makes others comfortable in their work.

Creates an accepting work environment.



Importance of Monitoring

Necessity of ensuring the behavior does not recur,

Anecdotal evidence of a significant level of recidivism,

Prior behavioral issues are a significant risk factor for later disruption (Papadakis and colleagues, see for example, Papadakis, Arnold et al. 2008)

Facilitates earlier identification

25FSPHP April 23-26, 2012Williams BW, Swiggart WH, Ghulyan MA, Vickers, K, Williams MV

Importance of Feedback

Form of guided self-assessment,Reinforces behavioral changes,Provides a standard by which to assess

gains.


Methods-Subject Group

Participants in the The Course for Distressed Physicians, a remedial CME course developed at the Center for Professional Health at Vanderbilt University. Cases: referred for workplace difficulties

that relate to team behavior Comparison: Physicians of similar

specialties as the cases .



Methods-Data Analysis

Specificity and sensitivity using the two classes of participants were analyzed.

The means and distributions were analyzed for consistency with other measures of performance more consistent with process measurement.

The outcome of an analysis of outliers is reported.



Reliability and Validity

Neither reliability or validity is a simple issue in the context of measures across time Consider the issue of the WAIS, while

valid and reliable, serial tests can be problematic due to lagged time effects;

In the case of “disruptive behavior”, serial measures are core to the value of the assessment instrument.



Measures of Validity

Types of validity: Face validity; Construct validity; Predictive validity

In general these are summarized by Messick: “ … the degree to which the empirical evidence

and theoretical rationales support the adequacy and appropriateness of interpretations and actions based on test scores.”

Expanded by Kane to four domains: Scoring, generalization, extrapolation, and

interpretation/decisions.



Today’s focus

While we touch briefly on our first two elements: Scoring, and, Generalization.

Most of the focus of this discussion is on: Extrapolation, and, Interpretation and decisions.

The interpretation and decisions element most clearly differentiate useful instruments from interesting academic exercises



Scoring

An extension of face validity Is the item appropriate to the construct

of interest; We selected a scale shown to be valid in

other 360 medical applications



Generalization

Convergence: Our scale(s) generate a Cronbach’s α in excess

of 0.9 the degree of exceeding depending on the item set.

Our scales, using underlying factor structures, demonstrate factor invariance across at least 3 sets of raters

The degree of coherence is clear, individual differences – the contribution to formal error – are being examined but two seem theoretically appropriate: time of remedial training and identification as disruptive.



Extrapolation

Extrapolation – Do the scores predict real-world outcomes of interest, a broad restatement of predicative validity. Four groups

distressed physician class participants at Vanderbilt

Distressed physician class participants at PRC

Comparison sample at Vanderbilt Comparison sample from PRC.

.FSPHP April 23-26, 2012


Extrapolation

No significant different was found between the two comparison samples.

The data were pooled in subsequent analyses



Extrapolation

The 360 demonstrates known group discrimination:

The method is valid as far as discriminating between professionals identified as demonstrating behavioral issues.



Extrapolation

Interprofessional Behavior

Note the reduction in mean difference across time is minimal;

However, the reduction in variance, and particularly skew is marked.

Extreme performance, particularly low performance is reduced over time.



Disruptive behavior – Social systems

Preliminary results suggest that disruptive physicians may not differ significantly from normal physicians in mean performance but may differ significantly in skew.

37


Extrapolation

Wave analysis for inter-professional behavior

Again the change is mostly in the reduction of variance across time, not in means.



Disruptive Behavior and Institutional Functioning

The presence of the system disruption ultimately results in breakdown:Communications;Affiliation;Roles; and,Protocols and

duties.

39

40FSPHP April 23-26, 2012Williams BW, Swiggart WH, Ghulyan MA, Vickers, K, Williams MV


Results suggest that disruptive physicians may not differ significantly from normal physicians in mean performance but may differ significantly in skew.



Over time as the disruptive behavior is extinguished the pattern of data will modify.

Results suggest the proportion of extreme reports falls and moderate to good reports increase.

Williams BW, Swiggart WH, Ghulyan MA, Vickers, K, Williams MV

Competencies and Team Function

As outlying performance decreases team cohesion increases.

FSPHP April 23-26, 2012 42


Discussion

The instrument discriminates between participants in the remediation exercise and the comparison sample.

The instrument shows appropriate sensitivity and specificity and appears to be valid.

Analysis of outliers and serial results means and distributions appear to be

consistent with expectation means and distributions change over time

appropriately. FSPHP April 23-26, 2012


Discussion

The tail of the distribution appears very sensitive to behavioral

change as reported by other observers.

The 3C 360° survey is a promising measure of CME efficacy in changing practice patterns.



Interpretation/Decisions

The data support the discrimination between: Physicians identified as being disruptive,

and, Physicians from comparison samples.

As well as: Physicians identified as being disruptive,

and, Physicians in a general remediation

program. These data suggest that general

interpretation is valid.FSPHP April 23-26, 2012


Interpretation/Decisions

The instrument is intended to address: Application within referral sources

(hospitals, clinics, academic medical centers); and,

A methodology that both tracks improvement and indicates adequate performance.

These elements are met through a core competency structure and sensitivity to changing outliers.FSPHP April 23-26, 2012


Discussion

The instrument provides a consistent measurement with the

literature and our experience of those areas of functioning related to interpersonal skill and communications, professionalism, and team behavior for healthcare professionals.

demonstrates appropriate sensitivity and specificity

provides the basis of effectively assessing intervention efficacy.

shows promise as a monitoring instrument and as a mean of identifying relapse behaviors.



Discussion

Sample Case Report



Discussion

Sample Comment page


federation of state physician health program 2012 annual meeting fsphp april 23-26, 2012williams bw,...

Documents