evidence-based practice: if doctors can do it, managers can do it?
DESCRIPTION
Evidence-Based Practice: If Doctors Can Do It, Managers Can Do It? Eric Barends AUPHA 2013 Annual Meeting Monterey, CATRANSCRIPT
Evidence-Based Practice
AUPHA Annual Meeting, June 20, 2013, Monterey
If doctors can do it managers can do it?
Evidence?
outcome of scientific research,
organizational facts & data,
benchmarking, best practices,
collective experience, personal
experience, intuition
All managers base their
decisions on ‘evidence’
However ...
Many managers pay little
or no attention to the
quality of the evidence they
base their decisions on
Trust me, 20 years of management experience
Teach managers how to
critically evaluate the validity,
and generalizability of the
evidence and help them find
‘the best available’ evidence
Evidence-based decision
What is the added value of evidence- based
practice for managers within the field of
hospital care?
Proof of concept
Teaching Hospital
- 6 managers
University Hospital
- 4 managers
Evidence based pilot
Phase 1: Training managers in the principles of EBP
Phase 2: Examination of the current decision making
processes that managers are using
Phase 3: Evaluation of 4 completed projects from an
EB perspective (retrospective)
Phase 4: Making EB recommendations for 4 new
projects (prospective)
Phase 5: Evaluation
Evidence based pilot
Some preliminary results
Decision making process
Decision making process
Focus on procedures instead of evidence
Internal politics and power struggles
No critical appraisal of the evidence at hand
Relying on anecdotal evidence (workshops,
best practices, popular management books,
consultants)
One option (sometimes two)
Bias: Outcome, Halo, Confirmation, etc.
Post mortem analysis
Evidence-based perspective
NOT: Did we made the right decision?
BUT: Is there evidence from scientific research to
support (or call into question) the approach
taken?
Best available evidence?
Post mortem: leadership training
leadership training: dm process
No problem definition
No organizational evidence consulted
Selection of training companies based on
experience, recommendation or reputation
No explicit selection criteria / procedure
‘Best’ presentation has won: one size fits all
leadership training: scientific evidence
15 meta analyses, 5 relevant
37 (‘systematic’) reviews, 2 relevant
Lots of relevant primary studies
leadership training: scientific evidence
Long history (30 yrs): moderate effect sizes
Senior & middle managers tend to benefit more
than managers at the supervisory level
Effect on ‘poor’ leaders is limited.
Leadership trainings that focus on
interpersonal / social skills show higher effect
sizes than those based on a specific leadership
‘model’
Reactions
Who knew?
Denial
Anger
Bargaining
Acceptance
Prospective /
EB recommendations
Questions / projects
360 degree feedback
Financial incentives
Lean Six Sigma
Hand Hygiene
Goal setting
Value Based Health Care
Downsizing
Evidence-based perspective
NOT: What works?
BUT: What are, given the target group, the problem and
the context involved, the main factors determining the
success or failure of the project that need to be taken into
account?Best available scientific
evidence?
Prospective: Multi Source Feedback
Multi Source Feedback: background
IFMS: based on multi source feedback
Regulating bodies and insurance companies
(KPI’s – prices/ revenue)
Based on CANMEDS, no standard method
New market: consulting firms
Process
Scoping session: inventory of the aspects
relevant to the question
Session with leading academic
Search in relevant databases
Critical Appraisal
Summary / research synthesis
Recommendations / guidelines
Multi Source Feedback: scientific evidence
223 primary studies on MSF, 42 relevant
6 meta analyses or systematic reviews on
MSF, 3 relevant
18 meta analyses or systematic reviews on
‘feedback’ or ‘performance appraisal’,
5 relevant
Content of the feedback (neg vs pos)
Way of the delivery of the feedback
Interpretation of the feedbackPersonality of the
rateeFeedback orientation
of the ratee
Type and number of ratersSelection of
raters
Rater reliablity (patients, nurses,
colleagues)Type of
response scale
Development vs perfomance appraisal
Organizational culture
Perceived procedural justice
Multi Source Feedback: main factors
Reactions
Who knew?
Wow!
Great!
Good stuff!
Relevant!
Lessons learned I
New approach
Recalibrates the power dynamics (accountability!)
The profit is in the process
Different (better?) decisions were made
Doctors love it!
Lessons learned II
Hard for individual managers
It starts with the senior management team
It’s all about accountability
Support system
EBP > Planning & Control
One day, maybe …
Chief Evidence Officer