one or two things about evidence-based management
DESCRIPTION
One or two things about Evidence-Based Management. Pitstop Evidence-Based HR, VOV lerend netwerk , Gent , 27 september 2012. Evidence-Based Management?. VOV leden , n= 86. Evidence-Based Practice. 1991 Medicine 1998 Education 1999 Social care, public policy 2000 Nursing - PowerPoint PPT PresentationTRANSCRIPT
Postgraduate Course
One or two things about Evidence-Based Management
Pitstop Evidence-Based HR, VOV lerend netwerk,Gent, 27 september 2012
Postgraduate Course
Evidence-Based Management?
VOV leden, n= 86
Postgraduate Course
Evidence-Based Practice
1991Medicine
1998Education
1999Social care, public policy
2000Nursing
2000Criminal justice
????Management?
Postgraduate Course
Definition
Evidence-based management means making decisions about the management of employees, teams or organizations through the conscientious, explicit and judicious use of four sources of information:
1. The best available scientific evidence
2. Organizational facts, metrics and characteristics
3. Stakeholders’ values and concerns
4. Practitioner expertise and judgment
Postgraduate Course
Four sources
Postgraduate Course
Four sources
Postgraduate Course
Trust me: 20 years of management experience!
Postgraduate Course
Bounded rationality
Postgraduate Course
Bounded rationality
Postgraduate Course
Seeing order in randomness Mental corner cutting Misinterpretation of incomplete data Halo effect False consensus effect Group think Self serving bias Sunk cost fallacy Cognitive dissonance reduction
Confirmation bias Authority bias Small numbers fallacy In-group bias Recall bias Anchoring bias Inaccurate covariation detection Distortions due to plausibility
Het feilbare brein
Postgraduate Course
if you’re hyperventilating breathe into a bag
Postgraduate Course
elderly people who have an irregular heartbeat are much more likely to die of
coronary disease
give them a drug that reduces the number of
irregular beats
Postgraduate Course
How 40,000 cardiologists can be wrong
In the early 1980s newly introduced anti-arrhythmic drugs were found to be highly successful at suppressing arrhythmias.
Not until a RCT was performed was it realized that, although these drugs suppressed arrhythmias, they actually increased mortality.
The CAST trial revealed Excess mortality of 56/1000.
By the time the results of this trial were published, at least 100,000 such patients had been taking these drugs.
Postgraduate Course
Errors and Biases of Human Judgment
Doctors and managers hold many erroneous beliefs,
not because they are ignorant or stupid, but because
they seem to be the most sensible conclusion
consistent with their own professional experience!
available evidence.
Postgraduate Course
Half of what you learn will be shown to be either dead wrong or out-of-date within 7 years of your graduation; the trouble is that nobody can tell you which half
Sackett: remember that your teachers are full of crap, just like your parents.
Problem II: false information
Postgraduate Course
1. Incompetent people benefit more from feedback than highly competent people.
2. Task conflict improves work group performance while relational conflict harms it.
3. Being intelligent is a disadvantage for performing low skilled jobs.
Evidence-based?
Postgraduate Course
Evidence-based?
Competentie management Excellente zorg Kwaliteits management Het nieuwe werken Kennis management Magnet, Investors in People Plain Tree, Healing Environment Balanced Score Card / INK Lean / Six Sigma / TOC
Postgraduate Course
Think critically about experience, question your assumptions, and challenge what you think you know.
Don’t be a parrot!
(Show me the evidence!)
Postgraduate Course
Postgraduate Course
1. Formulate a focused question
2. Searching for the best available evidence
3. Critical appraisal
4. Turning evidence into practice
5. Monitor the outcome
5-step approach
Postgraduate Course
Step 3: Critical appraisal of studies
Postgraduate Course
Best available evidence?
Postgraduate Course
Research designs
What is the BEST car?
Postgraduate Course
Which design for which question?
Research designs
Postgraduate Course
Twee type vragen
Effect vs Non-effect
Postgraduate Course
Type vraag: effect
Werkt het?
Werkt het beter dan ....?
Heeft het een effect op ....?
Wat zijn de succesfactoren voor ....?
Wat is nodig om het te laten werken?
Effect
Postgraduate Course
Type: non-effect
Needs: Wat hebben mensen nodig, waar hebben ze
behoefte aan?
Attitude: Wat denken of vinden mensen van ...?
Experience: Wat zijn de ervaringen van mensen met ...?
Prevalence: Hoeveel mensen / organisaties ....?
Procedure: Hoe kunnen we .... implementeren?
Explanation: Waarom werkt het?
Economics: Hoeveel kost het (tijd en geld)?
Postgraduate Course
Explanation
Which design for which question?
Postgraduate Course
Best research design?
Postgraduate Course
Best available?
Postgraduate Course
Evidence is not the same as ‘proof’ or ‘hard facts’
Evidence can be
- so strong that no one doubts its correctness, or
- so weak that it is hardly convincing at all
What is evidence?
Postgraduate Course
The best available evidence =
Studies with the highest internal validity
Studies with the highest external validity
Postgraduate Course
internal validity = indicates to what extent the
results of the research may be biased and is thus a comment on the degree to which alternative explanations for the outcome found
are possible (confounding).
Internal validity
Postgraduate Course
Confounding is the idea that a 3rd variable can distort or confuse (or confound..) a relationship between two other variables. For instance, when factor X causes disease Y, that relationship could be confounded by factor C that is associated with both factor X and disease Y. C would be an alternative explanation for the relationship observed between X and Y.
Confounding
Postgraduate Course
What are the confounders?
1. Shoe size & quality of handwriting
2. Body length & body weight
3. Number of storks & birth rate
4. Smoking youngsters & better lung function
Postgraduate Course
?Successful companies
Charismatic leaders
Reverse causation
Postgraduate Course
Levels of internal validity
1. Were there enough subjects in the study?
2. Was a control group used?
3. Were the subjects randomly assigned?
4. Was a pretest used?
5. Was the study started prior to the intervention or event?
6. Was the outcome measured in an objective and reliable way?
6x yes = very high (A)
5x yes = high (A)
4-3x yes = limited (B)
2x yes = low (C)
1-0x yes = very low (D)
Postgraduate Course
Levels of internal validity
Postgraduate Course
Levels of internal validity
It is shown that …
It is likely that …
Experts are of the opinion that …
There are signs that …
Postgraduate Course
Best available evidence: external validity
Postgraduate Course
Ecological validity: Is your organization so different from those in the study that its results may be difficult to apply?
Population validity: Is your population so different from those in the study that its results may be difficult to apply?
External validity: generalizability
Always ask yourself to what extent the evidence is generalizable to your situation:
Postgraduate Course
Generalizability
Keep in mind:
What works in one narrowly defined setting
might not work in another,
but some psychological principles
are generalizable to all human beings.
Postgraduate Course
Step 4: Turning evidence into practice
Postgraduate Course
Postgraduate Course
Feasible?
organizational facts and characteristics cultural aspects stakeholders’ values and concerns political aspects financial aspects /cost-effectiveness / ROI priorities change readiness / resistance to change implementation capacity timing
Postgraduate Course
Tot slot:
Focus on the decision making process! (not the outcome)