teaching evidence based medicine - conference … gp 15/1 presentations...are we evidence-based? •...
TRANSCRIPT
![Page 1: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/1.jpg)
Teaching Evidence Based Medicine
![Page 2: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/2.jpg)
What’s
Ahead?
• Practicing EBM in the
Trenches
– Staying up to date
– Answering questions
• Teaching EBM to
beginners
• Teaching EBM to
advanced learners
– Tough topics
• Making EBM practical
![Page 3: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/3.jpg)
But Evidence-Based Practice
Seems Pretty Far Away,…
![Page 4: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/4.jpg)
Barriers to EBM
• Time
• Evidence base poor
• Lack of Internet (or resources)
• Understanding EBM
• Teachers attitude
Med Educ. 2004;38:987-97. Acad Med. 2003;78:1183-90. Fam Med. 2001
Sep;33(8):602-6. Med Educ. 2004 Sep;38(9):987-97. BMJ 1998; 316: 361-5. J Gen
Intern Med 1999; 14: 236-42. The Cochrane Database of Systematic Reviews 2001,
Issue 3 Art. No.: CD 001270. Intern Med J. 2005;35(1):9-17.
![Page 5: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/5.jpg)
Are we Evidence-based?
• So do doctors understand EBM:
– Our residents tested poorly (50%) and use
Evidence based sources for 18% of their
questions.
– Multiple studies1 find doctors struggle with EBM
concepts (e.g. RR and AR), even when they think
they understand.
Ann Int Med 1992; 117: 916-21. Lancet 1994; 343: 1209-11. Am J Card. 2001; 87: 203-
7. Br J Gen Pract 1996; 46: 661-3. JAMA 2002; 287: 2813-4. BMJ 1998; 316: 361-
5. J Eval Clin Pract 2001; 7: 201-10.
![Page 6: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/6.jpg)
Staying Up-To-Date: Is it Possible?
• Reading all relevant journals to General
Practice?
• There are 341 journals, 90,000 articles/yr
• Even if narrowed down to the top 85, still
would review 16,000 articles/yr.
• If trained in epidemiology, still would take us
627.5 hours/month, or 21 hours a day
Alper et al. J Med Libr Assoc 2004;902(4):429-37.
![Page 7: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/7.jpg)
Answering Daily Questions: Is it Possible?
• We have ~1 “uncertainties” (Questions) for every 2
patient encounters1
• If we did a search (30-60 minutes2) and appraised a
paper (~30-60 minutes)
– 30 patients = 15 questions
– Assuming 1.5 hours per question = 22.5 hours/day
• In truth, Doctors1,3
– Spend 2-3 minutes getting answers to their questions
– Search pubmed (or embase) for <1% of their question
– Do critical appraisals < 0.1% of their questions
1. JAMA Intern Med. 2014;174(5):710-718. 2. J Fam Pract. 1996; 43:140-4. Bull Med
Libr Assoc 1994; 82: 140-146 3. BMJ 1999; 319: 358-61.
![Page 8: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/8.jpg)
So, Should We Quit Now?
• There are many barriers to EBM
• Most clinicians struggle understanding EBM
• Staying up-to-date is overwhelming
• Answer questions in practice by critical
appraisal is impossible.
• We NEED to Rethink EBM education and
Practice
![Page 9: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/9.jpg)
A New Approach to EBM
• Teaching:
– For >90%: Understand Basics and Terms.
• Needed for Practice.
– For <10%: Advanced understanding.
• Needed for people presenting or writing on Evidence
• Staying up-to-date requires
– Pre-Filtered & Pre-Appraised:
• Answering Clinical Questions requires
– Quick Search, Thorough and Unbiased
![Page 10: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/10.jpg)
How To Stay Up-To-Date
![Page 11: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/11.jpg)
Staying Up-to-Date: Options
• Studies – InfoPOEMS: Regular 1 pg summary up-dates of recent
articles. ~1/day (5 Minutes) US. Cost
– ACP Journal Club or Evidence Based Medicine: 30 articles
q2 months, 1 p summary (methods, findings & commentary)
(ACP = Internal Med, EBM = Family Med). US. Cost
– Goodfellow Gems: q 2 weeks, NZ. The briefest. FREE
• Questions – Tools for Practice: Focused Clinical Question with Bottom-
Line, Evidence, Context. ~300 words. Family Doctor
written. q 2 weeks FREE
![Page 12: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/12.jpg)
Intensive lipid lowering with statins unnecessary
with stable CAD (IDEAL) • Clinical question: Is intensive lowering of serum lipids with statin drugs
beneficial in patients with stable coronary artery disease?
• Bottom line: The intensive reduction of low-density lipoprotein (LDL) levels to well below 100 mg/dL (2.5 mmol/L) did not result in a significant reduction in the recurrence of major coronary events or all-cause mortality among patients with stable coronary artery disease. Intensive lowering is associated with an increased risk of discontinuing medication because of adverse events and significant drug costs. Aiming for an LDL of approximately 100 mg/dL (2.5 mmol/L) seems optimal for the majority of patients with stable disease. (LOE = 1b-)
• Reference: Pedersen TR, Faergeman O, Kastelein JJ, et al, for the Incremental Decrease in End Points Through Aggressive Lipid Lowering (IDEAL) Study Group. High-dose atorvastatin vs usual-dose simvastatin for secondary prevention after myocardial infarction. The IDEAL study: A randomized controlled trial. JAMA 2005; 294:2437-45.
• Study design: Randomized controlled trial (single-blinded)
• Funding: Industry
• Allocation: Uncertain
• Setting: Outpatient (specialty)
• Synopsis: Intensive,… (250 Words)
![Page 13: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/13.jpg)
ACP Sample
![Page 14: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/14.jpg)
Goodfellows
Gems
Sample
![Page 15: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/15.jpg)
![Page 16: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/16.jpg)
Staying Up-to-Date: Other Options
• Journal Watch: Summaries of Articles (less
structured) - Cost
• Get 1-2 Journals Table of Contents emailed to you,
– Example: NEJM. Abstracts for Free
– BMJ excellent choice (more Primary Care): Free
![Page 17: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/17.jpg)
P
r
e
v
i
o
u
s
N
e
x
t
P
r
e
v
i
o
u
s
N
e
x
t
Example:
NEJM
Abstract
![Page 18: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/18.jpg)
Where to Get Answers?
![Page 19: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/19.jpg)
Answering Clinical Questions:
• Resources like Tools for Practice (if answer there)
1.UpToDate (Cost (big)): Most complete, most
expensive. Very much like a medical textbook.
2.DynaMed (Cost): more Evidence-based, slightly
less cumbersome. More point form. Less
comprehensive.
![Page 20: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/20.jpg)
Answering Clinical Questions:
3. TRIP database (Free): Huge Search Engine, can
limit but often like PubMed (too broad). Answers
there, time inefficient.
4. Clinical Evidence (Cost): Quite Evidence-based,
but not very comprehensive (sometimes within
topics too) some old ones free via Pubmed.
5. BMJ Best Practice (Cost): Relatively evidence-
based but not as comprehensive & a bit
cumbersome with sub-categories
![Page 21: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/21.jpg)
UpToDate: Plus and Minus
• Positives:
– Full summary (pathophysio to f/u)
– Left Margin topics & Everything linked (pictures, drugs, etc)
– Links quick, good search
– Everything cited
• Negatives:
– Cost !
– Some Bias (single authors in field)
– Not Completely “Up-To-Date” (but very good)
– Can be cumbersome
![Page 22: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/22.jpg)
Teaching EBM:
The Basics?
For Learners like Medical
Students and Residents
![Page 23: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/23.jpg)
Primary Goals
• To help residents become competent self-
directed life-long learners with the skills to
effectively and efficiently keep up-to-date.
• To learn how to use Evidence Based Medicine
(EBM) to rapidly (as possible) solve problems
encountered in daily practice.
![Page 24: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/24.jpg)
Secondary Goals
• Review EBM components within the family medicine residency
curriculum, maximizing educational opportunities,
– Eliminating or modifying components with poor evaluation or performance.
– Enhancing other areas to promote EBM.
– Adding education time only to achieve the core features of life-long learning
skills.
• Promote the development of the skills in Residents needed to
critique educational opportunities.
• Promote and encourage residents seeking additional skills, training
or experience in EBM, knowledge transfer and/or research.
![Page 25: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/25.jpg)
Learning Objectives:
Knowledge Component
• To understand the rationale and benefits of EBM
• Provide a strong foundation in the basic principles of
EBM including
– Recognizing and formulating clinical questions
– Finding and Accessing information
– Interpreting information
– Applying information
• Provide and identify online resources, educational tools
and web-links
![Page 26: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/26.jpg)
Learning Objectives:
Skill Components
• Learn to identify problems/questions encountered in
practice and seek solutions based upon the best evidence.
(Via EBM “assignment” to promote rapidly answering
clinical questions).
• Build skills and comfort in knowledge transfer (primarily
through oral presentation skills).
![Page 27: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/27.jpg)
Learning Objectives:
Attitude Components
• To facilitate appreciation and enthusiasm for
– the judicious use of current evidence to optimize
patient care
– maintenance of skills and knowledge through life long
self-directed learning
![Page 28: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/28.jpg)
Some Background Research • Critical Appraisal based curriculum are often less
successful.1
• Workshops (3 day) have been shown to improve knowledge.2
• Classroom EBM teaching alone only improve knowledge while Integration improves skills, attitudes and behavior.3
• Web-based Clinical Questions & Answers helpful getting resident participation in EBM.4
1. Acad Med. 1999;74(6):686-94. & CMAJ 1998;158(2):177-81. 2. BMJ. 2002
7;325(7376):1338-41. & Acad Med. 2000;75(12):1212-4. 3. Acad Med. 2002;77(7):
741-2. & BMJ. 2004;329(7473):1017. 4. Acad Med. 2003 Mar;78(3):270-4.
![Page 29: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/29.jpg)
EBM Curriculum Components
• Family Medicine Evidence Based Medicine Workshop
– 2 day mixed lecture & small group work + computer lab
• Informatics/Evidence based resources
– UpToDate, DynaMed, Clinical Evidence, others
• Brief Evidence-based Assessment of Research (BEARs)
– 1 page assignment, ≥4 during Family block (1st yr)
• Resident Journal Club
– Selected by EBM faculty, rapid review summary articles in teams, shared presentations, focus on clinical application.
• Other components
– q 2 monthly Lit review lecture
![Page 30: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/30.jpg)
Our Curriculum Research 1
• IMG residents (vs Canadian grads) reported little
or poor quality EBM training & indicated
computer skills were a barrier1
• After introducing the curriculum, stat. sign.
improvement of resident comfort, self-reported
skills & attitudes towards EBM.2
– E.g. EBM curriculum improved my comfort with
evidence-based practice (1-5 scale: 2.76 before; 3.86
after, p<0.001)
1. Allan GM, Manca DP, Szafran O, Korownyk C. EBM a Challenge for International Medical
Graduates. Fam Med 2007; 39(3): 612. 2. Allan GM, Korownyk C, Tan A, Hindle H, Kung L,
Manca D. Developing an Integrated Evidence-Based Medicine Curriculum for Family Medicine
Residency at the University of Alberta. Acad Med 2008; 83: 581-587.
![Page 31: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/31.jpg)
Our Curriculum Research 2
• How do residents address questions in clinics
– 38 residents: 325 questions x114 clinical ½ days (420 pts)
– 0.8 questions/pts, 83% answered with one resource (1-6)
– Attempts: colleagues 65.5% (94% were preceptor), electronic resources 20.7% and paper resources 13.8%.
– Colleagues least likely need second resource (ss).
– Utility of answers: Colleagues (ss) superior to electronic
– EBM workshop training did not influence electronic resource use but semi-independence did: 17% to 51% (ss).
• Studied practicing graduates and presently analyzing
Can Fam Physician. 2012 Jun;58(6):e344-51.
![Page 32: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/32.jpg)
Curriculum Research: yet unpublished
• Our EBM Workshop (unpublished)
– Pre-workshop survey / post-workshop eval
• No diff year to year, Strongly positive feedback
– Pre-Post Workshop Quiz
• 15 questions, no difference year to year
• Mean score: pre 50% and post 75%
• BEARs Study1:
– Reviewed 317 BEARs first two years.
– Search: Pubmed (38%); Summary (22%); Prefiltered (19%)
– 48% from review, CPG, summary & filtered sites
– 19% large change, 50% small change, 24% reassured, 8% no help
Med Teach. 2013 Oct;35(10):e1546-50.
![Page 33: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/33.jpg)
Summary
• Curriculum involves training physicians to be
good evidence
• While successful on many levels, not perfect.
– Still need additional factors (e.g. independence)
• When you make an education program,
– Evaluate it (Study it).
![Page 34: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/34.jpg)
Teaching Teachers: Advanced EBM
![Page 35: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/35.jpg)
Advanced Education Components
• Tough topics
– Allocation Concealment
– Intention to Treat
– Numbers
– Funding
– P-values (hypothesis testing)
• Advanced Workshop
![Page 36: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/36.jpg)
Extra information for workshop
![Page 37: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/37.jpg)
What is in the DM trials?
Number Mean Age Duration of
DM
Past CVD Weight in
2nd Meta
UKPDS 33
(1998)
3,867
(4209*) 53.2 New 2%
23.9 UKPDS 34
(1998)
753
(4209*) 53.2 New 2%
ACCORD
(2008) 10,251 62.2 8.3 35% 21.5
ADVANCE
(2008) 11,140 66 7.5 32% 28.2
VADT (2009) 1791 60.4 11.5 40% 11.9
UGDP
(1978) 414 52.7 New 6% 11.8
* The n used in the BMJ meta for UKPDS combined
![Page 38: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/38.jpg)
What is in those Vitamin D trials
Study Country Endpoint Age
Camargo (2012) Mongolia Acute RI 10
Jorde (2012) Norway Influenza 63
Laaksi (2010) Finland URTI 18-28
Li-Ng (2009) USA URI* 59
Manaseki-Holland (2010) Afghanistan pneumonia 0-3
Manaseki-Holland (2012) Afghanistan pneumonia 0-1
Murdoch (2012) New Zealand Colds 47
Urashima (2010) Japan Flu 10.2
URI* = 2 or more of fever, cough, productive sputum or change in sputum
color and quantity, muscle aches, nausea or vomiting
![Page 39: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/39.jpg)
What is in those Vitamin D trials
• Let’s look specifically at Urashima (Am J
Clin Nutr 2010;91:1255-60)
Vitamin D Placebo Relative Risk*
n (167) % n (167) %
Influenza A 18 11% 31 19% 0.58 (0.34-0.996)
Influenza B 39 23% 28 17% 1.39 (0.90-2.15)
Influenza Like
Illness 8 5% 9 5% -
Total 65 38.9% 68 40.7% 0.96 (0.73-1.24)
* Using http://www.hutchon.net/ConfidRR.htm for RR (& CI) estimation
![Page 40: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/40.jpg)
Randomization, Blinding &
Allocation Concealment
![Page 41: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/41.jpg)
EBM TP: Cornerstones of Quality
Randomization, Allocation
Concealment & Blinding
• 1) Hypothesis
• 2) Randomize
• 3) Allocation to groups
• 4) Blind
(Do the Study)
• 5) Report (Baseline Characteristics)
![Page 42: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/42.jpg)
EBM TP: Cornerstones of Quality
Randomization & Blinding
• Many quality scores weight randomization and blinding heavily
– JADAD - 4/5 points (5th for follow-up)
• Randomization: ensures compared groups are equivalent.
• Blinding: prevents study personnel & pts from determining the group which pts are assigned
– cannot always be implemented
• Inadequate blinding favors treatment by about 17%1
1. JAMA. 1995;273:408-12
![Page 43: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/43.jpg)
EBM TP: Cornerstones of Quality
• BUT Randomization & Blinding mean little
without their forgotten cousin,…
![Page 44: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/44.jpg)
EBM TP: Cornerstones of Quality
Allocation concealment
• Allocation concealment prevents clinicians from
circumventing the randomization process
– AC can always be implemented
– Reporting AC poor (only 7% - 55%)1
• Poor Allocation concealment gives bias,2
– If Unclear AC = 33%
– If Inadequate AC = 41%
– Overall (unclear/inadequate) AC = 37%
1. BMJ 2004 3;328:22-4. ACP J Club. 2000 Mar-Apr;132:A11 .
2. JAMA 1995; 273: 408-412. Lancet. 1998;352:609-13.
![Page 45: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/45.jpg)
EBM TP: Cornerstones of Quality
Allocation concealment Examples
• If a trial sets it’s treatment effect at 25-50%,
– Doesn’t Conceal allocation,
– Any effect or prove of benefit may be entirely the
fault of poor AC (41%)
![Page 46: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/46.jpg)
Reporting Baseline Characteristics:
The Final Check
• Authors must report (usually Table 1) the
characteristics of the pts in each group (so we
can compare them)
• Ex Post-MI trial.
– ex if one group is older & has 10% higher risk
of MI, analysis must account for known > risk
![Page 47: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/47.jpg)
Analysis:
Intention to Treat
![Page 48: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/48.jpg)
EBM TP: “What is Intention to Treat”
• “Analysis was by intention to treat”
• Simply: Subjects were analyzed by what group they were
randomized to, not by what happened to them.
• The Analysis is by how the investigator intended to treat
them.
![Page 49: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/49.jpg)
Intention to Treat: Who Cares?
• The alternatives are: Per Protocol Analysis,
Explanatory Analysis, Analysis by Treatment
Administered, Efficacy Analysis
• Using these, you can hide outcomes or
select the best patients
• Example 1: Quiters (also known as drop-
outs)
![Page 50: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/50.jpg)
Intention to Treat: Example
Imagine a Stroke Study
Assume 10% have strokes a year
Assume it takes 6 months to get to surgery
![Page 51: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/51.jpg)
Intention to Treat: Example
• Intention to Treat
• 10% CER
• 10% EER
• = 0% AR
• = 0% RR
• NNT = infinite
• No difference
• Per Protocol
• 10% CER
• 5% EER (approx)
• = 5% AR
• = 50% RR
• NNT = 20
• publishable
![Page 52: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/52.jpg)
Intention to Treat: Summary
• Definition: Subjects are analyzed by the
group they were randomized to (aka: How
you Intended to Treat them), regardless of
what actually happened to them.
• The only non-bias way to analyze study
results (others bias trials up to 51%)
• Only 7-48% of articles do ITT (& less
properly)
BMJ. 1999; 319: 670-4. J Fam Pract. 2002; 51: 969-71. BMJ 2003;326: 117-5. J Clin
Epidemiol. 2003; 56: 833-42
![Page 53: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/53.jpg)
Therapy Stats
![Page 54: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/54.jpg)
EBM TP: (Liars, Damn Liars &) Stats
• RR vs AR
• RR when event unlikely
• The Media
![Page 55: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/55.jpg)
RR, AR and NNT
e.g. Treating Hypertension for MI Risk (Framingham data)
• AR = CER – EER
• RR= EER / CER
• NNT = 1 / AR
AR= 20% - 10% = 10%
RR= 10% / 20% = 50%
NNT= 1 / 0.1 = 10
CER: 60 y.o. M, HDL 1.0, BP>160 = 20%
EER: 60 y.o. M, HDL 1.0, BP<130 = 10%
![Page 56: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/56.jpg)
RR when event unlikely &
RR vs AR (NNT)
Common Diseases
• RR = 50%
• CER = 20%
• EER = 10%
• AR = 10%
• NNT = 10
Uncommon Diseases
• RR = 50%
• CER = 1%
• EER = 0.5%
• AR = 0.5%
• NNT = 200
![Page 57: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/57.jpg)
The Media: Women’s Health Initiative
• TV Evening News
(September 24, 2003)
• 41% increase in stroke
• 29% increase in MI
• Double rate of VTE
• 26% in Breast Ca
• No benefits mentioned
• Actual Risk per 10,000
(JAMA 288(3):321-33)
• 8 strokes in 10,000 p. y.
• 7 MI’s in 10,000 p. y.
• 18 VTE in 10,000 p. y.
• 8 Breast Ca in 10,000
• 6 Colorectal Ca & 5 Hip
Fractures Less/10,000
![Page 58: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/58.jpg)
Bottom Line
• Absolute Excess Adverse Events in the
Global Index are 19 in 10,000 patient years
OR
• 100 women on HRT for 5 years = 1 additional
adverse event
![Page 59: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/59.jpg)
Funding Bias
![Page 60: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/60.jpg)
EBM TP: Funding Bias
“He who pays, Calls the Tune”
Some Facts (& Myths)
• Is industry research more likely to be published
• No
• Is industry research comparatively poor quality
• No (?)
• Does Funding Influence Outcomes &
Recommendations
• Yes
![Page 61: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/61.jpg)
Funding Influence
• Funding gives an OR of 4-5.3 that,
– Study outcomes favor the drug being studied
– Drug will be recommended as Treatment of
choice
• Resources – Lexchin et al, BMJ, 2003; 326: 1167-70.
– Als-Nielsen et al, JAMA, 2003; 290(7): 921-8
– Melander et al, BMJ, 2003; 326: 1171-73
![Page 62: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/62.jpg)
How does it Happen?
• 1) “Pick your Battles.” Fund studies likely to be +ve (& don’t fund harm trials!)
• 2) “Pick your enemies.” Comparators should be poor drug, too low dose, not absorbed, etc.
• 3) “Give them what they want” Good Methods but ITT, reporting & others
• 4) “Let Sleeping Dogs Lie.” Don’t publish trials with bad outcomes; And Get good mileage out of good results.
• 5) “Buy what you need” Give Docs $12,000/pt + $30,000 bonus after 6.
![Page 63: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/63.jpg)
Selective Publication and Reporting - Circles: trials done by industry (red = favorable
outcome, blue = no better than placebo)
- Green Diamonds: publications off one trial.
-Yellow Boxes: publications off more than one trial
-Three Trials find their way into 15 publications (5 each)
![Page 64: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/64.jpg)
Selective Publication and Reporting
Rare cross referencing
Changing authors &
Definitions
Publications from
Single Trails:
- if trial +ve = 90%
- if trial –ve = 29%
![Page 65: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/65.jpg)
Bottom Line
• Don’t Blame Industry Entirely
– The authors of those papers are Doctors!
• Be Skeptical (Don’t “buy” in to advertising)
• Always “Cheque” Funding Source
• Then, Check Methods, Including ITT
• Then, Check that Recommendation Matches
Outcome & Treatment Effect
![Page 66: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/66.jpg)
P-Value meaning
![Page 67: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/67.jpg)
EBM TP: Alpha Errors, P-Values
and Confusion Intervals
• Can an X-ray have the appearance of a
fracture even when there isn’t one?
• Any investigation, whether a test or study,
can suggest something falsely.
• In research, they change false positive
to,…
![Page 68: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/68.jpg)
EBM TP:
Alpha error
• Alpha error (also Type I error): The null hypothesis is considered false when it is actually true (probability of error = p)
• In English: The study shows there is difference but,… it is a fluke and there really is no difference.
• This leads to,…
![Page 69: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/69.jpg)
P-value
• P-value 0.05 = There is 5% we will falsely reject the null hypothesis.
• English: If a study is showing there is a difference,… there is a 5% chance that it’s wrong (and there is no difference).
• When the p-value gets smaller (0.01 or 0.001, etc), so does the chance of making the mistake (i.e. assuming there is a difference when there is not).
![Page 70: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/70.jpg)
Shortfalls of P-Values
• Using P-value alone to examine a study turns the results
into a yes or no answer
• It also doesn’t give you an idea of the range of
possibilities (ie the effect could be is high or low as this)
• Therefore we need,…
![Page 71: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/71.jpg)
Confidence Intervals
• The range of values reflecting the statistical
precision of the estimate.
![Page 72: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/72.jpg)
Hypothetical Example:
Beer Damaged Neurons
• Imagine a university with 1000 students
• If I randomly selected 5 and used them to estimate the
mean # of damaged neurons, how confident should I be
that the next person will have a similar #.
• If I randomly selected 250, how confident would I be.
![Page 73: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/73.jpg)
CI: What does it tell us?
• Is the intervention helpful
• Is it statistically and clinically significant
• What is the degree of “precision”
![Page 74: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/74.jpg)
Comparing 2 trials
![Page 75: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/75.jpg)
Summary
• Alpha Error is when a study shows there is
difference but,… it is a fluke and there really is
no difference
• The probability of that error is the P-value
• A 0.05 P-Value means that if the study shows
a difference there is a 5% chance that it’s just
luck (and there is no difference).
• Confidence Intervals show the range of values
reflecting the precision of the estimate
![Page 76: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/76.jpg)
Questions?
![Page 77: Teaching Evidence Based Medicine - Conference … gp 15/1 presentations...Are we Evidence-based? • So do doctors understand EBM: –Our residents tested poorly (50%) and use Evidence](https://reader031.vdocuments.site/reader031/viewer/2022030509/5ab8a1f67f8b9a684c8cfe62/html5/thumbnails/77.jpg)
When Epidemiology Speaks,
No One Understands