on snow, chocolate and repair shops -...
TRANSCRIPT
![Page 1: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/1.jpg)
On snow, chocolate and repair shops
J. C. Villasboas
Larry Young
![Page 2: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/2.jpg)
To shun fear and learn to love forest plots
To review principles of when it is appropriate to combine
studies
Interpret the results of a meta-analysis including forest plots
Understand what systematic reviews can and cannot do for you
Demonstrate teaching techniques and identify resources to
assist
Teaching Objectives
![Page 3: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/3.jpg)
Extra-Special Teaching Objective
CERTIFIED to be JARGON FREE
Need a volunteer
![Page 4: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/4.jpg)
Are Systematic Reviews Scary? Raise your hand if you…
Always skip the methods section on Systematic
Reviews because it sounds like Gibberish
Make your colleagues teach the systematic review
session so you don’t look like an idiot
Think a “Forest Plot” might be a good buy given
the current housing market
![Page 5: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/5.jpg)
How to decrease fear
What strategies have you seen this week (small or
large group) to decrease fear in the learning
environment?
![Page 6: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/6.jpg)
Teaching Time Out
Simplify concepts
Avoid numbers and Jargon
There are tools to assist your teaching
TIPS Series
![Page 7: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/7.jpg)
![Page 8: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/8.jpg)
Gleeful to Grief-stricken Scale
Validated instrument for assessing levels of comfort (glee)
or fear (grief) in EBM workshop participants
Glee (0) Indifference (5) Grief (10)
![Page 9: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/9.jpg)
Reactions of people on hearing that they need to look at a meta-analysis:
![Page 10: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/10.jpg)
Ridicularia Exemplarum…
The best way to understand just about anything in
medicine is to reflect on a patient…or other life
dilemmas
The following tale is one such dilemma…
![Page 11: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/11.jpg)
Disclaimer
No conflict of interests for presenters
All characters and events in this presentation are fictional
Any use of names (or initials) is purely unintentional and any
similarities to actual persons, in this room or elsewhere, are
purely coincidence
We cannot emphasize enough how coincidental similarities
of our fictional entities to any real life individual, entity, or
character are
![Page 12: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/12.jpg)
Winds of change
![Page 13: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/13.jpg)
Winds of change blow over Miami
It’s 2pm and another hard day of work is over
LY drives back home one breezy Floridian afternoon
He thinks about life
He thinks about his friend and mentor SK
Fleetwood Mac bursts through his stereo
Winds of change
![Page 14: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/14.jpg)
Nostalgia
![Page 15: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/15.jpg)
Nostalgia
![Page 16: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/16.jpg)
Nostalgia
![Page 17: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/17.jpg)
Nostalgia
![Page 18: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/18.jpg)
Nostalgia
![Page 19: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/19.jpg)
LY understands
He decides to buy her a farewell surprise gift
He wants something meaningful.
He decides to call Manny
Manny is smart and lives in Cleveland, OH
He may know one or two things about cold weather
Nostalgia
![Page 20: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/20.jpg)
Finding Manny
Manny has left Cleveland
He succumbed to Winter Blues
He returned to Brazil
He now runs a repair shop
He still does some EBM on the side
![Page 21: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/21.jpg)
Seasonal Affective Disorder - SAD
![Page 22: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/22.jpg)
Manny vs. S.A.D.
LY worries for SK
After so many years in Miami it
may be hard for her to adapt
He asks for Manny’s help
![Page 23: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/23.jpg)
Manny vs. S.A.D.
![Page 24: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/24.jpg)
Manny vs. S.A.D.
Manny tried several interventions
Lights, exercise, binge drinking
Nothing seemed to work
He is convinced prevention is key
He thinks he has found the answer
![Page 25: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/25.jpg)
Chocolate vs. S.A.D.
After an extensive search, he has
found 4 RCTs on chocolate for
S.A.D. prevention
But he knows LY wants the best
evidence
He is not awfully busy lately
He could put them together for LY
![Page 26: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/26.jpg)
The Systematic Review
Somewhere in those 4 studies.
![Page 27: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/27.jpg)
The Evidence Pyramid (Of Truth)
Unsystematic Clinical Experience
Case-Control
Case Series
Observational Studies
(Cohort)
RCT
Systematic Review/Meta-analysis
![Page 28: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/28.jpg)
Let’s help Manny summarize each study
P I C O F/U Cearras et al
Ribeiro et al
Pastorini et al
Brown et al
![Page 29: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/29.jpg)
P I C O F/U Cearras et al
Ribeiro et al
Pastorini et al
Brown et al
![Page 30: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/30.jpg)
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al
Pastorini et al
Brown et al
![Page 31: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/31.jpg)
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al
Brown et al
![Page 32: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/32.jpg)
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al
![Page 33: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/33.jpg)
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al Healthy pedia
tric volunteers Chocolate bar Placebo
Change in GGs
Scale 2y
![Page 34: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/34.jpg)
Next Exercise
Instructions to be given by a visiting celebrity scholar….
Go to video and then do what the video tells you to do….
(use discretion in your interpretation)
![Page 35: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/35.jpg)
Celebrity Scholar
![Page 36: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/36.jpg)
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al Healthy pedia
tric volunteers Chocolate bar Placebo
Change in GGs
Scale 2y
One of these things is not like the other things…
![Page 37: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/37.jpg)
One of these things is not like the other things…
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al Healthy pedia
tric volunteers Chocolate bar Placebo
Change in GGs
Scale 2y
![Page 38: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/38.jpg)
One of these things is not like the other things…
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al Healthy pedia
tric volunteers Chocolate bar Placebo
Change in GGs
Scale 2y
![Page 39: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/39.jpg)
One of these things is not like the other things…
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al Healthy pedia
tric volunteers Chocolate bar Placebo
Change in GGs
Scale 2y
![Page 40: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/40.jpg)
One of these things is not like the other things…
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al Healthy pedia
tric volunteers Chocolate bar Placebo
Change in GGs
Scale 2y
![Page 41: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/41.jpg)
One of these things is not like the other things…
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al Healthy pedia
tric volunteers Chocolate bar Placebo
Change in GGs
Scale 2y
![Page 42: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/42.jpg)
One of these things is not like the other things…
![Page 43: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/43.jpg)
Is pooling sensible?
A priori determination of whether it makes sense
to combine on a principled basis
You have not looked at the data yet…
![Page 44: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/44.jpg)
Is pooling sensible?
MA of all kinds of treatments for all cancers?
MA of steroid treatments for COPD?
MA of studies looking at CT-angiogram for
diagnosis of pulmonary embolism?
![Page 45: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/45.jpg)
P I C O F/U Cearras et al
Adults at risk
of S.A.D. Chocolate bar Placebo
Incidence of
S.A.D. 2y
Ribeiro et al Adults at risk
of S.A.D. Hot chocolate Placebo
Incidence of
S.A.D. 2y
Pastorini et al Adults at risk
of S.A.D. Chocolate bar Cinnamon rolls
Incidence of
S.A.D. 2y
Brown et al Healthy pedia
tric volunteers Chocolate bar Placebo
Change in GGs
Scale 2y
![Page 46: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/46.jpg)
Does combining RESULTS make sense?
Comes from a review of dots and lines
What are the dots and lines in a MA?
![Page 47: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/47.jpg)
Favors New Treatment Favors Control No Difference
Result 1
![Page 48: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/48.jpg)
Eye-ball test
![Page 49: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/49.jpg)
Favors New Treatment Favors Control No Difference
Can we combine?
![Page 50: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/50.jpg)
Result 2
Favors New Treatment Favors Control No Difference
![Page 51: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/51.jpg)
Eye-ball test
![Page 52: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/52.jpg)
Can we combine?
Favors New Treatment Favors Control No Difference
![Page 53: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/53.jpg)
Result 3
Favors New Treatment Favors Control No Difference
![Page 54: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/54.jpg)
Eye-ball test
![Page 55: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/55.jpg)
Can we combine?
Favors New Treatment Favors Control No Difference
![Page 56: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/56.jpg)
Okay: What is the Eye-ball test?
![Page 57: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/57.jpg)
Eye-ball test
Visual impression of degree of overlap of each
study’s results (find a common neighborhood of
truth)
Visual impression of where the estimates of study
“effects” themselves lie (where are the dots)
![Page 58: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/58.jpg)
Favors New Treatment Favors Control No Difference
![Page 59: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/59.jpg)
Back to result 3: if not, why not?
Favors New Treatment Favors Control No Difference
![Page 60: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/60.jpg)
Why might things be different?
Different age groups
Chocolate vehicle
Active control / placebo
incidence of SAD vs. GGsS
![Page 61: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/61.jpg)
Let’s help Manny build a forest plot
![Page 62: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/62.jpg)
1 2 5 10 0.5 .2 .1
Cearras
Pastorini
Ribeiro
Brown
RR Cocoa product better Cocoa product worse
![Page 63: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/63.jpg)
To Combine Or Not To Combine?
Systematic review vs. meta-analysis
Disbelievers be comforted…
![Page 64: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/64.jpg)
1 2 5 10 0.5 .2 .1
Cearras
Pastorini
Ribeiro
Brown
RR Cocoa product better Cocoa product worse
We Don’t Have To Combine
![Page 65: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/65.jpg)
1 2 5 10 0.5 .2 .1
Cearras
Pastorini
Ribeiro
RR Cocoa product better Cocoa product worse
Brown
Or we can try to explain the differences in Brown’s trial (pediatric and GGsS) and find a more similar grouping
![Page 66: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/66.jpg)
Back To Our Scenario
I guess it works!
Who cares about calories?
LY will make sure she takes enough chocolate
with her
![Page 67: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/67.jpg)
Tips for minimizing learner fear
Use non-medical examples
Use common sense approach explanations rather
than statistical ones
![Page 68: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/68.jpg)
Now to a real life example Right after this commercial message
(5 min bathroom break)
![Page 69: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/69.jpg)
![Page 70: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/70.jpg)
Case Scenario: Take Tai Chi twice a week
80yo aunt calls
She was urged by her PCP to start exercising
He is trying to decrease her fall risk
He recommends she starts Tai Chi classes
PCP’s wife owns the only Tai Chi gym in town
She is suspicious and wants my advice
![Page 71: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/71.jpg)
Being the EBM nerd that you are, you reply, “I’ll find out and
get back to you!”
What’s our first step?
Case Scenario: Take Tai Chi twice a week
![Page 72: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/72.jpg)
Those 6 Letters Again
elderly adults
exercise [Tai Chi]
no intervention
falls
Treatment Question
RCT or Meta-analysis of well done RCTs
![Page 73: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/73.jpg)
Success!
Let’s appraise the article together… in 5 minutes?
![Page 74: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/74.jpg)
RAME IT
Did the authors define (frame) a sensible clinical question using
PICO framework?
IND IT
Did the authors do a detailed and exhaustive search (find) that
is unlikely to systematically miss important relevant studies?
Words that start with the letter “F”
![Page 75: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/75.jpg)
Words that start with the letter “F”
ILTER AND ETCH
Did authors select studies based on inclusion criteria, assess
risk of bias, collect data from individual studies in a
reproducible manner?
INDINGS AND OREST PLOTS
Review findings of a systematic review and the forest plots or
summary statistics if graphs are not provided for meta-analysis.
![Page 76: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/76.jpg)
Success… Appraisal
![Page 77: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/77.jpg)
Words that start with the letter “F”
RAME IT
IND IT
ILTER AND ETCH (AND IGHT)
INDINGS OREST PLOTS
![Page 78: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/78.jpg)
What are the results?
Everyone: look at Figure 2 (page 820)
Describe the dots and lines
![Page 79: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/79.jpg)
Exercise or Physical Therapy Interventions
Favors Control Favors Intervention
![Page 80: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/80.jpg)
I2 what?
From 0 to 100
The variability that is due to a real difference
I2 = 0
I2 = 100
What is the range of possible values for I2
Explain in a sentence?
Is the variability between studies due ONLY to chance?
Is the variability between studies due ONLY to a real difference in treatment?
![Page 81: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/81.jpg)
Favors New Treatment Favors Control No Difference
Result 1: what is I2?
I2= ____ 10%
![Page 82: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/82.jpg)
Result 2: what is I2?
Favors New Treatment Favors Control No Difference
I2= ____ 5 %
![Page 83: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/83.jpg)
Result 3: what is I2?
Favors New Treatment Favors Control No Difference
I2= ____ 90%
![Page 84: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/84.jpg)
Exercise or Physical Therapy Interventions
Favors Control Favors Intervention
![Page 85: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/85.jpg)
Single Clinical Intervention: Vitamin D
Favors Control Favors Intervention
![Page 86: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/86.jpg)
Back to your aunt…
Tai Chi classes are reasonable fall prevention strategies
Certainly other exercises are equally reasonable
Vitamin D supplementation can also be considered
![Page 87: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/87.jpg)
What Systematic Reviews Can Do for You
Save time
Increase power to detect rare events
Obviate need for large, expensive trials
Detect harm
Increase the precision of the estimate of effect
Enhance generalizability of the results if samples from
different populations are included
![Page 88: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/88.jpg)
What Systematic Reviews Can’t Do for You
Allow you to determine “equivalency” of different
interventions
“Get rid of” bias from individual studies
![Page 89: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/89.jpg)
To shun fear and learn to love forest plots
To review principles of when it is appropriate to combine
studies
Interpret the results of a meta-analysis including forest plots
Understand what systematic reviews can and cannot do for you
Demonstrate teaching techniques and identify resources to
assist
Teaching Objectives
![Page 90: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/90.jpg)
Extra-Special Teaching Objective
JARGON FREE (at least for
the first half…)
Avoided words
Point Estimate
Confidence Interval
Summary Statistic
Heterogeneity
I2
![Page 91: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/91.jpg)
Take Home Points
A well done meta-analysis seeks to get us closer to the
truth by combining results of well done individual trials
Sometimes it’s okay to combine the results of different
trials… sometimes it’s not
Differences between trials can be assessed using
different methods
![Page 92: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/92.jpg)
Take Home Points
I2 = 0 is EXCELLENT
I2 = 100 is AWFUL
We can all still learn a lot from Cookie Monster
![Page 93: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/93.jpg)
Gleeful to Grief-stricken Scale
Validated instrument for assessing levels of comfort (glee)
or fear (grief) in EBM workshop participants
Glee (0) Indifference (5) Grief (10)
![Page 94: On snow, chocolate and repair shops - Sites@Dukesites.duke.edu/.../files/2012/06/Systematic-Reviews.pdf · 2014. 3. 17. · Reviews because it sounds like Gibberish ... Seasonal Affective](https://reader034.vdocuments.site/reader034/viewer/2022052104/603ec547a0ddd427d82f3b0e/html5/thumbnails/94.jpg)
We Hope It Wasn’t So Scary After All