making sense of results - a workshop for healthcare librarians
DESCRIPTION
Making sense of results - a workshop for healthcare librarians. Dr Amanda Burls 2 nd UK Clinical Librarian Conference. Objectives. To look at how results can be presented To understand what a meta-analysis is To be able to interpret a “blobbogram” - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/1.jpg)
Making sense of results- a workshop for healthcare librarians
Dr Amanda Burls2nd UK Clinical Librarian Conference
![Page 2: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/2.jpg)
Objectives
• To look at how results can be presented• To understand what a meta-analysis is• To be able to interpret a “blobbogram”• To be able to make sense of tests for “statistical significance” • To explore how uncertainty in results can be summarised and
understand:• P-values• Confidence intervals
• To have fun!
![Page 3: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/3.jpg)
Making sense of results
• How are results summarised?
![Page 4: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/4.jpg)
![Page 5: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/5.jpg)
How many of you have attended a critical appraisal skills workshop?
• What sort of study design were you appraising?
• What are the key things you remember?
![Page 6: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/6.jpg)
Critical appraisal of any study design must consider
• Validity– Can the study (results) be trusted?
• ResultsResults– What are the results and how are they (or can they
be) expressed?
• Relevance– Do these results apply to the local context?
![Page 7: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/7.jpg)
Warning!• Everything I say from now onwards assumes
that the results being considered come from an unbiased study!
![Page 8: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/8.jpg)
How are results summarised?
• Most useful studies compare at least two alternatives.
• How can the results of such comparisons be expressed?
![Page 9: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/9.jpg)
Expressing results:What did the study show?
• Patients with backache:– 100 randomised to receive a firm mattress– 100 randomised to receive a medium mattress
• After 3 months:– 80 get better in the firm mattress group– 20 get better in the medium mattress group
• How would you summarise this for a friend?
![Page 10: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/10.jpg)
Summarise
• 80 out of 100 (80%) better in firm mattress group
• 20 out of 100 (20%) better in the medium mattress group
• 4 times as likely to get better with a firm mattress
• An extra 60% of people get better with a firm mattress
![Page 11: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/11.jpg)
How were the results summarised?
• There are two basic ways to summarise results of studies that compare two or more groups:
1. Difference (take them away)
2. Ratio (divide)
![Page 12: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/12.jpg)
The blobbogram!
![Page 13: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/13.jpg)
Blobbogram
Line of no differencebetween treatments
less more
![Page 14: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/14.jpg)
Blobbogram - Difference (taking away)
Line of no differencebetween treatments
less more0
![Page 15: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/15.jpg)
Blobbogram - ratio (dividing)
Line of no differencebetween treatments
less more1
![Page 16: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/16.jpg)
A randomised placebo-controlled trial
![Page 17: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/17.jpg)
Well conducted RCT – no bias
• Five people with backache received Potters
• Five people received placebo
• 4 out of 5 with Potters got better
• 2 out of 5 with placebo got better
![Page 18: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/18.jpg)
<0.00010.010.030.090.29p-value
0.27 to 0.520.10 to 0.640.04 to 0.67-0.04 to 0.71-0.23 to 0.7995% CI
0.40.40.40.40.4Proportion responding in control arm
408642Responders in control arm
1002015105Number in control arm
0.80.80.80.80.8Proportion responding in treatment arm
80161284Responders in treatment arm
1002015105Number in treatment arm
<0.00010.010.030.090.29p-value
0.27 to 0.520.10 to 0.640.04 to 0.67-0.04 to 0.71-0.23 to 0.7995% CI
0.40.40.40.40.4Proportion responding in control arm
408642Responders in control arm
1002015105Number in control arm
0.80.80.80.80.8Proportion responding in treatment arm
80161284Responders in treatment arm
1002015105Number in treatment arm
![Page 19: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/19.jpg)
No backache at 3 months(Results of our Potters tablet
versus placebo trial) Potters PlaceboPotters Placebo
Favours placebo Favours PottersFavours placebo Favours Potters
![Page 20: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/20.jpg)
No backache at 3 months(Results of our Potters tablet
versus placebo trial) Potters PlaceboPotters Placebo
Favours placebo Favours PottersFavours placebo Favours Potters
![Page 21: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/21.jpg)
No backache at 3 months(Results of our Potters tablet
versus placebo trial) Potters PlaceboPotters Placebo
Favours placebo Favours PottersFavours placebo Favours Potters
![Page 22: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/22.jpg)
![Page 23: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/23.jpg)
No backache at 3 monthsDo you think this study proves Potters works?
Potters PlaceboPotters Placebo
Favours placebo Favours PottersFavours placebo Favours Potters
![Page 24: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/24.jpg)
It could be due to chance!
• What if there had 1000 people in each arm and 800 got better with Potters and only 200 got better on placebo?
• Would you believe Potters works now?
• So how many people would you want in each arm to believe the trial?
![Page 25: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/25.jpg)
P-value in a nutshell
![Page 26: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/26.jpg)
The Null Hypothesis
![Page 27: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/27.jpg)
0 1
Impossible Absolutely certain
![Page 28: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/28.jpg)
• p = 0.5
quite likely - evens chance - 50:50 - 1 in 2
• p = 0.001
very unlikely - 1 in 1000• p = 0.01
unlikely - 1 in 100• p = 0.05
fairly unlikely - 1 in 20 - 5 times in 100
![Page 29: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/29.jpg)
Odds ratio (12b)
0
10
20
30
40
50
60
70
80
90
100
Pre and Post Workshop Scores
Perc
enta
ge 5
4
3
2
1
![Page 30: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/30.jpg)
MAAG (9b)
0
10
20
30
40
50
60
70
80
90
100
Pre and Post Workshop Scores
Perc
enta
ge 5
4
3
2
1
![Page 31: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/31.jpg)
Moral:
Any observed difference between two groups, no matter how small, can be made to be “statistically
significant” - at any level of significance - by taking a sufficiently large sample.
![Page 32: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/32.jpg)
• Question: How can we express uncertainty due to chance?
• Answer: the p-value
• But is there a better answer?
![Page 33: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/33.jpg)
Introduction to confidence intervals
• CIs are a way of showing the uncertainty surrounding our point estimate.
![Page 34: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/34.jpg)
No backache at 3 months(Results of our Potters tablet
versus placebo trial) Potters PlaceboPotters Placebo
Favours placebo Favours PottersFavours placebo Favours Potters
![Page 35: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/35.jpg)
No backache at 3 months(Results of our Potters tablet
versus placebo trial) Potters PlaceboPotters Placebo
Favours placebo Favours PottersFavours placebo Favours Potters
![Page 36: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/36.jpg)
No backache at 3 months(Results of our Potters tablet
versus placebo trial) Potters PlaceboPotters Placebo
Favours placebo Favours PottersFavours placebo Favours Potters
![Page 37: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/37.jpg)
No backache at 3 months(Results of our Potters tablet
versus placebo trial) Potters PlaceboPotters Placebo
Favours placebo Favours PottersFavours placebo Favours Potters
![Page 38: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/38.jpg)
Hypothermia vs. control In severe head injury
Mortality or incapacity (n=158)
RR 0.63 (0.46, 0.87)Marion 1997
.1 .2 1 5 10
Total (95%CI)
Clifton 1992
Hirayama 1994
Clifton 1993
RR
![Page 39: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/39.jpg)
Hypothermia vs. control In severe head injury
Mortality or incapacity (n=158)
RR 0.63 (0.46, 0.87)Marion 1997
.1 .2 1 5 10
Total (95%CI)
Clifton 1992
Hirayama 1994
Clifton 1993
RR
![Page 40: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/40.jpg)
Hypothermia vs. control In severe head injury
Mortality or incapacity (n=158)
RR 0.63 (0.46, 0.87)Marion 1997
.1 .2 1 5 10
Total (95%CI)
Clifton 1992
Hirayama 1994
Clifton 1993
Favours intervention RR Favours control
![Page 41: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/41.jpg)
Hypothermia vs. control In severe head injury
Mortality or incapacity (n=158)
RR 0.63 (0.46, 0.87)
Marion 1997
.1 .2 1 5 10
Total (95%CI)
Clifton 1992
Hirayama 1994
Clifton 1993
Favours intervention RR Favours control
![Page 42: Making sense of results - a workshop for healthcare librarians](https://reader035.vdocuments.site/reader035/viewer/2022070410/56814657550346895db372b5/html5/thumbnails/42.jpg)