confidence intervals kristin tolksdorf (based on previous epiet material) 18 th epiet/euphem...
TRANSCRIPT
![Page 1: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/1.jpg)
Confidence intervals
Kristin Tolksdorf(based on previous EPIET material)
18th EPIET/EUPHEM Introductory course01.10.2012
![Page 2: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/2.jpg)
Inferential statistics
• Uses patterns in the sample data to draw inferences about the population represented, accounting for randomness.
• Two basic approaches: – Hypothesis testing– Estimation
2
![Page 3: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/3.jpg)
Criticism on significance testing
“Epidemiological application need more than a decision as to whether chance alone could have produced association.” (Rothman et al. 2008)
→ Estimation of an effect measure (e.g. RR, OR) rather than significance testing.
→ Estimation of a mean
→ Estimation of a proportion3
![Page 4: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/4.jpg)
Why estimation?
Norovirus outbreak on a Greek island: “The risk of illness was higher among people who ate raw seafood (RR=21.5).”
How confident can we be in the result?What is the precision of our point estimate?
4
![Page 5: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/5.jpg)
The epidemiologist needs measurements rather than probabilities
2 is a test of association
OR, RR are measures of association on a continuous scale infinite number of possible values
The best estimate = point estimate
Range of “most plausible” values, given the sample data
Confidence interval precision of the point estimate
5
![Page 6: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/6.jpg)
Confidence interval (CI)
Range of values, on the basis of the sample data, in which the population value (or true value) may lie.
• Frequently used formulation: „If the data collection and analysis could be replicated many times, the CI should include the true value of the measure 95% of the time .”
6
![Page 7: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/7.jpg)
α/2
Lower limit upper limitof 95% CI of 95% CI
= 5%
s
α/2
Confidence interval (CI)
Indicates the amount of random error in the estimate Can be calculated for any „test statistic“, e.g.: means, proportions, ORs, RRs
95% CI = x – 1.96 SE up to x + 1.96 SE
1 - α
7
![Page 8: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/8.jpg)
CI terminology
RR = 1.45 (0.99 – 2.13)
Confidence intervalPoint estimate
Lower confidence limit
Upper confidence limit
8
![Page 9: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/9.jpg)
• amount of variability in the data
• size of the sample
• level of confidence (usually 90%, 95%, 99%)
Width of confidence interval depends on …
A common way to use CI regarding OR/RR is :If 1.0 is included in CI non significant If 1.0 is not included in CI significant
9
![Page 10: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/10.jpg)
Study A, large sample, precise results, narrow CI – SIGNIFICANTStudy B, small size, large CI - NON SIGNIFICANT
Looking at the CI
Study A, effect close to NO EFFECTStudy B, no information about absence of large effect
RR = 1
A
B
Large RR
10
![Page 11: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/11.jpg)
More studies are better or worse?
1RR
20 studies with different results...
11
clinical or biological significance ?
![Page 12: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/12.jpg)
Norovirus on a Greek island
• How confident can we be in the result?• Relative risk = 21.5 (point estimate)• 95% CI for the relative risk:
(8.9 - 51.8)
The probability that the CI from 8.9 to 51.8 includes the true relative risk is 95%.
12
![Page 13: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/13.jpg)
Norovirus on a Greek island
“The risk of illness was higher among people who ate raw seafood (RR=21.5, 95% CI 8.9 to 51.8).”
13
![Page 14: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/14.jpg)
Example: Chlordiazopoxide use and congenital heart disease (n=1 644)
Cases Controls
C use 4 4
No C use 386 1 250
OR = (4 x 1250) / (4 x 386) = 3.2
p = 0.080 ; 95% CI = 0.6 - 17.5
From Rothman K
![Page 15: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/15.jpg)
3.2
p=0.080
0.6 – 17.515
![Page 16: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/16.jpg)
Example: Chlordiazopoxide use and congenital heart disease – large study (n=17 151)
Cases Controls
C use 240 211
No C use 7 900 8 800
OR = (240 x 8800) / (211 x 7900) = 1.3
p = 0.013 ; 95% CI = 1.1 - 1.5
![Page 17: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/17.jpg)
Precision and strength of association
Strength
Precision17
![Page 18: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/18.jpg)
Confidence interval provides more information than p value
• Magnitude of the effect (strength of association)
• Direction of the effect (RR > or < 1)
• Precision of the point estimate of the effect (variability)
p value can not provide them ! 18
![Page 19: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/19.jpg)
2 Test of association, depends on sample size
p value Probability that equal (or more extreme) results can be observed by chance alone
OR, RR Direction & strength of associationif > 1 risk factor if < 1 protective factor(independently from sample size)
CI Magnitude and precision of effect
What we have to evaluate the study
19
![Page 20: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/20.jpg)
Comments on p-values and CIs
• Presence of significance does not prove clinical or biological relevance of an effect.
• A lack of significance is not necessarily a lack of an effect: “Absence of evidence is not evidence of absence”.
20
![Page 21: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/21.jpg)
Comments on p-values and CIs
• A huge effect in a small sample or a small effect in a large sample can result in identical p values.
• A statistical test will always give a significant result if the sample is big enough.
• p values and CIs do not provide any information on the possibility that the observed association is due to bias or confounding.
21
![Page 22: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/22.jpg)
Cases Non-cases Total 2 = 1.3E 9 51 60 p = 0.13NE 5 55 60 RR = 1.8Total 14 106 120 95% CI [ 0.6 - 4.9 ]
Cases Non-cases Total 2 = 12E 90 510 600 p = 0.0002NE 50 550 600 RR = 1.8Total 140 1060 1200 95% CI [ 1.3-2.5 ]
2 and Relative Risk
22
![Page 23: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/23.jpg)
Exposure Cases Non-cases AR%Yes 15 20 42.8%No 50 200 20.0%
Total 65 220
Common source outbreak suspected
REMEMBER: These values do not provide any information on the possibility that the observed association is due to a bias or confounding.
2 = 9.1 p = 0.002RR = 2.195%CI = 1.4 - 3.4
23%
23
![Page 24: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/24.jpg)
The ultimative (eye) test
24
• Hypothesis testing: X²-Test– Question: Is the proportion of facilitators wearing
glasses equal to the proportion of fellows wearing glasses?
• Estimation of quantities: Proportion– What is the proportion of fellows/facilitators
wearing glasses?
![Page 25: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/25.jpg)
The ultimative (eye) test
25
Proportion = 11/38 = 0.29SE = 0.07495%CI = 0.14 - 0.44
Glasses among fellows : Yes 11No 27
Total 38
Glasses among facilitators :
Yes 6No 8
Total 14
Proportion = 6/14 = 0.43SE = 0.13295%CI = 0.17 - 0.69
![Page 26: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/26.jpg)
Recommendations
• Always look at the raw data (2x2-table). How many cases can be explained by the exposure?
• Interpret with caution associations that achieve statistical significance.
• Double caution if this statistical significance is not expected.
• Use confidence intervals to describe your results.
26
![Page 27: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/27.jpg)
Suggested reading
• KJ Rothman, S Greenland, TL Lash, Modern Epidemiology, Lippincott Williams & Wilkins, Philadelphia, PA, 2008
• SN Goodman, R Royall, Evidence and Scientific Research, AJPH 78, 1568, 1988
• SN Goodman, Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy, Ann Intern Med. 130, 995, 1999
• C Poole, Low P-Values or Narrow Confidence Intervals: Which are more Durable? Epidemiology 12, 291, 2001
27
![Page 28: Confidence intervals Kristin Tolksdorf (based on previous EPIET material) 18 th EPIET/EUPHEM Introductory course 01.10.2012](https://reader035.vdocuments.site/reader035/viewer/2022062417/55152a64550346a80c8b6680/html5/thumbnails/28.jpg)
Previous lecturers
28