inference after anova, multiple comparisons 3/21/12 inference after anova the problem of multiple...
TRANSCRIPT
![Page 1: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/1.jpg)
Inference after ANOVA, Multiple Comparisons
3/21/12
• Inference after ANOVA• The problem of multiple comparisons• Bonferroni’s Correction
Section 8.2 Professor Kari Lock MorganDuke University
![Page 2: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/2.jpg)
• Clicker: overwhelmingly positive
• Textbook: positive
• Homework: surprisingly positive
• Lecture: mostly positive, some complaints• most common complaint: too fast• not always open to questions• full lecture slides not posted in advance
• Lab: less positive, varied complaints• TA too busy, TA not helpful• can do at home• too long
Midterm Evaluation
![Page 3: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/3.jpg)
Midterm Evaluation
![Page 4: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/4.jpg)
• Project 1 (due TOMORROW, 5pm)
• Homework 7 (due Monday, 3/26)NO LATE HOMEWORK ACCEPTED!Turn in by Friday, 3/23, 5pm to get it graded
before Exam 2.
• Start preparing for Exam 2 (next Wednesday and Thursday)
To Do
![Page 5: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/5.jpg)
• Conclusion: What have you learned? Make sure to answer the research question posed in your introduction.
• This is a paper, there should be text. Do not just give R output, formulas, and numbers.
Project 1 Comments
![Page 6: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/6.jpg)
• Exam 2:• In-class portion: Wednesday, 3/28• Lab portion: Thursday, 3/29
• In-class portion: (75%)• Open only to a calculator and two double sided
pages of notes prepared by you
• Lab portion: (25%)• Open to everything except communication of any
form with other humans
Exam 2
![Page 7: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/7.jpg)
• The emphasis will be on material we have learned since the first exam, although you are still responsible for everything we learned prior to the first exam
Exam 2
![Page 8: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/8.jpg)
• On the course website, under documents:• Last semester’s in-class exam, and solutions• Last semester’s labs exam, and solutions• Full solutions to all essential synthesis
problems from Unit 3• Full solutions to all review problems from
Unit 3• Full solutions to all odd problems from
Chapters 7 and 8
• Doing problems is the key to success!!!
Practice
![Page 9: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/9.jpg)
• Work lots of practice problems! • Take last year’s exams under realistic
conditions (time yourself, do it all before looking at the solutions, etc.)
• Prepare a good cheat sheet and use it when working problems
• Read the corresponding sections in the book if there are concepts you are still confused about
Keys to In-Class Exam Success
![Page 10: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/10.jpg)
• Primarily, make sure you know how to summarize, visualize, create an interval, and conduct a test for any one variable or relationship between two variables
• For practice, try doing both intervals and tests for any one or two variables in our class survey
• Beyond that, make sure you are comfortable with the content from the labs
• Open-book does NOT mean you don’t have to study. You will not have time to look up every command you need during the exam.
Keys to Lab Exam Success
![Page 11: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/11.jpg)
You have LOTS of opportunities for help!
• Today, 3 – 5pm (Prof Morgan)• Today, 6 – 8 pm (Michael)• Friday, 1:30 – 3 pm (Prof Morgan)• Sunday, 5 – 7 pm (Jessica)• Sunday, 7 – 9 pm (Michael)• Monday, 3 – 4 pm (Prof Morgan)• Monday, 4 – 6 pm (Christine)• Tuesday, 3 – 6 pm (Prof Morgan)• Tuesday, 6 – 8 pm (Yue)
Office Hours before Exam
![Page 12: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/12.jpg)
Cuckoo Birds•Cuckoo birds lay their eggs in the nests of other birds (typically small birds).
•When the cuckoo baby hatches, it kicks out all the original eggs/babies
•If the cuckoo is lucky, the mother will raise the cuckoo as if it were her own
•Do cuckoo birds found in nests of different species differ?
http://opinionator.blogs.nytimes.com/2010/06/01/cuckoo-cuckoo/
![Page 13: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/13.jpg)
Length of Cuckoo Eggs
![Page 14: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/14.jpg)
Length of Cuckoo EggsBird Sample
MeanSample
SDSample
Size
Pied Wagtail 22.90 1.07 15
Pipit 22.50 0.97 60
Robin 22.58 0.68 16
Sparrow 23.12 1.07 14
Wren 21.13 0.74 15
Overall 22.46 1.07 120
35.90
101.29
137.19
SSG
SSE
SST
Is there a significant difference between the groups?
(a) Yes(b) No
![Page 15: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/15.jpg)
Source
Groups
Error
Total
df
4
115
119
Sum ofSquares35.90
101.29
137.19
MeanSquare
8.97
0.88
FStatistic10.19
p-value
4.3 × 10-7
ANOVA Table
We have very strong evidence that average length of cuckoo eggs differs for nests of different species
Equal variability Normal(ish) data
![Page 16: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/16.jpg)
Cuckoo Birds•How long are cuckoo bird eggs found in robins’ nests?
•Is there a significant difference between the average length of eggs found in robins’ nests and the average length of eggs found in sparrows’ nests?
•While we could proceed with formulas from Chapter 6 or simulation methods from Chapters 3 and 4, there are special ways of doing inference after ANOVA…
![Page 17: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/17.jpg)
Inferences after ANOVA•If the ANOVA assumption of equal variability across groups is satisfied, we can use the data from all groups to estimate variability:
1. Estimate any within-group standard deviation with MSE
2. Use the error degrees of freedom, , for -distributionsn k t
1
11
Single Mean:
SM
nnE
SE
2 21 2
1 2 1 2
Difference in Two Means:
1 1
n nSE MSE
n n
![Page 18: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/18.jpg)
Cuckoo Birds•How long are cuckoo bird eggs found in robins’ nests? Give a 90% confidence interval.
•Is there a significant difference between the average length of eggs found in robins’ nests and the average length of eggs found in sparrows’ nests?
(a) Yes
(b) NoBird Sample Mean Sample SD Sample Size
Pied Wagtail 22.90 1.07 15
Pipit 22.50 0.97 60
Robin 22.58 0.68 16
Sparrow 23.12 1.07 14
Wren 21.13 0.74 15
Overall 22.46 1.07 120
(22.19, 22.97)
![Page 19: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/19.jpg)
Cuckoo Birds•How long are cuckoo bird eggs found in robins’ nests? Give a 90% confidence interval.
1.66
22.58 0.3
0
8
.8
9
822.58
16
22.19,22.97
*statistic t SE
*R
R
MSEX t
n
t*
We are 90% confident that the average length of Cuckoo eggs found in Robins’ nests is between 22.19 and 22.97 mm.
![Page 20: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/20.jpg)
Cuckoo Birds•Is there a significant difference between the average length of eggs found in robins’ nests and the average length of eggs found in sparrows’ nests?
statistic nullt
SE
22.58 23.12
1 10.88
1
0.541.57
0.3416
3114
R S
R S
X Xt
MSEn n
This study does not provide evidence for a difference in average mean length of Cuckoo eggs between those found in Robins and Sparrows nests.
0 :
:R S
a R S
H
H
![Page 21: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/21.jpg)
Pairwise Comparisons•Pairwise comparisons test for a difference in means between each pair of groups
•Only do pairwise comparisons if the overall ANOVA is significant
•If there are lots of categories, the number of possible pairwise comparisons grows quickly
•Automate the process with RStudio
![Page 22: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/22.jpg)
Cuckoo Birds
![Page 23: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/23.jpg)
Length of Cuckoo Eggs
![Page 24: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/24.jpg)
Extrasensory Perception•Is there such a thing as extrasensory perception (ESP), or a “sixth sense”?
•Do you believe in ESP or a sixth sense?
(a) Yes (b) No (c) Not sure
![Page 25: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/25.jpg)
Extrasensory Perception
![Page 26: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/26.jpg)
Extrasensory Perception
How would you test whether American belief in ESP differs between current Duke students who take STAT 101 and all Americans in 2001?
a) z-testb) t-testc) Chi-square testd) ANOVA
![Page 27: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/27.jpg)
Extrasensory Perception
Based on the available data, how would you test whether belief in ESP differs between 1990 and 2001?
a) z-testb) t-testc) Chi-square testd) ANOVA
![Page 28: Inference after ANOVA, Multiple Comparisons 3/21/12 Inference after ANOVA The problem of multiple comparisons Bonferroni’s Correction Section 8.2 Professor](https://reader031.vdocuments.site/reader031/viewer/2022032415/56649f0a5503460f94c1e647/html5/thumbnails/28.jpg)
Summary
• When performing inference after ANOVA, use √MSE as an estimate for standard deviation within groups, and use n – k as the degrees of freedom for the t-distribution