hypothesis testing: p-value 2/13/12 randomization distribution p-value statistical significance...
Post on 28-Dec-2015
230 Views
Preview:
TRANSCRIPT
Hypothesis Testing: p-value2/13/12
• Randomization distribution• p-value• Statistical significance
Section 4.2 Professor Kari Lock MorganDuke University
• Project 1 Proposal (due Wednesday)
• Homework 4 (due Monday)NO LATE HOMEWORK ACCEPTED!
To Do
In the actual experiment, the people who exercised for 5 seconds had an average pulse of 85.5. Those who did not exercise had an average pulse of 69.6.
Is this sample difference larger than we would see, just by random chance, if exercising for 5 seconds did not increase pulse rate?
Exercise and Pulse
85.5 69.6 15.9E NEX X
Exercise and Pulse
www.lock5stat.com/statkey/ p-value
If 5 seconds of exercise does not increase pulse rate, we would see a sample difference as extreme as 15.9 in only 0.002 of all such experiments.
• In a randomized experiment on treating cocaine addiction, 48 people were randomly assigned to take either Desipramine (a new drug), or Lithium (an existing drug), and then followed to see who relapsed
• Question of interest: Is Desipramine better than Lithium at treating cocaine addiction?
Cocaine Addiction
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R
R R R R R R
R R R R R R
R R R R R R
R R R R
R R R R R R
R R R R R R
R R R R R R
Desipramine Lithium
1. Randomly assign units to treatment groups
R R R R
R R R R R R
R R R R R R
N N N N N N
RRR R R R
R R R R N N
N N N N N N
RR
N N N N N N
R = RelapseN = No Relapse
R R R R
R R R R R R
R R R R R R
N N N N N N
RRR R R R
R R R R RR
R R N N N N
RR
N N N N N N
2. Conduct experiment
3. Observe relapse counts in each group
LithiumDesipramine
10 relapse, 14 no relapse 18 relapse, 6 no relapse
1. Randomly assign units to treatment groups
10 18
24
ˆ ˆ
24.333
D Lp p
• Two options:1. H0 is true (the drugs cause the same proportion of
relapses) 2. Ha is true (Desipramine causes a smaller
proportion of relapses than Lithium)
• In situation (1), how would you explain the observed difference in the proportion of relapses?
• How can we see whether this is a plausible explanation?
Cocaine Addiction
To see if a statistic provides evidence against H0, we need to
see what kind of sample statistics we would observe,
just by random chance, if H0 were true
Measuring Evidence against H0
• Assume the null hypothesis is true
• Simulate new randomizations
• For each, calculate the statistic of interest
• Find the proportion of these simulated statistics that are as extreme as your observed statistic
Randomization Test
R R R R
R R R R R R
R R R R R R
N N N N N N
RRR R R R
R R R R N N
N N N N N N
RR
N N N N N N
10 relapse, 14 no relapse 18 relapse, 6 no relapse
R R R R R R
R R R R N N
N N N N N N
N N N N N N
R R R R R R
R R R R R R
R R R R R R
N N N N N N
R N R N
R R R R R R
R N R R R N
R N N N R R
N N N R
N R R N N N
N R N R R N
R N R R R R
Simulate another randomization
Desipramine Lithium
16 relapse, 8 no relapse 12 relapse, 12 no relapse
ˆ ˆ16 12
24 240.167
LDp p
R R R R
R R R R R R
R R R R R R
N N N N N N
RRR R R R
R N R R N N
R R N R N R
RR
R N R N R R
Simulate another randomization
Desipramine Lithium
17 relapse, 7 no relapse 11 relapse, 13 no relapse
ˆ ˆ17 11
24 240.250
D Lp p
www.lock5stat.com/statkey
Cocaine Addiction
p-valueProportion as extreme as observed statistic
observed statistic
The probability of getting a sample difference in proportions as low as -0.33 just by random chance, if the drugs really are equally effective, is 0.02
• Based on a randomization distribution, the p-value is the proportion of statistics that are more extreme than that observed
• This is the area in the tail(s) beyond the observed statistic in the randomization distribution
• Which tail(s) to include depends on the alternative hypothesis
p-value
• A one-sided alternative contains either > or < • A two-sided alternative contains ≠
• The alternative hypothesis depends on the research question of interest
• For a one-sided alternative, the p-value is the proportion in the tail specified by Ha
• For a two-sided alternative, the p-value is twice the proportion in the smallest tail
Alternative Hypothesis
• Students were given words to memorize, and then randomly assigned to either take a 90 min nap, or to stay awake and take a caffeine pill. 2 ½ hours later, all students were tested on their recall ability. Is sleep or caffeine better for memory?
Sleep or Caffeine for Memory?
Mednick, Cai, Kanady, and Drummond (2008). “Comparing the benefits of caffeine, naps and placebo on verbal, motor and perceptual memory,” Behavioral Brain Research, 193, 79-86.
Mean nMea
umbn numb
er of er of words
words recallrecalled if student slee
ed if student consumes caffeinepsS
C
0 : 0: 0
S C
a S CHH
3S CX X
How extreme would this be if H0 were true???
www.lock5stat.com/statkey
Sleep or Caffeine for Memory?
Actual experiment: 3S CX X
p-value = 2 × 0.022
= 0.044
If sleep and caffeine are equally effective for memory, we would get a sample difference in means as extreme as 3 in about 0.044 of all experiments.
0 when trueS C HX X
• The Center for Disease Control (CDC) conducted a randomized trial in South Africa in which half of women in labor are randomly assigned to be treated with a wipe containing chlorohexidine, and the other half with a sterile wipe (control)
Infections in Childbirth
Source: Eriksen, Sweeten, Blanco (1997). “Chlorohexidine vs Sterile Vaginal Wash During Labor to Prevent Peripartum Infection,” American Journal of Obstetrics and Gynecology, 176:426-430.
0
:
:
0
0C S
a C S
H
H pp
p p
: Proportion of infection among women receiving chlorohexidine wipe
: Proportion of infection among women receiving sterile wipeS
Cp
p
0. 07ˆ 0ˆC Sp p
What can you conclude about the p-value?
a) p-value < 0.5b) p-value > 0.5c) Nothing
Infections in Childbirth
0
:
:
0
0C S
a C S
H
H pp
p p
0. 07ˆ 0ˆC Sp p
• The p-value is the probability of getting results are extreme as your sample statistic, if the null hypothesis is true
• “As extreme as” is defined in the direction of the alternative hypothesis. (for two-sided alternatives, consider both tails)
• If your sample statistic does not support your alternative hypothesis, there is no point in going through the test!
Alternative Hypothesis
What can you conclude about the p-value?
a) p-value = 0b) p-value = 0.5c) p-value = 1d) Nothing
p-value
1
1 2
0 2:
0
0
:a
H
H
21 0X X
What can you conclude about the p-value?
a) p-value < 0.5b) p-value > 0.5c) Nothing
p-value
1
1 2
0 2:
0
0
:a
H
H
21 0.2X X
• It is believed that sunlight offers some protection against multiple sclerosis, but the reason is unknown
• To find out, researchers randomly assigned mice to one of three treatments:• Control (nothing)• Vitamin D Supplements• UV Light
• All mice were injected with proteins known to induce a mouse form of MS, and they observed which mice got MS
Multiple Sclerosis and Sunlight
Seppa, Nathan. “Sunlight may cut MS risk by itself”, Science News, April 24, 2010 pg 9, reporting on a study appearing March 22, 2010 in the Proceedings of the National Academy of Science.
• In testing whether UV light provides protection against MS in mice, what are the null and alternative hypotheses?
pUV = proportion of mice exposed to UV light that get MSpC = proportion of mice not exposed to UV light that get MS
a) H0 : pUV – pC > 0, Ha: pUV – pC = 0b) H0 : pUV – pC < 0, Ha: pUV – pC = 0c) H0 : pUV – pC = 0, Ha: pUV – pC > 0d) H0 : pUV – pC = 0, Ha: pUV – pC < 0
Multiple Sclerosis and Sunlight
• In testing whether UV light provides protection against MS in mice, the experiment yielded a p-value of 0.002. What would you conclude?
a)H0 is probably not true UV light does provide protection against MSb)H0 is probably not true UV light does not provide protection against MSc)Ha is probably not true UV light does provide protection against MSd)Ha is probably not true UV light does not provide protection against MSe)Nothing
Multiple Sclerosis and Sunlight
• In testing whether Vitamin D provides protection against MS in mice, the experiment yielded a p-value of 0.47. What would you conclude?
a) H0 is probably not true Vitamin D does provide protection against MSb) H0 is probably not true Vitamin D does not provide protection against MSc) Ha is probably not true Vitamin D does provide protection against MSd) Ha is probably not true Vitamin D does not provide protection against MSe) Nothing
Multiple Sclerosis and Sunlight
• The p-value is the probability of getting results as extreme as those observed, if the null hypothesis is true
• The p-value measures evidence against the null hypothesis
Strength of Evidence
p-value
The smaller the p-value, the stronger the evidence against Ho.
The smaller the p-value, the stronger the evidence against Ho.
The smaller the p-value, the stronger the evidence against Ho.
p-value
• If the p-value is small enough, we reject the null hypothesis, in favor of the alternative hypothesis
Hypothesis Testing
How small is small enough?
• The significance level, , is the threshold below which the p-value is deemed small enough to reject the null hypothesis
• If the p-value is less than , the results are statistically significant, and we reject the null hypothesis in favor of the alternative
Statistical Significance
Statistical ConclusionsStrength of evidence against H0:
Formal decision of hypothesis test, based on = 0.05 :
statistically significant
not statistically significant
A formal hypothesis test has only two possible conclusions:
1. The p-value is small: reject the null hypothesis in favor of the alternative
2. The p-value is not small: do not reject the null hypothesis
Formal Decisions
• Example:H0 : X is an elephantHa : X is not an
elephant
Would you conclude, if you get the following data?• X has four legs• X walks on two legs
Elephant Example
• A randomization distribution shows the distribution of statistics that would be observed if H0 were true
• A p-value is the probability of getting a statistic as extreme as that observed, if H0 is true
• The p-value measures the strength of evidence against the null hypothesis
• Results are statistically significant if the p-value is less than the significance level, α
• In making formal decisions, reject H0 if the p-value is less than α, otherwise do not reject H0
Summary
top related