1 paired differences paired difference experiments 1.rationale for using a paired groups design...
TRANSCRIPT
1
Paired Differences
Paired Difference Experiments
1. Rationale for using a paired groups design
2. The paired groups design
3. A problem
4. Two distinct ways to estimate μ1 – μ2
5. Formal statement – large sample test
6. Formal statement – small sample test
7. Examples
2
Paired Differences
Rationale for using a paired groups design
The basic problem:
When you measure two samples of cases that have been treated differently, the differences between the two resulting sets of scores will be produced by either or both of two types of effect:* the treatment* everything else that matters
3
Paired Differences
Rationale for using a paired groups design
We’re not interested in the effect on performance of “everything else that matters”. We want to know whether the treatment effect is real.
But suppose that the variability due to “everything else that matters” is much larger than the variability due to the treatment. In that case, we may not be able to detect the signal (treatment effect) because of the noise (error variability – that is, variability due to everything else that matters).
4
Paired Differences
Rationale for using a paired groups design
We have to do something to reduce that part of the difference between the groups that is due to “everything else that matters.”
We do this by matching or testing the same people twice. Both approaches remove the effects of nuisance variables.
5
Paired Differences
Rationale for using a paired groups design
* If the two samples of cases are more alike in things that matter, then the contribution of the treatment to any difference between the means is proportionally larger.* That is, if the contribution of the treatment (to the difference between group means) stays the same, but the contribution of other differences between the groups goes down, then we have a more sensitive test.
6
Paired Differences
Total variability in the data set
Variability due to the treatment effect
Variability due to all other causes
Here, most of the variability in the data set is produced by things other than the treatment effect.
7
Paired Differences
Total variability in the data set
Numerator of Z or t-test
Denominator of Z or t-test
8
Paired Differences
Total variability in the data set
Variability due to the treatment effect
Variability due to all other causes
Here, variability due to treatment effect is the same, but variability due to other causes has decreased.
9
Paired Differences
Total variability in the data set
Numerator of Z or t-test
Denominator of Z or t-test
10
Paired Differences
The paired groups design
One way to reduce variability due to “everything else that matters” is to use the paired groups design.
* Matched pairs: select people in pairs matched on some relevant variable (e.g., IQ), then randomly assign one to each condition.* Repeated measures: every person gets both treatments, so acts as their own control.
11
Paired Differences
The paired groups design
Suppose that for each person with IQ = 110 in the treatment condition we have a person with IQ = 110 in the control condition. Similarly with all other IQs represented in the treatment condition – each has a matched-IQ case in the control condition.
* Now, if we subtract score for one member of pair from score for other member, the effect of IQ cannot contribute to that difference.
12
Paired Differences
A problem
When we match pairs or used repeated measures on the same people, we violate one of the assumptions of the independent groups tests of difference between two population means* The statistical test is based on the assumption that the observations in one group are independent of the observations in the other group.
13
Paired Differences
A problem
Why is that assumption a problem?
* Because here, once we have selected Group 1, we do not then independently select Group 2.* As a direct result the sample mean difference X1 – X2 is not a good estimator of the population mean difference μ1 – μ2.
14
Paired Differences
Two distinct ways to estimate μ1 – μ2
1. Choose a random sample from Population A. Independently choose a random sample from Population B. Compute the means for each sample and find the difference between these means.
2. Choose a random sample from Population A and a matching sample from Population B. Find the difference between each score in sample A and its matched score in sample B. Compute the mean of these differences.
15
Paired Differences
Two distinct ways to estimate μ1 – μ2
In the first case, we work with a sampling distribution based on differences between independent sample means.
* Anything that could make one sample mean different from the other will contribute to the variability (σX1-X2
) of that sampling distribution.
* With a more variable sampling distribution, we need a larger sample difference to be confident that the inferred population difference is real.
16
Paired Differences
Two distinct ways to estimate μ1 – μ2
In the second case, because of matching, many of the (random) things that could drive sample means apart are eliminated from the differences X1i – X2i.
* As a result, the variability in the sampling distribution (σD) has fewer sources.* So we can infer a real population difference with a smaller sample difference (samples are less likely to be different just by chance).
17
Paired Differences
Paired groups test: large samples
HO: μD = DO HO: μD = DO
HA: μD < DO HA: μD ≠ DO
or μD > DO
(DO = historical value of the difference between the population means.)
Test statistic: Z = XD – DO
σD ∕√nD
18
Paired Differences
Paired groups test: large samples
Rejection region:
Z < –Zα │Z│> Zα/2
or Z > Zα
Assumptions: 1. Distribution of differences is normal. 2. Difference scores are randomly selected from the population of differences (between matched pairs or repeated measures).
19
Paired Differences
Important note
Z = XD – DO
σD ∕√nD
Notice that the numerator does not have an X1 and an X2. It just has XD.
* We begin by finding the differences between each pair of observations. From then on, we work only with these difference scores.
20
Paired Differences
Paired groups test, small samples
HO: μD = DO HO: μD = DO
HA: μD < DO HA: μD ≠ DO
or μD > DO
(DO = historical value of the difference between the population means.)
Test statistic: t = XD – DO
sD ∕√nD
21
Paired Differences
Paired groups test – small samples
Rejection region:
t < –tα │t│> tα/2
or t > tα
Assumptions: 1. Distribution of differences is normal. 2. Difference scores are randomly selected from the population of differences (between matched pairs or repeated measures).
22
Paired Differences
Example 1
A psychologist was studying the effectiveness of several treatments to help people quit smoking. In one treatment, smokers heard a lecture about the effects of cigarette smoke on the human body, accompanied by graphic slides of those effects. In the other treatment, smokers had daily phone conversations with a therapist who encouraged them not to smoke that day. To control for effects of age and sex, the psychologist assigned people to the experimental groups in pairs matched on those variables. The data, in the form of number of hours without a cigarette appear on the next slide
23
Paired Differences
Example 1
Pair L+S DE Diff Diff2
1 105 122 17 289
2 98 86 -12 144
3 121 127 6 36
4 99 92 -7 49
5 65 85 20 400
6 130 152 22 484
7 108 92 -16 256
8 57 63 6 36
∑ = 36 ∑ = 1694
Notice the negative signs!!
Without negative signs, Σ would be 106
24
Paired Differences
Example 1
sD2 = 1694 – (36)2 = 218.857
8
7
sD = √218.857 = 14.79
25
Paired Differences
Example 1
HO: μD = DO
HA: μD ≠ DO
Test statistic: t = XD – DO
sD ∕√nD
26
Paired Differences
Example 1
Rejection region: tcrit = t7,.025 = 2.365
tobt = 4.5 – 0 = 4.5 = 0.861
14.79/√8 5.229
Decision: do not reject HO – no evidence treatment effects differ
27
Paired Differences
Example 2
Tetris is a computer game requiring some spatial information-processing skills and good eye-hand coordination, either or both of which may improve with practice. Six people who had never previously played Tetris were tested on the game at the beginning (Test 1) and at the end (Test 2) of a 2-week period during which they played Tetris for one hour each day. Their Tetris scores on the two testing sessions appear on the next slide.
28
Example 2
a. Did the subjects’ Tetris scores improve significantly from Test 1 to Test 2 (α = .05)?
b. Is the variance of the subjects’ Test 2 scores significantly different from 400,000, the variance among the population of experts at Tetris (α = .05)?
Paired Differences
29
Paired Differences
Example 2a
Subject Test 1 Test 2 Diff D2
1 3025 5642 2617 68486892 4120 5117 997 9940093 2675 4333 1658 27489644 6715 6026 -689 4747215 1997 5429 3432 117786246 4807 4807 0D ∑= 8015
D2 ∑=22845007 SD = 1558.095
30
Paired Differences
Example 2a
Rejection region: tcrit = t5,.05 = 2.015
tobt = 1335.83 – 0 = 2.100
1558.095/√8
Decision: Reject HO – scores improved significantly