chapter 9: the analysis of variance for simple experiments (single factor, unrelated groups designs)

Chapter 9: The analysis of variance for simple experiments (single factor, unrelated groups

designs).

Overview of experimental research• Groups start off similar to each other on every possible

measure. • During the experiment, groups are TREATED

DIFFERENTLY• Responses thought to be effected by the different

treatments are then measured• If the group means become more different that they should

be if only random sampling fluctuation were at work, the differences may have been caused, in part, by the different ways the groups were treated.

• Determining whether the differences between group means result simply from sampling fluctuation or are (probably) due in part to the treatment differences is the job of the statistical analysis.

Null hypothesis

• Null hypothesis: The different ways the groups are treated have no effects or all have the same effect.– Corollary: Any differences among the group

means at the end of the experiment continue to reflect the same thing as they did before the experiment, random sampling fluctuation.

Experimental hypothesis

• Experimental hypothesis: The different ways the groups are treated affect the groups differently.– Corollary: At the end of the experiment, the

group means differ too much for their differences to be explained by random sampling fluctuation. Being treated differently causes the groups to become more different. That is, the group means will be further apart.

Groups will always differ simply because of sampling fluctuation caused

by ID and MP• In this course, we will study the analysis of

experiments in which groups treated differently are measured once and compared to each other.

• Groups differ from each other because they are composed of different people (ID) and because there are always measurement problems (MP)

• Just by chance, if there are three groups, one will have the highest mean, another the lowest mean, and the third will fall in the middle.

• Sigma2 as estimated by MSW is our basic measure of the effects of ID & MP.

The logic of the F test

• We are asking whether the difference between the treatment group means at the end of the experiment is solely the result of random sampling fluctuation (H0) or of random sampling fluctuation plus the effects of treating the groups differently (H1) .

• So we need two measures.• One measure should only be able to reflect random

sampling fluctuation.• The other should be able to reflect random sampling

fluctuation plus the effects of the IV.

Our two estimates of sigma2

One way to estimate sigma2 is to find the difference between each score and its group mean, square and sum those differences. This yields a sum of squares within group (SSW). To estimate sigma2 you divide SSW by degrees of freedom within group (dfW=n-k). This estimate of sigma2 is called the mean square within groups, MSW. You have been calculating it since Chapter 5.

The other way to estimate sigma2 is to find the difference between each participant’s group mean and the overall mean. Then you square and sum the differences. This yields a sum of squares between the group and grand means (SSB). To estimate sigma2

you divide SSB by the number of degrees of freedom between groups (dfB=k-1). The result is called the mean square between groups,MSB. It is new.

What is indexed by sigma2 and its best estimate:MSW

• Sigma2 indexes random sampling fluctuation. Its best estimate is MSW. Remember that everyone in a treatment group is treated the same way. Thus, any differences among the scores within each group, the basis of MSW, can only reflect ID + MP.

• Thus MSW can not reflect any effects of the independent variable.

• Because it can only index random individual differences and measurement problems, MSW is always your best estimate of the effects of random sampling fluctuation as indexed by sigma2, the population variance. We will use MSW as the denominator in the F ratio.

What is indexed by the mean square between groups (MSB)?

• Group means will differ from each other and from mu because they are made up of different people and because there are always measurement problems. This is the basis of random sampling fluctuation.

• However, since we treat the groups differently, the distance between each group’s mean and the overall mean can also reflect the effects of the independent variable (as well as the effects of random individual differences and random measurement problems).

What is indexed by the mean square between groups (MSB)?

Thus MSB reflects individual differences, measurement problems, and the possible effect of the independent variable (MSB = ID + MP + (?)IV)

• If treating the groups differently (the IV) has an effect, it will push the group means apart. When that happens MSB reflects three sources of variation (ID + MP + IV), not just ID + MP. In that case, MSB will overestimate sigma2 and be larger than MSW.

• In the single factor analysis of variance (Chapter 9), MSB will serve as the numerator in the F ratio.

Testing the Null Hypothesis (H0)• H0 says that the IV has no effect. • If H0 is true, groups differ from each other and from the

overall mean only because of sampling fluctuation based on random individual differences and measurement problems (ID + MP).

• These are the same things that make scores differ from their own group means.

• So, according to H0, MSB and MSW are two ways of measuring the same thing (ID + MP) and are both good estimates of sigma2.

• Two measurements of the same thing should be about equal to each other and a ratio between them should be about equal to 1.00.

In simple experiments (Ch.9), the ratio between MSB and MSW is

the Fisher or F ratio.

In simple experiments, F=MSB/MSW.

H0 says F should be about 1.00.

The Experimental Hypothesis (H1)• The experimental hypothesis says that the groups’ means will

be made different from each other (pushed apart) by the IV, the independent variable (as well as by random individual differences and measurement problems).

• If the means are pushed apart, MSB will increase, reflecting the effects of the independent variable (as well as of the random factors). MSW will not.

• So MSB will be larger than MSW

• Therefore, H1 suggests that an F ratio comparing MSB to MSW should be larger than 1.00.

As usual, we set up 95% confidence intervals around the prediction of the null.

• In Ch. 9, the ratio MSB/MSW is called the F ratio.• If the F ratio is about 1.00, the prediction of the null is

correct.• It is rare for the F ratio to be exactly 1.00.• At some point, the ratio gets too different from 1.00 to

be consistent with the null. • We are only interested in the case where the ratio is

greater than 1.00 which indicates that the means are further apart than the null suggests.

• The F table tells us when the difference among the means is too large to be explained as reflecting only ID + MP, sampling fluctuation alone.

So, to conduct an F test in a simple experiment

• You compute MSW and MSB.• F= MSB/ MSW

• You compare the results to the F table.• If your ratio is less than the critical value for F found

in the table, your data are nonsignificant, consistent with H0.

• If your ratio equals or exceeds the critical value in the F table, your data are significant. You must reject H0 and extrapolate your results to the population from which your samples were drawn.

REMEMBER, YOU NEVER TEST THE EXPERIMENTAL HYPOTHESIS

STATISTICALLY.• You can only examine the data in light of the

experimental hypothesis after rejecting the null.

• Good research design makes the experimental hypothesis the only reasonable alternative to the null.

• Accepting the experimental hypothesis is based on good research design and logic, not statistical tests.

What comes next

• We need to do two things– Learn to compute the F and t tests– Review the logic we’ve just gone through so

that you fully understand it

We will do them in that order, first analyzing data from typical single factor, unrelated groups experiments. Then we will review the concepts.

Analyzing the results of an experiment

An experiment• Population: Male, self-selected, “social drinkers”• Number of participants (9) and groups (3)• Design: Single factor, unrelated groups• Independent variable: Stress

– Level 1: No Stress: Participants labeled 1.1, 1.2, 1.3– Level 2: Moderate stress: Participants 2.1, 2.2, 2.3– Level 3: High stress: Participants 3.1, 3.2, 3.3

• Dependent variable: ounces alcohol beverage consumed

• H0: Stress does not affect alcohol consumption.• H1: Stress will cause increased alcohol consumption.

Computing MSW and MSB

1.11.21.3

2.12.22.3

3.13.23.3

81012

121215

131718

999

000

999

2MX -3-3-3

000

333

MX 101010

131313

161616

131313

131313

131313

MX404

114

914

2XX -202

-1-12

-312

XX X#S101010

131313

161616

X

101 X

132 X

163 X

13M

28WSS

6Wdf

67.4WMS

54BSS

2Bdf

27BMS

ANOVA summary table

Between GroupsStress level

Within GroupsError

54 2 27 26 6 4.67

5.78

SS df MS F p

?

We need to look at the F table to determine significance.

Divide MSB by MSW

to calculate F.

Using the F tableTo use the F table, you must specify the

degrees of freedom (df) for the numerator and denominator of the F ratio.

In both Ch 9 and Ch 10 the denominator is MSW. As you know, dfW = n-k.

In Ch 9, the numerator is MSB and dfB=k-1.

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49

Degrees of freedom in Numerator 1 2 3 4 5 6 7 8 Df in

denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03

9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47


denominator

36 4.41 3.26 2.86 2.63 2.48 2.36 2.28 2.21 7.39 5.25 4.38 3.89 3.58 3.35 3.18 3.04

40 4.08 3.23 2.84 2.61 2.45 2.34 2.26 2.19 7.08 4.98 4.13 3.65 3.34 3.12 2.95 2.82

60 4.00 3.15 2.76 2.52 2.37 2.25 2.17 2.10 7.08 4.98 4.13 3.65 3.34 3.12 2.95 2.82

100 3.94 3.09 2.70 2.46 2.30 2.19 2.10 2.03 6.90 4.82 3.98 3.51 3.20 2.99 2.82 2.69

400 3.86 3.02 2.62 2.39 2.23 2.12 2.03 1.96 6.70 4.66 3.83 3.36 3.06 2.85 2.69 2.55

3.84 2.99 2.60 2.37 2.21 2.09 2.01 1.94 6.64 4.60 3.78 3.32 3.02 2.80 2.64 2.51

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49


denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03

9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47

These are related to the number of different

treatment groups.They relate to theMean Square

between groups.

k-1

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49


denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03

9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47

These are related to the number of subjects.

They relate to theMean Squarewithin groups.

n-k

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49


denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03

9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47

The critical values in thetop rows are alpha = .05.

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49


denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03

9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47

The critical values in thebottom rows are for

bragging rights (p< .01).

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49


denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03

9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47

In an experiment with 3 treatment groups, we have 2 df between groups (k-1).

If we have 9 subjects, and 3 groups, we have 6df within groups (n-k) . Since this is the ratio of MSB to

MSW, the variance estimate between groups must be 5.14 times larger than the variance estimate within groups.

5.14

If we find an F ratio of 5.14 or larger , we reject the null hypothesis and declare that there is a treatment

effect, significant at the .05 alpha level.

ANOVA summary table

Between GroupsStress level

Within GroupsError

54 2 27 26 6 4.67

5.78

SS df MS F p

?

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49


denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03

9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47

5.1410.92

5.78 is as large or larger than the criticalvalue at the .05 alpha level (5.14).

So it is statistically significant. It does not equal or exceed the critical value for .01

F (2,6)=5.78, p<.05

Critical values in the F table

• The critical values in the F table depend on how good MSB and MSW are as estimates of sigma2.

• The better the estimates, the closer to 1.00 the null must predict that their ratio will fall.

• What makes estimates better??? DEGREES OF FREEDOM. Each degree of freedom corrects the sample statistic back towards its population parameter.

• Thus, the more degrees of freedom for MSW and MSB, the closer the critical value of F will be to 1.00.

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49


denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44 11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.03

9 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47

Notice the usual effect of higher df

The more dfB and dfW, ...

the better our estimates of sigma2, ...

the closer F should be to 1.00 when the null is true.

H0 says that F should be about 1.00.

Remember the pattern of critical values in the F table

If the null is true, as df increase, each mean square becomes a better estimate of sigma2 and the null must predict an F ratio closer and closer to 1.00. Whether an F ratio is significant depends on dfW and dfB as well as on the size of the ratio.

The F test and the t test

Another experiment• Population: Female, moderately depressed outpatients• Number of participants (10) and groups (2)• Design: Single factor, unrelated groups• Independent variable: Dose of new drug, Feelbetter

– Level 1: Placebo (n=5): Participants 1.1 – 1.5– Level 2: Moderate dose (n=5): Participants 2.1 – 2.5

• Dependent variable: HAM-D scores. Lower scores are better.

• H0: Feelbetter does no more good than placebo.

• H1: The average response in the treatment groups will differ more than they would unless Feelbetter effectively helps depression.

The results:

• The five participants who received the placebo scored 18, 21, 22, 30, 24.

• The five participants who received Feelbetter scored 11, 15, 13, 17 and 19.

• Obviously, the Feelbetter group had lower Ham-D scores. But was that attributable to sampling fluctuation or are the groups too dissimilar for that explanation to cover the data?

• We need to compute F, where F = MSB/MSW.

Here is the computation of MSW and MSB: Now you set up an Anova summary table, compute F, determine significance,

and write the results properly

1.11.21.31.41.5

2.12.22.32.42.5

1821223024

1115131719

1616161616

1616161616

2MX 44444

-4-4-4-4-4

MX 2323232323

1515151515

1919191919

1919191919

MX2541

491

16044

16

2XX -5-2-171

-40-224

XX X#S2323232323

1515151515

X

231 X

152 X 00.19M

120WSS8Wdf

00.15WMS

160BSS

1Bdf

160BMS

ANOVA summary table

Between GroupsDrug dose

Within GroupsError

160 1 160.00 120 8 15.00

10.67

SS df MS F p

?

We need to look at the F table to determine significance.

Divide MSB by MSW

to calculate F.

3 10.13 9.55 9.28 9.12 9.01 8.94 8.88 8.84 34.12 30.82 29.46 28.71 28.24 27.91 27.67 27.49


denominator

4 7.71 6.94 6.59 6.39 6.26 6.16 6.09 6.04 21.20 18.00 16.69 15.98 15.52 15.21 14.98 14.80

5 6.61 5.79 5.41 5.19 5.05 4.95 4.88 4.82 16.26 13.27 12.06 11.39 10.97 10.67 10.45 10.27

6 5.99 5.14 4.76 4.53 4.39 4.28 4.21 4.15 13.74 10.92 9.78 9.15 8.75 8.47 8.26 8.10

7 5.59 4.74 4.35 4.12 3.97 3.87 3.79 3.73 12.25 9.55 8.45 7.85 7.46 7.19 7.00 6.84

8 5.32 4.46 4.07 3.84 3.69 3.58 3.50 3.44

11.26 8.65 7.59 7.01 6.63 6.37 6.19 6.039 5.12 4.26 3.86 3.63 3.48 3.37 3.29 3.23 10.56 8.02 6.99 6.42 6.06 5.80 5.62 5.47

ANOVA summary table


Within GroupsError

160 1 160.00 120 8 15.00

10.67

SS df MS F p

.05

F(1,8)=10.67, p<.05.

Now let us use the same data to do a t test.

(t for 2, F for more). One way to do a t test is to compare estimates of sigma (rather than of sigma2). So,

we take the square roots of MSB and MSW to get sB and s, find their ratio

and compare it to the t table.

t test summary table


Within GroupsError

160 1 12.65 120 8 3.87

3.226

SS df s t p

?

We need to look at the t table to determine significance.

df 1 2 3 4 5 6 7 8

.05 12.706 4.303 3.182 2.776 2.571 2.447 2.365 2.306

.01 63.657 9.925 5.841 4.604 4.032 3.707 3.499 3.355

df 9 10 11 12 13 14 15 16.05 2.262 2.228 2.201 2.179 2.160 2.145 2.131 2.120.01 3.250 3.169 3.106 3.055 3.012 2.997 2.947 2.921

df 17 18 19 20 21 22 23 24.05 2.110 2.101 2.093 2.086 2.080 2.074 2.069 2.064.01 2.898 2.878 2.861 2.845 2.831 2.819 2.807 2.797

df 25 26 27 28 29 30 40 60.05 2.060 2.056 2.052 2.048 2.045 2.042 2.021 2.000.01 2.787 2.779 2.771 2.763 2.756 2.750 2.704 2.660

df 100 200 500 1000 2000 10000.05 1.984 1.972 1.965 1.962 1.961 1.960.01 2.626 2.601 2.586 2.581 2.578 2.576

Example 2 – t test summary table


Within GroupsError

160 1 12.65 120 8 3.87

3.266

SS df s t p

.05

With two groups, t = the square root of F.F and t always give you

the same level of significance.

t(8)=3.266, p<.05.

The t Test: t for 2, F for More

• The t test is a special case of the F ratio.

• If there are only two levels (groups) of the independent variable, then

Ft W

B

MS

MS

s

sB

t table and F table

when there are only two groups:

df in the numerator is always 1

Bdf 1k 12 1

Relationship between t and F tables

• Because there is always 1 df between groups, the t table is organized only by degrees of freedom within group (dfW).

• By the way, the values in the t table are the square root of the values in the first column of the F table.

df 1 2 3 4 5 6 7 8.05 12.706 4.303 3.182 2.776 2.571 2.447 2.365 2.306.01 63.657 9.925 5.841 4.604 4.032 3.707 3.499 3.355

df 9 10 11 12 13 14 15 16.05 2.262 2.228 2.201 2.179 2.160 2.145 2.131 2.120.01 3.250 3.169 3.106 3.055 3.012 2.997 2.947 2.921

df 17 18 19 20 21 22 23 24.05 2.110 2.101 2.093 2.086 2.080 2.074 2.069 2.064.01 2.898 2.878 2.861 2.845 2.831 2.819 2.807 2.797

df 25 26 27 28 29 30 40 60.05 2.060 2.056 2.052 2.048 2.045 2.042 2.021 2.000.01 2.787 2.779 2.771 2.763 2.756 2.750 2.704 2.660

df 100 200 500 1000 2000 10000.05 1.984 1.972 1.965 1.962 1.961 1.960.01 2.626 2.601 2.586 2.581 2.578 2.576

t tablefrom Chapter 6

Let’s look atthese values.

df 3 4 5.05 3.182 2.776 2.571.01 5.841 4.604 4.032

t tablefrom Chapter 6

F table.

3 10.13 34.12

4 7.71 21.20

5 6.61 16.26

1

Ft

2tF

Back to the basic concepts

It would be good to understand the logic behind the F test well enough so you could explain how the test is created by the logic of the situation. So we will review the concepts now.

Let’s take that one point at a time. At the beginning of an experiment:

• Participants are randomly selected from a population. Then they are randomly assigned to treatment groups.

• Thus, at the beginning of the study, each treatment group is a random (sub)sample from a specific population.

Groups start off much the same in every possible way

• Since each treatment group is a random sample from the population, each group’s mean and variance will be similar to that of the population.

• That is, each group’s mean will be a best estimate of mu, the population mean.

• And the spread of scores around each group’s mean will yield a best estimate of sigma2 and sigma.

So: At the beginning of an experiment the treatment groups differ only because of random sampling fluctuation.When there are different people in each group, the random sampling fluctuation is caused by 1.) random individual differences and 2.) random measurement problems.

Sampling fluctuation is the product of the inherent variability of the data.

That is what is indexed by sigma2, the average squared distance of scores

from the population mean, mu.

To summarize:• Since the group means and variances of random

samples will be similar to that of the population, they will be similar to each other.

• This is true for any and all things you can measure.

• The only differences among the groups at the beginning of the study on any and all measures will be the (mostly minor) differences associated with random sampling fluctuation caused by the fact that there are different people in each group and that there are always random measurement problems (ID + MP).

The ultimate question

• If we then treat the groups differently, will the treatments make the groups more different from each other at the end of the experiment than if only sampling fluctuation created their differences?

In the simplest experiments - (Ch 9)

• In the simplest experiments, the groups are exposed to treatments that vary on a single dimension.

• The dimension on which treatments of the groups vary is called the independent variable.

• We call the specific ways the groups are treated the “levels of the independent variable.”

The independent variable• An independent variable can be any preplanned difference in

the way groups are treated. Which kind of difference you chose relates to the experimental hypothesis, H1.

• For example, if you think you have a new medication for bipolar disorder, you would compare the effect of various doses of the new drug to placebo in a random sample of bipolar patients. Thus, the groups would differ in terms of the dose of drug.

• Proper experimental design would ensure that the differences in dose received is the only way the groups will be systematically treated differently from each other.

Why is it called the “independent variable”?Remember, we call the different treatments the “levels”

of the independent variable.

Who gets which level is random. It is determined solely by the group to which a participant is randomly assigned.

• So, any difference in the way a person is treated during the experiment is unrelated to or “independent of” the infinite number of pre-existing differences that precluded causal statements in correlational research.

The dependent variable

• Relevant responses (called dependent variables) are then measured to see whether the independent variable caused differences among the treatment conditions beyond those expected given ordinary sampling fluctuation.

• That is, we want to see whether response are related to (dependent on) the different levels of the independent variable to which the treatment groups were exposed.

Differences after the experiment among group means on the dependent variable

may well be simple sampling fluctuation!• The groups will always differ somewhat from each other

on anything you measure due to sampling fluctuation.• With 3 groups, one will score highest, one lowest and one

in the middle just by chance. In four groups one will score highest, one lowest, with two in the middle, one higher than the other. Etc.

• So the simple fact that the groups differ somewhat is not enough to determine that the independent variable, the different ways the groups were treated, caused the differences.

• We have to determine whether the groups are more different than they should be if only sampling fluctuation is at work.

H0 & H1: If one is wrong, the other must be right.

• Either the independent variable would cause differences in responses (the dependent variable) in the population as a whole or it would not.

• H0: The different conditions embodied by the independent variable would have NO EFFECT if administered to the whole population.

• H1: The different conditions embodied by the independent variable would produce different responses if administered to the whole population

H0 & H1 AGAIN: If one is wrong, the other must be right.

• Either the independent variable would cause differences in responses (the dependent variable) in the population as a whole or it would not.

H0: The different conditions embodied by the levels of the independent variable would have the same effects as each other on the whole population. Any differences among the means of the treatment groups at the end of the experiment is purely the result of random sampling fluctuation.

H1: The different conditions embodied by the levels of the independent variable would produce different responses, if administered to the whole population.

Generalizing from samples to the population they represent

The population can be expected to respond to the different levels of the IV similarly to the samples• Remember, random samples are

representative of the population from which they are drawn.

• If the different levels of the independent variable cause the groups to differ (more than they would from simple sampling fluctuation), the same thing should be true for the rest of the population.

For example:

• Say a new psychotherapy causes a random sample of anxious patients to become less anxious in comparison to treatment groups given more conventional approaches or pill placebo.

• Then, we would expect all anxious patients to respond better to the new treatment than to the ones to which it was compared.

However!

• As in the case of correlation, we don’t want to toss out treatments that we know work JUST because the new treatment happens to do better in an experiment.

• We would want to be sure that any differences among the treatment groups is not just a chance finding based on random sampling fluctuation.

The Null Hypothesis

• The null hypothesis (H0) states that the only reason that the treatment group means are different is sampling fluctuation. It says that the independent variable causes no systematic differences among the groups.

• A corollary: Try the experiment again and a different group will score highest, another lowest. If that is so, you should not generalize from which group in your study scored highest or lowest to the population from which the samples were drawn.

• Your best prediction remains that everyone will score at the mean on the dependent variable, that treatment condition will not predict response.

THIS QUESTION IS IMPORTANT!

If the null hypothesis (H0) is true, what is the relationship between the way the treatment groups respond to different levels of the IV and the way the whole population can be expected

to respond to different levels of the independent variable?

THIS QUESTION IS IMPORTANT!

IF H0 is TRUE, the way the treatment groups respond to the

independent variable has nothing to do with the way the population can be expected to

respond.

Notice that people often respond to random sampling fluctuation as if

something was causing a difference.

• People take all kinds of food supplements because they believe the supplements (e.g.. Echinacea) will make colds go away more quickly.

• If you tried it, and it worked wouldn’t you tell your friends? Wouldn’t you try it again with your next cold?

• Having recovered quickly after taking something provides the evidence. After all, its what happened to you!

But did the food supplement really make a difference?

• When carefully tested, NO food supplement has proved to shorten colds.

• The mistake lies in taking random variation in the duration of a cold as evidence that the Echinacea (or whatever) did something beneficial.

• That’s ok if it is just your pocket book that is affected. But what if you were an FDA scientist? Wouldn’t people expect better evidence of efficacy before they gave the food supplement company an enormous amount of their money?

Another example of what happens if no one tests the null.

• Drugs have to be tested by the FDA for safety and efficacy. They must be shown to be safe and better than placebo.

• Food supplements don’t.• NEITHER DOES SURGERY.• New operations require only that a qualified

surgeon and a hospital board think a new procedure is PLAUSIBLE. So, the surgeon must just persuade a hospital board that an operation should be beneficial for it to be employed..

Angioplasty as a placebo!

• Heart attacks occur when the blood vessels that feed the heart with oxygen and fuel are blocked by cholesterol based plaque that forms on the artery walls.

• As people age, the coronary arteries often narrow by the plague on the artery walls.

• So surgeons open up the coronary arteries with tiny balloons and keep them open with a stint, a little mesh screen. This year, over 900,000 Americans will have stints implanted in a procedure known as angioplasty.

You must ask, again and again, what is the evidence.

• Implanting stints causes about 4% of people to have minor heart attacks.

• That’s over 35,000 heart injuring operations this year.

• No one ever proved that implanting lowered the risk of heart attacks. It just made sense.

You must ask, again and again, what is the evidence? Did they test the null

hypothesis?It’s almost certainly not true. The places that the

plague breaks off and really blocks blood vessels is in the areas that are still soft, not yet firmly calcified.

• In calcified areas, the plague just sits there, narrowing the vessels but doing little harm.

• The stints are placed in the calcified areas. That solves nothing.

• By the way, drastically lowering blood cholesterol with drugs called “statins” seemingly works a lot better.

Comparisons with proper control groups are the answer.

• But, as you will learn when you study experimental design, when real world problems are studied, suitable control groups are not the easiest thing in the world to obtain for a variety of both scientific and political reasons.

We call rejecting a true null hypothesis a “Type 1 Error.”

• The first rule in science is “Do not increase error.”• Scientists don’t like to say something will make a

difference when it isn’t true.• So, before we toss away proven treatments,

approve a new treatment, or say that something will cause illness or health, we want to be fairly sure that we are not just responding to sampling fluctuation.

The scientist’s answer: test the null hypothesis

• So, as we did with correlation and regression, we assume that everything is equal, all treatments have the same effect, unless we can prove otherwise.

• The null hypothesis says that the treatments do not systematically differ; one is as good as another.

• As usual, we test the null hypothesis by asking it to make a prediction and then establishing a range of results for the test statistic consistent with that prediction.

• As usual, that range is a 95% CI for the test statistic.

chapter 9: the analysis of variance for simple experiments (single factor, unrelated groups designs)

Documents