Factorial ANOVA

Two-factor, crossed,additive models

1

Applied Stats Algorithm

No Unacceptable

Numerical Categorical

Predictor(s)

CategoricalNumerical Both

1 Factor 2+ Factors

Today

Scientificquestion?

Classify Study

Response Variable Multi-Var

Univariate

Fixed Effects Random Effects

Censored Complete

2

Experiment Description One (numerical) response variable Dependent, Outcome

Two (categorical) independent variables Treatments, Predictors, Explanatory

If the independent variables are both factors and are crossed, called a Factorial Design If there are observations at each treatment

combination, called a complete design There are also incomplete and fractional factorial

designs3

Farming Example(Factorial setup) Suppose we continue with the farming example 16 observations of crop yield (Y) 4 fertilizers (Factor A) with levels {𝑎𝑎1, 𝑎𝑎2, 𝑎𝑎3, 𝑎𝑎4} 2 types of crop (Factor B) with levels {𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐,𝑠𝑠𝑐𝑐𝑠𝑠𝑠𝑠𝑠𝑠𝑎𝑎𝑐𝑐}

There are 4 � 2 = 8 different treatments, each with 2 replications

This is a 4 x 2 factorial experiment

4

Single-Factor setup Why can’t we just label the treatments as a

single factor and analyze it as before? We can Factor AB with 8 levels:

{𝑎𝑎1 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐, 𝑎𝑎1 𝑠𝑠𝑐𝑐𝑠𝑠𝑠𝑠𝑠𝑠𝑎𝑎𝑐𝑐, 𝑎𝑎2 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐, … , 𝑎𝑎4 𝑠𝑠𝑐𝑐𝑠𝑠𝑠𝑠𝑠𝑠𝑎𝑎𝑐𝑐}y <- c(65, 64, 56, 60, 55, 58, 62, 65, 66, 69, 72, 76, 60, 64, 68, 70)

a <- factor(rep(c("a1", "a2", "a3", "a4"), each=4))

b <- factor(rep(c("corn", "soybean"), times=8))

farm <- data.frame(yield=y, fertil=a, type=b)

farm <- within(farm, x <- paste(fertil, type))

# farm$x <- paste(farm$fertil, farm$type)with(farm, tapply(yield, x, mean))

boxplot(yield ~ x, data=farm)

anova(lm(yield ~ x, data=farm))5

R output

6

R output> with(farm, tapply(yield, x, mean))

a1 corn a1 soybean a2 corn a2 soybean a3 corn a3 soybean a4 corn a4 soybean

60.5 62.0 58.5 61.5 69.0 72.5 64.0 67.0

> anova(lm(yield ~ x, data=farm))Analysis of Variance Table

Response: yield

Df Sum Sq Mean Sq F value Pr(>F)

x 7 315.75 43.857 1.8992 0.194

Residuals 8 190.00 23.750

No significant effects detected, despite the fact that we already know fertilizer is important Effect hidden by non-importance of crop type

7

Two-factor ANOVA Single-factor analysis doesn’t give us the effects

separated by the two factors, which we want It is also impossible to spot interactions with only

one factor Think of an 𝑚𝑚 𝑥𝑥 𝑐𝑐 factorial design as a series of 𝑐𝑐 single-factor experiments, each with 𝑚𝑚 groups Or a series of 𝑚𝑚 experiments with 𝑐𝑐 groups

Define simple effects as the results of one of those 𝑐𝑐 (or 𝑚𝑚) experiments

8

Farming example We have 4 fertilizers, and 2 types of crop For the 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 crops, we have a simple effect of

fertilizer on crop yield For the 𝑠𝑠𝑐𝑐𝑠𝑠𝑠𝑠𝑠𝑠𝑎𝑎𝑐𝑐 crops, we have another simple

effect of fertilizer on crop yield These two simple effects, averaged together,

are called the main effect of fertilizer If the simple effects are the same as the main

effect, then there is no interaction present Otherwise, there is an interaction

9

Interactions An interaction means “it depends” The effect of Factor A depends on the level of

Factor B The effect of Factor B depends on the level of

Factor A Important to recognize before setting model If present, the statistical model will change

10

Farming Example(Interaction) Do we believe that the effect of fertilizer depends

on what crop type is planted? This is like saying “Fertilizer 𝑎𝑎1 is best for corn,

but 𝑎𝑎3 is best for soybeans” It’s very possible!

If so, cannot say something like “Fertilizer 𝑎𝑎1 is better than 𝑎𝑎3” because … it depends!

Just check the group means to find out Such a plot is called an interaction plot

11

Plot the group means

with(farm, interaction.plot(fertil, type, y, col=c("red", "blue"),

main="Interaction Plot", xlab="Fertilizer mean", ylab="Yield"))

12

Another way to do it

with(farm, interaction.plot(type, fertil, y, col=c("red", "blue"),

main="Interaction Plot", xlab="Fertilizer mean", ylab="Yield"))

13

Interactions Other, equivalent definitions of interaction The values of one or more contrasts change at

different levels of the other factor The main effect is not representative of the simple

effects The differences among cell means representing

effect of Factor A at one level of Factor B are not the same as at another level of Factor B

The effects of one factor are conditionally related to the levels of another factor

14

Additive Model It doesn’t look like the effect of Fertilizer

depends on crop type, so fit an additive model𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝛼𝛼𝑖𝑖 + 𝛽𝛽𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖

𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 – 𝑖𝑖𝑡𝑡𝑡 obs. from 𝑗𝑗𝑡𝑡𝑡 level of Factor A and 𝑘𝑘𝑡𝑡𝑡 level of Factor B𝜇𝜇 – overall grand mean𝛼𝛼𝑖𝑖 – deviation from mean of 𝑗𝑗𝑡𝑡𝑡 group of Factor A𝛽𝛽𝑖𝑖 – deviation from mean of 𝑘𝑘𝑡𝑡𝑡 group of Factor B𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 – error/noise/uncertainty We will study models with interactions later

15

Additive Models An additive model means that the ‘effects’ are

added together There is an effect of being in group 𝑗𝑗 of Factor A If you’re in group 𝑗𝑗, you get 𝛼𝛼𝑖𝑖 added to your

expected outcome There is an effect of being in group 𝑘𝑘 of Factor B If you’re in group 𝑘𝑘, you get 𝛽𝛽𝑖𝑖 added to your

expected outcome If you’re in both groups, you get 𝛼𝛼𝑖𝑖 + 𝛽𝛽𝑖𝑖 added to

your expected outcome16

R output> with(farm, tapply(yield, list(fertil, type), mean))

corn soybean

a1 60.5 62.0

a2 58.5 61.5

a3 69.0 72.5

a4 64.0 67.0

> anova(lm(yield ~ fertil + type, data=farm))Analysis of Variance Table

Response: yield

Df Sum Sq Mean Sq F value Pr(>F)

fertil 3 283.25 94.417 5.4023 0.01571 *

type 1 30.25 30.250 1.7308 0.21507

Residuals 11 192.25 17.477

17

Conclusions It seems like Fertilizer has an effect on crop

Yield, but crop Type does not We did not allow the effect of Fertilizer to depend

on Type, but it doesn’t look like it does anyway According to the interaction plot, at least Should probably do a statistical test for this Coming up!

Of course, we should also follow-up our main effect conclusion with some pairwise tests, contrasts, etc…

18

Factorial ANOVA

Models with interactions

19

Consider Consider 8 hypothetical experiments, each

involving 2 levels of 2 different factors (A and B) Group means:Exp’t 1𝑠𝑠1 𝑠𝑠2 �𝑌𝑌𝐴𝐴

𝑎𝑎1 5 5 5𝑎𝑎2 5 5 5�𝑌𝑌𝐵𝐵 5 5

Exp’t 3𝑠𝑠1 𝑠𝑠2 �𝑌𝑌𝐴𝐴

𝑎𝑎1 7 3 5𝑎𝑎2 7 3 5�𝑌𝑌𝐵𝐵 7 3

Exp’t 2𝑠𝑠1 𝑠𝑠2 �𝑌𝑌𝐴𝐴

𝑎𝑎1 4 4 4𝑎𝑎2 6 6 6�𝑌𝑌𝐵𝐵 5 5

Exp’t 4𝑠𝑠1 𝑠𝑠2 �𝑌𝑌𝐴𝐴

𝑎𝑎1 6 2 4𝑎𝑎2 8 4 6�𝑌𝑌𝐵𝐵 7 3

Exp’t 5𝑠𝑠1 𝑠𝑠2 �𝑌𝑌𝐴𝐴

𝑎𝑎1 6 4 5𝑎𝑎2 4 6 5�𝑌𝑌𝐵𝐵 5 5

Exp’t 7𝑠𝑠1 𝑠𝑠2 �𝑌𝑌𝐴𝐴

𝑎𝑎1 8 2 5𝑎𝑎2 6 4 5�𝑌𝑌𝐵𝐵 7 3

Exp’t 6𝑠𝑠1 𝑠𝑠2 �𝑌𝑌𝐴𝐴

𝑎𝑎1 5 3 4𝑎𝑎2 5 7 6�𝑌𝑌𝐵𝐵 5 5

Exp’t 8𝑠𝑠1 𝑠𝑠2 �𝑌𝑌𝐴𝐴

𝑎𝑎1 7 1 4𝑎𝑎2 7 5 6�𝑌𝑌𝐵𝐵 7 3 20

No Interactions

21

Interactions

22

Possible Outcomes As you’ve just seen, it is possible to have No interaction, and No main effects Main effect for one factor but not the other Main effects for both factors

Interaction between factors, and No main effects Main effect for one factor but not the other Main effects for both factors

Yikes! The combinations get more numerous as the number of levels (or factors) increases 23

Removable Interactions Sometimes we apply transformations to our data Perhaps to meet ANOVA assumptions Or to speak to Americans

These can dampen out or create interactions:

24

Removable Interactions An interaction is removable if there exists some

transformation on the response that will make the interaction disappear

You can tell from the interaction plot, sort of If the lines cross, it is not removable If the lines would cross if we plotted the axes

reversed, it is not removable Removable interactions are less convincing

since they depend on the way the response variable is collected/measured

25

Practice!

26

Interactive Model(Setup) Two-factor design 2 levels of Factor A; 3 levels of Factor B 4 observations at each treatment combination

Data Table Table of Sums𝑎𝑎1𝑠𝑠1 𝑎𝑎1𝑠𝑠2 𝑎𝑎1𝑠𝑠3 𝑎𝑎2𝑠𝑠1 𝑎𝑎2𝑠𝑠2 𝑎𝑎2𝑠𝑠3𝑌𝑌111 𝑌𝑌112 𝑌𝑌113 𝑌𝑌121 𝑌𝑌122 𝑌𝑌123𝑌𝑌211 𝑌𝑌212 𝑌𝑌213 𝑌𝑌221 𝑌𝑌222 𝑌𝑌223𝑌𝑌311 𝑌𝑌312 𝑌𝑌313 𝑌𝑌321 𝑌𝑌322 𝑌𝑌323𝑌𝑌411 𝑌𝑌412 𝑌𝑌413 𝑌𝑌421 𝑌𝑌422 𝑌𝑌423

Sum 𝐴𝐴𝐵𝐵11 𝐴𝐴𝐵𝐵12 𝐴𝐴𝐵𝐵13 𝐴𝐴𝐵𝐵21 𝐴𝐴𝐵𝐵22 𝐴𝐴𝐵𝐵23

𝑠𝑠1 𝑠𝑠2 𝑠𝑠3 𝑆𝑆𝑆𝑆𝑚𝑚𝑎𝑎1 𝐴𝐴𝐵𝐵11 𝐴𝐴𝐵𝐵12 𝐴𝐴𝐵𝐵13 𝐴𝐴1𝑎𝑎2 𝐴𝐴𝐵𝐵21 𝐴𝐴𝐵𝐵22 𝐴𝐴𝐵𝐵23 𝐴𝐴2𝑆𝑆𝑆𝑆𝑚𝑚 𝐵𝐵1 𝐵𝐵2 𝐵𝐵3

27

Interactive Model Suppose we want to include an interaction now

𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝛼𝛼𝑖𝑖 + 𝛽𝛽𝑖𝑖 + 𝛼𝛼𝛽𝛽 𝑖𝑖𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 – 𝑖𝑖𝑡𝑡𝑡 obs. from 𝑗𝑗𝑡𝑡𝑡 level of Factor A and 𝑘𝑘𝑡𝑡𝑡 level of Factor B𝜇𝜇 – overall grand mean𝛼𝛼𝑖𝑖 – deviation from mean of 𝑗𝑗𝑡𝑡𝑡 group of Factor A𝛽𝛽𝑖𝑖 – deviation from mean of 𝑘𝑘𝑡𝑡𝑡 group of Factor B𝛼𝛼𝛽𝛽 𝑖𝑖𝑖𝑖 – deviation remaining after main effects are

removed𝜀𝜀𝑖𝑖𝑖𝑖𝑖𝑖 – error/noise/uncertainty

28

Deviations of means (Two-factor model) 𝐴𝐴𝑖𝑖 effect = �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝑇𝑇 𝐵𝐵𝑖𝑖 effect = �𝑌𝑌𝐵𝐵𝑘𝑘 − �𝑌𝑌𝑇𝑇 Interaction effect = �𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑇𝑇 − �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝑇𝑇 − �𝑌𝑌𝐵𝐵𝑘𝑘 − �𝑌𝑌𝑇𝑇

= �𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝐵𝐵𝑘𝑘 + �𝑌𝑌𝑇𝑇

We can write the between group deviation as:�𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑇𝑇 = �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝑇𝑇 + �𝑌𝑌𝐵𝐵𝑘𝑘 − �𝑌𝑌𝑇𝑇 + ( �𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝐵𝐵𝑘𝑘 + �𝑌𝑌𝑇𝑇)�𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑇𝑇 = (𝐴𝐴𝑖𝑖 effect) + (𝐵𝐵𝑖𝑖 effect) + (Interaction effect)

29

Deviations of individuals(Two-factor model) That was for group means, not individuals Deviation of an individual from the grand mean Sum of the deviations from the group mean to

grand mean, and individual to group mean𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑇𝑇 = �𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑇𝑇 + (𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑖𝑖𝑖𝑖)

Replace with formula on last slide gives:

𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑇𝑇 = �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝑇𝑇 + �𝑌𝑌𝐵𝐵𝑘𝑘 − �𝑌𝑌𝑇𝑇 + (�𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝐵𝐵𝑘𝑘 + �𝑌𝑌𝑇𝑇) + 𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑖𝑖𝑖𝑖

Does this relationship hold for SS as well?30

SS revisited Need to partition the between-group SS into

three parts instead of just one Factor A, Factor B, Interaction

𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑇𝑇 = �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝑇𝑇 + �𝑌𝑌𝐵𝐵𝑘𝑘 − �𝑌𝑌𝑇𝑇 + �𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝐵𝐵𝑘𝑘 + �𝑌𝑌𝑇𝑇 + 𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑖𝑖𝑖𝑖

Σ 𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑇𝑇2 = Σ �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝑇𝑇

2+ Σ �𝑌𝑌𝐵𝐵𝑘𝑘 − �𝑌𝑌𝑇𝑇

2 + Σ �𝑌𝑌𝑖𝑖𝑖𝑖 − �𝑌𝑌𝐴𝐴𝑗𝑗 − �𝑌𝑌𝐵𝐵𝑘𝑘 + �𝑌𝑌𝑇𝑇2

+ Σ 𝑌𝑌𝑖𝑖𝑖𝑖𝑖𝑖 − �𝑌𝑌𝑖𝑖𝑖𝑖2

𝑆𝑆𝑆𝑆𝑇𝑇 = 𝑆𝑆𝑆𝑆𝐴𝐴 + 𝑆𝑆𝑆𝑆𝐵𝐵 + 𝑆𝑆𝑆𝑆𝐴𝐴 𝑥𝑥 𝐵𝐵 + 𝑆𝑆𝑆𝑆𝐸𝐸

31

Homework example Consider an experiment where we randomly

assign a group of 16 students to 4 treatments Half will read the lecture notes, half won’t Half will do the practice problems, half won’t

Measure their score on the midterm, out of 100 4 treatment combinations (2 x 2 factorial) Check for interaction

32

Homework example(R code & output)df <- as.data.frame(expand.grid(readLecture = c(rep("no read",2),

rep("read", 2)), pracProbs = c(rep("no practice",2), rep("practice",2)), KEEP.OUT.ATTRS = F))

require(plyr)

df <- arrange(df, pracProbs, readLecture)

df$score <- c(50,45,56,60, 65,58,63,70, 60,70,68,65, 88,89,86,94)

> with(df, tapply(score, list(readLecture, pracProbs), mean))no practice practice

no read 52.75 65.75

read 64.00 89.25

with(df, interaction.plot(readLecture, pracProbs, score, col=c("red", "blue"), main="", type = "b"))

with(df, interaction.plot(pracProbs, readLecture, score, col=c("red", "blue"), main="", type = "b"))

33

Homework example(R output) Interaction plots (both ways, just because) When increasing one level of Factor A also

increases the effect of Factor B, this is called a reinforcement effect

The other way is called an interference effect

34

Homework example(R code & output)> anova(lm(score ~ readLecture*pracProbs, data=df))Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

readLecture 1 1207.56 1207.56 48.9139 1.447e-05 ***

pracProbs 1 1463.06 1463.06 59.2633 5.561e-06 ***

readLecture:pracProbs 1 150.06 150.06 6.0785 0.02974 *

Residuals 12 296.25 24.69

40

Homework example(Conclusions) When the interaction is not significant, it should

be removed and the main effects tested with an additive model Some statisticians leave it there and test main

effects, but this is less powerful It looks like the two types of studying reinforce

each other Need to be careful how we interpret main effects

in the presence of an interaction, usually Here it is pretty clear – both study methods are

clearly beneficial 41

Farming example(R code & output)> anova(lm(yield ~ fertil + type, data=farm))Analysis of Variance Table

Response: yield

Df Sum Sq Mean Sq F value Pr(>F)

fertil 3 283.25 94.417 5.4023 0.01571 *

type 1 30.25 30.250 1.7308 0.21507

Residuals 11 192.25 17.477

> anova(lm(yield ~ fertil*type, data=farm))Analysis of Variance Table

Response: yield

Df Sum Sq Mean Sq F value Pr(>F)

fertil 3 283.25 94.417 3.9754 0.05262 .

type 1 30.25 30.250 1.2737 0.29178

fertil:type 3 2.25 0.750 0.0316 0.99186

Residuals 8 190.00 23.750

42

Testing Contrasts

Differences between marginal means are definitely contrasts (main effect for fertilizer) 𝐻𝐻0: 𝜇𝜇11+𝜇𝜇21

2= 𝜇𝜇12+𝜇𝜇22

2= 𝜇𝜇13+𝜇𝜇23

2= 𝜇𝜇14+𝜇𝜇24

2

How to form these contrasts in factorial ANOVA?

𝑓𝑓1 𝑓𝑓2 𝑓𝑓3 𝑓𝑓4 𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐

𝑡𝑡1 𝜇𝜇11 𝜇𝜇12 𝜇𝜇13 𝜇𝜇14𝜇𝜇11 + ⋯+ 𝜇𝜇14

4

𝑡𝑡2 𝜇𝜇21 𝜇𝜇22 𝜇𝜇23 𝜇𝜇24𝜇𝜇21 + ⋯+ 𝜇𝜇24

4

𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐 𝜇𝜇11 + 𝜇𝜇212

𝜇𝜇12 + 𝜇𝜇222

𝜇𝜇13 + 𝜇𝜇232

𝜇𝜇14 + 𝜇𝜇242 𝜇𝜇

43

𝐹𝐹 𝑇𝑇 𝒇𝒇𝟐𝟐 𝒇𝒇𝟑𝟑 𝒇𝒇𝟒𝟒 𝒕𝒕𝟐𝟐 𝒇𝒇𝟐𝟐𝒕𝒕𝟐𝟐 𝒇𝒇𝟑𝟑𝒕𝒕𝟐𝟐 𝒇𝒇𝟒𝟒𝒕𝒕𝟐𝟐 𝑬𝑬(𝒀𝒀|𝑿𝑿 = 𝒙𝒙)1 C 0 0 0 0 0 0 0 𝛽𝛽01 S 0 0 0 1 0 0 0 𝛽𝛽0 + 𝛽𝛽42 C 1 0 0 0 0 0 0 𝛽𝛽0 + 𝛽𝛽12 S 1 0 0 1 1 0 0 𝛽𝛽0 + 𝛽𝛽1 + 𝛽𝛽4 + 𝛽𝛽53 C 0 1 0 0 0 0 0 𝛽𝛽0 + 𝛽𝛽23 S 0 1 0 1 0 1 0 𝛽𝛽0 + 𝛽𝛽2 + 𝛽𝛽4 + 𝛽𝛽64 C 0 0 1 0 0 0 0 𝛽𝛽0 + 𝛽𝛽34 S 0 0 1 1 0 0 1 𝛽𝛽0 + 𝛽𝛽3 + 𝛽𝛽4 + 𝛽𝛽7

Testing Contrasts(Reference Coding) First, write out the model and expected value for

each cell (use reference coding to start)E 𝑌𝑌𝑖𝑖𝑠𝑠𝑌𝑌𝑌𝑌 = 𝛽𝛽0 + 𝛽𝛽1𝑓𝑓2 + 𝛽𝛽2𝑓𝑓3 + 𝛽𝛽3𝑓𝑓4 + 𝛽𝛽4𝑡𝑡2 +𝛽𝛽5𝑓𝑓2𝑡𝑡2 + 𝛽𝛽6𝑓𝑓3𝑡𝑡2 + 𝛽𝛽7𝑓𝑓4𝑡𝑡2

44

Testing Contrasts(Reference Coding)

To test 𝐻𝐻0: 𝜇𝜇11+𝜇𝜇212

= 𝜇𝜇12+𝜇𝜇222

corresponds to

𝐻𝐻0: 𝛽𝛽0 + 12𝛽𝛽4 − 𝛽𝛽0 + 𝛽𝛽1 + 1

2(𝛽𝛽4 + 𝛽𝛽5) = 0

𝐻𝐻0:𝛽𝛽1 = 𝛽𝛽5 = 0 (can you see why?)

𝑓𝑓1 𝑓𝑓2 𝑓𝑓3 𝑓𝑓4 𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐

𝑡𝑡1 𝛽𝛽0 𝛽𝛽0 + 𝛽𝛽1 𝛽𝛽0 + 𝛽𝛽2 𝛽𝛽0 + 𝛽𝛽3 𝛽𝛽0 +14

(𝛽𝛽1 + 𝛽𝛽2 + 𝛽𝛽3)

𝑡𝑡2 𝛽𝛽0 + 𝛽𝛽4𝛽𝛽0 + 𝛽𝛽1

+𝛽𝛽4 + 𝛽𝛽5𝛽𝛽0 + 𝛽𝛽2

+𝛽𝛽4 + 𝛽𝛽6𝛽𝛽0 + 𝛽𝛽3

+𝛽𝛽4 + 𝛽𝛽7

𝛽𝛽0 + 𝛽𝛽4 +14 (𝛽𝛽1 + 𝛽𝛽2 + 𝛽𝛽3)

+14 (𝛽𝛽5 + 𝛽𝛽6 + 𝛽𝛽7)

𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐 𝛽𝛽0 +12𝛽𝛽4

𝛽𝛽0 + 𝛽𝛽1 +12 (𝛽𝛽4 + 𝛽𝛽5)

𝛽𝛽0 + 𝛽𝛽2 +12 (𝛽𝛽4 + 𝛽𝛽6)

𝛽𝛽0 + 𝛽𝛽3 +12 (𝛽𝛽4 + 𝛽𝛽7)

45

Reference Coding(R code & output)> fit <- lm(yield ~ fertil*type, data=farm)> model.matrix(fit)

> L <- rbind(c(0, 1, 0, 0, 0, 0, 0, 0),c(0, 0, 0, 0, 0, 1, 0, 0))

> glh.test(fit, L)

Test of General Linear Hypothesis

Call:

glh.test(reg = fit, cm = L)

F = 0.0895, df1 = 2, df2 = 8, p-value = 0.9153

46

Testing Contrasts(Reference Coding)

Interactions are also sets of contrasts 𝐻𝐻0: 𝜇𝜇21 − 𝜇𝜇11 = 𝜇𝜇22 − 𝜇𝜇12 = 𝜇𝜇23 − 𝜇𝜇13 = 𝜇𝜇24 − 𝜇𝜇14 𝐻𝐻0: 𝜇𝜇12 − 𝜇𝜇11 = 𝜇𝜇22 − 𝜇𝜇21 and

𝜇𝜇13 − 𝜇𝜇12 = 𝜇𝜇23 − 𝜇𝜇22 and 𝜇𝜇14 − 𝜇𝜇13 = 𝜇𝜇24 − 𝜇𝜇23

𝑓𝑓1 𝑓𝑓2 𝑓𝑓3 𝑓𝑓4 𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐

𝑡𝑡1 𝜇𝜇11 𝜇𝜇12 𝜇𝜇13 𝜇𝜇14𝜇𝜇11 + ⋯+ 𝜇𝜇14

4

𝑡𝑡2 𝜇𝜇21 𝜇𝜇22 𝜇𝜇23 𝜇𝜇24𝜇𝜇21 + ⋯+ 𝜇𝜇24

4

𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐 𝜇𝜇11 + 𝜇𝜇212

𝜇𝜇12 + 𝜇𝜇222

𝜇𝜇13 + 𝜇𝜇232

𝜇𝜇14 + 𝜇𝜇242 𝜇𝜇

47

Testing Contrasts(Reference Coding)

𝐻𝐻0: 𝜇𝜇21 − 𝜇𝜇11 = 𝜇𝜇22 − 𝜇𝜇12 = 𝜇𝜇23 − 𝜇𝜇13 = 𝜇𝜇24 − 𝜇𝜇14 To test the interaction corresponds to𝐻𝐻0:𝛽𝛽4 = 𝛽𝛽4 + 𝛽𝛽5 = 𝛽𝛽4 + 𝛽𝛽6 = 𝛽𝛽4 + 𝛽𝛽7𝐻𝐻0:𝛽𝛽5 = 𝛽𝛽6 = 𝛽𝛽7 = 0

𝑓𝑓1 𝑓𝑓2 𝑓𝑓3 𝑓𝑓4 𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐

𝑡𝑡1 𝛽𝛽0 𝛽𝛽0 + 𝛽𝛽1 𝛽𝛽0 + 𝛽𝛽2 𝛽𝛽0 + 𝛽𝛽3 𝛽𝛽0 +14

(𝛽𝛽1 + 𝛽𝛽2 + 𝛽𝛽3)

𝑡𝑡2 𝛽𝛽0 + 𝛽𝛽4𝛽𝛽0 + 𝛽𝛽1

+𝛽𝛽4 + 𝛽𝛽5𝛽𝛽0 + 𝛽𝛽2

+𝛽𝛽4 + 𝛽𝛽6𝛽𝛽0 + 𝛽𝛽3

+𝛽𝛽4 + 𝛽𝛽7

𝛽𝛽0 + 𝛽𝛽4 +14 (𝛽𝛽1 + 𝛽𝛽2 + 𝛽𝛽3)

+14 (𝛽𝛽5 + 𝛽𝛽6 + 𝛽𝛽7)

𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐 𝛽𝛽0 +12𝛽𝛽4

𝛽𝛽0 + 𝛽𝛽1 +12 (𝛽𝛽4 + 𝛽𝛽5)

𝛽𝛽0 + 𝛽𝛽2 +12 (𝛽𝛽4 + 𝛽𝛽6)

𝛽𝛽0 + 𝛽𝛽3 +12 (𝛽𝛽4 + 𝛽𝛽7)

48

Reference Coding(R code & output)> L <- rbind(c(0, 0, 0, 0, 0, 1, 0, 0),

c(0, 0, 0, 0, 0, 0, 1, 0),c(0, 0, 0, 0, 0, 0, 0, 1))

> glh.test(fit, L)Test of General Linear Hypothesis

Call:

glh.test(reg = fit, cm = L)

F = 0.0316, df1 = 3, df2 = 8, p-value = 0.9919

> anova(fit)Response: yield

Df Sum Sq Mean Sq F value Pr(>F)

fertil 3 283.25 94.417 3.9754 0.05262 .

type 1 30.25 30.250 1.2737 0.29178

fertil:type 3 2.25 0.750 0.0316 0.99186

Residuals 8 190.00 23.750

49

Testing Contrasts(Reference Coding) Reference coding gets worse the higher the

order of the factorial experiment Cell means is no better

Effect coding gets easier Better learn it now!

50

𝐹𝐹 𝑇𝑇 𝒇𝒇𝟏𝟏 𝒇𝒇𝟐𝟐 𝒇𝒇𝟑𝟑 𝒕𝒕𝟏𝟏 𝒇𝒇𝟏𝟏𝒕𝒕𝟏𝟏 𝒇𝒇𝟐𝟐𝒕𝒕𝟏𝟏 𝒇𝒇𝟑𝟑𝒕𝒕𝟏𝟏 𝑬𝑬(𝒀𝒀|𝑿𝑿 = 𝒙𝒙)1 C 1 0 0 1 1 0 0 𝛽𝛽0 + 𝛽𝛽1 + 𝛽𝛽4 + 𝛽𝛽51 S 1 0 0 -1 -1 0 0 𝛽𝛽0 + 𝛽𝛽1 − 𝛽𝛽4 − 𝛽𝛽52 C 0 1 0 1 0 1 0 𝛽𝛽0 + 𝛽𝛽2 + 𝛽𝛽4 + 𝛽𝛽62 S 0 1 0 -1 0 -1 0 𝛽𝛽0 + 𝛽𝛽2 − 𝛽𝛽4 − 𝛽𝛽63 C 0 0 1 1 0 0 1 𝛽𝛽0 + 𝛽𝛽3 + 𝛽𝛽4 + 𝛽𝛽73 S 0 0 1 -1 0 0 -1 𝛽𝛽0 + 𝛽𝛽3 − 𝛽𝛽4 − 𝛽𝛽7

4 C -1 -1 -1 1 -1 -1 -1 𝛽𝛽0 − 𝛽𝛽1 − 𝛽𝛽2 − 𝛽𝛽3+𝛽𝛽4 − 𝛽𝛽5 − 𝛽𝛽6 − 𝛽𝛽7

4 S -1 -1 -1 -1 1 1 1 𝛽𝛽0 − 𝛽𝛽1 − 𝛽𝛽2 − 𝛽𝛽3−𝛽𝛽4 + 𝛽𝛽5 + 𝛽𝛽6 + 𝛽𝛽7

Testing Contrasts(Effect Coding)E 𝑌𝑌𝑖𝑖𝑠𝑠𝑌𝑌𝑌𝑌 = 𝛽𝛽0 + 𝛽𝛽1𝑓𝑓1 + 𝛽𝛽2𝑓𝑓2 + 𝛽𝛽3𝑓𝑓3 + 𝛽𝛽4𝑡𝑡1 +𝛽𝛽5𝑓𝑓1𝑡𝑡1 + 𝛽𝛽6𝑓𝑓2𝑡𝑡1 + 𝛽𝛽7𝑓𝑓3𝑡𝑡1

51

Testing Contrasts(Effect Coding)

To test 𝐻𝐻0: 𝜇𝜇11+𝜇𝜇212

= 𝜇𝜇12+𝜇𝜇222

corresponds to

𝐻𝐻0: 𝛽𝛽0 + 𝛽𝛽1 − 𝛽𝛽0 + 𝛽𝛽2 = 0𝐻𝐻0:𝛽𝛽1 = 𝛽𝛽2 as expected

𝑓𝑓1 𝑓𝑓2 𝑓𝑓3 𝑓𝑓4 𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐

𝑡𝑡1𝛽𝛽0 + 𝛽𝛽1+ 𝛽𝛽4 + 𝛽𝛽5

𝛽𝛽0 + 𝛽𝛽2+ 𝛽𝛽4 + 𝛽𝛽6

𝛽𝛽0 + 𝛽𝛽3+ 𝛽𝛽4 + 𝛽𝛽7

𝛽𝛽0 − 𝛽𝛽1 − 𝛽𝛽2 − 𝛽𝛽3+𝛽𝛽4 − 𝛽𝛽5 − 𝛽𝛽6 − 𝛽𝛽7

𝛽𝛽0 + 𝛽𝛽4

𝑡𝑡2𝛽𝛽0 + 𝛽𝛽1− 𝛽𝛽4 − 𝛽𝛽5

𝛽𝛽0 + 𝛽𝛽2− 𝛽𝛽4 − 𝛽𝛽6

𝛽𝛽0 + 𝛽𝛽3− 𝛽𝛽4 − 𝛽𝛽7

𝛽𝛽0 − 𝛽𝛽1 − 𝛽𝛽2 − 𝛽𝛽3−𝛽𝛽4 + 𝛽𝛽5 + 𝛽𝛽6 + 𝛽𝛽7

𝛽𝛽0 − 𝛽𝛽4

𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐 𝛽𝛽0 + 𝛽𝛽1 𝛽𝛽0 + 𝛽𝛽2 𝛽𝛽0 + 𝛽𝛽3 𝛽𝛽0 − 𝛽𝛽1 − 𝛽𝛽2 − 𝛽𝛽3 𝛽𝛽0

52

Testing Contrasts(Effect Coding)

𝐻𝐻0: 𝜇𝜇21 − 𝜇𝜇11 = 𝜇𝜇22 − 𝜇𝜇12 = 𝜇𝜇23 − 𝜇𝜇13 = 𝜇𝜇24 − 𝜇𝜇14 To test the interaction corresponds to𝐻𝐻0: 2𝛽𝛽4 + 2𝛽𝛽5 = 2𝛽𝛽4 + 2𝛽𝛽6 = 2𝛽𝛽4 + 2𝛽𝛽7 = 2𝛽𝛽4 − 2𝛽𝛽5 − 2𝛽𝛽6 − 2𝛽𝛽7𝐻𝐻0:𝛽𝛽5 = 𝛽𝛽6 = 𝛽𝛽7 = −𝛽𝛽5 − 𝛽𝛽6 − 𝛽𝛽7𝐻𝐻0:𝛽𝛽5 = 𝛽𝛽6 = 𝛽𝛽7 = 0

𝑓𝑓1 𝑓𝑓2 𝑓𝑓3 𝑓𝑓4 𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐

𝑡𝑡1𝛽𝛽0 + 𝛽𝛽1+ 𝛽𝛽4 + 𝛽𝛽5

𝛽𝛽0 + 𝛽𝛽2+ 𝛽𝛽4 + 𝛽𝛽6

𝛽𝛽0 + 𝛽𝛽3+ 𝛽𝛽4 + 𝛽𝛽7

𝛽𝛽0 − 𝛽𝛽1 − 𝛽𝛽2 − 𝛽𝛽3+𝛽𝛽4 − 𝛽𝛽5 − 𝛽𝛽6 − 𝛽𝛽7

𝛽𝛽0 + 𝛽𝛽4

𝑡𝑡2𝛽𝛽0 + 𝛽𝛽1− 𝛽𝛽4 − 𝛽𝛽5

𝛽𝛽0 + 𝛽𝛽2− 𝛽𝛽4 − 𝛽𝛽6

𝛽𝛽0 + 𝛽𝛽3− 𝛽𝛽4 − 𝛽𝛽7

𝛽𝛽0 − 𝛽𝛽1 − 𝛽𝛽2 − 𝛽𝛽3−𝛽𝛽4 + 𝛽𝛽5 + 𝛽𝛽6 + 𝛽𝛽7

𝛽𝛽0 − 𝛽𝛽4

𝑀𝑀𝑠𝑠𝑎𝑎𝑐𝑐 𝛽𝛽0 + 𝛽𝛽1 𝛽𝛽0 + 𝛽𝛽2 𝛽𝛽0 + 𝛽𝛽3 𝛽𝛽0 − 𝛽𝛽1 − 𝛽𝛽2 − 𝛽𝛽3 𝛽𝛽0

53

A Common Practice We want to predict the Grade in a class from Sex

{Male, Female} and PoST {Stats, Other} A 2x2 factorial design

Main interest is the effect of sex, but recognize that maybe PoST has an effect too, so

Split the data by PoST, and run two-sample t-tests comparing sex within each PoST

If one is significant and the other isn’t, then the effect of sex depends on PoST Good idea? Discuss …

54

Visualize

55

A Common Practice(R code & output)> with(subset(fact, post == "Stats"), t.test(score[sex == "Male"], score[sex == "Female"]))

Welch Two Sample t-test

t = -2.0979, df = 17.684, p-value = 0.05057

95 percent confidence interval:

-20.42798825 0.02798825

mean of x mean of y

68.8 79.0

> with(subset(fact, post == "Other"), t.test(score[sex == "Male"], score[sex == "Female"]))

Welch Two Sample t-test

t = -2.556, df = 17.62, p-value = 0.02007

95 percent confidence interval:

-18.961573 -1.838427

mean of x mean of y

50.4 60.8

56

A Common Conclusion At the 5% significance level, Males and females have similar grades in the

Stats program Males and females have different grades in the

Other programs Therefore, the effect of sex depends on program

of study

What do you think?

57

The correct solution> fit <- lm(score ~ sex*post, data= fact)> anova(fit)Analysis of Variance Table

Response: score

Df Sum Sq Mean Sq F value Pr(>F)

sex 1 1060.9 1060.9 10.557 0.00251 **

post 1 3348.9 3348.9 33.326 1.398e-06 ***

sex:post 1 0.1 0.1 0.001 0.97501

Residuals 36 3617.6 100.5

58

Three Factors: A, B and C Three main effects: one each for A, B and C There are many subsets of simple effects The effect of A, at level 𝑠𝑠𝑖𝑖 and 𝑐𝑐𝑖𝑖, etc…

Three two-factor interactions AxB (averaging over C) AxC (averaging over B) BxC (averaging over A)

One three-factor interaction: AxBxC

59

Three-factor interaction The form of the AxB interaction depends on the

value of C The form of the AxC interaction depends on the

value of B The form of the BxC interaction depends on the

value of A These statements are equivalent Use the one that is easiest to understand

60

Graph a 3-factor interaction Make a 2-factor interaction plot, at each level of

the third factor Maybe pick the factor that has the fewest levels

61

Higher order factorial designs For F factors, There will be one intercept

There will be 𝐹𝐹1 main effects

There will be 𝐹𝐹k k-factor interactions

There is an F test for each one

62

Higher order factorial designs As the number of factors increases The higher-way interactions get harder and harder

to understand All the tests are still tests of sets of contrasts Differences between differences of differences …

It gets harder and harder to write down the contrasts

Effect coding becomes easier

63