unbalanced data in factorials types i, ii, iii ss part...

Unbalanced Data in FactorialsTypes I, II, III SS

Part 2

Chapter 10 in Oehlert

STAT:5201

Week 9 - Lecture 2b

1 / 29

Unbalanced factorials: Types I, II, III SS

Types of sums of squares

Type II SS

The Type II SS relates to the extra variability explained when a termis entered into the model after all terms at the same level or at amore fundamental level have already been entered. These SS couldbe called model building sums of squares. The Type II SS followsthe “hierarchy principle.”

If you fit the full model (i.e. all possible interactions), you can look atthe table of Type II SS in order to test for higher interactions, such asa 3-way interaction, and if it is not significant, you can use the sametable to test for 2-way interactions and the p-value coincides with amodel that does not include the 3-way interaction (but you don’tactually have to remove the 3-way interaction and re-fit the model).

2 / 29


Type II SS for 3-way ANOVA

Source Type II SS

A SS(A|1,B,C )B SS(B|1,A,C )C SS(C |1,A,B)AB SS(AB|1,A,B,C ,AC ,BC )AC SS(AC |1,A,B,C ,AB,BC )BC SS(BC |1,A,B,C ,AB,AC )ABC SS(ABC |1,A,B,C ,AB,AC ,BC )

So, you can look at the Type II SS and essentially get tests from a“new model” without actually re-fitting the model in SAS.

As discussed earlier, start with the highest-order interaction tests andwork your way toward the main effects, continuing while terms arenot significant.

3 / 29


Example (Unbalanced 3-way ANOVA)

[Problem 10.1 in Oehlert]An experiment investigated the release of the hormone ACTH from ratpituitary glands under eight treatments from three factors.

Response: amount of ACTHFactors: CRF(0 or 100nM)

Calcium(0 or 2mM)Verapamil (0 or 50mM)

The control treatment (all factors at 0 moles) received 8 EUs, while allother treatments received 4 EUs (N=36). This is a completely randomizeddesign.

All factors coded as 1,2.

We will use Type II SS to decide on what factors should be in the model.

4 / 29


Example (Unbalanced 3-way ANOVA)proc freq data=p10_1;

tables calcium*crf*verapamil/nocol norow nocum nopercent;

run;

The FREQ Procedure

Table 1 of crf by verapamil

Controlling for calcium=1

crf verapamil

Frequency| 1| 2| Total

---------+--------+--------+

1 | 8 | 4 | 12

---------+--------+--------+

2 | 4 | 4 | 8

---------+--------+--------+

Total 12 8 20

Table 2 of crf by verapamil

Controlling for calcium=2

crf verapamil

Frequency| 1| 2| Total

---------+--------+--------+

1 | 4 | 4 | 8

---------+--------+--------+

2 | 4 | 4 | 8

---------+--------+--------+

Total 8 8 165 / 29



Due to nonconstant variance, the log transformation was used.

proc transreg data=p10_1;

model boxcox(response)=class(superfactor);

run;

/*Use the lambda=0 transformation from box-cox, which is log(y)*/

6 / 29


Example (Unbalanced 3-way ANOVA)data p10_1; set p10_1;

logy=log(response);

run;

proc glm data=p10_1 plot=diagnostics;

class verapamil crf calcium;

model logy=verapamil|crf|calcium/ss2;

run;

Source DF Type II SS Mean Square F Value Pr > F

verapamil 1 0.34131932 0.34131932 14.91 0.0006

crf 1 11.27564925 11.27564925 492.51 <.0001

verapamil*crf 1 0.03314380 0.03314380 1.45 0.2390

calcium 1 7.34125531 7.34125531 320.66 <.0001

verapamil*calcium 1 0.05613136 0.05613136 2.45 0.1286

crf*calcium 1 1.11913661 1.11913661 48.88 <.0001

verapami*crf*calcium 1 0.03096207 0.03096207 1.35 0.2547

7 / 29



2

1

Starting with the 3-way interaction, we see that this term is not significant1© and is not needed in the model. Moving onto 2-way interactions, we

see that the only 2-way interaction that is needed is crf-by-calcium 2©.Note that the p-values for the 2-way interaction here 2© relate to a modelwithout the 3-way interaction included.

8 / 29



Because the 2-way interaction for crf-by-calcium was significant, weneed to keep the main effects for crf and calcium in the model,regardless of their p-values in the table, because of the hierarchy principle.

We will re-fit the final parsimonious model and perform some relevanttests and plots.

proc glm data=p10_1 plot=diagnostics;

class verapamil crf calcium;

model logy= verapamil crf calcium crf*calcium/ss2;

lsmeans verapamil/pdiff;

lsmeans calcium*crf/adjust=tukey pdiff;

run;

9 / 29


Example (Unbalanced 3-way ANOVA)Dependent Variable: logy

Sum of

Source DF Squares Mean Square F Value Pr > F

Model 4 22.91219129 5.72804782 235.02 <.0001

Error 31 0.75553614 0.02437213

Corrected Total 35 23.66772743

R-Square Coeff Var Root MSE logy Mean

0.968077 11.41904 0.156116 1.367153


verapamil 1 0.34131932 0.34131932 14.00 0.0007

crf 1 11.43485091 11.43485091 469.18 <.0001

calcium 1 7.28089585 7.28089585 298.74 <.0001

crf*calcium 1 1.12256063 1.12256063 46.06 <.0001

NOTE: The Type II SS test for verapamil above puts the interaction into the error term, which will inflate the σ̂2 for that test,

but the main effect is strong enough here to be significant already (the Type III tests for this model would have removed that

interaction from the error term before testing verapamil, probably preferred). 10 / 29


Example (Unbalanced 3-way ANOVA)symbol1 value=star interpol=std1mj color=black line=1;

symbol2 value=circle interpol=std1mj color=blue line=2;

proc gplot data=p10_1;

plot logy*crf=calcium/haxis=.5 to 2.5;

run;

Looking at the plot, we see that calcium has a positive effect at both levels of crf, but it has a

larger positive effect when crf is set to the high level of 100 nM.

11 / 29


Example (Unbalanced 3-way ANOVA)Least Squares Means

Adjustment for Multiple Comparisons: Tukey-Kramer

LSMEAN

crf calcium logy LSMEAN Number

1 1 0.59678787 1

1 2 1.18026526 2

2 1 1.41154042 3

2 2 2.71481285 4

Least Squares Means for effect crf*calcium

Pr > |t| for H0: LSMean(i)=LSMean(j)

Dependent Variable: logy

i/j 1 2 3 4

1 <.0001 <.0001 <.0001

2 <.0001 0.0282 <.0001

3 <.0001 0.0282 <.0001

4 <.0001 <.0001 <.0001

All four crf*calcium means are significantly different from each other.

12 / 29


Example (Unbalanced 3-way ANOVA)Least Squares Means

H0:LSMean1=

LSMean2

verapamil logy LSMEAN Pr > |t|

1 1.37662585 0.0007

2 1.57507735

verapamil also has a positive effect on ACTH.

13 / 29

Unbalanced factorials: Types I, I, III SS


Interpretation of verapamil on original scale...We can take a closer look at the ‘positive’ main effect for verapamil byconsidering the log transformation, and backtransforming to the originalscale.

Changing verapamil from 0nM (low) to 50 nM (high) is associated witha positive change in the mean response (on the log-scale) of1.575-1.376=0.199 units. Thus, changing from the low quantity to thehigh quantity of verapamil is associated with a multiplicative change inthe mean response on the raw scale of exp(0.199) = 1.22 units.

log(y2)− log(y1) = 0.199 ⇒ log(y2

y1) = 0.199 ⇒

y2

y1= exp(0.199) ⇒ y2 = y1 exp(0.199)

14 / 29

Unbalanced factorials: Types I, I, III SS


Interpretation of calcium and crf on original scale...In the same manner as the slice option, we can consider the effect ofcalcium when crf is at the low level, and the effect of calcium whencrf is at the high level, separately (both significant).

Least Squares Means

LSMEAN

crf calcium logy LSMEAN Number

1 1 0.59678787 1

1 2 1.18026526 2

2 1 1.41154042 3

2 2 2.71481285 4

On the raw scale, changing calcium from low to high when crf is at the low level changes themean response by a multiplicative factor of exp(1.180-0.597)=1.79 units.On the raw scale, changing calcium from low to high when crf is at the high level changes themean response by a multiplicative factor of exp(2.715-1.412)=3.68 units.

15 / 29


Because the Type II SS takes the “hierarchy principle” intoconsideration, it can be used as a model building tool.

The summation of the Type II SS does not equal SSmodel when thedata are unbalanced.

16 / 29


Types of sums of squares

Type III SS

The Type III SS relates to the extra variability explained when a termis entered into the model after ALL other terms have already beenentered. These SS could be called fully adjusted sums of squares.The Type III SS does not follow the “hierarchy principle, so youshould only look at relevant rows of the table.

Source Type III SS

A SS(A|1,B,C ,AB,AC ,BC ,ABC )B SS(B|1,A,C ,AB,AC ,BC ,ABC )C SS(C |1,A,B,AB,AC ,BC ,ABC )AB SS(AB|1,A,B,C ,AC ,BC ,ABC )AC SS(AC |1,A,B,C ,AB,BC ,ABC )BC SS(BC |1,A,B,C ,AB,AC ,ABC )ABC SS(ABC |1,A,B,C ,AB,AC ,BC )

17 / 29


The Type III SS does not honor the “hierarchy principle”, but thetable is still relevant if you start with the highest order interaction,and only proceed if interaction terms are not significant.

In practice, I find that I look at the Type III SS, and then removenonsignificant interaction terms and then re-fit the smaller model(but Type II SS could me this same information).

The summation of the Type III SS does not equal SSmodel when thedata are unbalanced.

18 / 29


Example (SAS unbalanced 2x2 factorial)

A professor wished to investigate if the version of exam (2 version)had any impact on scores.

Exams were randomly handed out to students.

The factors for the data are status (1:undergrad or 2:grad) andexam (1 or 2). The data are unbalanced with respect to the factors,and status would be a ‘controlled for’ factor here.

status1 2

exam 1 11 112 6 17

What is noticeable here is that exam version 2 was taken by adisproportionate number of graduate students, and we wouldnaturally think that the grad students would perform better than theundergrads (which the data confirms).

19 / 29



The first statistical test would naturally be on the interaction.

Source DF Type I SS Mean Square F Value Pr > F

status 1 5912.815873 5912.815873 30.13 <.0001

exam 1 13.797977 13.797977 0.07 0.7922

status*exam 1 243.243786 243.243786 1.24 0.2720 <--


status 1 5417.733042 5417.733042 27.61 <.0001

exam 1 13.797977 13.797977 0.07 0.7922

status*exam 1 243.243786 243.243786 1.24 0.2720 <--

Source DF Type III SS Mean Square F Value Pr > F

status 1 5602.998709 5602.998709 28.55 <.0001

exam 1 0.299684 0.299684 0.00 0.9690

status*exam 1 243.243786 243.243786 1.24 0.2720 <--

20 / 29



As this is an unbalanced design, order matters in the the Type I SS.

proc glm data=grades plot=diagnostics;

class status exam;

model grades = status exam/ ss1;

run;


status 1 5912.815873 5912.815873 29.96 <.0001

exam 1 13.797977 13.797977 0.07 0.7928


class status exam;

model grades = exam status/ ss1;

run;


exam 1 508.880808 508.880808 2.58 0.1158

status 1 5417.733042 5417.733042 27.45 <.0001

The exam factor looks much more significant when added first comparedto being added second.

21 / 29



Let’s look at order of entry, Type II SS (hierarchical), for main effects.


class status exam;


run;


status 1 5417.733042 5417.733042 27.45 <.0001

exam 1 13.797977 13.797977 0.07 0.7928


class status exam;


run;


exam 1 13.797977 13.797977 0.07 0.7928

status 1 5417.733042 5417.733042 27.45 <.0001

Order of entry did not matter here because A and B are at the samehierarchical level. SS(A|B) and SS(B|A) is outputted.

22 / 29



Type III SS (fully adjusted), for main effects.


class status exam;


run;


status 1 5417.733042 5417.733042 27.45 <.0001

exam 1 13.797977 13.797977 0.07 0.7928


class status exam;


run;


exam 1 13.797977 13.797977 0.07 0.7928

status 1 5417.733042 5417.733042 27.45 <.0001

In this particular case of a 2-way ANOVA main effects, same as Type II SS.23 / 29



Just to emphasize the need to account for other factors (or knowncovariates) prior to testing a factor of interest in an observational study oran unbalanced ANOVA, I’m going to consider two models: exam only, andexam and status.

Really, the instructor wanted to know if the exam version mattered, butthey wisely kept track of what type of student was taking the test.

1) exam as the only predictor


class exam;

model grades = exam;

lsmeans exam;

run;

24 / 29



In this one factor model, exam is not significant but it’s effect (version 1mean 62.3, version 2 mean 69.0) is inflated compared to the model thatfits both exam and status (next model).


exam 1 508.8808081 508.8808081 1.60 0.2132

Least Squares Means

exam LSMEAN

1 62.2727273

2 69.0000000

25 / 29



2) exam and status as predictors


class status exam;

model grades = exam status/ss3;

lsmeans exam;

run;


exam 1 13.797977 13.797977 0.07 0.7928

status 1 5417.733042 5417.733042 27.45 <.0001

Least Squares Means

exam LSMEAN

1 62.2727273

2 63.4157549

Given a student’s status, the estimated effect of exam version is muchmuch smaller here (version 1 mean 62.3, version 2 mean 63.4)

26 / 29



20

40

60

80

1.00 1.25 1.50 1.75 2.00

Status: Undergrad=1, Grad=2

grades exam

1

2

Model accounting for Status & Exam version

20

40

60

80

1.00 1.25 1.50 1.75 2.00

Exam version (only predictor)grades status

1

2

Model accounting for only Exam version

The status affect is expected. Because this is an unbalanced design, weshould account for status before we test for an exam effect. Otherwise,we could get a false impression (bias) of the exam effect.

27 / 29


Example (R code for plot 1)lmBoth <- lm(grades~statusNum+examNum,data=df)

library(ggplot2)

ggplot(df, aes(x = statusNum, y = grades, colour=exam)) +

geom_point() +

scale_colour_manual(values = c("#A31727","#17A319")) +

theme(axis.title.x = element_text(face="bold", colour="black", size=15),

axis.title.y = element_text(face="bold", colour="black", size=15)) +

xlab("Status: Undergrad=1, Grad=2") +

ggtitle("Model accounting for Status & Exam version") +

geom_abline(intercept = lmBoth$coefficients[1],

slope = lmBoth$coefficients[2],size=1,col="#A31727") +

geom_abline(intercept = (lmBoth$coefficients[1]+lmBoth$coefficients[3]),

slope = lmBoth$coefficients[2],size=1,col="#17A319")

28 / 29


Example (R code for plot 2)ggplot(df, aes(x = examNum, y = grades,colour=status)) +

geom_point() +

geom_abline(intercept = lmExam$coefficients[1],

slope = lmExam$coefficients[2],size=1) +

theme(axis.title.x = element_text(face="bold", colour="black", size=15),

axis.title.y = element_text(face="bold", colour="black", size=15)) +

xlab(" Exam version (only predictor)") +

ggtitle("Model accounting for only Exam version")

29 / 29

unbalanced data in factorials types i, ii, iii ss part...

Documents