8. multi-factor designs

8. Multi-Factor Designs

Chapter 8. Experimental Design II: Factorial Designs

1

• Identify, describe and create multifactor (a.k.a. “factorial”) designs

• Identify and interpret main effects and interaction effects

• Calculate N for a given factorial design

Goals

2

• As experimental designs increase in complexity:

• More information can be obtained.

• Care in design becomes ever more important.

• Designs with multiple factors and levels:

• Allow detection of interaction effects

• Allow detection of non-linear effects

• Involve more complexity around potential sequence effects and equivalent groups problems

Complexity and Design

3

8.1 Describing Multi-Factor Designs

4

• Have more than one IV (or factor). a.k.a. “factorial design”

• Described by a numbering system that gives the number of levels of each IV Examples: “2 × 2” or “3 × 4 × 2” design

• Also described by factorial matrices

Multi-Factor Designs

5

• Number of digits = number of IVs:

• “3 × 3” or “5 × 2” means two IVs.

• “2 × 2 × 2” or “3 × 4 × 2” means three IVs.

• Value of each digit = # of levels in each IV:

• 3 × 3 means two IVs, each with three levels.

• 3 × 4 × 2 means three IVs with 3, 4 and 2 levels, respectively

Numbering System for Factorial Designs

6

2 x 2 Factorial Design

Drug TherapyDrug Therapy

Placebo Prozac

Psycho-therapy

None Control ProzacPsycho-therapy

CBT CBT Combined Therapy

7

2 x 3 Factorial Design


Placebo Prozac

Psycho-therapy

None Control Prozac

Psycho-therapy

CBT CBT CBT + Prozac

Psycho-therapy

EFT EFT EFT + Prozac

8

Going 3D: 2 x 2 x 2 Factorial Design


Placebo Prozac

Psycho-therapy

None♀

Control

♀ProzacPsycho-

therapyCBT

♀CBT

♀Combo


Placebo Prozac

Psycho-therapy

None♂

Control

♂ProzacPsycho-

therapyCBT

♂

CBT

♂

Combo

MaleFemale

9

2 x 2 x 3 Factorial Design


Placebo Prozac

Psycho-therapy

None♀

Control

♀Prozac

Psycho-therapy

CBT ♀ CBT♀CBT +

ProzacPsycho-therapy

EFT ♀ EFT♀ EFT +

Prozac


Placebo Prozac

Psycho-therapy

None♂

Control

♂

Prozac

Psycho-therapy

CBT ♂ CBT♂ CBT +


EFT ♂ EFT♂ EFT +

Prozac

MaleFemale

10


Placebo Prozac

Psycho-therapy

None♀Intro

Control

♀Intro

Prozac Psycho-therapy

CBT♀Intro

CBT

♀Intro

Combo


Placebo Prozac

Psycho-therapy

None♂ Intro

Control

♂Intro


CBT♂Intro

CBT

♂ Intro

Combo


Placebo Prozac

Psycho-therapy

None♀ Extro

Control

♀ Extro


CBT♀Extro

CBT

♀ Extro

Combo


Placebo Prozac

Psycho-therapy

None♂Extro

Control

♂Extro


CBT♂Extro

CBT

♂Extro

Combo

MaleFemaleIn

trov

erts

Extr

over

ts

11

• Level: One level of one IV. A row or column in the Factorial Matrix. Also, for 3+ IVs, one of the sub-matrices

• Condition: A particular combination of one level of each IV. One cell in the Factorial Matrix.

• In single-factor designs: level = condition

Levels vs. Conditions

12

Placebo Level of Drug Therapy IV


Placebo Prozac

Psycho-therapy

None Control Prozac Psycho-therapy

CBT CBT Combo

13

Prozac Level of Drug Therapy IV


Placebo Prozac

Psycho-therapy


CBT CBT Combo

14

None Level of Psychotherapy IV


Placebo Prozac

Psycho-therapy


CBT CBT Combo

15

CBT Level of Psychotherapy IV


Placebo Prozac

Psycho-therapy


CBT CBT Combo

16

One-factor Designs

Study TimeStudy Time

2 Hours 5 Hours

Study TimeStudy TimeStudy TimeStudy Time

2 Hours

3 Hours

4 Hours

5 Hours

2-level

Multilevel

17

Discussion / Questions

• Why are the terms level and factor interchangeable in a single-factor design?

• How many IVs are there in a 3×2×2 design? How many levels of each IV? How many total conditions?

18

8.2 Interpreting Data From Multi-Factor Designs

19

• Two types of effects can emerge in multi-factorial designs:

• Main Effects: When one IV has an effect on its own. That is, the mean for some pair of levels of the IV differ significantly from one another.

• Interaction Effects: When the effect of one IV is different for different levels of another IV.

• These are NOT mutually exclusive

Interpreting Data from Factorial Designs

20

A Simple 2x2 Design


Placebo Prozac

Psycho-therapy


CBT CBT Combo

21

Main Effect of Psychotherapy


Placebo Prozac

Psycho-therapy

None (Control+ Prozac ) / 2

(Control+ Prozac ) / 2Psycho-

therapyCBT (CBT + Combo)

/ 2(CBT + Combo)

/ 2

We collapse across the levels of all other IVs to evaluate a main effect

22

Main Effect of Drug Therapy


Placebo Prozac

Psycho-therapy

None(Control+

CBT ) /2

(Prozac + Combo)

/2

Psycho-therapy

CBT

(Control+ CBT )

/2

(Prozac + Combo)

/2

We collapse across the levels of all other IVs to evaluate a main effect

23

Numerical Example


Placebo Prozac

Psycho-therapy

None 12 ± 2 18 ± 1Psycho-therapy

CBT 17 ± 1 23 ± 3

24

Main Effect of Psychotherapy?


Placebo Prozac

Psycho-therapy

None (12+18)/2 = 15(12+18)/2 = 15Psycho-therapy

CBT (17+23)/2 = 20(17+23)/2 = 20

25

Main Effect of Drug Therapy?


Placebo Prozac

Psycho-therapy

None 12+172

14.5

18+232

20.5

Psycho-therapy

CBT

12+172

14.5

18+232

20.5

26

Numerical ExampleDrug TherapyDrug Therapy

Placebo Prozac µ ∆

Psycho-therapy

None 12 ± 2 18 ± 1 15 -6Psycho-therapy

CBT 17 ± 1 23 ± 3 20 -6

µ 14.5 20.5

∆ -5 -5

27

Numerical ExampleDrug TherapyDrug Therapy

Placebo Prozac µ ∆

Psycho-therapy

None 12 ± 2 18 ± 1 15 -6Psycho-therapy

CBT 17 ± 1 30 ± 3 20 -13

µ 14.5 20.5

∆ -5 -12

28

Evidence of Interaction


• In a 3x3x2 design, how many potential main effects are there? How many IVs would you collapse across to evaluate each main effect?

29

• Multi-factorial experiments manipulate several IVs to see if their effects interact

• Example Question: Does gender interact with psychotherapy in affecting depression?

• Two IVs:

• Gender. 2 Levels = male; female

• Psychotherapy. 2 levels: control (none); experimental (therapy)

• One DV: Depression (measure = BDI)

Example Multi-Factorial Experiment

30

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

31

Another 2-Factor Design, 3 Levels Per Factor

ArousalArousalArousal

Low Med High

Task Difficulty

Easy LowEasy

MedEasy

HighEasy

Task Difficulty

Average LowAverage

MedAverage

HighAverage

Task Difficulty

Hard LowHard

MedHard

HighHard

32

Another 2-Factor Design, 3 Levels Per Factor


Low Med High µ ΔLM ΔMH ΔLH

Task Difficulty

Easy 40 40 40 40 0 0 0

Task Difficulty

Avrge 15 30 15 20 15 -15 0Task Difficulty

Hard 8 5 2 5 -3 -3 -6

µ 21 25 19

ΔEA -25 -10 -25

ΔAH -7 -25 -13

ΔEH -32 -35 -38

33

Results: 3x3 Design

!"

#!"

$!"

%!"

&!"

'!"

()*" +,-./0" 1.23"

!"#$%#&

'()"*

+#%,-'.*

4567"

89,:52,"

15:-"

34

3x3 Results: Main Effects, No Interaction


Low Med High µ ΔL

M

ΔM

H

ΔLH

Task Difficult

y

Easy 30 40 50 40 10 10 20

Task Difficult

yAvrge 15 25 35 25 10 10 20

Task Difficult

y

Hard 6 16 26 16 10 10 20

µ 17 27 37

ΔEA -15 -15 -15

ΔAH -9 -9 -9

ΔEH -24 -24 -2435

3x3 Results: 2 Main Effects, No Interaction

!"

#!"

$!"

%!"

&!"

'!"

(!"

)*+" ,-./01" 2/34"

!"#$%#&

'()"*

+#%,-'.*

5678"

9:-;63-"

26;."

36

• If one IV has an effect--that is, there’s a significant effect of going from one level of that IV to another, while ignoring (“collapsing across”) all other IVs--then that IV is said to produce a “main effect”.

• If the effect of one IV differs depending on the level of another IV, there’s an interaction.

Interpreting Data from Factorial Designs

37

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'!"

()*+,)-"""""""""""""

.*)"+/0,1234"

5/0,123"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'38

The Importance of Interactions

• Interpretation of interaction fx overrides interpretation of main fx

• Example: What’s most important in these results: Main effect of gender? Main effect of therapy? Interaction of the two?

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

If the gender factor is ignored, the therapy seems to simply be effective for all people. But this is not true. It is effective for females only.

39

X-Way Interactions

• When there are 2 IVs, a 2-way interaction is possible,with 3 IVs, may have a 3-way interaction, etc.

• 3-way interaction means the 2-way interaction changes depending on a 3rd variable.

40

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

Introverts Extroverts



41


42

8.3 Mixed Multi-Factor Designs

43

All Participants (N = 20)

Condition 1(n = 10)

Condition 2(n = 10)

Review: Between-Subjects Design

44

All Participants (N=10)

Level 1 (N = 10)

Level 2 (N = 10)

Review:Within-Subjects Design

45

• With multiple factors/IVs, one can mix different kinds of variables (within/between; subject/manipulated, etc.)

• If all IVs are within-subjects then the design is “fully within”

• If all IVs are between-subjects then the design is “fully between”

• Otherwise, it’s a “mixed” design

Within, Between & Mixed Multi-Factor Designs

46

All Participants (N = 20)

Condition A1B1 (n=5)




2x2 Fully Between Subjects Design

47

All Participants (n = 20)

Condition A1B1(n = 20)




2x2 Fully Within

Subjects Design

Note that orders are not shown, there would be 24 for a fully-counterbalanced design!

48

All Participants (20)Level B1 (10) Level B2 (10)

A1B1(10)

A2B1(10)

A1B2(10)

A2B2(10)

2x2 Mixed Design

49

• a.k.a., Repeated-measures factorial design.

• All subjects are run through all conditions (i.e., all cells of the factorial matrix).

• Same advantages/disadvantages as single-factor repeated measures design

Fully Within-Subjects Factorial Design

50

• Question: Is face recognition more impaired by inversion than object recognition?

• Method

• Subjects are 20 undergraduates

• Materials are pictures of 25 famous faces and 25 common objects, either inverted or not. (So 100 images in all).

Example Experiment 1:Fully Within-Subjects

51

• Design: 2x2 Fully within-subjects factorial, with factors being Type of Image (Face or Object) and View (upright or inverted).

• Procedure: All 20 subjects are shown all 100 images several times in random order and asked to identify each as quickly as possible. Repeated-measures factorial design.

• DV is reaction time to name picture.

Example Experiment 1

52

Image TypeImage Type

Face Object

View

Upright

View

Inverted

53

• Expected results: RT will be higher for inverted images than upright ones (main effect). But this effect will be greater for faces (interaction).

• Implications: Implies that there’s something different about how people process faces as compared to objects


54

Possible Results

!"#!!"$!!"%!!"&!!"'!!"(!!")!!"

*+,-./0" 1234,045"

!"#$%&'%()*"+,%

-./)010*%23/"$.#./4$%

67849" :;<4809"

55

• Each subject run through only one condition (i.e., one cell of the factorial matrix)

• If all IVs are subject variables, you have a Nonequivalent groups factorial design

• If all IVs are manipulated, decide how equivalent groups are formed:

• Random assignment: Independent groups factorial design

• Matching: Matched groups factorial design

Fully Between-Subjects Factorial Designs

56

• Question: Same as before, “are faces more affected by inversion than objects?”

• Method

• Subjects are 80 undergraduates (note higher N than within-Ss design).

• Materials: Same as before, 25 pictures of faces, 25 pictures of objects, shown both upright and inverted.

Example Experiment 2: Fully Between-Subjects

57

• Design 2×2 fully between-subjects factorial design. Assign subjects randomly to one of four groups of 20. Independent groups factorial design.

• Procedure: Each group sees 25 pictures (upright faces, inverted face, upright objects, or inverted objects).


58


Face Object

View

Upright

View

Inverted

59


60

• At least one IV within-subjects and one between-subjects.

• Subjects run through all levels of some IVs, but only single level of other IVs. That is, each subject goes through one row or column of the factorial matrix.

• Random assignment, matching, counterbalancing can all be used.

Mixed Factorial Designs

61

• Question: Is face recognition more impaired by inversion than object recognition?

• Method

• Subjects are 40 undergraduates (note higher N than fully within, but lower than fully between).

• Materials are pictures of 25 famous faces and 25 objects, either inverted or not.

Example Experiment 3:Mixed Factorial Design

62

• Design: 2x2 Mixed factorial with factors being Type of Image (face or object, within) and View (upright or inverted, between)

• Procedure: 20 subjects are shown the 50 inverted images (25 faces and 25 objects), while 20 other subjects are shown the 50 upright images (25 faces, 25 objects).


63


Face Object

View

Upright

View

Inverted

64

• “Person by Environment”

• Variety of fully-between or mixed factorial design

• At least one subject IV (person) and at least one manipulated IV (“environment”)

PxE Factorial Designs

65

• Question: Does the effect of assigned study style interact with preferred study style?

• Method

• Person IV: Ss assigned to groups based on preferred study style: Crammers or Distributers. This is a subject IV

• Enviro IV: Half of subjects in each above group are assigned to study by cramming or by distributing study. This is manipulated

Example Experiment 4: PxE Design

66

Possible Results

Preferred Style (subject)Preferred Style (subject)

Crammer Distributer

Assigned Style (manipulated)

Cramming 65 65

Assigned Style (manipulated)

Distributing 80 90

67

Possible Results

!"#

$"#

%"#

&"#

'"#

(""#

)*+,,-./# 0-12*-342-./#

!"#$%&'"

()*+,-,."/$*'0)%)*10"

)*+,,5*1# 0-12*-3425*1#

Assigned Study Style

68

• Cannot draw causal links for the subject variables, can draw causal links for the manipulated (”environment”) variable.

• So a causal link can be established for assigned style but not preferred style.

• Cannot draw causal links for interaction effects.

Interpreting Results From PxE Designs

69

Example 2x3x2 StudyCaspi et al., 2007, PNAS, 104 (47), 18860-18865

70

How Many Participants?

• If I need 50 participants per cell in a 2×2 factorial design, what is the total N?

• What if the design is fully within?

• What if the design is mixed?

• Answer the same questions for a 3×2×3 design with 10 participants per cell.

71

Analyzing Data From Multi-Factor Designs

• As for multi-level designs, multi-factor designs are generally analyzed via ANOVA procedures:

• Pre-tests for normality and other assumptions

• 2-way (or X-way) ANOVA/MANOVA/ANCOVA...

• Post-hoc tests to examine effects in greater detail

• Planned comparison techniques may also be involved

• Note that there are no well-established techniques for dealing with multi-factor ordinal-scale data

72


73

8.4 Summary: Design Complexity

74

• Can’t detect non-linear effects.

• Can’t detect interactions.

• Involve only simple counter-balancing or simple equivalent groups problems.

Single-Factor, 2-Level Experimental Designs

75

• Can detect non-linear effects

• Can’t detect interactions

• May involve relatively complex counter-balancing or equivalent groups problems

Single-Factor, Multilevel Designs

76

• Multi-factor Designs

• Can detect interactions and main effects

• Can detect non-linear effects where IVs have ≥ 3 levels

• May involve both complex counter-balancing and equivalent groups problems.

Multi-Factor Designs

77

Conclusion: Experimental Design

• Experiments and quasi-experiments are just one way of doing research

• True experiments (not quasi) allow conclusions about causality

• Next we will turn to observational research, which is simpler in some ways

78

8. multi-factor designs

Documents