KNR 295Exptl

DesignSlide 1

Experimental Design

Ch. 7 – Let’s not kid ourselves, this

is going to hurt

KNR 295Exptl

designSlide 2 Experimental Design

How on Earth can you ensure that 2 groups of different people are equal (in all respects, not just on the measure of choice) at the beginning of an experiment? You can’t But you can make it more probable

(and to experimenters, good enough)

Remember though, even if you achieve this, groups can still grow different after they have been formed

KNR 295Exptl

designSlide 3 Experimental Design

Searching for group equivalence What we do:

Random assignment

Does it work? Maybe! Sample size, power &c.

KNR 295Exptl

designSlide 4 Experimental Design

If random assignment is the solution, and increased internal validity is the benefit, is there a cost? Undoubtedly

Sample size big enough? Control of social threats, & mortality Its unreal, so improved internal validity

comes at the cost of external validity

KNR 295Exptl

designSlide 5 Experimental Design

2-group experimental designs

Two-group, post-test only randomized experimental design

KNR 295Exptl

designSlide 6 Experimental Design

More on probabilistic equivalence Random assignment will

distribute folk to groups such that their scores on any measure will be distributed randomly (duh)…this means they will probably be different, but that it is statistically improbable that this will be a significant difference

KNR 295Exptl

designSlide 7 Experimental Design

More on probabilistic equivalence

KNR 295Exptl

designSlide 8 Experimental Design

Random selection Random assignment

External validity control

Internal validity control

KNR 295Exptl

designSlide 9 Experimental Design

Classifying experimental designs Signal enhancing vs. noise

reducing The signal vs. noise idea:

Strong treatment enhances signal

Good measurement reduces noise

KNR 295Exptl

designSlide 10 Experimental Design

Classifying experimental designs Signal enhancing vs. noise

reducing Designs differ in their strengths ~

Factorial designs focus on isolating aspects or combinations of treatments that seem to affect the measurement most (signal enhancer)

Covariance/blocking designs focus on lessening the effects of known sources of noise (noise reducers)

KNR 295Exptl

designSlide 11 Experimental Design

Factorial designs Imagine an educational

program… You are interested in (IV’s)

Time of instruction (1 hour vs. 4 hr) Setting (in-class or pulled out of class)

You measure via study scores (DV)

Note – we are now dealing with 2 independent

variables for the first time

KNR 295Exptl

designSlide 12

Experimental Design: Factorial

KNR 295Exptl

designSlide 13

Experimental Design: Factorial

KNR 295Exptl

designSlide 14

Experimental Design: Factorial

KNR 295Exptl

designSlide 15

Experimental Design: Factorial

KNR 295Exptl

designSlide 16

Experimental Design: Factorial

KNR 295Exptl

designSlide 17

Experimental Design: Factorial

KNR 295Exptl

designSlide 18

Experimental Design: Factorial

KNR 295Exptl

designSlide 19

Experimental Design: Factorial

A silly example - The marshmallow peeps study Factor 1: Alcohol

(presence/absence) Factor 2: Smoking (yes/no)

KNR 295Exptl

designSlide 20

Experimental Design: Factorial

Does alcohol have an effect? Imbibed liberally Moderate

headache Nausea No permanent

damage

KNR 295Exptl

designSlide 21

Experimental Design: Factorial

Does tobacco have an effect?

No marketing to young chicks Peep grabs a ciggie… …lights up…

…begins smoking… …& continues ‘til

satiated It can give up any

time it wants to…no effect

KNR 295Exptl

designSlide 22

Experimental Design: Factorial

So, alcohol & nicotine are benign?

Wait..what if you combined them? Sum of the parts? More than the sum of the parts?

KNR 295Exptl

designSlide 23

Experimental Design: Factorial

Is there an interaction? Combine the elements

Faint flame…blackening …smell of caramel…

Metamorphosis “ball of charred

goo…” “less sweet” “crunchier” “gross”

KNR 295Exptl

designSlide 24

Experimental Design: Factorial

Variations – i. 2 x 3

KNR 295Exptl

designSlide 25

Experimental Design: Factorial

Variations – i. 2 x 3

KNR 295Exptl

designSlide 26

Experimental Design: Factorial

Variations – ii. 2 x 2 x 3 (3 factor)

KNR 295Exptl

designSlide 27

Experimental Design: Factorial

Variations – iii. 2 x 3 + control

KNR 295Exptl

designSlide 28

Experimental Design: Blocking

Reducing noise – Randomized block designs Key point – unexplained variation in a

sample reduces power The solution is to reduce the variation within the

sample by splitting the sample up You split across some factor that you know causes

the sample to differ with respect to the measure of interest (making multiple blocks)

You do not include this as a factor in the experiment, because it is not of interest

Each block will have less variability on the measure, and therefore more power

KNR 295Exptl

designSlide 29

Experimental Design: Blocking

Reducing noise – Randomized block designs

Here is the design

notation for what was

described on the last slide

KNR 295Exptl

designSlide 30

Experimental Design: Blocking

Reducing noise – Randomized block designs

“+’s” show scores for all treatment group

members (average of all “+” gives treatment

group score – average on x-axis is for pretest, and on y-axis is for posttest

“o’s” show scores for all control group members (average of all “o” gives treatment group score – average on x-axis is for pretest, and on y-axis is

for posttest

KNR 295Exptl

designSlide 31

Experimental Design: Blocking

Reducing noise – Randomized block designs

Note that, regardless of the block, the spread of

scores on the post-test is less within the block than

across the entire measure

KNR 295Exptl

designSlide 32

Experimental Design: Covariates

Reducing noise – Covariance designs Design can vary, but basic is this –

Lingo – “controlling for”, “removing the effect of” Both terms imply use of covariates

KNR 295Exptl

designSlide 33

Experimental Design: Covariates

Reducing noise – Covariance designs

KNR 295Exptl

designSlide 34

Experimental Design: Hybrids

Solomon 4 group

To examine & control testing effects in pre-

post arrangements

KNR 295Exptl

designSlide 35

Experimental Design: Hybrids

Switched replication design

To examine & control social interaction

threats

KNR 295Exptl

designSlide 36

Experimental Design: Hybrids

Reducing social interaction threats (other than via switched

replication) Blind & double blind set ups Placebos Isolation of groups