research methods in psychology repeated measures designs
TRANSCRIPT
Research Methods in Psychology
Repeated Measures Designs
Repeated Measures Designs
Each individual participates in each condition of the experiment• completes the DV with each condition• hence “repeated measures”
Also called “within-subject” design• entire experiment is conducted “within” each
subject
Repeated Measures Designs, continued
Why Use a Repeated Measures Design?• no need to balance individual differences
across conditions of experiment all participants are in each condition
• fewer participants needed• convenient and efficient• more sensitive
Sensitivity
A sensitive experiment• can detect the effect of an independent
variable• even if the effect is small
Repeated measures designs are more sensitive than independent groups designs• “error variation” is reduced
same people participate in each condition variability due to individual differences eliminated
Practice Effects
Main disadvantage of repeated measures designs is practice effects• People change as they are tested repeatedly
performance may improve over time people may become bored or tired over time
Practice effects become a potential confounding variable if not controlled
Practice Effects, continued
Example:• Suppose a researcher compares two different
study methods, A and B Condition A: participants use a highlighter to mark
key points while reading a text, then take a test on the material
Condition B: participants read a text, then make up sample test questions and answers, then take a test on the material
Practice Effects, continued
• Suppose all participants first experience Condition A and
then Condition B results indicate test scores are higher in Condition
A compared to Condition B
• Is marking text with highlighter (A) better than writing sample questions/answers (B)? impossible to know
• confounding of IV with order of presentation• practice effects (boredom, fatigue) may account for
poorer performance in Condition B
Practice Effects, continued
Practice effects must be balanced, or averaged, across conditions• Counterbalancing the order of conditions
distributes practice effects equally across conditions half of the participants do Condition A, then B the remaining participants to Condition B, then A Conditions A and B then have equivalent practice
effects practice effects aren’t eliminated, but they are
averaged across the conditions of the experiment
Counterbalancing Practice Effects
Two types of repeated measures designs• Complete and Incomplete• purpose of each type of design is to
counterbalance practice effects• each design uses different procedures for
counterbalancing practice effects
Complete Design
Practice effects are balanced within each participant in the complete repeated measures design• each participant experiences each condition
several times, using different orders each time• a complete repeated measures design is used
when each condition is brief (e.g., simple judgments
about stimuli)
Complete Design, continued
Two methods for generating orders of conditions• block randomization• ABBA counterbalancing
Complete Design, continued
Block randomization• a block consists of all conditions (e.g., 4
conditions: A, B, C, D)• generate a random order of the block (ACBD)• participant completes condition A, then C,
then B, then D• generate a new random order for each time
the participant completes the conditions of the experiment (e.g., DACB, CDBA, ADBD)
Complete Design, continued
Block randomization• balances practice effects only when
conditions are presented many times• practice effects are averaged across the
many presentations of the conditions• practice effects are not balanced if conditions
are presented only a few times to each participant
Complete Design, continued
ABBA counterbalancing• used when conditions are presented only a
few times to each participant• procedure: present one random sequence of
conditions (e.g., DABC), then present the opposite of the sequence (CBAD)
• each condition has the same amount of practice effects
Complete Design, continued
ABBA counterbalancing• balance practice effects that are “linear”
linear practice effects • participants change in the same way following each
presentation of a condition
nonlinear practice effects• participants change dramatically following the
administration of a condition• example: participant experiences insight about how to
complete an experimental task (“aha … now I get it”)• likely to use this insight in subsequent conditions
Complete Design, continued
• Example of linear practice effects suppose participants gain “one unit” of practice
with each administration (“trial”) of a condition • there are zero practice effects with the first
administration
Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6Condition A B C C B A
Practice Effects +0 +1 +2 +3 +4 +5
Practice effects are balanced because total practice effects is +5 for each condition:
A: 0 + 5 B: 1 + 4 C: 2 + 3
Complete Design, continued
• Example of nonlinear practice effects: Suppose a participant figures out a method for
completing the task on the third trial, and then uses the new method for subsequent trials
Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6
Condition A B C C B A
Practice Effects +0 +1 +5 +5 +5 +5
Practice effects are not balanced across the conditions:
A: 0 + 5 = 5 B: 1 + 5 = 6 C: 5 + 5 = 10
Complete Design, continued
• Nonlinear practice effects create a confounding differences in scores on the DV may not be caused
by the IV (conditions A, B, C) differences on DV may be due to different amounts
of practice effects associated with each condition
• ABBA counterbalancing should not be used when practice effects are likely to vary or change
over time (i.e., nonlinear practice effects) use block randomization instead
Complete Design, continued
• ABBA counterbalancing should not be used when anticipation effects can occur participants develop expectations about which
condition will appear next in a sequence responses may be influenced by expectations
rather than actual experience of each condition if anticipation effects are likely, use block
randomization
Incomplete Design
Each participant experiences each condition of the experiment exactly once• complete design: more than once
Practice effects are balanced across participants in the incomplete design• complete design: practice effects balanced
within each subject
Incomplete Design, continued
General rule for balancing practice effects• each condition (e.g., A, B, C) must appear in
each ordinal position (1st, 2nd, 3rd) equally often
• if this rule is followed, practice effects will be balanced across conditions will not confound the experiment
Incomplete Design, continued
Two techniques for balancing practice effects in an incomplete repeated measures design• all possible orders• selected orders
Incomplete Design, continued
All possible orders• use when there are four or fewer conditions• two conditions (A, B) → two possible orders: AB, BA
half of the participants would be randomly assigned to do condition A first, followed by B
other half of participants would complete condition B first, followed by A
• three conditions (A, B, C) → six possible orders:ABC, ACB, BAC, BCA, CAB, CBA
participants would be randomly assigned to one of the six orders
Incomplete Design, continued
• four conditions (ABCD) → 24 possible orders (ABCD, ABDC, ACBD, ACDB, ADBC, etc.)
• five conditions → 120 possible orders• six conditions → 720 possible orders• at least one participant must receive each
order of the conditions therefore, all possible orders is used for
experiments with four or fewer conditions of the IV
Incomplete Design, continued
Selected orders• select particular orders of conditions to
balance practice effects• two methods
Latin Square random starting order with rotation
• each condition appears in each ordinal position exactly once
• each participant is randomly assigned to one of the orders of conditions
Incomplete Design, continued
• Procedure for Latin Square randomly order the conditions of the experiment
(e.g., ABCD) number the conditions (A = 1, B = 2, C = 3, D = 4) use this rule for generating the 1st order:
1, 2, N, 3, N – 1, 4, N – 2, 5, N – 3, 6, etc.
where N = last number of conditions
• the first order of four conditions would be 1 2 4 3
Incomplete Design, continued
to generate 2nd order of conditions add 1 to each number in the first order (1 2 4 3)
• “N” represents the number of conditions (e.g., 4); we can’t use N + 1 because this would create a 5th condition
• additional rule: N + 1 always is “1” -- the first condition• the second order of conditions is 2 3 1 4
to generate 3rd order of conditions add 1 to each number in the second order (again, N + 1 = 1)
• the third order of conditions is 3 4 2 1 follow the same procedure for each subsequent
order• The number of orders is the same as the number of
conditions (e.g., 4 conditions → 4 orders)
Incomplete Design, continued
Match letters of conditions to their numbers to create the Latin Square
1st 2nd 3rd 4th 1st 2nd 3rd4th
1 2 4 3 A B D C
2 3 1 4 B C A D
3 4 2 1 C D B A
4 1 3 2 D A C B
Incomplete Design, continued
• Each condition appears in each ordinal position equally often, which balances practice effects For example, condition “A” appears in each ordinal
position:
1st 2nd 3rd 4th
A B D C
B C A D
C D B A
D A C B
Incomplete Design, continued
• Another advantage of Latin Square each condition precedes and follows every other
condition once (e.g., AB and BA, BC and CB)
1st 2nd 3rd 4th
A B D C
B C A D
C D B A
D A C B
this helps to control for potential order effects
Incomplete Design, continued
Random starting order with rotation• generate a random order of conditions (e.g., ABCD)• rotate the sequence by moving each condition one position to
the left each time
1st 2nd 3rd 4th
A B C D
B C D A
C D A B
D A B C each condition appears in each ordinal position to balance
practice effects unlike Latin Square, order of conditions is not balanced
Data Analysis of Repeated Measures Designs
Complete repeated measures designs require an additional step• Because participants complete each condition
many times, a summary score (e.g., mean) is computed for each participant for each condition
• this represents each participant’s average performance in each condition
Data Analysis, continued
• Suppose two participants complete two conditions (A, B) of
an experiment four times each the DV is their rating on a 1–5 scale assume IV (conditions A, B) represents two types
of stimuli participants are asked to judge (e.g., size of the stimuli)
Data Analysis, continued
Suppose the following data are observed in an ABBA design:
Condition Participant 1 Participant 2
A 2 1
B 4 4
B 5 4
A 1 1
B 3 5
A 1 2
A 2 3
B 5 5
Data Analysis, continued
To analyze these data we first need to compute the average rating for each condition (A, B) for each participant:
Condition Participant 1 Participant 2
A 2 1
B 4 4
B 5 4
A 1 1
B 3 5
A 1 2
A 2 3
B 5 5
A: (2+1+1+2)/4 = 1.50 A: (1+1+2+3)/4 = 1.75
B: (4+5+3+5)/4 = 4.25 B: (4+4+5+5)/4 = 4.50
Data Analysis, continued
Next calculate the mean for each condition across all participants
In this example with two participants, the means for conditions A and B are
Condition A Condition Bparticipant 1 1.50 4.25participant 2 1.75 4.50
mean 1.625 4.375
Null hypothesis testing or confidence intervals would be used to determine whether this difference between means is reliable
The Problem of Differential Transfer
Repeated measures designs should not be used when differential transfer is possible• occurs when the effects of one condition
persist and affect participants’ experience of subsequent conditions
• use independent groups design instead• assess whether differential transfer is a
problem by comparing results for repeated measures design and random groups design
Comparison of Two Designs
Differences between repeated measures design and independent groups design• Independent variable
repeated measures: each participant experiences every condition of the IV
independent groups: each participant experiences only one condition of the IV
• What is balanced (averaged) across conditions to rule out alternative explanations for findings? repeated measures: practice effects independent groups: individual differences variables