psy2174 s11...

5. Experimental Research1

• Know the difference between observational research (including quasi-experimental research), and experimental research

• Explain the basic logic of experiments

• Know the various sorts of error that can come up in an experiment and how to deal with them

• Understand the importance of control groups

• Recognize the difference between manipulated (true experiment) and subject (quasi-experiment) variables

• Understand the various threats to an experiment’s validity and how to deal with them

Goals

2

5.1 Experiment vs. Observation

3

• All research involves careful observation.

• Experimental research is distinct in terms of:

• Manipulation: The level of certain variables of interest is modified purposefully by the experimenter

• Control: All other variables (ideally) are held constant

• Because only the manipulated variable changes, any change in the observed variable(s) can be attributed to the manipulated variable(s)

• Thus experiments (and only experiments) allow conclusions about causality

Experimental Research

4

• Observational research does not manipulate variables and may not control any (usually does not)

• Examples include: Survey research, naturalistic observation, correlational research, case studies, etc.

• Quasi-experimental research:

• A form of observational research

• Set up like an experiment, with different groups

• But: Manipulation of variables is not possible and/or has already occurred prior to the experiment

• Examples: Research on gender, age, ethnicity, etc. etc.

Observational Research

5

Basic Structure of a True Experiment

• Some variables (independent variables or IVs) are manipulated

• Some variables (dependent variables or DVs) are measured

• All other variables (extraneous variables or EVs) are held constant

6

• The variables manipulated by the experimenter

• Always have 2+ levels. One level is often the control level, and the other(s) are the experimental levels(s).

• Example: If the IV is “therapy”, there would be (at least) two levels: No therapy (control) and therapy (exp).

Independent Variables

7

Basic Structure of a Quasi-Experiment

• Subjects are categorized according to some variable(s). These are the IVs. Example: Participants divided by sex

• Some variables (dependent variables or DVs) are measured

• Where possible, other variables (EVs) are held constant

8

• IV is “manipulated” when the experimenter can decide which levels each participant experiences. Example: If you make one group of P’s watch a scary movie while another group watches a neutral movie, you are manipulating anxiety.

• Manipulated variables allow causal conclusions to be made regarding the effect of the IV on the DV.

• Only with manipulated IVs do you have a true experiment (as opposed to quasi-experiment)

Manipulated vs. Subject IVs

9

• An IV is a Subject IV* when the experimenter cannot decide which levels each participant will go through Example: In a study on gender effects, one can’t decide whether P’s go through study as men or women

• Not truly manipulated, so do not allow causal conclusions about the effect of the IV on the DV

• Studies with subject IVs are not true experiments, they are called “Quasi-experimental”.

* A.K.A.: natural groups, non-manipulated, or ex post facto IV

Manipulated vs. Subject IVs

10

In an experiment on the effects of claustrophobia on intelligence, an experimenter has one group of participants do an IQ test in a small room while the other group does the test in a large room. What is the IV, and is it a subject or manipulated IV? Is this an experiment or a quasi-experiment?Note: The name of the IV is not usually there in the research description, you have to come up with it on your own based on the description of the levels

Subject or Manipulated?

11

In an experiment on the effects of gender on perceived intelligence, an experimenter has one group of participants judge pictures of female faces while the other group does the task with pictures of male faces. What is the IV,and is it a subject or manipulated IV? Is this an experiment or a quasi-experiment?


12

In an experiment on the effects of anxiety on intelligence, an experimenter divides people into anxious and non-anxious groups based on their scores on a standard anxiety scale. He then has all of them do an IQ test. What is the IV, and is it a subject or manipulated IV? Is this an experiment or a quasi-experiment?


13

Experiment or Quasi-Experiment?

TreatmentTreatment

Placebo Drug

GenderGender

Male Female

TreatmentTreatmentTreatment

Placebo Dose1 Dose2

GenderGender

M F

Anxiety

High HighM HighF

Anxiety Med MedM MedFAnxiety

Low LowM LowF

14

• a.k.a. the DVs or “measured variables”

• The variables thought to be causally* affected by the IVs.

• The variables measured by the experimenter.

• Vary by scale: Nominal, Ordinal, Interval, Ratio

• DVs are measures representing constructs

* Note: This can only be established in an experiment, not a quasi-experiment

Dependent Variables

15

• EV: All variables besides the IVs and DVs.

• The experimenter seeks to hold constant (control) these.

• Uncontrolled EVs produce systematic or random measurement error, which can vary between experimental conditions or not

Extraneous Variables

16

Error from EVs

Random Systematic

No

Yes

Homogeneity of Variance

Equal absolute offset

Inhomogeneity of Variance

CONFOUND

Type of ErrorD

iffer

s be

twee

n ex

peri

men

tal g

roup

s?

17

• Confound: An EV that produces systematic error whose magnitude varies between experimental conditions

• Severely weakens interpretability of an experiment.

• Any observed effects are due to an unknown combination of IV’s effect and EV’s effect.

• Example: A study tests kids before taking a reading program and 6 months after. The kids read better after. Was this due to the program? Who knows? The kids’ maturity is systematically greater after than before, thus it constitutes a confound.

Confounds

18

• Equal Offset: When an EV produces equal systematic error across experimental conditions

• May not weaken interpretability of an experiment if one is interested in differences between conditions

• Does produce problems if one is interested in the correct absolute value of the DVs

• May also produce problems indirectly by generating ceiling or floor effects

Equal Offset

19

Example: In a study of claustrophobia effects on accuracy, one group is tested in a small room, the other in a large one. If both have a large window, this might reduce the sense of crowding. However, the group differences should be the same as if there was no window. Exception: If the window greatly reduces the sense of crowding in both rooms, it will produce a ceiling effect.

Equal Offset

20

Acc

urac

y

5060708090

100

Small Large

with slightly effective window

ROOM SIZE

5060708090

100

Small Large

If there were no window

ROOM SIZE

5060708090

100

Small Large

with highly effective window

ROOM SIZE

Example 2: A theory suggest that women should on average obtain a score of around 3 on a certain political engagement (PE) measure, while men should obtain about a 5 on average. A study is done to test this theory. However, because the sample is drawn from Ottawa, and the people there are higher in PE, the scores do not reflect this. Geographic location produces a problematic equal offset in this case.

Equal Offset

21

Polit

ical

Eng

agem

ent

General Population

0

3

6

Male Female

Ottawa Population

0

3

6

Male Female0

3

6

Male Female

Halifax Population

• All experiments involve random variations between subjects.

• When one conditions produces greater variability than the other, we have inhomogeneity of variance.

• Not generally a major problem, except that it affects what kinds of statistical procedures one can use. Parametric inferential statistical procedures are based on having (roughly) equal variance in all conditions.

• Nonparametric statistical procedures (which are less powerful) must be used in this case.

Inhomogeneity of Variance

22

• Variability within each experimental condition is equal

• This is the ideal situation (although one always wants as little measurement error as possible).

• Parametric stats can be used

Homogeneity of Variance

23

• Avoid confounds at all costs

• Avoid equal systematic error at all costs if interested in absolute values, otherwise it’s less worrisome

• Try to make variability equal between groups

• Minimize random measurement error for greater experimental power

Error: Summary

24

Experimental Power

25

Questions

• What is the logic of a “true” experiment?

• What is a quasi-experiment?

• Why is it a problem if systematic error is different in magnitude across conditions?

26

• Question: Does distributed study (DS) lead to better retention than last-minute cramming (LMC)?

• One IV: Study style. OpDef = Assigned distribution of study time in learning a textbook chapter

• LMC Group: 3 hrs. study on Monday

• Baseline Group: 3 hrs. study on both Mon & Tues

• DS Group: 3 hours study on Mon, Tue, and Wed

• DV = Retention. OpDef: Multiple choice test of material on Friday

Example Experiment: Study Style

27

Results

!"

#!"

$!"

%!"

&!"

'!"

(!"

)!"

*!"

+!"

#!!"

,-./01234/5"

67-11238"

9-.5:235"

;/7-/58<"

=2./72>4/5?"

;/4?<238"

!"#$%&$%&'(%")*(+$"

)&,-.'%/")&+0&$/."

28

Study Style Experiment: Schedules for Groups

Mon Tue Wed Thr Fri

LMC 3 hrs. Test

Base-line 3 hrs. 3 hrs. Test

DS 3 hrs. 3 hrs. 3 hrs. Test!

29

• Total study time: Distributed Studiers have more total study time!

• Retention interval: Distributed Studiers have shorter time to hold on to info.

• Day of the week: It might be harder to study on Monday than Wednesday.

Confounds in Example Experiment

30

IV: Distribution of Study

EV1: Study hours

EV2: Retention Interval

DV: Retention

1 day 3 hours 3 days Low

2 days 6 hours 2 days Average

3 days 9 hours 1 day High

...but causal link could be this...

Would like to draw a causal link between these two...

...or this...

31

...or some combination of these (we have no way of knowing).

Version 2

Mon Tue Wed Thr Fri

LMC 3 hrs. Test

Base-line 1.5 hrs. 1.5 hrs. Test

DS 1 hrs. 1 hrs. 1 hrs. Test"32

Version 3

Mon Tue Wed Thr Fri

LMC 3 hrs. Test

Base-line 1.5 hrs. 1.5 hrs. Test

DS 1 hrs. 1 hrs. 1 hrs. Test"33

Question your Question

• Beyond questions about methodological details, we have to ask about theory and rationale

• Are we really asking the right question with any of these designs?

• Should we instead be measuring long-term retention beyond a few days?

34

Discussion / Questions

• Which version of the example studying experiment is better? Why?

• We didn’t solve the day-of-week issue, how problematic is that?

• What other critiques might we launch at this study? Are we asking the right question here?

35

• Multi-factorial experiments manipulate several IVs to see if their effects interact

• Example Question: Does gender interact with psychotherapy in affecting depression?

• Two IVs:

• Gender. 2 Levels = male; female

• Psychotherapy. 2 levels: control (none); experimental (therapy)

• One DV: Depression (e.g., measure with BDI)

Example Multi-Factor Experiment

36

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"!"#$%&#'()*)+'

,-)./$"'()*)+'

!"

#!"

$!"

%!"

&!"

'()*+(,"""""""""""""

-)("*./+0123"

4./+012"

!"#$%&#'()*)+'

,-)./$"'()*)+'

37

Some possible patterns of results... Two-Factor Design, with 2 levels per factor

PsychotherapyPsychotherapy

Control Therapy

M ControlM TherapyM

F ControlF TherapyFGe

nd

er

38

Example Results 1:Main Effect Only


Control Therapy µ Δ

M 30 ± 3 20 ± 3 25 10

F 30 ± 2 20 ± 4 25 10

µ 30 20

Δ 0 0

Ge

nd

er

39

Possible Results 2:Main Effect+Interaction



M 30 ± 3 20 ± 3 25 10

F 30 ± 2 10 ± 4 20 20

µ 30 15

Δ 0 10

Ge

nd

er

40

Possible Results 3:Interaction Only



M 40 ± 3 30 ± 3 35 10

F 30 ± 2 40 ± 4 35 -10

µ 35 35

Δ 10 -10

Ge

nd

er

41

Another 2-Factor Design, 3 Levels Per Factor

ArousalArousalArousal

Low Med High

Task Difficulty

Easy LowEasy

MedEasy

HighEasy

Task Difficulty

Average LowAverage

MedAverage

HighAverage

Task Difficulty

Hard LowHard

MedHard

HighHard

42

Another 2-Factor Design, 3 Levels Per Factor

ArousalArousalArousal

Low Med High µ ΔLM ΔMH ΔLH

Task Difficulty

Easy 40 40 40 40 0 0 0

Task Difficulty

Avrge 15 30 15 20 15 -15 0Task Difficulty

Hard 8 5 2 5 -3 -3 -6

µ 21 25 19

ΔEA -25 -10 -25

ΔAH -7 -25 -13

ΔEH -32 -35 -3843

Results: 3x3 Design

!"

#!"

$!"

%!"

&!"

'!"

()*" +,-./0" 1.23"

!"#$%#&

'()"*

+#%,-'.*

4567"

89,:52,"

15:-"

44


• If a multi-factorial study has 3 factors, what the minimum number of conditions it must have?

• Can a study with 2 factors have 2 levels in one factor and 3 in the other?

45

5.2 Threats to Experimental Validity

46

• Does the experiment really test the hypothesis it’s supposed to? Several sub-questions here:

• Construct Validity: Are you really measuring the variables you think you are?

• External Validity: Do your findings generalize?

• Internal validity: Is the experiment well-controlled?

• Statistical Validity: Are you using the right stats procedure?

Experimental Validity

47

• Using a measure that has poor construct validity means your experiment isn’t testing what you think it is.

• Obviously, this leads to poor experimental validity

• Review Chapter 4 material for more

Threats to Construct Validity

48

• Research has poor external validity when results do not generalize to the population of interest

• Due to sampling and testing methods, data may apply only to...

• Limited groups (e.g., only students)

• Limited environments (e.g., the lab)

• Limited times or places (e.g., North America)

• Related terms: Ecological validity, Transferability, Generalizability, etc.

Threats to External Validity

49

• There are LOTS! Some examples:

• Time-Based: History, maturation, regression to the mean

• Materials-Based: Testing effects, placebo effects, and instrumentation effects

• Subject-Based: Group differences, differential attrition, participant bias and experimenter bias

• We will spend some time on these, and methods for avoiding them

Threats to Internal Validity

50

• Using the wrong test for your measurement scale Example: Using parametric inferential statistical procedure with ordinal data

• Failing to test for assumptions of inferential test before using it. Example: Most parametric tests assume your data is normally distributed. Is it?

• Running multiple tests on the same data set for the same hypothesis: Causes Alpha inflation, increasing chance of Type I Error (saying an effect is there when it isn’t)

• Running experiment with low statistical power, thus causing high chance of Type II Error (saying no effect is there when it is)

Threats to Statistical Validity

51

• Floor Effect: No effect is measured because scores in all conditions are near the minimum possible value (e.g., 20% on mult-choice test with 5 answers).

• Ceiling Effect: No effect is measured because scores in all conditions are near the maximum possible value (e.g., 100%).

• Can lead to impression of an interaction effect where none exists

Floor & Ceiling Effects

52

• It is not always so obvious where the floor and ceiling lie in a data range.

• How can you confirm that the data are showing a ceiling/floor effect? Your old friend, exploratory stats! Specifically, frequency distribution.

• Normally distributed? Ceiling/floor effect unlikely.

• Skewed away from maximum? Probable ceiling effect

• Skewed away from minimum? Probable floor effect

DetectingFloor & Ceiling Effects

53

• In a study of face recognition, participants are asked to memorize then identify faces of different levels of distinctiveness (Distinctive, Average, or Nondescript)

• Inferential stats show no significant difference between the conditions.

Example: DetectingFloor & Ceiling Effects

!"#

$"#

%"#

&"#

'"#

(""#

)*+,*-.,*/0# 1/02340# 56-70+.2*8,#

!"#$%&''()#'*%

+#'"%,-./-$'/-0"$"..%

54

Is this data to be trusted? Here are the frequency distributions for the three conditions.

Example: DetectingFloor & Ceiling Effects

!"

#"

$"

%"

&"

'"

("

)!" *!" +!" #!!"

!"#$%#&'()

*#"'#&+),-""#'+)

!"

#"

$"

%"

&"

'"

("

)!" *!" +!" #!!"

!"#$%#&'()

*#"'#&+),-""#'+)

!"

#"

$"

%"

&"

'"

("

)!" *!" +!" #!!"

!"#$%#&'()

*#"'#&+),-""#'+)

Distinctive Average Nondescript

55

Control Groups

• Control group: Group of participants identical (ideally) to the experimental group

• Undergoes same events as exp group, except for variable of interest

• By using the control group as a baseline, one can “factor out” almost all confounds

56

• Placebo: Control groups gets a non-active substitute for a treatment, to determine effects of simply administering treatment.

• Waiting List: Control group participants are those waiting for treatment of some sort. Ethical?

• Yoked: Each control group participant’s treatment is equated to particular test group participant’s treatment. Useful if length or type of treatment varies in the test group.

Varieties of Control Groups

57

• Powerful method for dealing with confounds

• Pre-test gives a baseline for comparing control and experimental groups post-treatment

• Difference pre-to-post should be greater for exp group than control if treatment is effective

Pre-Post StudiesExperimental

Grouppre-test treatment post-test

Control Group

pre-test post-test

58


• What are two major techniques for avoiding confounds?

• What is the relationship between experimental validity and construct validity?

59

• P’s experience an event between pre- and post-test --other than the exp manipulation--that affects the measured variable.

• Example: P’s in a study on therapy for depression might get worse because hospital where they’re being treated experiences some disaster

History Confound

Experimental Group:

pre-testTherapy &

Event post-test

Control Group:

pre-test Event post-test

60

• When subjects grow or change naturally over time and this affects the measured behaviour.

• Example: Kids in 1-year fitness program will grow up and become stronger and faster, regardless of the fitness program’s existence.

Maturation Confound

Experimental Group:

pre-test Program + growth post-test

Control Group:

pre-test growth post-test

61

• Occurs when a sample is selected from a population based on a given range of scores Example: Choosing kids from an average school because they are poor readers

• Over time, the sample’s mean will move towards the population mean.

• The math behind why this happens is complex, just take my word for it.

Regression To The Mean

62

1. One population is broken into two groups: A “high score” group and a “low score” group Example: Class is divided into tall and short groups. RttM will make the short group seem to grow and the tall one seem to shrink (believe it or not!)

2. Samples from two populations with different means are drawn such that the samples have the same mean Example: Two samples, one of scientists and one of politicians, are selected such that the samples have the same mean IQ. RttM will make the scientist sample get smarter and the politician sample get (even) dumber.

Bottom Line: Don’t do either of these things! These are surprisingly common errors in research.

When Regression Becomes a Problem

63

• Occurs when taking the pre-test makes you better or worse on the post-test

• Example: P’s given same IQ test before and after a cognitive enrichment program might get better scores because of practice.

Testing Confound

ExperimentalGroup:

pre-test(practice) program post-test

Control Group:

pre-test (practice) post-test

64

• P’s receiving treatment tend to report feeling better, even if treatment is in fact ineffective

• Psychological Effect: P’s think they should be getting better because they’re getting treated, so they report feeling better.

• Psycho-neuro-immunological Effect: Feeling of well-being that comes from being treated may lead to physiological changes in the body that are not related to the treatment.

Placebo Effect

65

• To determine the true effect of a treatment, the control group should get a placebo.

Placebo Effect

Experimental Group:

pre-test treatment + placebo fx post-test

Control Group:

pre-test placebo fx post-test

66

• When the measuring instrument changes from one condition to another

• Example: In an experiment looking at the effects of enrichment on IQ, an experimenter uses the WAIS IQ test kit for pre-test, and the WAIS-Revised for post-test. Changes in IQ measures could be due to either enrichment or differences in the test.

Instrumentation Confound

Experimental Group:

pre-test(WAIS)

enrich-ment

post-test (WAIS-R)

Control Group:

pre-test(WAIS)

post-test (WAIS)

67

!

When Pre-Testing Backfires

• Pre-testing can be problematic if the effects of the pre-test interact with those of treatment. Example: Taking an IQ test before the cognitive enrichment program might “clue people in” to what to pick out of the program in order to do well on the second IQ test. Thus, when the CEP is given to those who haven’t taken an IQ test, it might not have the same effect.

• A control group will not save you here

68

• Comparing Exp1 and Exp2 tells you if results are due to the treatment, the pretest, or an interaction of the two.

• Comparing Control 1 and Control 2 tells you the effect of the pre-test alone.

• Very solid design, but requires twice the subjects and twice the testing time of basic pre-post control-group design.

Evaluating Pre-Test FX:The Solomon Design

Experimental 1 pretest treatment post-test

Experimental 2 treatment post-test

Control 1 pretest post-test

Control 2 post-test

69

The Downside of Pre-Testing

• Pre-testing can result in confounds if pre-test interacts with IV

• It also takes up time and resources

• Consider eliminating pre-testing, especially in cases where P’s are randomly assigned to groups

70

• In true experiments, subjects are randomly assigned to levels. This balances out individual differences.

• When this is not possible, subject self-assignment may become a problem.

• Participants with particular characteristics may have a tendency to choose one level over another, leading to a confound

Subject Self-Assignment

71

• Prof offers two sections of course: One with labs, one without.

• Hypothesis: Labs will enable greater learning, leading the section with labs to do better on the final exam.

• Problem: Students can’t be randomly assigned, they get to choose their section. Better (more motivated, assiduous) students might choose the section with labs, causing group differences.

• Partial solution: If labs are between midterm and final, that provides a pre-test, which would reveal group differences

• However: Group diffs can have an effect on their own, or might interact with other things, like practice effects (e.g., better students might learn more from the midterm than poorer students)

Subject Self-Assignment Example

72


• What are two situations in which regression to the mean becomes a problem?

• What’s the benefit of pre-testing? When can it be a liability?

73

• A.K.A. “differential subject mortality”

• Occurs when particular types of people drop out of one group (control or experimental) more than the other.

• Example: Poor students might drop out of the section with labs, causing the average for that section to rise.

Attrition

74

Participant Behaviour Entanglement

• Ps’ behaviour can block or alter other Ps’ behaviour in unpredictable ways if they are allowed to interact.

• It is generally best to test subjects one at a time.

• In testing group effects, it may be best to use confederates as the group.

• Alternative: Treat each group as a single “participant” (but might need lots of subjects!).

75

• Hawthorne effect: Observation itself changes behaviour. Solution: Use naturalistic observation.

• The “Good subject” effect & demand characteristics. Solution: Use “blind” procedures.

• Leakage: Previous subjects tell others about experiment. Solutions: Limit time to complete observations, request that participants don’t tell others, and use Manipulation Checking

Participant Bias

76

Manipulation Checking

• Aspect of experiment designed to see if manipulation worked as intended

• Often a questionnaire asking Ps if they “bought” the manipulation and/or if they found it effective.

• Can also involve behaviour inventory

77

• Experimenter can give subtle cues to subjects as to what expected results are.

• Experimenter may unconsciously tend to measure differently for different groups.

• Solutions:

• Remove experimenter. Automate as much as possible.

• Use naive observer. Double-blind procedure.

Experimenter Bias

78

• With all these confounds to, uh, confound us, is it even possible to design a good experiment?

• Yes, but it’s not easy (Remember, psych research is not for the weak!)

• It is, however, impossible to design a perfect experiment, so don’t expect this of yourself or others.

Despairing?

79

Think Twice...

• Carpenter’s adage: “Measure twice, cut once”

• Scientist’s adage: “Think twice, measure once”

• Do not rush experimental design, there are many pitfalls to be avoided and careful design will save time in the long run.

80

A university puts in place a mandatory anti-anxiety seminar that all students must attend. They test students’ anxiety levels at the beginning of the semester (pre-test, before the seminar) and near the end of the semester (post-test, after the seminar). They find that there’s no difference in anxiety levels between pre- and post-test and decide not to implement the anti-anxiety seminar in later years. Was this a good decision? Why or why not?

Find the Confound

81

In a study of the effects of caffeine on reaction time, participants are asked to do a reaction time test, drink 3 cups of regular coffee, then do the same test again. The results show faster reaction times on the post-test than the pre-test. Can it be concluded that caffeine speeds reaction time? Why or why not?

Find the Confound

82

A study is done to test the effects Ziepfzer Corp’s latest product, Smartzac™ on IQ. One group (experimental) of participants receives the drug and the other (control) receives a placebo. Due to annoying side-effects (dizziness, sexual nightmares, and sleep crime) half the people in the experimental group drop out. How might this affect the experiment?

Find the Confound

83

• Participant differences: random assignment, block random assignment, matching

• Order effects: Full counterbalancing & various partial counterbalancing techniques (more later...)

• Participant bias: Blind procedures. Removal of demand characteristics. Naturalistic observation.

• Experimenter bias: Automation, double-blind procedures

• Floor & ceiling effects: Use of tasks that are neither too difficult nor too easy (pilot testing).

Summary: Confounds & Controls

84


85

psy2174 s11...

Documents