Experimental ResearchExperimental Research

Validity and ConfoundsValidity and Confounds

What is it?What is it?

Systematic inquiry that is Systematic inquiry that is characterized by:characterized by: An investigator’s direct manipulation An investigator’s direct manipulation

(variation) of some factor or factors (IV (variation) of some factor or factors (IV or IV’s) thought to be causally related to or IV’s) thought to be causally related to some outcome or outcomes (DV or DV’s) some outcome or outcomes (DV or DV’s) which are observed as data while which are observed as data while holding all other factors constant.holding all other factors constant.

Independent VariableIndependent Variable

The independent variable: The independent variable: conditions conditions in the experimentin the experiment Different situations (rooms, experiences, etc)Different situations (rooms, experiences, etc) Different instructionsDifferent instructions Different tasksDifferent tasks Different treatmentsDifferent treatments Experimental vs. control groupsExperimental vs. control groups Bystander study: Manipulate the number of Bystander study: Manipulate the number of

bystandersbystanders

Essential Features of Essential Features of ExperimentsExperiments

Independent VariableIndependent Variable Types?Types? Control groupControl group

Dependent variableDependent variable

Manipulated vs. Subject Manipulated vs. Subject VariableVariable

Subject variablesSubject variables are characteristics that you cannot are characteristics that you cannot manipulate: gender, choosing subjects with high or low manipulate: gender, choosing subjects with high or low anxiety.anxiety.

Subject variable studies also called “natural groups studies”Subject variable studies also called “natural groups studies”

Manipulated variablesManipulated variables true experimental designs true experimental designs Create groups where you place/force subjects to be in one Create groups where you place/force subjects to be in one

group or anothergroup or another Create anxiety for ½ of the group.Create anxiety for ½ of the group.

Limitation with a natural groups studyLimitation with a natural groups study can’t draw can’t draw causal conclusions because you can’t rule out other causal conclusions because you can’t rule out other possible factors that might be the “true” causal factor.possible factors that might be the “true” causal factor.

Example: genderExample: gender

Extraneous variables vs. Extraneous variables vs. ConfoundsConfounds

Extraneous VariableExtraneous Variable: uncontrolled, : uncontrolled, unsystematic factors or variance. Can unsystematic factors or variance. Can be ANY variable that is related to the IVbe ANY variable that is related to the IV

Creates “noise” and may affect the outcome Creates “noise” and may affect the outcome variable, just not systematically. variable, just not systematically.

Confounding variableConfounding variable: uncontrolled, : uncontrolled, systematic variancesystematic variance

Affects the different levels of the IV Affects the different levels of the IV differently.differently.

2 things to control for confounds:2 things to control for confounds: Random assignmentRandom assignment Within Subjects designsWithin Subjects designs

Internal vs External ValidityInternal vs External Validity

External validityExternal validity: the extent to which the : the extent to which the results of ones study can be generalized to results of ones study can be generalized to subjects and/or conditions outside of the subjects and/or conditions outside of the study. Often done by replication studiesstudy. Often done by replication studies

Internal validity:Internal validity: found when results are found when results are characterized by the ABSENCE of competing characterized by the ABSENCE of competing explanations. You are confident that explanations. You are confident that changes in the IV did indeed lead to changes in the IV did indeed lead to changes in the DV.changes in the DV.

Experimental control is the key to internal validityExperimental control is the key to internal validity An absence of confounds and extraneous An absence of confounds and extraneous

variablesvariables Control groups help establish internal validityControl groups help establish internal validity

Threats to Internal ValidityThreats to Internal Validity

Post test studyPost test study TreatmentTreatmentobservationobservation Ritalin and ADHDRitalin and ADHD

Pre test-post test studyPre test-post test study ObservationObservation treatment treatmentobservation- observation-

Offers a baseline measurement from Offers a baseline measurement from which we can comparewhich we can compare

Threats to Internal ValidityThreats to Internal Validity

In pre test-post test studies we find..In pre test-post test studies we find.. History and maturationHistory and maturation

HistoryHistory: Something external happens and affects changes in your : Something external happens and affects changes in your independent variable independent variable

maturationmaturation: external to your study, but internal to your subjects. : external to your study, but internal to your subjects. Development or experienceDevelopment or experience

Regression to the meanRegression to the mean When people are chosen because they are ‘extreme’ in some way When people are chosen because they are ‘extreme’ in some way

and you measure their change over time in that extreme dimensionand you measure their change over time in that extreme dimension you’re likely to see a change in the score closer to the mean. you’re likely to see a change in the score closer to the mean.

InstrumentationInstrumentation and testingand testing Change in the researchers (or subjects) instrumentation or Change in the researchers (or subjects) instrumentation or

measuring over time.measuring over time. Testing:Testing: when actually taking the test affects the behavior or helps when actually taking the test affects the behavior or helps

“practice”. “practice”. IDU studyIDU study

Instrumentation:Instrumentation: changes in the instrument used to measure from pre changes in the instrument used to measure from pre to post. to post.

Controlling Against TtIVControlling Against TtIV

Use a Control GroupUse a Control Group ObservationObservation Treatment Treatment Observation ObservationANDAND Observation----------------Observation---------------- Observation ObservationOROR ObservationObservation Placebo--- Placebo--- Observation Observation

But, what if these threats don’t affect But, what if these threats don’t affect both groups equally?both groups equally?

ExampleExample

Want to know the sexual assault opinions Want to know the sexual assault opinions of UWT studentsof UWT students Group 1: Wild life videoGroup 1: Wild life video

2 weeks later…2 weeks later… Group 2: Assault videoGroup 2: Assault video

But during those 2 weeks there was a string of But during those 2 weeks there was a string of sexual assaults on campussexual assaults on campus

What kind of threat?What kind of threat? Solution?Solution?

Other ThreatsOther Threats

Other threats come from HOW a subject is chosen Other threats come from HOW a subject is chosen for the study and how comparison groups are for the study and how comparison groups are formedformed

Subject selectionSubject selection- if subjects chosen for one - if subjects chosen for one group are different than those in another group group are different than those in another group before the study takes place. before the study takes place.

Solution: Random assignment of subjects into groupsSolution: Random assignment of subjects into groups

Attrition/subject mortality-Attrition/subject mortality- When subjects drop When subjects drop out or refuse to take part in your experiment. out or refuse to take part in your experiment.

Possible solution: try to interview the people who Possible solution: try to interview the people who dropped out to find out why and see if that is related to dropped out to find out why and see if that is related to the IVthe IV