copyright © allyn & bacon (2007) single-variable, independent-groups designs graziano and...

Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)

Single-Variable, Single-Variable, Independent-Groups Independent-Groups DesignsDesigns

Graziano and RaulinGraziano and RaulinResearch Methods: Chapter 10Research Methods: Chapter 10This multimedia product and its contents are protected under copyright law. The following are This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.part, of any images; (3) Any rental, lease, or lending of the program.


Experimental DesignExperimental Design

1.1. Tests hypotheses about causal Tests hypotheses about causal effects of the independent variable effects of the independent variable (IV)(IV)

2.2. Includes at least two levels of the IVIncludes at least two levels of the IV3.3. Randomly assigns participants to Randomly assigns participants to

conditionsconditions4.4. Includes specific procedures for Includes specific procedures for

testing hypothesestesting hypotheses5.5. Includes control for the major threats Includes control for the major threats

to internal validityto internal validity


VarianceVariance

Variance is necessary in any Variance is necessary in any researchresearch– Without variance, there is nothing to Without variance, there is nothing to

testtest– Variance was defined in Chapter 5Variance was defined in Chapter 5

Research designs control unwanted Research designs control unwanted sources of variance in order to sources of variance in order to evaluate the effects of the evaluate the effects of the independent variableindependent variable


Forms of VarianceForms of Variance

Systematic between-groups varianceSystematic between-groups variance– Experimental variance (due to the IV)Experimental variance (due to the IV)– Extraneous variance (due to confounding variables)Extraneous variance (due to confounding variables)

Nonsystematic within-groups Nonsystematic within-groups variancevariance– Due to chance factors and individual differencesDue to chance factors and individual differences

We analyze the results of our study We analyze the results of our study using the using the FF-test (ANOVA)-test (ANOVA)– Ratio of between-groups variation to within-groups Ratio of between-groups variation to within-groups

variationvariation


Controlling VarianceControlling Variance

Maximizing experimental varianceMaximizing experimental variance– Make sure that there are real differences between the Make sure that there are real differences between the

groups (using a groups (using a manipulation checkmanipulation check))

Controlling extraneous varianceControlling extraneous variance– Make sure the groups are as similar as possible at the Make sure the groups are as similar as possible at the

start of the studystart of the study– Therefore, the only difference is the IV manipulationTherefore, the only difference is the IV manipulation

Minimizing error varianceMinimizing error variance– Control with careful measurement or with special Control with careful measurement or with special

designs (e.g., correlated-group designs)designs (e.g., correlated-group designs)


Controlling VarianceControlling Variance

The relationship of The relationship of the various sources the various sources of variance to the of variance to the FF ratio is shown hereratio is shown here

You want toYou want to– Maximize Maximize

experimental experimental variancevariance

– Minimize error Minimize error variancevariance

– Control extraneous Control extraneous variancevariance


Manipulation CheckManipulation Check

An specific test of whether the An specific test of whether the independent variable independent variable manipulation actually worked the manipulation actually worked the way it was intendedway it was intended

Example: A study testing the Example: A study testing the hypothesis that females, but not hypothesis that females, but not males, tend to turn anger inward males, tend to turn anger inward rather than express it externallyrather than express it externally


Expressed HostilityExpressed Hostility

The data on the The data on the dependent dependent measure measure suggests that suggests that females really do females really do respond with less respond with less hostility than hostility than males when males when frustratedfrustrated


Reported AngerReported Anger

However, the However, the manipulation manipulation check of reported check of reported anger suggests anger suggests that the females that the females may not have may not have been angered by been angered by the frustration the frustration manipulationmanipulation


Physiological ArousalPhysiological Arousal

And the second And the second manipulation manipulation check of check of physiological physiological arousal seems to arousal seems to indicate that the indicate that the report of less report of less anger by the anger by the female female participants is realparticipants is real


Nonexperimental Nonexperimental DesignsDesigns Do not include the critical controls Do not include the critical controls

of experimental designsof experimental designs May still be used, but caution is May still be used, but caution is

necessarynecessary Four designs covered in this sectionFour designs covered in this section

– Ex post facto designEx post facto design– Single-group, posttest-only designSingle-group, posttest-only design– Single-group, pretest-posttest designSingle-group, pretest-posttest design– Pretest-posttest, natural control-group designPretest-posttest, natural control-group design


Ex Post Facto DesignEx Post Facto Design

A very weak designA very weak design– What we do when we try to figure out, after the What we do when we try to figure out, after the

fact, what caused something to happenfact, what caused something to happen– Not good scienceNot good science– Does not control confounding variablesDoes not control confounding variables


Single-Group, Single-Group, Posttest-Only DesignPosttest-Only Design

Even with the manipulation, virtually no Even with the manipulation, virtually no control over confounding variablescontrol over confounding variables

We tend to use an implicit control group We tend to use an implicit control group (what we think would have happened if there (what we think would have happened if there had been no manipulation)had been no manipulation)


Single-Group, Single-Group, Pretest-Posttest Pretest-Posttest DesignDesign

The pretest documents change, but The pretest documents change, but factors other than the treatment could factors other than the treatment could have accounted for the changehave accounted for the change– History, maturation, regression to the mean, History, maturation, regression to the mean,

etc.etc.


Pretest-Posttest, Pretest-Posttest, Natural Control-Group Natural Control-Group DesignDesign


Pretest-Posttest, Pretest-Posttest, Natural Control-Group Natural Control-Group DesignDesign Like an experiment except that Like an experiment except that

participants are not randomly participants are not randomly assigned to the groupsassigned to the groups

A reasonably strong design A reasonably strong design except that it does not control for except that it does not control for selectionselection– Selection could be a powerful Selection could be a powerful

confounding factor in many studiesconfounding factor in many studies


Experimental DesignsExperimental Designs

Meet all criteria for an experimentMeet all criteria for an experiment Provide more powerful tests of Provide more powerful tests of

hypotheseshypotheses Designs discussed in this chapterDesigns discussed in this chapter

– Randomized, posttest-only, control-group designRandomized, posttest-only, control-group design– Randomized, pretest-posttest, control-group designRandomized, pretest-posttest, control-group design– Multilevel, completely randomized, between-Multilevel, completely randomized, between-

subjects designssubjects designs– Solomon’s four-group designsSolomon’s four-group designs


Randomized, Posttest-Randomized, Posttest-Only,Only,Control-Group DesignControl-Group Design


Randomized, Posttest-Randomized, Posttest-Only,Only,Control-Group DesignControl-Group Design Random assignment controls for Random assignment controls for

selectionselection Other confounding variables are Other confounding variables are

controlled by comparing the controlled by comparing the treatment and no treatment treatment and no treatment groupsgroups– For example, history and maturation For example, history and maturation

should be the same in both groupsshould be the same in both groups


Randomized, Pretest-Randomized, Pretest-Posttest,Posttest,Control-Group DesignControl-Group Design


Randomized, Pretest-Randomized, Pretest-Posttest,Posttest,Control-Group DesignControl-Group Design Adding a pretest allows us to Adding a pretest allows us to

quantify the amount of change quantify the amount of change following treatmentfollowing treatment

Also allows us to verify that the Also allows us to verify that the groups were equal initiallygroups were equal initially

A strong basic research design, A strong basic research design, with excellent control over with excellent control over confoundingconfounding


Multilevel, Randomized,Multilevel, Randomized,Between-Subjects DesignBetween-Subjects Design


Multilevel, Randomized,Multilevel, Randomized,Between-Subjects DesignBetween-Subjects Design

May or may not include a pretestMay or may not include a pretest Multi-group extension of the basic Multi-group extension of the basic

experimental designsexperimental designs Controls virtually all sources of Controls virtually all sources of

confounding variablesconfounding variables


Solomon’sSolomon’sFour-Group DesignFour-Group Design


Solomon’sSolomon’sFour-Group DesignFour-Group Design

Combines two basic experimental Combines two basic experimental designsdesigns– Randomized, posttest-only, control-Randomized, posttest-only, control-

group designgroup design– Randomized, pretest-posttest, Randomized, pretest-posttest,

control-group designcontrol-group design Allows the assessment of an Allows the assessment of an

interaction between the pretest interaction between the pretest and the treatment and the treatment


Statistical Analysis Statistical Analysis IssuesIssues If the data are nominal, use chi-If the data are nominal, use chi-

squaresquare If the data are ordinal, use the Mann-If the data are ordinal, use the Mann-

Whitney Whitney UU-test (two groups only)-test (two groups only) If the data are interval or ratioIf the data are interval or ratio

– If two groups, a If two groups, a tt-test of the posttest -test of the posttest measures will test the hypothesis measures will test the hypothesis

– More complex designs require an ANOVAMore complex designs require an ANOVA


Analysis of VarianceAnalysis of Variance

Evaluates differences in group meansEvaluates differences in group means– It does this evaluation by comparing It does this evaluation by comparing

different variance estimates (termed different variance estimates (termed mean mean squaressquares))

– The The FF statistic is a ratio of statistic is a ratio of the mean square between-groups andthe mean square between-groups and the mean square within-groupsthe mean square within-groups

The larger the differences between the The larger the differences between the group means, the greater the group means, the greater the FF value value


SpecificSpecificMean ComparisonsMean Comparisons A significant A significant FF-test means that at -test means that at

least one group is significantly least one group is significantly different from at least one other different from at least one other groupgroup– If you have more than two groups, you have If you have more than two groups, you have

to do follow-up tests to see which groups to do follow-up tests to see which groups differdiffer

Specific mean comparisons can beSpecific mean comparisons can be– Planned comparisons Planned comparisons – Post hoc testsPost hoc tests


Other Experimental Other Experimental DesignsDesigns Other experimental designs covered in Other experimental designs covered in

later chapterslater chapters Correlated-groups designs Correlated-groups designs (Chapter 11)(Chapter 11)

– Within-subjects designsWithin-subjects designs– Matched-subjects designsMatched-subjects designs– Single-subject designsSingle-subject designs

Factorial designs Factorial designs (Chapter 12)(Chapter 12)

– Many variations on factorial designs are Many variations on factorial designs are possiblepossible


SummarySummary

Research is designed to measure and Research is designed to measure and control sources of variancecontrol sources of variance

There are several non-experimental There are several non-experimental and experimental designs availableand experimental designs available

Experimental designs have two Experimental designs have two elementselements– Random assignment of participants Random assignment of participants

to conditionsto conditions– At least one control groupAt least one control group

copyright © allyn & bacon (2007) single-variable, independent-groups designs graziano and...

Documents