Research DesignsResearch Designs

AgendaAgenda

Experimental DesignsExperimental Designs

Natural ExperimentsNatural Experiments

Time Series and Panel DesignsTime Series and Panel Designs

Cross Sectional DesignsCross Sectional Designs SurveysSurveys

AnnouncementsAnnouncements

Next week- half of you meet in library Next week- half of you meet in library Tuesday, half come here, flip for ThursdayTuesday, half come here, flip for Thursday

A through McGrath meet in Library A through McGrath meet in Library Tuesday, Here ThursdayTuesday, Here Thursday

Mickelson-Z meet here Tuesday, Library Mickelson-Z meet here Tuesday, Library ThursdayThursday

Don’t read Johnson 6, 11Don’t read Johnson 6, 11

RecapRecap

HypothesesHypotheses

VariablesVariables

Units of AnalysisUnits of Analysis

Positive, negative, spurious, Positive, negative, spurious, antecedent/intervening/mediating, antecedent/intervening/mediating, moderatingmoderating

Intro to Iyengar and KinderIntro to Iyengar and Kinder

Media and Public Opinion: Hypodermic Media and Public Opinion: Hypodermic Needles to Minimal Effects to ??Needles to Minimal Effects to ??

How might you study media effectsHow might you study media effects

Why might experiments be useful?Why might experiments be useful?

ExperimentExperiment

Allows for the greatest level of researcher Allows for the greatest level of researcher controlcontrol

Systematically vary one variable at a timeSystematically vary one variable at a time

Control GroupControl Group

Treatment GroupTreatment Group

Stimulus/TreatmentStimulus/Treatment

ExperimentExperiment

AssignmentAssignment MatchingMatching RandomRandom

Within Subjects DesignWithin Subjects Design

Between Subjects DesignBetween Subjects Design

Classic ExperimentClassic Experiment

Randomly assign groupsRandomly assign groups

Test at time 1Test at time 1

Provide treatment to one groupProvide treatment to one group

Provide no treatment to other groupProvide no treatment to other group

Test at time 2Test at time 2

Both between and within subjects designBoth between and within subjects design

Iyengar and KinderIyengar and Kinder

Hypotheses- MultipleHypotheses- Multiple Exposure influences agendaExposure influences agenda Exposure influences accountabilityExposure influences accountability Qualities of story matterQualities of story matter

Recruit a random sample of citizensRecruit a random sample of citizens

Participants watch the newsParticipants watch the news

Mild deception- watch with us to avoid Mild deception- watch with us to avoid distractionsdistractions

Iyengar and KinderIyengar and Kinder

PretestPretest

What are key IVs/DVs?What are key IVs/DVs?

Show TV newsShow TV news Control group- Unaltered NewsControl group- Unaltered News Treatment group- Added StoryTreatment group- Added Story

Post Test measuresPost Test measures

Findings- Agenda Setting and Priming on Findings- Agenda Setting and Priming on Most issuesMost issues

Iyengar and KinderIyengar and Kinder

Interesting Features?Interesting Features?

Create natural environmentCreate natural environment

Tight Control of StimulusTight Control of Stimulus

Number of ConditionsNumber of Conditions

No pre testNo pre test

Is it desirable to pretest?Is it desirable to pretest?

Between Subjects TestBetween Subjects Test

Example- Druckman and Holmes 2005Example- Druckman and Holmes 2005

Comparison between groupsComparison between groups

Random Assignment makes comparison Random Assignment makes comparison validvalid

Quasi Experimental ApproachesQuasi Experimental Approaches

Increased control over non experimental Increased control over non experimental techniquestechniques

Less control than true experimentLess control than true experiment

Sidanius and PrattoSidanius and Pratto

Social Dominance TheorySocial Dominance TheorySecret Shopper ParadigmSecret Shopper ParadigmSend different races, genders to do various Send different races, genders to do various taskstasksLook at and bargain for apartmentLook at and bargain for apartmentBargain for carBargain for carApply for jobApply for jobControlsControlsResults- White men do best, African-American Results- White men do best, African-American Men do worstMen do worst

Natural ExperimentsNatural Experiments

Two similarly matched entitiesTwo similarly matched entities

Differ in “one” key wayDiffer in “one” key way

E.g. Mondak Study of Newspapers in E.g. Mondak Study of Newspapers in Cleveland and PittsburghCleveland and Pittsburgh

E.g. political coverage on different E.g. political coverage on different networksnetworks

Social ProgramsSocial Programs

Concern- Lack of ControlConcern- Lack of Control

Experimental Methods-AdvantagesExperimental Methods-Advantages

Allow the researcher control over the Allow the researcher control over the situationsituation

Allows for more certainty of measurementAllows for more certainty of measurement

Allow for stronger causal inferences- High Allow for stronger causal inferences- High Internal ValidityInternal Validity Control of extraneous factorsControl of extraneous factors

Experimental Methods-Experimental Methods-DisadvantageDisadvantage

Limits to External ValidityLimits to External Validity Subject PopulationSubject Population Reality of SituationReality of Situation Cannot create some circumstancesCannot create some circumstances Demand EffectsDemand Effects

Cannot manipulate all factorsCannot manipulate all factors PersonalityPersonality Ethical ConsiderationsEthical Considerations

Somewhat weak relationship between attitudes Somewhat weak relationship between attitudes and behaviors? and behaviors?

Experiments- When AppropriateExperiments- When Appropriate

When feasibleWhen feasible

When ethicalWhen ethical

When factors of interest are controllableWhen factors of interest are controllable

Public Opinion ChangePublic Opinion Change

Media StudiesMedia Studies

Time SeriesTime Series

Look at cases at multiple timesLook at cases at multiple times

Panel Study- Re-interview same cases at Panel Study- Re-interview same cases at set pointsset points

Presidential ApprovalPresidential Approval Track from 50s to presentTrack from 50s to present Look at Approval rating of president as Look at Approval rating of president as

Dependent VariableDependent Variable Look at fluctuations in other factorsLook at fluctuations in other factors

Time Series- AdvantagesTime Series- Advantages

Causality- follows temporal patternsCausality- follows temporal patterns

Gives best insight into over time changeGives best insight into over time change

Panels- good for understanding changePanels- good for understanding change

Time Series- DisadvantagesTime Series- Disadvantages

Must have data over timeMust have data over time

Consistency of measures over timeConsistency of measures over time

Causality- confusion?Causality- confusion?

Challenge- empirical generalizationChallenge- empirical generalization

PanelsPanels Drop outsDrop outs Systematic bias in sampleSystematic bias in sample LearningLearning

Time Series- When AppropriateTime Series- When Appropriate

When looking at change over timeWhen looking at change over time

When looking at historical eventsWhen looking at historical events

When looking at development processesWhen looking at development processes

When looking at factors that cannot be When looking at factors that cannot be easily manipulated in lab or do not vary easily manipulated in lab or do not vary cross-sectionallycross-sectionally

Cross SectionalCross Sectional

One timeOne time

Look at multiple casesLook at multiple cases

Cases should have variationCases should have variation

Example- National Election StudyExample- National Election Study

Cross Sectional Studies- SurveyCross Sectional Studies- Survey

Random sample of population of interestRandom sample of population of interest

Ideal Sample greater than 500Ideal Sample greater than 500

Fixed questionnaire Fixed questionnaire

Ask all at roughly same timeAsk all at roughly same time

Ideal for public opinion/voting behaviorIdeal for public opinion/voting behavior

Cross Sectional Designs-Cross Sectional Designs-AdvantagesAdvantages

Does not require as much advanced Does not require as much advanced planningplanning

Good for comparing groups of thingsGood for comparing groups of things

SnapshotSnapshot

Cross Sectional Design- Cross Sectional Design- DisadvantagesDisadvantages

Does not work for certain types of Does not work for certain types of phenomenaphenomena

Limited ability to make causal connectionsLimited ability to make causal connections Temporal issuesTemporal issues

Limited number of casesLimited number of cases

Case StudyCase Study

Focus intense study on one caseFocus intense study on one case

Often qualitativeOften qualitative

Challenge- generalityChallenge- generality

SurveysSurveys

More a tool than a design More a tool than a design Randomly select large number of peopleRandomly select large number of peopleAdminister same questionnaireAdminister same questionnaireMay be cross sectional or time seriesMay be cross sectional or time seriesAllows rich data collectionAllows rich data collectionOften most generalizableOften most generalizableExpensiveExpensiveNot socialNot socialInflexibleInflexible

Focus GroupsFocus Groups

4-10 people together 4-10 people together

Have them discussHave them discuss

Relatively unstructuredRelatively unstructured

Advantage- study more social phenomenaAdvantage- study more social phenomena

Generate new ideasGenerate new ideas

Concern- generalizabilityConcern- generalizability

InterpretationInterpretation