psychology 242, dr. mckirnanresearch ethics quasi-experiments studying naturally occurring events ...
TRANSCRIPT
Psychology 242, Dr. McKirnan Research Ethics
Quasi-experiments
Studying naturally occurring events
Measurement studies
Retrospective designs
Evaluate existing groups or program
Single shot survey or measure
Non-equivalent groups
Time series designs
Quasi-experimental designsExperimental designs for “studies in nature”.
Back NextHomepage
2Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
True v. quasi-experimental designs
True experiments: Quasi-experiments:
Emphasize internal validity Assess cause & effect (in
relatively artificial environment) Test clear, a priori hypotheses
Emphasize external validity Describe “real” / naturally
occurring events Clear to exploratory hypotheses
Participants assigned to experimental v. control groups
Random or matching Participants & experimenter
Blind to assignment
Existing or non-equivalent groups
Non-random assignment Participants not blind Control group not possible?
Control study procedures Create / manipulate independent
variable Control procedures & measures
Control often not possible May not be able to manipulate
the independent variable Partial control of procedures &
measures
Back NextHomepage
3Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Research Ethics
Quasi-experiments: naturally occurring events
Studying naturally occurring events
Measurement studies
Retrospective designs
Evaluate existing groups or program
Single shot survey or measure
Non-equivalent groups
Time series designs
Back NextHomepage
4Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Natural disaster / stressor3-mile island Stress -> immune systemS.F. earthquake Stress & coping
Crime / traumaIraq service, 9 / 11 / 01 PTSD & treatment
Historical event9/11 & air travel ban Contrails & climate change Economic collapse Voting patterns
Publicity / cultural event Info. re: Hormone replacement Health behavior
Measurement studies
1. Naturally occurring events; examples
Event Study question(“Predictor”) (“Outcome”)
Back NextHomepage
5Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Naturally occurring events, 2
Independent variable: Predictor variable (e.g., natural disaster) often assessed after
the event (post-hoc).
Researcher has little control over dose / type of predictor
Participant selection No control over who is exposure to event Some control over selection of sample (e.g., via targeted
sampling)
Many potential confounding variables Outcome (dependent) variables:
No control with archival data Some control with surveys Use retrospective (measured) variables to clarify
interpretation of outcomes or test hypothesis.
Back NextHomepage
6Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Retrospective Event Outcomevariable(s) (“Predictor variable”) variable Social support earthquake [v. control city?] stress & coping
Psych. history crime / trauma mental health[archive? Self-report?] [v. control people?]
Personal attitudes historical event voting patterns
Demographics cultural event health behavior
Naturally occurring events: Retrospective designs
Using retrospective (measured) variables to clarify interpretation of outcomes or test hypothesis.
Back NextHomepage
7Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Research Ethics
Quasi-experiments: Existing groups
Studying naturally occurring events
Measurement studies
Retrospective designs
Evaluate existing groups or program
Single shot survey or measure
Non-equivalent groups
Time series designs
Back NextHomepage
8Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Existing groups
Single self-selected group; no comparison possible users of psychotherapy (or any product) members of group or cult [contrast with demographically
matched controls?]
Two or more groups, with self-selection and / or "non-blind" assignment Psychological interventions: therapy v. wait list, etc.
Two or more groups, no random assignment Comparing schools / cities / existing groups…
Existing groups:
Back NextHomepage
9Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
May control selection of study group, or must use Convenience sample. Other data may be available about group.
Group
Typically no control over event. Not a true Independent Variable
Naturally occurring event or social change Observe1
Dependent Variable(s):May or may not have control over measures (e.g., surveys v. archival measures).
“One shot” case studies
Typical use: Surveys or measures after an event.
Heuristic value: generating hypotheses for later study or confirm controlled data in “real world” setting.
Internal / External validity:
No control over selection of people into the event.
Potentially no control over selection into measurement group.
No control group; uncontrollable event, or other groups may not “need” the intervention (e.g., therapy)
Back NextHomepage
10Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Example of one-shot case study
Research questions: Does psychotherapy “work” from consumer view? Who gets therapy / what does it consist of? Do consumer responses vary by type of therapy?
Research approach: One shot case study / survey
Sampling frame: Any therapy or psychological service user No real information re: population of therapy users.
Sampling procedure: 4,100 Consumer reports readers responding to “in
magazine” mail-back survey form
Example: Consumer Reports psychotherapy survey[Click for paper]
Back NextHomepage
11Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
One shot: Consumer reports survey, 2
Negatives: Selection bias
no control over who got therapy (self-selection)
of those who got therapy, no control over who returned a survey (secondary self-selection)
Cursory outcome measures: satisfaction rather than mental health
Positives: Huge, national sample Wholly anonymous, 3rd party data collection; less bias “Real world” assessment of product quality
Experimental Controls Evaluate by gender, type of treatment, medications, to
provide more differentiated analysis
Back NextHomepage
12Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
One shot: Consumer reports survey, 3
Key distinction in psychological interventions: “efficacy” v. “effectiveness” research EfficacyEfficacy; “true” experimental design / Lab-basis
Rigorous controls; High internal validity Test basic theory or highly specific technique
Do the specific ingredients (or theory…) of this treatment validly induce the key outcome?
EffectivenessEffectiveness; quasi-experimental; “natural” or applied setting
Less or no control; naturally occurring treatment
High external validity
Does treatment “work” in real patients w/real therapists?
Back NextHomepage
13Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Comparisons of study types: (Consumer reports 4)
Efficacy experiment v.Efficacy experiment v.• One specific diagnosis • Rigorous control group
• “no treatment” condition • “attention control”
• Random assignment • Manualized / uniform treatments
• High Fidelity to treatment method • Fixed number of sessions.
• Well operationalized outcomes, e.g., • clinician-diagnosed disorder• Standard / validated self-report
symptom scales
• “Blind" raters or diagnosticians ("single-blind“: patient & therapist know what the treatment is..)
• Patients followed for a fixed period
Effectiveness research• Multiple diagnoses & severities • No control group, 2nd controls
• Archival, via pt. characteristics
• Self-selection; “shopping” • Multiple / mixed treatments
• Highly tailored to patient • # sessions is patient based.
• Diverse, self-referenced outcomes • Subjective sense of “wellness” • Lessening of “problem” behaviors
or moods • Personal assessment of functioning
• Self-rated: cannot be "blind"
• Diverse times since treatment• Retrospective rather than
prospective
Back NextHomepage
14Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Consumer reports survey, 5.
Survey findings on therapy effectiveness: People who got more treatment (> 6 months) did better.
For general ψ health MH specialists did best, marriage counselors worst.
For patients’ presenting problem(s) all specialists did about the same.
Back NextHomepage
15Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
3/4 went to mental-health specialist. Patients who rated themselves worse at
outset made the most progress. AA very highly evaluated Therapy did as well as medications 40% got drugs;
MDs gave medications to 83% of patients MH pros; 20% drug treatments 50% who got drugs got no counseling 20% got no information about side effects 40% of anti-anxiety drugs given > 1 year
Consumer reports survey, 6.
Other effectiveness / descriptive findings:Other effectiveness / descriptive findings:
Back NextHomepage
16Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Observation + Archival Controls
Dependent Variable(s):Combine survey or other measures with archival or ancillary data as Control variables.
More complex case studies
Basic selection / convenience biases, uncontrollable event.
Group Naturally occurring event
Example 1: San Francisco earthquake & coping
Sampling frame: - Randomly selected survey participants
Outcomes: - Standardized mental health scales- Self-reports of stress
Quasi-controls: - population norms on outcomes- ancillary measures, e.g., social support
Findings: - High rates of stress Rx, - Social support ‘buffers’ stress
Back NextHomepage
17Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
“One Shot” + archival control: examples
Event: - Three-mile Island nuclear accident
Sampling frame: - Randomly selected residents of geographic area around TMI
Outcomes: - Blood draws for immune markers- Self-reports of stress
Quasi-controls: - Demographically matched sample- Archival data on health & illness
Findings: - Long-term suppression of key immune markers (natural killer cells, T cells)
Example 2: Stress and immune functioning
Back NextHomepage
18Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Event: - Loss of spouse
Sampling frame: - Hospital records, self-selected spouses
Outcomes: - Blood draws for immune markers- Standardized mental health scales- Occupational functioning
Quasi-controls: - Population norms on MH scales- Archival data: occupation & illness
Findings: - Long-term immune suppression- Social support ‘buffered’ stress- impact of bereavement > other stressors
One shot designs with archival controls, 3
Example 3: Psych. & Health effects of bereavement
Back NextHomepage
19Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
One shot designs; Summary
Virtue: Assess naturally occurring or uncontrollable socially or
politically important events
Provides “real world” look at processes that are typically studied in experiments: “Effectiveness” v. Efficacy data
Archival data can help interpret the findings / “control” some alternate interpretations.
Liability: lack of control group creates multiple threats to internal validity
No pre-measure makes interpretation (e.g., of change…) difficult.
“One Shot” designs: no control over independent variable(s), only partial control over measurement:
An experiment is not possible There cannot be a control group “Pre-” measures not possible or practical
Back NextHomepage
20Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Research Ethics
Quasi-experiments: Existing groups
Studying naturally occurring events
Measurement studies
Retrospective designs
Evaluate existing groups or program
Single shot survey or measure
Non-equivalent groups
Time series designs
Back NextHomepage
21Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
One group pre-test — post-test
Selected or convenience sample.
Group
Event or intervention May or may not be controllable by researcher, e.g., policy change.
Intervention or event Observe2
Outcome AssessmentTypically controllable, but may be archival.
Observe1
Baseline AssessmentMay or may not have control over measures (e.g., surveys v. archival measures).
Educational & social environments
Political or health policy change
Not feasible to have a control group
System-wide intervention / social change (school, public health campaign..)
Uses:
Back NextHomepage
22Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Key design feature: no control group.
Maturation
Reactive measures
Statistical regression
Mortality / drop-out
History Historical / cultural events occur between baseline & follow-up.
Individual maturation or growth occurs between baseline & follow-up.
People respond to being measured or being a measured a second time.
Extreme scores at baseline “regress” to a more moderate level over time.
People leave the experiment non-randomly (i.e., for reasons that may affect the results…).
Group Intervention or event Observe2Observe1
Confound Observe2Observe1
Threats to internal validity (confounds):
Back NextHomepage
23Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Examples: One group pre- post-, HIV testing
Event: - Receipt of HIV testing & counseling
Sampling frame: - Participants in testing centers
Study structure: - Baseline retrospective interview at testing session
- Follow-up interview 3 months later
Quasi-controls: - Population characteristics to predict between-group differences
Outcomes: - Self-reports of sexual risk
Example 1: Effects of HIV testing on sexual risk.
Back NextHomepage
24Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Example: One group pre- post-, HIV, 2
Findings:
Effects of HIV testing on sexual risk, cont.
Threats to internal validity
- Self-selection into testing group
- Mortality: non-random drop-out(?)
- History: general shift in norms & behavior during study time may account for observed change
- Instrument change; people may answer more conservatively during a follow-up interview
- Significant shifts toward safety
- Few demographic predictors of risk or risk change
Back NextHomepage
25Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Examples: One group pre- post- , Education
Intervention: - Standardized testing becomes integral to educational programs & school evaluation.
Sampling frame & - Longitudinal data across multipleStudy structure: years in target
school grades.
- No control group possible.
Quasi-controls: - Population characteristics to predict between-group differences
Outcomes: - Standardized test scores
Example 2: Educational reform & “No Child left behind” testing requirements.
Back NextHomepage
26Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Example: One group pre- post-, education, 2
Findings: Modest, statistically significant increase in scores
Usual demographic predictors of change; more affluent, better schools..
Internal validity?: Reactive measures; teachers & students do better
when measured; (they also cheat; see Houston Miracle article)
Instrumentation: kids get better at taking standardized tests, teacher better at teaching them
History: General cultural shift Education more prominent in city
More affluent families sending kids to public schools
Education reform & test scores.
Back NextHomepage
27Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
One group pre- post- designs; Summary
Virtues: provide data on naturally occurring socially or politically
important events Pre-measure allows researcher to interpret change & examine
status of groups at baseline.
History maturation statistical regression reactive measures mortality / drop-out
One group pre- post- test design useful where: An experiment is not possible There cannot be a control group Researchers have control over measurement and the
independent variable
Liability: lack of control group creates multiple threats to internal validity:
Back NextHomepage
28Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Groups are not equivalent at baseline, due to..
Self-selection Non-random assignment Use of existing groups Participants not blind
#1; Static Group Design
Non-equivalent two-group designs
Group1
Group2
Observe1
Observe1
Assessments may or may not be controlled
Survey or interviews Archival / existing data,
e.g., clinic records, grades
Intervention or event
Intervention or event may or may not be controlled by researcher;
Existing program Experimental intervention Naturally occurring event (..9/11..)
(No baseline)Contrast group
Back NextHomepage
29Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Non-equivalent designs; pre- post-
Observation1 used to Assess equivalence of groups at baseline Test for threats to internal validity:
Reactive measuresHistory, mortality effectsRegression effects
Group
Group
Observe1
Observe1
#2 Two Group Pre- Post- Design
Non-equivalent groups Self-selection Non-random assignment Use of existing groups Participants not blind
Observe2
Intervention or event Observe2
Intervention & Assessments often controlled by researcher in these designs.
Similar to true experimental
design, except for non-equivalent
groups
Contrast group
Back NextHomepage
30Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Examples: Non-equivalent groups, condoms
Study structure: - NY = intervention schools, Chicago are contrast schools.
- Baseline, sexual health programming, end of year Follow-up
Example Non-equivalent control group design: Effects of condom distribution on sexual safetyIntervention: - Condom education & distribution in
High School health classes
Outcomes: - Clinical measures: STDs
- Self-reports: sexual activity & safety
Sampling frame: - Schools in New York & Chicago
- Schools matched for SES, race, size
Back NextHomepage
31Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Examples: Non-equivalent groups, condoms, 2
Findings: NY (intervention) students; lower STD rate, safer sex NY and Chicago students; similar levels of sexual
activity Thus; sexual health classes appeared to increase
safety without increasing sexual activity.
Internal validity?: Reactive measures; Study is not blind; NY students
know they are the intervention group
Non-equivalent groups: Possible differences between cities = unmeasured confounds
Condom distribution, cont.
Back NextHomepage
32Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Non-equivalent designs
Groups 1 & 2:Observation1 used to Assess equivalence of groups at baseline Test threats to internal validity
Groups 3 & 4:Post-test only tests for reactive effects of assessment Compare 1+2 versus 3+4 Test interaction of treatment group x pre- post- versus post- only
Group 2
Group 1
Observe1
Observe1
Soloman 4-group design
Observe2
Intervention Observe2
Contrast group
Group 4
Group 3
Observe2
Intervention Observe2
Contrast group
Back NextHomepage
33Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Non-equivalent 2 group designs:
Summary Most common quasi-experimental approach. Used where:
Some form of control or contrast group is possible Groups cannot be equivalent:
Participants cannot be blind re: group assignment Random assignment not possible Must use existing or self-selected groups.
Virtue: Study natural / “real world” interventionsContrast group lessens major threats to internal
validityLiability: non-equivalent groups = possible confound.
Back NextHomepage
34Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Research Ethics
Quasi-experiments: Existing groups
Studying naturally occurring events
Measurement studies
Retrospective designs
Evaluate existing groups or program
Single shot survey or measure
Non-equivalent groups
Time series designs
Back NextHomepage
35Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Group Measure1 M2 M3 M4 M5 M6…
Intervention or event
Interrupted time series design
Intervention may be experimental or observed
Policy shift, e.g., educational policy
Uncontrolled event; e.g., 9/11/01, Media event
Assessments may be experimental or archival Successive cross-sectional surveys
Traffic data, clinic or crime reports, test scores
Test effect of intervention or event on ongoing series of measurements.
Back NextHomepage
36Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Group Measure1 M2 M3 M4 M5 M6…
Intervention or event
Time series designs
Multiple baseline
Demonstrate highly stable effect
long-term crime ratesdisease prevalenceeconomic performance…
Show steady rate of change
Hypothesis; tested by: Shift in stable rate after
intervention Increase / decrease in rate
of change after intervention
Threats to internal validity: sensitive to very local history Single group possibly prey to confound
Advantage for internal validity Eliminates carryover effects of repeated measurement tests maturation, history, reactive measurement, etc
Back NextHomepage
37Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Example of interrupted time series: Shift in Baboon culture.
Core question: Do baboon troops develop and transmit a learned “culture”?
Baseline: Long-term observational data on aggressiveness in a specific baboon troop.
Intervention: Tuberculosis outbreak due to infected food. Dominant / aggressive males fed first
are selectively infected are naturally culled from troop
Naturally occurring event in >20yr. ongoing field study.
Back NextHomepage
38Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Baboon culture: findings
Quasi-controls: Parallel data from other baboon troops.
Outcome measures: Standardized indices of aggression & dominance behavior
Core finding: With dominant males gone,
remaining males showed more cooperative behavior
Enhanced cooperation was transmitted across generation, showing learned “culture”.
Back NextHomepage
39Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Example: Interrupted time series data
Data: Archival records of HIV tests reported to CDC, collected monthly
Data show stable baseline over multiple observations Timing of intervention precise relative to data collection
Intervention: Magic reports infection on national TV. Uncontrollable, “naturally occurring” event Tests hypothesis re: modeling effects in health behavior
Finding: Initial spike in testing rates, followed by leveling off at higher base rate.
Initial increase expected Hypothesis tested by longer-term shift in base rate, available
due to archival time-series data Effect found for both genders.
The “Magic Johnson effect” on HIV testing
Back NextHomepage
40Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Example of time-series data: “Magic” / HIV effect.
Tesoriero, J.M., Sorin, M.D., Burrows, K.A., LaChance-McCullough, M.L. (1995). Harnessing the heightened public awareness of celebrity HIV disclosures: “Magic” and “Cookie” Johnson and HIV testing. AIDS Education and Prevention, 232-250.
Magic’sAnnouncement
Low & variable baserate of testing
Initial spike
New, higher base rate
Multiple (monthly) measures.
Time-series data showing shift in HIV testing after Magic’s announcement.
Back NextHomepage
41Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Multiple time series study
Group 1 Measure1 M2 M3 M4 M5 M6…
Group 2 Measure1 M2 M3 M4 M5 M6…
Multiple time series data
Groups typically formed by blocking variable measured post-hoc;
Health claims in NYC v. other cities post- 9/11/01
Younger v. older voting patterns post- Iraq invasion
Heterosexual v. gay HIV testing rates post- Magic Johnson media event.
Hypothesis; tested by interaction of blocking variable by repeated measure:
Is shift in stable rate ( rate of change) greater in one group than another?
Intervention or event
Back NextHomepage
42Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Blocking variables
Testing blocking variables in the HIV testing time-series data.
Core questions:Both heterosexuals and Ethnic minorities had low HIV
testing rates May feel HIV is not relevant to them – it is a “white gay” problem. They may lack resources or venues for testing.
Will having a prominent African-American Heterosexual disclose HIV+ status may change those perceptions?
Hypotheses:Heterosexuals will respond more strongly to the Magic
Johnson media event than will gay/bisexual men. African-American and Latino men and women will
respond most strongly.
Back NextHomepage
43Psychology 242Introductionto Research
Psychology 242, Dr. McKirnanWeek 12-13, quasi-experimental designs.
Blocking variables: sexual orientation, 1.
High base-line and high variability in testing rates among men with risky partners, and IDUs.
Testing blocking variables: Gay / IDU data.
Gay / bisexual men show less variable, but generally lower baserates.
Risky men & IDUs slightly increase, with substantial variability.
Gay & bisexual men show no change.
Back NextHomepage
44Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Blocking variables: sexual orientation, 2.
Testing blocking variables: Heterosexuals.
In contrast to gay / bisexual men or IDUs, heterosexual show an initially low baserate.
Followed by a large spike after the announcement
And a much higher new baserate.
The hypothesis that heterosexuals would be more affected by the “Magic” announcement was supported by the interaction of Time x the blocking variable of sexual orientation.
Back NextHomepage
45Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Blocking variables: ethnic differences
African-Americans and Hispanics show low baserates and a high spike post-announcement
Testing blocking variables: Ethnic differences.
Both groups go back toward their baselines shortly post-announcement.
Back NextHomepage
46Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Blocking variables: ethnic differences, 2.
HIV testing among Whites was similar to African-Americans & Hispanics at baseline,
They showed stable, much higher testing rate after Magic’s HIV announcement.
Ethnic differences: White participants.
Back NextHomepage
47Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Summary: Blocking variables in time series data
A series of measures before & after an event allows us to clearly identify patterns of behavior, and to test group differences (via blocking variables).
The hypothesis that ethnic groups would differ was supported by interaction of Time x the blocking variable of ethnicity (but in a direction that was not predicted: Whites showed more change).
Back NextHomepage
48Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Time series designs: Summary
Time series is most common with archival data: existing, standard records collected for other purposes.
Used where: The hypothesis concerns changes in long-term trends Typically an experiment cannot be run
Simple practicality or cost, e.g., health care issues Ethics; crime rates, rates of domestic violence, etc. The target events are not controllable.
Virtue:
Study natural / “real world” processes or interventions
Blocking variables – comparing time trends across groups -- lessens major threats to internal validity
Liability: lack of control = possible confound.
Back NextHomepage
49Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Quick quiz
Researchers often use _____ to help interpret “single shot” surveys
A = paradigm change
B = measurement studies
C = experimental controls
D = retrospective measures
Back NextHomepage
50Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Quick quiz, 2
B = Maturation
E = Reactive measures
D = Statistical regression
A = Mortality / drop-out
C = History
Historical / cultural events occur between baseline & follow-up.
Individual maturation or growth occurs between baseline & follow-up.
People respond to being measured or being a measured a second time.
Extreme scores at baseline “regress” to a more moderate level over time.
People leave the experiment non-randomly (i.e., for reasons that may affect the results…).
Match:
Back NextHomepage
51Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Quick quiz, 2
B = Maturation
E = Reactive measures
D = Statistical regression
A = Mortality / drop-out
C = History People respond to being measured or being a measured a second time.
Match:
Back NextHomepage
52Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Quick quiz, 3
B = Maturation
E = Reactive measures
D = Statistical regression
A = Mortality / drop-out
C = History
Growth or natural change between baseline & follow-up.
Match:
Back NextHomepage
53Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Quick quiz, 4
B = Maturation
E = Reactive measures
D = Statistical regression
A = Mortality / drop-out
C = HistoryPeople leave the experiment non-randomly (i.e., for reasons that may affect the results…).
Match:
Back NextHomepage
54Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Group Measure1 M2 M3 M4 M5 M6…
Intervention or event
Quick quiz 5
This is called a:
A = Threat to internal validity
B = Manipulation check
C = Multiple baseline
D = ..lot of work.
Back NextHomepage
55Psychology 242Introductionto Research
Psychology 242, Dr. McKirnan Week 12-13, quasi-experimental designs.
Quasi-experiments; Summary
2. Evaluate existing groups or program(s) Single shot survey or measure of an intervention
With or without control variables
Non-equivalent / pre-existing groups Static group or 2 group pre- post- design
Time series designs, often with archival data
1. Study naturally occurring events that could not be brought into a lab or a true experiment.
Measurement studies
Retrospective designs
Trade off internal for external validity
Back NextHomepage
56Psychology 242Introductionto Research
THANK YOU !!