research. refined.
• Telephone surveys appear remarkably resistant to systematic bias, despite low response rates.
• However, costs continue to rise and response rates continue to drop.
Why Talk About Mixed Samples & Modes?
research. refined.
• CDSF file• Enumerated/supplemented• Empaneled
• Single/Dual Frame• Telephone Recontacts• Empaneled
• Full Universe Samples• Internet Empaneled
• In-Person w/ w/o Interviewer
• Telephone w/ w/o Interviewer
• Internet
Samples & Modes
research. refined.Options Within Samples
• Appended Sample from available sources
• Listed sample of any kind with incomplete coverage
• High incidence areas or other geographically selected subsamples
• Internet could/should be considered incomplete at least due to coverage area
• Etc.
research. refined.
• Can have any combination of inbound phones, outbound phones, mail, and internet
• A single sample frame
Popular Mixed Samples/Modes Designs
• 2 samples• 2 modes
• 2 samples• 1 mode • 2 samples
• 1 mode
• multiple samples
• 1 mode• multiple
samples• 2 modes
research. refined.Problems of Combining Multiple Samples
The weight of academic research continues to find in nonprobability samples 1) greatersystematic non-response (bias) and 2) greater variability in the size of bias than in probabilitysamples.
One mode has more measurement error than another on variables of interest. However, in asingle sample study, mode becomes a sample type, potentially confounding with measurementerror.
research. refined.Problem 1: Probability & Nonprobability
0 5 10 15 20
Luth Web Unweighted
Research Now Web Unweighted
SSRS Omnibus Cell Phones…
SSRS Telephone Omnibus…
SportsPoll Telephone Unweighted
NHIS Unweighted
Luth Web Raked
Research Now Web Raked
Research Now Propensity Weighted
Research Now Propensity Weighted…
Research Now Matched
Research Now Matched and Raked
Luth Web Matched
Luth Web Matched and Raked
Cell Phones Raked
SSRS Omnibus Telephone Raked
SportsPoll Raked
NHIS Weighted
Race within Education
Age within Education
Region within Education
Race within Age
Education within Age
Region within Age
Age within Race
Region within Race
Education within Race
Race within Region
Age within Region
Education within Region
research. refined.Problem 2: Multiple Modes
• Two samples, different modes in each• One sample, sequential or concurrent mixed-mode offering• Multiple time points via panel or cross-sectional
1) Media-related factors: who has control over the Q and A2) Factors influencing the information transmission: visual/auditory, verbal, nonverbal, paralinguistic
differences and visual/graphical/layout 3) Interviewer effects: Privacy, locus of control, social desirability, etc.
1) Less social desirability2) More missing data3) Less extremeness (evidence weak)4) Inconsistent effects with regard to recency/primacy
research. refined.
Percent of Jews Percent of Sample Weight
Client List 40% 65% 0.62
Name Targeted 10% 15% 0.67
Residual 50% 20% 2.50
•
•
•
Standard Base Weighting for Selected & Residual Designs
research. refined.References