Designing Surveys for Mobile Devices: Pocket-Sized Surveys That Yield Powerful Results

Mario Callegaro, Tim Macer

Mobile Phone Penetration Up

Rules of Thumb

No horizontal scrollingVertical scrolling OK

Avoid long listsEspecially in check all that apply

Situation FluidAs Tablet become Popular

Depends on the Platform

Platform Considerations

Need to Test on Multiple Platforms

Apple (iPhone/iPad) can’t implement Flash

Default on all phones is to not enable Java

Useful Paradata

UserAgentStringDevice\Model

Operating System

Screen Resolution

Fonts

“Can You See It Now? Good”Usability Testing of a Mobile Health Application

Sarah Cook, Rita Sembajwe, Emily Geisen, Barbara Massoudi

New Way to Do a Diary

Benefits

Immediate Results

Cost Effective

Create Easy to Use Dashboard

Usability Suggestions

Don’t scroll vertically on Select All

Make it easy to trace any sliding

Hard to video what they do

Mobile Phone Effects at Event-Based Sampling

Dan Williams

Case Study

Three Modes of Collection

WebMost Popular

Not all on Mobile Device

IVRCapture Older Population

SMSImmediate Response

Younger Respondents

Are you who you say you are? Using a Multisource Cross-validation Methodology for

Panel Membership Information.

Kumar Rao

Real, Unique, and Engaged

3rd Party Database ValidationInclude Demographics

Use Multiple Databases

Results

Cost could be worth the extra

All more likely to be established households

False Positives Too High

Still Important Part of Process

Differential Sampling Based on Historical Individual-Level Data in Online Panels

Richard Kelly

Quota Sampling

Way to Deal with Non-ResponseDidn’t Know Demographics

More Efficient to Screen Out

Just Transferred Over

to Online

Differential Sampling

Know the Demographics

Know the Response Rates

Oversample those Hard to Reach

More Efficient and Cost Effective

Designing Questions for Web Surveys: Effects of Check-List, Check-All, and Stand-Alone Response

Formats on Survey Reports and Data Quality

Jennifer Dykema, Nora Cate Schaeffer, Jeremy Beach, Vicki Lein, and Brendan Day

Three Types Web Designs

Check-ListMore Items Selected

Check-AllLower Break-offs

Stand AloneLess Primacy Effect

Category Selection Probing in Online Access Panels

Dorothée Behr, Lars Kaczmirek,

Michael Braun, Wolfgang Bandilla

Cognitive Testing OE

Face-to-Face too Expensive

Online TestingProbing Open Ends

Community vs PanelMore chatty?

Results

Topic Trumps Source

Use Communities Built Around the Topic

Face-to-Face More Involved

Response Quantity, Response Quality, and Costs of Building an Online Panel via Social Contacts

Vera Toepoel

Snowball Recruiting

No Online Panel in NL RepresentativeRequires More Commitment

Try Refer a Friend Program

Use Network Theory

Results

Snow Never RolledOnly got 120 recruits

Don’t Use Students

Incentives not Worth the Cost

Representativeness

The Use of Web Panels to Characterize Rare Conditions

John Boyle

Hard to Reach Population

Only 23 in a sample of 10,000 HH

Costs High

Variance Too High

Important Diseases

Clean the Online Data

Certain Improbable Conditions

Speeders

Straightliners

Results In Line

Prevalence In Line

Treatments Numbers Good

Got Much More Sample Size

Cost Less

Measuring Intent to Participate and Participation in the 2010 Census and Their Correlates and Trends:

Comparisons of RDD Telephone and Non-probability Sample Internet Survey Data

Josh Pasek and Jon Krosnick

Intent to Complete Census

Better Demographics Compositions

Intent Numbers Varied

Predictors for Intent to Complete Different

Trends Also Different

Can a Non-Probability Sample Ever be Useful for Representing a Population?: Comparing Probability and Non-Probability Samples of Recent College Graduates

Cliff Zukin, Jessica Godofsky, Carl Van Horn, Wendy Mansfield, and J. Micheal Dennis

Comparing Sampling

Probability Samples have a Prob Theory

Can’t Intelligently Trade Off Error

Compare KN Panel to volunteer Panel

Recent Graduates

Results

Differences between probability and non-probability panel

No mode effects or questionnaire effects

Differences mitigated a lot when weighting for other non-quota variables