mobile surveys: quality and validation

GREATER NEW YORK | LOS ANGELES | DALLAS | LONDON | NEW DELHI

Robert ClancyVP, Insights & Strategy

Lisa Wilding-BrownVP, Global Panel and Sampling Operations

Mobile Survey Data:Quality and ValidationMRMW Conference July 17, 2013

http://www.google.com/url?sa=i&rct=j&q=mobile+survey&source=images&cd=&cad=rja&docid=DtdmZJk8VRSlmM&tbnid=WkRojtt0-LIQDM:&ved=0CAUQjRw&url=http://www.gsmbird.com/using-a-mobile-survey-app-for-better-client-interaction/&ei=3evNUe6CDIX7ygG6hoGIAg&psig=AFQjCNEAkU3BTCsmQW68Q0DL8Se8CtQc9A&ust=1372536135818032


What we’ll be talking about today

2

1. The Emergence of Mobile

2. Case Study: Mobile vs. Online Data Quality

3. Ensuring Mobile Data Quality


Drive slow, be safe and…

3


…stay connected

4

GREATER NEW YORK | LOS ANGELES | DALLAS | LONDON | NEW DELHI 5

The creepy phone interviewer


The creepy mall interviewer

6


The New Dawn


Different Modes, Same Challenge

8

http://www.google.com/url?sa=i&rct=j&q=unsure+person&source=images&cd=&cad=rja&docid=xIfzaFr5qHXEYM&tbnid=TNdXRYdb4u9_JM:&ved=0CAUQjRw&url=http://www.huffingtonpost.com/briana-rognlin/life-coaching-for-skeptic_b_709037.html&ei=vc3NUYfuLYX88QT3_oDoBw&psig=AFQjCNHPTis6JtF3GAcKjUxQfmYKTq8A8Q&ust=1372528379770606

http://www.google.com/url?sa=i&rct=j&q=online+survey&source=images&cd=&cad=rja&docid=XlcJ9feZoakAkM&tbnid=R1_UHHah7QUJ5M:&ved=0CAUQjRw&url=http://www.marketresearchforbeginners.com/five-tips-for-writing-an-online-survey/&ei=nanNUc_mFJHO9ATRjIGoDA&bvm=bv.48572450,d.eWU&psig=AFQjCNFngZl7iucteh60CQR4p8dTm9QucA&ust=1372519186665373


A fragmented consumer makes connecting a challenge

9


Now, we’re connecting on a whole new level…


…and, we can’t get enough

11

56%

14%

19%

Within reach of my bed

In another room

In the bedroom

Where is your phonewhen you sleep?

24%

45%

64%

Percent of time during dayphone is within arm’s length

100% of the time

70+% of the time

90+% of the time

30%

31%

24%

15%

Within 5 minutes

2 hrs. +

30 mins. – 2 hrs.

6 – 30 minutes

How quickly do you use your smart phone after waking?

65%

35%NO

YES

Use smart phone within10-15 minutes before sleeping?

Source: uSamp Mobile Study, June 2013


Quality of Mobile Responses


Big Driver of Quality Data for Mobile

13

User Experience

http://www.google.com/url?sa=i&rct=j&q=happy+with+mobile+phone&source=images&cd=&cad=rja&docid=ImBDD_59YfL_4M&tbnid=_ORrQeKSdt4C6M:&ved=0CAUQjRw&url=http://footage.shutterstock.com/clip-2162681-stock-footage-successful-businessman-is-happy-using-mobile-phone-sending-text-message-outdoors.html&ei=HKrNUYvDEJTW8gSj9YCQCg&bvm=bv.48572450,d.eWU&psig=AFQjCNHoCKGLzSiOVZF_erZJrwQe4DVY9w&ust=1372519267651031


Surveys on Smartphones = A Great Experience

14

Easy 75%Interesting 59%Fun 50%

In the moment Convenient

Preferred Survey Mode

42%55%

Source: uSamp Mobile Study, November 2012

http://www.google.com/url?sa=i&rct=j&q=round+the+clock&source=images&cd=&cad=rja&docid=icuhHA5KPCQbuM&tbnid=sQVD0bV0xf6DXM:&ved=0CAUQjRw&url=http://rapgenius.com/1687646/Mac-miller-crushin-round-the-clock/Its-all-right-its-okay-seein-you-all-night-and-all-day-we-be-crushin-round-the-clock-crushin-crushin-round-the-clock-we-be-crushin-round-the-clock-crushin-crushin-round-the-clock-its-all-right-its-okay-seein-you-all-night-and-all-day-hey-we-be-crushin-round-the-clock-crushin-crushin-round-the-clock-we-be-crushin-round-the-clock-crushin-crushin-round-the-clock&ei=1rPNUaxghu70BMLmgfAO&bvm=bv.48572450,d.eWU&psig=AFQjCNHp_uJnYuDmxowqo5CJEE-oBO6nxA&ust=1372521728171740

http://www.google.com/url?sa=i&rct=j&q=computer&source=images&cd=&cad=rja&docid=R1ZurOuOtUJ2NM&tbnid=4LD4vNRi1acYyM:&ved=0CAUQjRw&url=http://www.buffalocomputerconsulting.com/&ei=UrXNUbHINIaA9gSp5oG4DA&bvm=bv.48572450,d.eWU&psig=AFQjCNGJ3X8f-pBbJYna1y8KXbyq5KPpyA&ust=1372522178228681






Mobile vs. Online Case Study: Response Rates

15

Mobile respondents responded well, despite a tougher task

MOBILE ONLINE

Initial response to invitation 9.0% 4.4%

Completed screener 86% 92%

Began main survey 92% 100%

Completed survey 86% 99%

Source: uSamp Mobile vs. Online Side-by-Side Case Study Grocery Shopping Survey, May 2013


Mobile: in-the-moment yields higher quality data

16

Mobile Online

Confidence in recalling specific brands

HIGHER LOWER

Social desirability errors

LESS MORE

“Satisficing”

LESS MORE



Mobile: Rich data collection

17


AUDIO TYPED

Open end word count 19.5 17.3

Percent 30 words or more

18.7% 15.3%

http://www.google.com/url?sa=i&rct=j&q=typing+on+computer&source=images&cd=&cad=rja&docid=j2gXmNYqk2LcuM&tbnid=iSPRd8TxsjAvlM:&ved=0CAUQjRw&url=http://footage.shutterstock.com/clip-2342456-stock-footage-high-speed-typing-on-computer-keyboard.html&ei=mrvNUZrRLpPA9gTZj4DoDw&psig=AFQjCNFAxL3ZjIp3EHi7HXtGMJoH_W6pMg&ust=1372523768249890

http://www.google.com/url?sa=i&rct=j&q=talking+into+smartphone&source=images&cd=&cad=rja&docid=4S0cz1Wy04F60M&tbnid=i1WcxqWgEV2ibM:&ved=0CAUQjRw&url=http://tech2.in.com/news/smartphones/over-1-billion-smartphones-in-use-worldwide-report/519692&ei=Sr7NUYK4HorY8gSOiYGADw&psig=AFQjCNHEK3Q26pAown21BEw5FaiB7tIKSA&ust=1372524446648302


Ensuring Data Quality

http://www.google.com/url?sa=i&rct=j&q=data+quality&source=images&cd=&cad=rja&docid=Urpo5j6GizWO7M&tbnid=ywKJm6uzngd2LM:&ved=0CAUQjRw&url=http://businessnetworking.com/tag/givers-gain/&ei=W6_RUe6RNoLW9ATfuYBo&bvm=bv.48572450,d.eWU&psig=AFQjCNGNciAXso62t9RRgJoQIl1Mo7YgNg&ust=1372782786290515


Learning from our past… there is no magic bullet

19


Quality techniques in the mobile environment

20

Source Testing

Registration Scoring

Algorithmic Analysis

Respondent on-boarding

Recruitment

User Modeling

Red Herrings

Geo-specific Tracking

Geo-fencing

Rich Media Capture

Respondent Management

http://www.google.com/url?sa=i&rct=j&q=crowd&source=images&cd=&cad=rja&docid=hT7_nFc4Qf9tZM&tbnid=rqmiYeefu8RRkM:&ved=0CAUQjRw&url=http://irevolution.net/2013/04/02/theory-of-crowd-capital/&ei=n87NUajjL4ni9gTmg4DIAg&psig=AFQjCNEAARx0X64SL5zUjvXJ5_nR6mdmqw&ust=1372528658542394


Caucasian CEO of a fortune 500 company….

21

21


…suffering from alopecia…

22


…and a super rare skin condition…

23


…currently living in New York…

24


…No! San Francisco…

25


…who drives a cherry red Lincoln…

26


…and owns a show dog…

27


…who won Best in Show at Westminster.

28


The Black Swan

29

http://www.google.com/url?sa=i&rct=j&q=black+swan+bird&source=images&cd=&cad=rja&docid=x46tt2Qfdpd5sM&tbnid=3cT5egjgyjXnaM:&ved=0CAUQjRw&url=http://www.thewesternisles.co.uk/birds/black-swan.htm&ei=fTrUUeO_DOHs0QGTqYCwBw&bvm=bv.48705608,d.dmQ&psig=AFQjCNGeLXpiX13-6wT8v2lYVDXFKI2Bfg&ust=1372949404509192

On the horizon…

30

• Image recognition

• Automated photo coding

• Dial testing

• Audio transcription

• Behavioral monitoring


Robert ClancyVP, Insights & Strategy

Lisa Wilding-BrownVP, Global Panel and Sampling Operations

mobile surveys: quality and validation

Technology

new dawn

new level

mobile data quality

usamp mobile study

mobile environment

mobile respondents

emergence of mobile

quality of mobile responses