imagine the possibilities - assembling modular mobile surveys to create complete datasets - ssi...

JULY 16 - 18, 2013, Minneapolis, USA

WWW.MRMW.NET

The original, premier event for the Mobile Marketing Research Industry

WWW.MRMW.NET

TITLE SPONSOR DIAMOND SPONSOR PLATINUM SPONSOR

GOLD SPONSORS

WORKSHOP HOST

SILVER SPONSORS

PREMIERE SPONSOR

NETWORKING EVENING SPONSOR BAG SPONSOR PREMIERE SPONSOR

SSI Confiden*al © 2013 Survey Sampling Interna6onal

Assembling Modular Mobile Surveys to Create Complete Datasets

Market Research in the Mobile World July 17, 2013


Edward Paul Johnson Director of Analy6cs SSI Chris1 Walters Principal Gongos Research

| 2 |


Current Environment

Researcher to Respondent Rela1onship

VS.

respondents researchers

I want to know what you think about soH drinks

Great! Here’s a 30-‐minute survey…

Sure I have some 6me to help you out

Wait, what does my hair color have

to do with beverages?

| 3 |


Problem

New Mobile World

Consumers want control

Where. When. How long?

How can we give them what they want?

| 4 |


Ques6ons

Can we modularize? → Within-‐respondent

→ Between-‐respondent

Can we fuse the data?

| 5 |


Key Hypotheses

Willingness to par1cipate in “long” surveys on mobile devices will be increased by offering incremental incen1ves 2 There are minimal data effects on modularizing surveys on smartphones

3 Data fusion techniques allow advanced analysis despite missing data

4 Bivariate analysis is possible with data fusion techniques

5 AStude and behavioral hooks will prove superior to demographic hooks

6

Modularizing surveys will create a beVer, more enjoyable experience for the respondent 1

This is the correct image.

| 6 |


Methods

Ques1onnaire Design

Full Ques1onnaire

Sec1on A

Sec1on B

Sec1on C

We started with a full ques1onnaire

with three sec1ons

| 7 |


Methods

Ques1onnaire Design

Full Ques1onnaire

Hook

Que

s6on

s

Sec1on C

Sec1on B

Sec1on A

Pieces from

each sec1on were removed as

hook ques1ons

| 8 |


Methods

Ques1onnaire Design

Full Ques1onnaire

Hook

Que

s6on

s

The rest of the ques1onnaire was

split into three modules

Mod

ule

Mod

ule

Mod

ule

Sec1on C

Sec1on B

Sec1on A

| 9 |


Methods

Ques1onnaire Design

Respondents were randomly

assigned to a star1ng module.

Respondents always saw a specific set of

‘hook’ ques1ons within their first

module.

Respondent Survey

Hook Ques1ons

1st Module

2nd Module

3rd Module

Choice to con1nue

Choice to con1nue

| 10 |


Need to Assemble the Pieces

| 11 |


Assembly Method

Hot Decking Imputa1on

•  Finds similar respondents •  Based on similari6es to link

respondents together •  Drops unlinked data

—  Only actual responses

Respondent Matching

•  Es6mates missing data •  Based on the similari6es to

the remaining sample •  Includes all respondents

—  actual responses and —  es6mated responses


Results

H1 – enjoyable experience

Average Difference in Sa6sfac6on Ra6ngs Sa6sfac6on -‐0.06 -‐0.03 -‐0.02 -‐0.14

Bad Data Checks No Straightlining 84.0% 81.7% 82.3% 87.0%

Failed Grid 1 Instruc6ons 17.6% 13.9% 12.8% 16.9%

Failed Grid 2 Instruc6ons 17.3% 18.1% 9.7% 12.7%

Failed Zip Code Match 2.2% 1.6% 3.4% 1.4%

Control [Online Complete]

Control [Mobile Complete]

Test [Mobile Modular]

Test [Online Modular]

| 13 |


11% 9% 25%

8% 9% 8%

4% 8%

80% 83% 71% 84%

Abandoned Removed Completes

Results

H2 – willingness to par1cipate

Control [Online Complete]

Control [Mobile Complete]

Test [Mobile Modular]

Test [Online Modular]

| 14 |


Results

H2 – willingness to add modules

9% 19%

72%

1 Module 2 Modules 3 Modules 1 Module 2 Modules 3 Modules

| 15 |


Results

H3 -‐ minimal sta1s1cal effects

Compared to Online Control

Online Modular Sta1s1cal:

No discernible differences

Mobile Modular Sta1s1cal:

25% of the ques6ons showed sta6s6cal

differences

| 16 |


Results

H3 -‐ minimal prac1cal effects

Compared to Online Control

Online Modular Prac1cal:

Findings and resul6ng insights are the same

Mobile Modular Prac1cal:

Resul6ng insights were similar, however, some differences do exist

| 17 |


Results

H4 – segmen1ng the data

Hot Deck Data Imputa1on

Data Matching Based on AStudes & Behaviors

Online Control

Segmenta1on analysis resulted in a three-‐segment solu1on.

| 18 |


Results

H4 -‐ checking segment algorithms

When applying the online control algorithm to each data set…

When applying algorithms from each data set to the online control data…

| 19 |


Results

H4 -‐ segment solu1on results

20% 26% 26%

53% 51% 50%

28% 23% 24%

Segment Sizing Across Techniques

Online Control

Mobile Hot Deck Data Imputa1on

Segment 1

Segment 2

Segment 3

Data Matching Based on AStudes

& Behaviors

| 20 |


Results

H4 – accuracy of segment assignment

83% 86% 86% 86%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Imputed Matching

Missing Full


Mobile Hot Deck Data Imputa1on

| 21 |


Results

H5 – bivariate rela1onships

Hot Deck Imputa1on

Respondent Matching

501 424 17% 20% 0.29 0.22 .24 0.29 .19 0.19

Control Modular No Imputa1on

Sample Size 333 416* % Significant Differences 0.0% 15%

Average Reasons r 0.39 0.31 Average AVribute r 0.30 0.24

Average Reason x AVribute r 0.22 0.20

Distance Correla1on

| 22 |


Results

H6 -‐ aStudinal vs. demographic s1tching

Online Control

Segment 1

Segment 2

Segment 3

Data Matching Based on AStudes

& Behaviors

20% 19% 26%

53% 55% 50%

28% 25% 24%

Data Matching Based on

Demographics

| 23 |


Results

H6: aStudinal vs. demographic s1tching

92% 86% 87% 86%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Demos Aetudes

Missing Full


Data Matching Based on Demographics

| 24 |


Conclusions

Allow respondents to choose mode (online vs. mobile vs. mul1modal)

1 Within-‐respondent modulariza1on key to reducing holes in data 2 Advanced analy1cs feasible (i.e. segmenta1on)

3 Both fusion techniques work with unique advantages

4

| 25 |


[email protected] [email protected]

| 26 |

WWW.MRMW.NET

TITLE SPONSOR DIAMOND SPONSOR PLATINUM SPONSOR

GOLD SPONSORS

WORKSHOP HOST

SILVER SPONSORS

PREMIERE SPONSOR

NETWORKING EVENING SPONSOR BAG SPONSOR PREMIERE SPONSOR

JULY 16 - 18, 2013, Minneapolis, USA

WWW.MRMW.NET

The original, premier event for the Mobile Marketing Research Industry

imagine the possibilities - assembling modular mobile surveys to create complete datasets - ssi...

Technology

respondent survey hookques1ons

respondent canwefusethedata

metohelp youout

premier event

imputed matching missing

resul6nginsightswere

assignedtoa star1ngmodule

howcanwegive themwhatthey