getting closer to in the moment insights generation - esomar congress 2016

The past is behind, learn from it. The future is ahead, prepare for it. The present is here, live it.

Thomas S. Monson

If one wants to run, they must learn how to walk. If one wants to teach, they must learn. If one wants to define the future, they must study the past.

Confucius

The longer you live in the past, the less future you have to enjoy.

Anonymous

1930 1935 1940 1945 1950 1955 1960 1965 1970 1975 1980 1985 1990

DIGITAL TRANSFORMATION IS DISRUPTING MARKET RESEARCH

GLOBAL CONSUMER RESEARCH & DATA COLLECTION COMPANIES SINCE 1930

1995 2000

DIGITAL TRANSFORMATION IS DISRUPTING MARKET RESEARCH

THINGS WILL NEVER BE THIS EASY AGAIN

2005 2010 20152000

COMBINING SURVEY BASED AND BEHAVIORAL DATA COLLECTIONDIGITAL TRANSFORMATION IS DISRUPTING MARKET RESEARCH

Use behavioral data to understand the context of survey data.

Use behavioral data to select the sample.

Use behavioral data to survey in the moment.

BEHAVIORAL DATASee what.

DECLARED DATAUnderstand why.

Happy panelists Data quality Faster and better decisions

R E S E A R C H N E E D S TO G E T C LO S E R TO T H E M O M E N T

Getting closer to in the moment insights generation

Gamechangers happen overnight

Our study

HYPOTHESES ABOUT DIFFERENT SAMPLE SELECTION METHODS

Using behavioral data will…

Improve

… the participation and efficiency

… the data quality

… the survey experience

Not affect

… the respondents’ profile

… the substantive results

Getting closer to in-the-moment will…

Improve even further

… the participation and efficiency

… the data quality

… the survey experience

Not affect

… the respondents’ profile

… the substantive results

OUR STUDY

DIFFERENT SAMPLE SELECTION METHODS

Providers

Panel: Netquest

Passive metering technology: Wakoopa

Data collection

February 23 to April 3, 2016

June 7 to June 28, 2016

OUR STUDY

1. Classic survey without previous information

2. Classic survey with sample selected using metered data

3. Survey in the next 48 hours with sample selected using metered data

Comparison

People who visited the website of at least one of 8 airline companies in the last 2 months

Population of interest

Should we use behavioral data to select the samples?

IMPROVEMENTS IN SURVEY PARTICIPATION AND SURVEY EFFICIENCY

58%65%

82%89%

0%

20%

40%

60%

80%

100%

%participants who match target pop.*

USING BEHAVIORAL DATA

29%

86%

67%73%

0%

20%

40%

60%

80%

100%

Observed vs Expected Participation*

* Exact definitions provided in paper

MUCH BETTER SURVEY EXPERIENCE AND SLIGHTLY BETTER DATA QUALITY

35%

20%

33%

16%

0%

20%

40%

60%

80%

100%

Don’t Know answers (Block D)


61%

79%80%

92%

0%

20%

40%

60%

80%

100%

Proportions who liked the survey

CHANGES IN RESPONDENT PROFILES AND SURVEY RESULTS

76%

64%

85%

71%

0%

20%

40%

60%

80%

100%

Average Brand Awareness (%)


51%

76%

65%

83%

0%

20%

40%

60%

80%

100%

Tertiary education (%)

Should we get closer to the moment?

REDUCED PARTICIPATION BUT IMPROVED EFFICIENCY

82%89%89%

96%

0%

20%

40%

60%

80%

100%

%participants who match target pop.*

GETTING CLOSER TO “IN-THE-MOMENT” INSIGHTS GENERATION

67%73%

21%

38%

0%

20%

40%

60%

80%

100%

Observed vs Expected Participation*

* Exact definitions provided in paper

SIMILAR SURVEY EXPERIENCE AND DATA QUALITY

33%

16%

33%

18%

0%

20%

40%

60%

80%

100%

Don’t Know answers (Block D)


80%

92%

76%

91%

0%

20%

40%

60%

80%

100%

Proportions who liked the survey

CHANGES IN RESPONDENT PROFILES AND SIMILAR SURVEY RESULTS

85%

71%

85%

70%

0%

20%

40%

60%

80%

100%

Average Brand Awareness (%)


65%

83%

59%

89%

0%

20%

40%

60%

80%

100%

Tertiary education (%)

KEY TAKE AWAYS

Should we use behavioral?

Overall, using behavioral data to select the

sample seems to help: recommended!

But we need to use quotas/weighting


Getting closer to the moment-of-truth does not improve as expected

But it does not harm except for response rates

CONCLUSIONS

In general, results are robust across countriesDifferences in values, but not in trends, except in a few cases

Can we use it across borders?

Sample selection

A key step for online surveys

Affect costs, data quality, inferences possible

Finding the best way to do it is crucial

KEY TAKE AWAYS

Should we use behavioral?

Overall, using behavioral data to select the

sample seems to help: recommended!

But we need to use quotas/weighting


Getting closer to the moment-of-truth does not improve as expected

But it does not harm except for response rates

CONCLUSIONS

In general, results are robust across countriesDifferences in values, but not in trends, except in a few cases

Can we use it across borders?

Sample selection

A key step for online surveys

Affect costs, data quality, inferences possible

Finding the best way to do it is crucial

We would like to ask you

to answer a couple of

questions about these

results.

NOTIFICATION

Your answers are very important for us.

Please go to ntq.io/esomar

CONCLUSIONS

WHAT NEXT?

Is it worth it to continue in the direction of in-the-moment?

Many challenges already todayto survey in the next 48 hours.

Little evidence getting closer is improving things

Is 48 hours too long already?

Our current point of view:

in-the-moment might be worth it but only for specific projects.

What about your point of view?

PHOTOPHOTO

Thank you!

Melanie RevillaResearcher, RECSM-UPF

Daan VersteegMarketing Director, Netquest

[email protected]/in/dsaversteeg