KEYS TO DRIVING LASTING PANEL SATISFACTION

CUSTOM PANEL DEFINED

2

A custom online panel is targeted network of people whom you engage with and learn from over time.

1. Typically Invitation Only, Private Environments 2. Customers, Partners, Recruited Participants 3. Company Owned and Sponsored: Branded or Blind 4. Category or Brand Specific: Branded or Blind 5. Mix Quant with Qual. 6.

3

In the telephone research world people are refusing to participate in surveys

In the online access panel world members stop responding and those who do respond are ever more closely monitored for cheating

In the custom panel world the respondents are customers and can no longer be treated as numbers in a set of tables

4

We  need  to  be  a  lot  more    Respondent  Centric  

PRESENTATION OVERVIEW

5

Panelist Satisfaction Study (PSAT)

Study Design Results from Angus Reid Forum (ARF) Combined results from custom panels Specific results from custom panels

Ideas for driving positive panelist satisfaction

Questionnaire Design Providing Feedback Incentives

Take Aways

PANEL SATISFACTION STUDY (PSAT)

PSAT STUDY DESIGN

7

Panelists asked to rate overall satisfaction on 10 point scale

Further 10 point ratings across key areas which are typically sources of satisfaction / dissatisfaction for panel members.

Also asked to indicate likelihood to recommend

Shapely regression analysis used to determine which of these variables have the greatest impact on overall satisfaction

PANEL  NAME  

SATISFACTION POINTS MEASURED

8

Level of satisfaction rated for:- Survey topics Frequency of surveys Quality of surveys Length of surveys Amount of time given to respond Incentives offered Newsletters / Communications received Look and feel of surveys That the input provided is valued

KEY DRIVERS OF SATISFACTION

9

Positive  and  Important    

Negative  and  Important  

Positive  but  LESS  Important  

 

Negative  but  LESS  Important  

 

RESULTS FROM ANGUS REID FORUM (CANADIAN NATIONAL PANEL)

10

the survey topics

the frequency of surveys

the quality of surveys you

receive

the length of each survey

the amount of time given to respond to

surveys

the incentives

with the look and feel of surveys with the

newsletters / communications that you receive

that the input you provide is valued

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

-2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0

Impo

rtan

ce

Performance

ARF Overall Satisfaction

RESULTS FROM COMMUNITY PANELS

11

the survey topics

the frequency of surveys

the quality of surveys you

receive

the length of each survey

the amount of time given to respond

to surveys

the incentives

with the look and feel of surveys

with the newsletters /

communications that you receive

that the input you provide is valued

-2.0

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5

Impo

rtan

ce

Performance

Custom Panels Average

RELATIVE IMPORTANCE

12

0% 2% 4% 6% 8% 10% 12% 14% 16%

Time  to  respond

Frequency

Length  of  survey

Look  and  feel

Newsletters

Incentives

Survey  topics

Input  is  valued

Quality  of  surveys

BLIND PANELS

13

0% 5% 10% 15% 20%

Frequency

Time  to  respond

Look  and  feel

Length  of  survey

Newsletters

Input  is  valued

Survey  topics

Quality  of  surveys

Incentives

All  Custom Average  Blind

BUSINESS PANELS

14

Panel of teachers with an Average Score of 8.33 Key Drivers: Input 21% (15%) & Newsletter 17% (11%)

Office Services Panel with an Average Score of 7.68

Key Drivers: Input 19% (15%) & Incentives 17% (13%) IT Decision maker panel with an Average Score of 5.90

Key Drivers: Newsletters 25% (11%) & NO INCENTIVE

NO INCENTIVES

15

A tale of 2 panels #1: IT decision makes with average score of 5.9 #2: Sports fan panel with average score of 8.1

Obviously brand affinity and connection make a huge difference in overall score HOWEVER...

Sports Fan panel has a sophisticated engagement strategy

PSAT - CONCLUSIONS

16

Delivering on 2 key promises are at the core of healthy and satisfied panels: 1. Thoughtful study design

Survey Quality Survey Topic

2. Creation of sharing feedback mechanism My input is valued

RESEARCH DESIGN

TOPIC VARIETY

18

When you are trying to nurture a ongoing research relationship with a person sometime you have to ask questions that interest them

even if the results are not directly useful to you

   

Recruit  Start   Product  Test   Concept  Test   CSAT  Dec  

Every  Two  Months  -­‐  Panel  Omnibus  

Offer  Test   Usability  Test  

September October November December January February

VISUAL QUESTIONS

19

Our research has consistently found that Visual Questions have a positive impact on respondent experience and willingness to participate again.

0% 50% 100%

Text

VQ

Text

VQ

Text

VQ

Text

VQ

Agree  Strongly Agree  Somewhat

Easy  to  complete  

Enjoyed  survey  

Will  complete  again  

Reasonable  time  

VISUAL QUESTIONS

20

Our research has consistently found that Visual Questions have a positive impact on respondent experience and willingness to participate again.

0% 50% 100%

Text

VQ

Text

VQ

Text

VQ

Text

VQ

Agree  Strongly Agree  Somewhat

Easy  to  complete  

Enjoyed  survey  

Will  complete  again  

Reasonable  time  

VISUAL QUESTIONS

21

VISUAL QUESTIONS

22

VISUAL QUESTIONS

23

FEEDBACK MECHANISM

SHARING FEEDBACK WITH MEMBERS

25

The company GETS insight

The company MAKES decisions

I GIVE a company insight and ideas

The company GIVES me feedback on how I compare

and how my insight is used

SHARING FEEDBACK WITH MEMBERS

26

We classify feedback into 5 types:

1. The results themselves 2. What you decided to do

Before you do it done

3. Recognition from the decision maker 4. Information about other members 5. Insights into your company / organisation

SHARING FEEDBACK WITH MEMBERS

27

We classify feedback into 5 types:

Newsletters Letters or emails At the beginning or end of surveys Community Portals Public Release

PUT IT ALL TOGETHER

29

A full plan that incorporates both research and feedback will make keeping the respondent in focus a lot easier

 

 Recruit  Start   Product  Test   Concept  Test   CSAT  Dec  

Every  Two  Months  -­‐  Panel  Omnibus  

Offer  Test   Usability  Test  

September October November December January February

Newsletter  #1  

Holiday    Letter    

from  President  Newsletter  

#2  

SHARING FEEDBACK WITH MEMBERS

30

If you manage your own panel:

Measure panelist satisfaction yearly Pay attention to the quality and mix of studies being fielded Implement some rules around DQs, questionnaire length Create a feedback plan and follow-through

If you are a researcher who uses panels Consider the people who will be taking your survey and treat them like adults Provide content for feedback

prepared to read the responses