tips to radically improve your surveys

Tips to radically improve your

Surveys @stephbeadell

Stephanie BeadellVP of Marketing

[email protected]@StephBeadell

? + =

@stephbeadell

Surveys are Powerful Tools

…unless the data is bad.

@stephbeadell

Radically Improve Your Data

3@stephbeadell

steps

Typical Survey Question

@stephbeadell

Do you feel like part of our community?

Do you feel like part of our community?

@stephbeadell

Yes

No

I Don't Know

1@stephbeadell

Use a 5 or 7-point

Scale@stephbeadell

A Better Question

@stephbeadell

I fit in with the community.

Strongly

AgreeStrongly

Disagree

Neutral

I Fit in with the Community

@stephbeadell

0

20

40

60

80

100

120

140

160

180

200

Strongly Disagree

Disagree Neutral Agree Strongly Agree

IFitinwiththeCommunity

Average: 3.67

“I Fit in with the Community” Score Over Time

@stephbeadell

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5


“Likert Scale”

@stephbeadell

Likert Scale Basics

@stephbeadell

Strongly

Agree

NeutralStrongly

Disagree

5 or 7 points

Likert Scale Basics

@stephbeadell

BraveShy

Unipolar

(Don’t Do This)

Analysis Opportunities

@stephbeadell

Likert Scale Question

@stephbeadell


Strongly

AgreeStrongly

Disagree

Neutral

Scoring the Answers

@stephbeadell


1 2 3 4 5

Correlation Analysis & More

@stephbeadell

“Heavy forum users are more likely to

be satisfied with their membership.”

Sat

isfa

ctio

nFrequency of Forum Visits

Forum Usage vs. Satisfaction

Use a 5 or 7-point

Scale

When in Doubt:

@stephbeadell

2@stephbeadell

@stephbeadell

into multiple questions

Break down

Big Concepts

Typical Survey Question

@stephbeadell

Are you happy with the community?

… it could mean

@stephbeadell

Do you like the people in the community?

… it could mean

@stephbeadell

Does the community meet your needs?

… it could mean

@stephbeadell

Does being a member make you feel good?

@stephbeadell

?

?? ?

Try this instead

@stephbeadell

1. Being a member is valuable to my career.

2. I enjoy being a member.

3. The company cares about the community.

Report it Like This

@stephbeadell

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Valuable to Career

Enjoy Being Part

Company Cares

Member Happiness Survey

Average Happiness

Score:

11 out of 15

Track it Over Time

@stephbeadell

“Our team increased member happiness by 3 points this quarter.”

0

2

4

6

8

10

12

14

16

Month 1 Month 2 Month 3

Happiness

@stephbeadell

Break down

Big Concepts

3@stephbeadell

Typical Survey Results

@stephbeadell

020406080100120140160180200

Strongly Disagree


IFitinwiththeCommunity2.

Freelancer

Small Business

Enterprise

BusinessSize1.

Compare and

Contrast@stephbeadell

Comparing Answers

@stephbeadell

EnterpriseSmall Business

01020304050607080


01020304050607080


Typical Survey Results

@stephbeadell

020406080100120140160180200

Strongly Disagree


IFitinwiththeCommunity2.

Freelancer

Small Business

Enterprise

BusinessSize1.

Comparing Answers

@stephbeadell

EnterpriseSmall Business

01020304050607080


01020304050607080


How to Do It in Excel: Pivot Tables

@stephbeadell

Export Raw Data

And in SurveyMonkey: Filter/Compare

@stephbeadell

Filter or Compare

@stephbeadell

Plan segments in advance

Comparing Answers

@stephbeadell

Ask identifying questions

3@stephbeadell

steps

Use a 5 or 7-point

Scale

When in Doubt:

@stephbeadell

@stephbeadell

Break down

Big Concepts

Compare and

Contrast@stephbeadell

How Not to Screw it Up

@stephbeadell

BiasIntroducing

@stephbeadell

Priming

Bargh, Chen, and Burrows (1996)

@stephbeadell

Sentence Scramble Task

Rudeaggressively, bold, rude, bother, disturb, intrude, annoyingly, interrupt,

audaciously, brazen, impolitely, infringe, obnoxious, aggravating, bluntly

Politehonor, considerate, appreciate, patiently, cordially, yield, polite, cautiously,

courteous, graciously, sensitively; discreetly, behaved, unobtrusively

Neutralexercising, flawlessly; occasionally rapidly, gleefully, practiced,

optimistically, successfully, normally, send, watches, encourages. gives,

clears, prepares

Bargh, Chen, and Burrows (1996)

@stephbeadell

People who Interrupted After the Task

Polite Neutral Rude

Introducing Bias: Priming

@stephbeadell

When are people prompted to take the survey?

@stephbeadell

Order Bias

Order Bias

@stephbeadell

How important is content marketing?

Order Bias

@stephbeadell

How important is content marketing?

How much content marketing do you plan to do this year?

Solving Order Bias with Randomization

@stephbeadell

Question Randomization

Solving Order Bias with Randomization

@stephbeadell

Shuffle Question Order

@stephbeadell

Stereotype Threat

Stone, Lynch, Sjomeling, and Darley (1999)

@stephbeadell

“natural athletic ability”

“sports intelligence”

Inzlicht (2000)

@stephbeadell

Female Math Test Performance When Others Present

2Women 2Men

Solving for Stereotype Threat

@stephbeadell

Demographic data always

goes at the end of a survey

Thanks!