creating a good survey

7/27/2019 Creating a Good Survey

1/23

Sampling of individuals from a population with a view

towards making inferences about the population Communication process between the surveyor and a

participant, where the surveyor tries to understand a

subject of his/her interest from the participantsperspective

Can be a questionnaire, 1-1 interview etc.


2/23

Customer retention and satisfaction measurements

Customer/employee profile census

Assess program/product response

Explore unknown interests of the consumers


3/23

Would you like Walmart with less clutter ?

In 2009, Walmart asked this question to its customers

YES was the overwhelming response

Spent hundreds of millions of dollars uncluttering their

stores, removing 15% of inventory, shortening shelves,

clearing aisles

Losses of $1.85 Billion

http://consumerist.com/2011/04/walmart-declutters-aisles-per-customer-request-then-loses-185-billion-in-sales.htmlhttp://dailyartifacts.com/walmarts-185-billon-dollar-mistake


4/23

Came up with the answer first, then asked customers to agree

Walmart paid attention to what people said and not what

people did, and they are more often than not, two differentthings

It was a leading question most people would say yes

(More is better, Less is worse)

Customers wanted less clutter; But they valued choice more

Surveys and focus groups bring out only what's easy to

verbalize and not what is important, Barry Schwartz

http://consumerist.com/2011/04/walmart-declutters-aisles-per-customer-request-then-loses-185-billion-in-sales.htmlhttp://dailyartifacts.com/walmarts-185-billon-dollar-mistake


5/23

Surveys results are supposed to be representative of the

population

Major decisions are made based on such results

Participants time is valuable

Surveys reveal latent / secret needs and demands

Few modes of direct communication Survey is one

among them Make best use


6/23


7/23


8/23

Establish your objectives

Information Requirements

Neither too broad nor too narrow

Clear objectives will shape your question(s)

Choosing your audience

Should be representative of the population

Minimum acceptable sample size

Irrelevant Audience Noisy data


9/23

Brief, Objective, Simple and Specific

Brevity Shortest question without losing intent

Objectivity Neutrality of the question

Leading, Loaded and Assumptive questions

Simplicity Dont show off; No jargon

Avoid personal Qs wherever possible Age, Income etc. Easy introductory questions; Avoid sensitive questions

Online/Paper Surveys - Use different pages for different

classes of questions Improves the flow of the survey

For more details on intelligent questions, contact the Placements Committee


10/23

Open Ended Free response

What do you think is your greatest strength ?

Closed Ended Multiple Choice Binary, Range, ExactHave you used our product before ?

How many times have you used our product before ?

Ranking / OrdinalRank the following features in order of importance to you

Rating / Matrix

Answer the following questions (1-Strongly Disagree, 2-Disagree,

3-Neutral, 4-Agree, 5- Strongly Agree)


11/23

Used mostly in recruitment process to determine

attitudes/values.

Account for Self-Good Time constraint is a good way to

do this

Prevailing moods/emotions may affect participant answers

For example, in a survey conducted in the US

88% believe they will go to heaven


12/23

Ask concrete questions

Did you enjoy the book ? vs. Have you recommended the book to

someone else ?

Avoid double-barreled questions One question, One focus

The government must take steps to curtail global warming and

terrorism. Do you agree ?

Ask specific questions for more accurate responses

Do you exercise regularly ?

How many times do you exercise in a week ?

How many times have you exercised in the last month ?

Use the right UI components (for online surveys)


13/23

Would you like daily updates to your application ?

Leading Question No one would really say no

Close to 70% of respondents said Yes, while more than 50% of thesame respondents said that internet consumption was top priority

Would you like the new streetlight placed on the left-side or

the right-side of the road ?

Hypothetical Question Too much visualization is needed to make a

proper decision. The user is not an expert on this matter

Have you stopped beating your wife ?

Loaded question

How does one say that he does not beat his wife !!!


14/23

What does the

survey mean by

Okay/Neutral ?Generally, in such

cases, we might see

participants going

for the least

controversial

response


15/23


16/23


17/23


18/23


19/23

Correlation vs. Causality

What the customer says vs. What he/she actually does

Sampling Bias / Size of sample

Selective Perception


20/23


21/23


22/23

http://www.socialresearchmethods.net/kb/survtype.php

http://www.questionpro.com/survey-design.html

SurveyMonkey Smart Survey design

HCI course material, prepared by Prof Scott Klemmer of Stanford

University

Organizational Behavior, 13th edition

http://w4.ub.uni-konstanz.de/srm/issue/archive
http://www.socialresearchmethods.net/kb/survtype.phphttp://www.questionpro.com/survey-design.htmlhttp://www.questionpro.com/survey-design.htmlhttp://www.questionpro.com/survey-design.htmlhttp://www.questionpro.com/survey-design.htmlhttp://www.questionpro.com/survey-design.htmlhttp://www.socialresearchmethods.net/kb/survtype.php


23/23

Questions ?

creating a good survey

Documents