Customer Opinion and Audience Response

Surveys and trivia games that are Easy to Make, Fun to Take

Bill Leath and Barry Fuchs

© 2012 Pinnion

www.pinnion.com

Customer Opinion and Audience Response

2

Table of Contents Introduction .................................................................................................................................................. 3

Why? ............................................................................................................................................................. 5

Survey justification statement .............................................................................................................. 7

Who? ............................................................................................................................................................. 9

Your Customers ......................................................................................................................................... 9

Your Colleagues ....................................................................................................................................... 11

What? .......................................................................................................................................................... 14

Multiple choice and structured communication .................................................................................... 14

Content ................................................................................................................................................... 16

Where? ........................................................................................................................................................ 18

When? ......................................................................................................................................................... 20

How? ........................................................................................................................................................... 22

Answer options ....................................................................................................................................... 23

Dichotomous ....................................................................................................................................... 23

Multiple choice ................................................................................................................................... 23

Rating scales – Likert and Semantic Differential ................................................................................. 24

Image select ........................................................................................................................................ 25

Demographic ....................................................................................................................................... 25

Skip logic and branching ..................................................................................................................... 26

Text piping ........................................................................................................................................... 27

Distribution options ................................................................................................................................ 28

Conclusion ................................................................................................................................................... 31

About Us ..................................................................................................................................................... 32

Bibliography ................................................................................................................................................ 33

www.pinnion.com

3

Introduction

“Let’s do a survey!” You’ve probably heard that phrase countless times around the office, at a meeting

of your favorite club or association, or while planning events at your child’s school. After all, what better

way is there to show that you really want to know what’s on the minds of your customers, members, co-

parishioners, fellow parents, etc? A survey says “I’m listening,” provides you with information, and can

take the pressure off of making big decisions. That’s the good side.

There is a bad side to surveys, though. Or, should we say, a dark side. After all, dark is the bottom of the

trash can where countless ignored surveys have ended up. And, dark are the desk drawers where many

survey results get stuffed away – quickly forgotten by managers who are too busy to remember why

they even created a survey in the first place. When survey requests appear on every sales receipt and

arrive in your inbox after each hotel stay or car repair, it’s easy to see why “survey fatigue” is causing

many of us to cringe each time we’re asked to give feedback (Peltz, 2012).

That’s why it’s time for a new type of survey guide. Sure, it helps know your dichotomous question from

your likert scale if you want to write a good survey, but we believe it’s more important than ever to

think of surveys as snapshots of customer opinion. Like a photo snapshot, they should be easy, quick,

and fun for everyone involved. Like a traditional survey, they should provide you, the creator, with

meaningful information that’s easy to process, understand, and share with others. Bottom line: surveys

should be easy to make and fun to take.

We’ve tried to organize this book so that it’s easy and fun for you, too. The easy part comes from our

“just the facts” approach, where you’ll learn about surveys through these familiar chapters:

Why

Who

What

Where

When

How

The fun should come naturally, because we actually want you to read this book and learn how to create

fun surveys. Just in case we’re not fun, or funny, enough on our own, we’re also going to explain how

you can include trivia, quizzes, and other fun stuff in your surveys so that your customers feel less like

they’re doing their chores and more like they’re actually have a good time interacting with your survey.

Customer Opinion and Audience Response

4

Note: We will use “customer” throughout this book. Please feel free to substitute “readers,”

“members,” “co-workers,” “classmates,” or whatever term best describes your target audience.

www.pinnion.com

5

Why?

One of our main goals is to help you create surveys that are

actionable – that allow decisions to be made based on the

responses that you receive. It’s for this reason that “Why?” should

be the first question that you ask, and answer, when you’re

considering whether to conduct a survey.

As you come up with answers to that question, you’ll probably be

surprised at how many different reasons there can be to create a

survey. Here are just a few examples:

Keep in touch with your customers between purchases

Remind customers of your brand at times when you would

like for them to make a purchase

Solicit opinions on a new product line before making a

major investment

Generate fresh content for your Facebook page or website

Get feedback on the quality of your company’s customer

service

We’ll cover more examples throughout this book, but the

important thing to notice is how the strategy behind surveys has

changed. Three of the five examples listed above are designed to

do something more than just collect customer opinions. They are

maintaining relationships, drawing customers in to your website or

brand, and perhaps even driving additional purchases. The nature

of online surveys allows you to do this in a way that doesn’t detract

from the primary reason that most surveys are created: to collect

customer opinions.

Until now, surveys have often felt like transactions, because that’s

how they’ve been designed. And, frankly, how many of us want to

go through another transaction – in the form of the survey – to ask

about an earlier transaction – such as going through the grocery

checkout – that we already completed? To add insult to injury,

If we were going to write

a news article about your

survey, then “Why” might

be the last thing that we

mention. For example, we

could write “Our most

awesome reader surveyed

his audience of 6,000

people in the Chicagoland

area through his blog and

a mobile survey app

during the second week of

March because he needed

their input to create the

biggest impact with his

upcoming project.” In this

case, you learned about

the who, what, when,

how, and where before

we got to the why:

“because he needed their

input to create the

biggest impact with his

upcoming project.”

Surveys are different,

though. It’s important to

always ask Why as the

first step of your research.

Customer Opinion and Audience Response

6

these transactional surveys often arrived at inconvenient times, which is exactly why so many of us got

in the habit of simply deleting the emails and hanging up the phone whenever we were asked to

participate.

Wegman’s, a popular East Coast grocery chain based in Rochester, NY, has moved away from this

transactional model through the creation of the Wegman’s Opinion Panel. Customers on the panel

“have agreed to participate in ongoing Wegmans surveys as part of their membership” and “directly

influence the direction of future products and services that benefit (the members of the panel).” Wow!

The word “panel” alone makes this online group of customers sound prestigious and Wegmans talks a

good game about how much influence the panel can have over product decisions. But, is it real or just

marketing-speak? Allow this Wegman’s customer to answer that question with her enthusiastic Tumblr

post:

Double wow! Imagine having customers who not only read your emails, but actually look forward to

having them arrive. The right mix of feedback opportunity and valuable content can make this possible.

This is not to say that you’ll only hear from happy customers who want to shower you with praise. That

shouldn’t be the goal, either, because hearing about the negative situations that some of your

customers have experienced provides important opportunities to improve your products and services.

Remember, this is the age of oversharing and constant status updates. If your unhappy customer can

quickly and easily provide direct feedback, knowing that someone on the other end is listening, you may

avoid an off-the-cuff, upset post to Facebook and get the chance to turn his experience around. This can

Figure 1 – Source: http://www.tumblr.com/tagged/wegmans?before=1314840187

Key Point – The Why of Surveys: Surveys are no longer a one-way proposition, e.g. “give us information that’s valuable to us.” Now, surveys and survey results can be offered as content that provides value back to your customers.

www.pinnion.com

7

be especially valuable if you or your staff are able to correct a simple misperception that the customer

had about your products or services.

Keeping all of this in mind, let’s get down to the reason(s) why you want to create a survey. You may

want to start by filling in the blanks of this statement:

Survey justification statement

This survey will provide (name(s) of decision makers or committee) with the necessary data to (decision that

will be made) by asking (customers, members, etc.) to provide opinions regarding (a past experience or upcoming

situation).

We recommend a survey for this project instead of other feedback mechanisms because (list of

reasons that other options were rejected). We have already checked for this data at (potential existing sources of

information that have already been explored) and have not found the information that we need.

That last sentence is very important, because surveys are just one way of getting the information that

you need. And, for all of the advantages of modern surveys, it is still possible to annoy your customers

by over-surveying them. This is especially true if you ask them a question that has already been asked

before, which gives the impression that the information they provided to you earlier was simply ignored.

So, think of all of the places where the information that you need could already be lurking: sales reports,

surveys conducted by other departments, trade publications, Census data, etc. You might find enough

existing data to realize that you don’t need to create a survey at all. That’s great! Seriously. You’ve saved

yourself, your customers, and your organization valuable time and money. You’ll find another reason to

do a survey soon enough, so don’t worry if you end up not needing to do one right at this moment. The

rest of this book will still be here when you need it.

Even if you’ve determined that the necessary data isn’t available, there’s still the question of whether a

survey is the most appropriate method of getting the information that you want or need. Let’s say that

one of your product lines has experienced a substantial drop in sales during the past year. You could

survey your customers to find out what they like and don’t like about that product, but the first step

may be to do some industry research to determine if your competitors are seeing a similar drop in sales.

The clues provided by other types of research may persuade you that a survey is not necessary. Or, you

may still want to do a survey, but with a different set of questions than you had originally planned to

use.

Customer Opinion and Audience Response

8

Other types of research include:

Primary sources

Focus groups

Secret shoppers

Committees and advisory boards

Individual interviews (aka depth interviews)

Mall Intercepts – quick, face-to-face feedback solicited from consumers at shopping malls

Secondary sources

Internet searches

Industry reports, such as Hoover’s and IBISWorld

Books, magazines, and trade publications

Census data

Survey data from other organizations

If you’ve made it this far, then you must have decided that a survey is right tool for getting the

information that you need. And, by taking a bit of time to get to the Why of your survey, you also figured

out a lot about the Who, What, and How. Let’s take a look at the first of these in the next section.

www.pinnion.com

9

Figure 2: The Two Whos - Your Audience and Your Colleagues

Your Customers

• Needs clear, concise survey questions

• To provide honest, thoughtful answers

Your Colleagues

• Need a clear, concise summary of answers

• To make good decisions based on audience opinion

Who? When you start to answer this question, it probably won’t take you long to figure out that there are

really two sets of Whos that you need to think about. In fact, it definitely won’t take you very long

because we are about to tell you:

One set of Whos are the

people who will be

responding to your survey,

your customers. The other

set of Whos are the people

who will make decisions

based on the survey results,

your colleagues. Your

ultimate goal is to provide

your colleagues with the

data that they need in the

clearest, most concise

format possible. This means

that you need to create a survey that’s equally clear and concise, so that your customers will provide

honest, thoughtful answers to each of the questions that you present.

Your Customers

We’ve written this book as a resource for people who want to gather customer opinions for use at a

business or non-profit organization. We are not scientists and assume that you are not, either, so you

are likely interested in gathering a “sufficiently reliable” (Shapiro, 2008) estimate of customer opinion

through your surveys. In other words, you’re interested in creating a simple tool that will give you a

Key Point – The Who of Surveys: Your colleagues and your customers both want to contribute and provide good information. Both groups are also made up of very busy individuals, so it’s critical to respect everyone’s time and good nature by ensuring that each survey question:

• Is presented only to the proper individuals • Will provide valuable data for your colleagues or yourself

Customer Opinion and Audience Response

10

good idea of customer opinions, not in developing a complex study that strives for a perfect set of

results.

One way to do get good results is to ensure that your sample, or the number of people who respond to

your survey, is large enough to provide you with good “ballpark” data that can be assumed to represent

the opinions of your customer base as a whole. One number that’s batted around quite frequently is 30.

As explained wonderfully in these charts

from Jed Campbell, 30 is the magic number

where things become a lot more stable,

even if you would need a much larger

sample in order to conduct a formal study.

If you’d like a better idea of the number of

responses you’ll need for a meaningful

sample, check out the Sample Size

Calculator available online from Creative

Research Systems.

These estimates assume that you will be

using a random sample of respondents

who, in turn, will be representative of a

larger group of random potential

customers. Of course, if you are asking

current customers to participate in your

survey, then your sample will not be

random because your respondents will all

have at least one trait in common: they are

all customers of your business. The sample

will be even less random if you undertake some of the relationship-building strategies that we

mentioned in the Why section, because now your respondents all have two things in common: they are

customers and they have agreed to participate in a series of surveys that you send to them over time.

This is not necessarily a problem in itself and does not suggest that the information that you receive isn’t

valuable. It just may not be scientifically-valid. You’ll need to assess whether the results you receive are

Figure 3 – Source: http://www.jedcampbell.com/?p=262

www.pinnion.com

11

indicative of your base of customers or potential customers as a

whole. Perhaps you’ll learn that your respondents are pickier or

more demanding, but also more loyal, and keep that in mind as you

review and evaluate the feedback that they provide.

You also want to be sure that you’re approaching a relevant group,

or segment, of customers with the invitation to participate in your

survey. By determining who you wish to invite, and why, you’ll help

to ensure the validity of the opinions you receive. You’ll also avoid

bothering a group of individuals who likely will not have any

interest in a particular survey, thereby increasing the chance that

they’ll read the next communication that you send their way.

Determining the right people doesn’t stop at the survey level. It

goes right down to the individual questions. You probably

remember old paper surveys that asked you to, for example, Skip

to Question 9 depending on your answer to Question 6. With

online and mobile surveys, there’s no need to ask individuals to

skip ahead, because you can automate your survey to do the

skipping for them. This is called Question Branching, or Skip Logic,

and we’ll talk about it more in the How section. For now, just

remember that each survey and each question should be

presented only to those who can help you to get the information

that you need.

Your Colleagues

There are a few phrases that you don’t want to hear from your

colleagues as you walk into the conference room, giddy with

excitement from all of the great data you were able to collect from

your survey respondents:

“So what?”

“I’ll take a look at this next week.”

Imagine that you run a

fashion brand for men

and women. If you’re

planning to ask for

opinions about ladies

handbags, then you’ll

probably want to invite

only women to answer

your survey.

Probably, but not

definitely.

If you’re nearing the

holiday season and

believe that men may be

interested in purchasing

handbags as gifts, then it

may be worth sending the

survey to them as well.

And, if you’re in touch

with a group of men and

women who blog about

fashion, then you may

want to include all of

them, or none of them, in

your invitation.

Customer Opinion and Audience Response

12

“Wait, we did a survey?”

“Just give a copy to my assistant.”

It’s not hard to imagine how deflating these comments would be. After all, you’ve worked to collect this

data and now it’s clear that the results won’t see the light of day. What’s more, your customers took the

time to share their opinions – how would they feel to know that the information was just filed away,

never to be seen again?

The key to avoiding this ugliness and despair is to remember the ultimate goal that we mentioned

earlier in this chapter: to provide clear, concise data. This starts by thinking about your colleagues as you

create your survey questions. When you come up with a question that you would like to ask, consider

how the data that comes

from the responses to that

question will be used.

Questions without a clear

purpose should be

eliminated to shorten the

survey and save time for

everyone involved

(Walonick, 2004). This way,

you’ll end up with survey

results full of must-have

information that your

colleagues will be

clamoring for, instead of a

bland report with nice-to-

know facts that are easily ignored.

Your colleagues also have a role to play during survey preparation: helping to test it out before

distribution to your customers. If your survey is longer than two or three questions, then there’s a good

chance it has some complexity to it. Testing will not only find any mistakes, it will also alert you to any

questions or multiple-choice answers that are confusing, ambiguous, or out of order. Think about all of

the ways that you plan to distribute your survey – online, mobile app, website embed, mobile website,

etc. – and make sure that at least one of your testers attempts to take the survey via each of these

Create

•Each question should provide data that your colleagues need

•Eliminate any question that only gives "nice to know" information

Test

•Ask colleagues to look for confusing or out-of-order questions

•Try taking the survey in multiple ways, including on a smartphone

Share •Explain the Why of the survey and give an overview of results

•Different audiences will need the results in different formats

Figure 4 - Basic steps for a clear, concise survey

www.pinnion.com

13

platforms (Flagg, 2011). Remember to also test the various types of online browsers that are available,

including Google Chrome, Mozilla Firefox, Microsoft Internet Explorer, and Apple Safari.

When it comes time to present the results, remember the importance of selling the recipients on the

importance of the data that you have collected. Give them a brief overview of the Why for your survey,

so that they can see why the survey was the right application for this research challenge, along with a

summary of the key learnings that you have identified from the results. Consider the ways in which your

results should be presented and keep in mind that you may need to make multiple versions available: a

PDF of simple charts and graphs may be appropriate for senior managers and committee members who

need a quick view of the key information, while department managers may appreciate having access to

the complete results set in spreadsheet format so that they can sort and calculate the data however

they wish.

Customer Opinion and Audience Response

14

What?

We mentioned in the last section that there are at least four different ways to distribute an online

survey via the Internet and mobile devices, plus at least four different web browsers. (We won’t even

get into all of the different versions available for each browser.) This is remarkable considering that

household telephone ownership did not become prevalent enough to make telephone surveys possible

until the 1960s and 70s (Kohut, 2009). In other words, it only took about three decades to go from an

age when most polling was done through personal interviews to today’s world of instantaneous

response by individuals who can share their opinions from nearly any location in the world.

The fact that it’s so easy to create surveys today means that there has been an immeasurable increase in

the number of surveys, not to mention in the number of people who create surveys. Many survey

creators today do not work with surveys on a full-time basis and have no formal training in survey

methodology or market research. That’s why we wrote this book and it’s probably why you’re reading it.

So, this section is devoted to exploring a topic that may seem obvious: What is a survey? In short, it’s a

questionnaire or poll that is distributed to a group of constituents – random or not – for the purpose of

collecting information that can aid the decision-making process. Now, let’s break it down and find out

about the elements that can ensure your online survey gets a lot of love, not hate (Or, as they say on

Twitter, an #FTW, not a #Fail.)

Multiple choice and structured communication

The fact that you have chosen to use a survey, instead of another type of primary or secondary research,

suggests that many of your questions will be structured with multiple choice options for the answer.

After all, if you were interested in detailed or open-ended responses, then focus groups or individual

interviews would be a better option for collecting information. These qualitative forms of research have

their advantages:

Complete, unfiltered opinions directly from customers

Ability to ask detailed follow-up questions

Face-to-face interaction can develop brand loyalty that is not possible with other research

methods.

www.pinnion.com

15

On the other hand, focus groups and individual interviews are extremely high-cost in both time and

money. Most organizations simply cannot afford to do them very often, if at all. The information

gathered through these methods is also unstructured, meaning that no two answers will be exactly the

same. It takes a lot of time and effort to attempt to summarize open-ended responses so that they can

be compared to one another. And, the original words spoken by respondents are often lost once the

summary is created anyway.

We believe that the quantitative nature of structured communication provides a framework to simplify

the research process in many ways:

Respondents can move quickly through each question by choosing from a list of pre-determined

answers

Creators get a data that’s easy to analyze, so that trends and preferences can be easily spotted

The data can be processed and calculated in a variety of ways.

Answer choices can be re-used at the same time or in future surveys to allow for comparisons

between different groups of respondents.

Sheetz, a regional convenience store chain, uses structured communication to take customer orders via

in-store kiosks. This speeds the ordering process by showing customers exactly what is available for each

sandwich. As the company states,

the method is “quick, easy, helps to

ensure the accuracy of (customer)

orders and prevents others from

knowing” about whatever unusual

food craving you may be having at

that moment (Sheetz, n.d.).

The speed and ease of structured

communication applies whether

you’re a sandwich maker juggling

several orders at once or a survey

maker who needs to collect and

Figure 5 - Your lunch order can be a form of structured communication

Key Point – The What of Surveys: Structured communication creates usable data and simplifies the research process by providing a list of potential responses for each question.

Customer Opinion and Audience Response

16

process responses from many individuals. Therefore, most of your survey questions will probably be

created in a multiple-choice format. You may need to include some options for short, open responses,

but these should be used sparingly. We’ll cover question types in detail in the How section.

Content

In the 1930s, IBM hired a teacher named R.B. Johnson to create a mass-produced version of his

invention: a test-scoring machine that could sense pencil markings on a sheet of paper (IBM, n.d.).

Sound familiar? Nearly 100 years later, we’re all too familiar with the ubiquitous “#2 pencil” form that

persists to this day in classrooms and polling stations. These forms may fit the bill for the quickness and

accuracy that we look for in a good survey, but they’re pretty lousy when it comes to customer

engagement. They may have been unusual eight decades ago, but any novelty value that these forms

may have had when they were invented has long since worn away.

With online surveys, we’re no longer bound by the size of paper, the cost of postage, or the complexities

of printing. Some may argue that this allowed survey makers to swing too far in the other direction –

using the free, unlimited nature of the web to create overly-complex surveys that seemed to go on

forever. Luckily, the advent of surveys that can be distributed through mobile devices has brought us an

ideal compromise. Mobile surveys still allow for the type of multimedia content that was never possible

with paper surveys. Yet, the small screens of mobile devices – and hurried nature of mobile users –

require survey designers to bring some discipline and restraint back into the design process.

Which sounds wonderful except for one problem: it’s still a survey. And, consumers are tired of filling

out surveys. Those decades of filling in little circles with #2 pencils, followed by another decade of

constant requests for interminable online surveys, have taken their toll. Think about your own

experience – how many times have you come up with an excuse to get off the phone with a pollster?

How quickly do you delete the survey request emails that arrive in your inbox? But, even if you can’t

stand the thought of another survey, does that mean that you’ve stopped sharing your opinions

altogether? Of course not!

People still love to tell you what’s on their mind. So, why not ask them that? Instead of “Please fill out

this customer survey,” what about “Hi Jodi, we’d like to know what’s on your mind today.” That is the

actual reason you are contacting Jodi, after all, and it avoids the negative connotations that come with

“survey,” “poll,” or “questionnaire.” We recommend banishing those words from your customer

communication because they don’t refer at all to the relationship that you are trying to build. There’s a

www.pinnion.com

17

reason that the audience at a movie theater is invited to “enjoy the show” and not “view this

projection,” after all. Talk to your audience about the experience, not the tool.

This means you’ll have to live up to your promise of meaningful interaction with a survey that’s

interesting and engaging to your respondents. Consider the options that your survey provider offers in

this area. Will you have the ability to:

Include pictures in various locations

o The start or end of the survey

o Each question

o Each answer option

Display results for each question

o Online

o On iPhones or Android devices

o In the same screen as the survey (instead of a link

to another page)

Use different types of input options for each question

o Touchscreen buttons

o Checkboxes

o Picker wheels

o Sliders

o Open response/free text

Include trivia questions or other diversions

Offer points or rewards for responses

Create surveys with unlimited questions and then collect

unlimited responses

You can check the About Us page if you’d like to learn about a survey provider that offers each of these

options, but it really is important to consider these types of features no matter which provider that you

use. The more that you can do to create surveys that feel new and different from what’s been offered

before, the more innovative your organization looks and the better responses you’ll see. What you offer

is only part of the calculation, though. You’ll also need to consider Where your surveys can be

completed. Let’s take a look at that on the next page.

Figure 6 - Touchscreens allow for new types of input options, such as

sliders

Customer Opinion and Audience Response

18

Where?

In the world of surveys, this used to be a fairly straightforward question. Your respondents might have

taken your survey while they were at your business or event location. Otherwise, consumers probably

completed your survey at home and business customers provided feedback when they had a few spare

moments at the office. How quickly things have changed.

Mobile devices, aka smartphones and tablets, are the most quickly-adopted consumer technology in

history (Kang, 2011), with nearly 100 million people owning one in the US as of December 2011

(comScore, Inc., 2011). About half of these smartphone owners are checking email on almost a daily

basis and this figure rose 33 percent during 2011 alone (comScore, Inc., 2012).

The number of locations where customers could be responding to your survey has increased as a result

of the explosive increase in iPhones and Android phones. No longer confined to home or the office, your

customers are very likely to be responding wherever they may be, simply because they’re bored. In

waiting rooms, on buses and trains, and at the cafeteria, mobile devices are the new way to connect

with friends, families, and brands whenever there’s a dull moment…no matter how short that moment

may be.

If you plan events, such as conferences, trade shows, festivals, or even a company picnic, then the

proliferation of mobile devices gives you a great opportunity to request feedback from your participants

while they are on-site and ready to provide their opinions about the event. You probably know how hard

it is to get people to provide feedback after they’ve left the event, and paper forms handed out at the

event are equally problematic, so mobile really has the potential to change the dynamic in this industry.

The only catch is that you’ll still need to capture feedback from your participants at other times of the

year, as we’ll explain in the When section.

All of this talk about mobile devices raises another point – Where is no longer just a physical or

geographic concept, but also digital and virtual one. When customers take your survey on a mobile

device, is it because they received the invitation:

In an email?

Via their Facebook or Twitter app?

From a survey app notification?

For those who respond via their computers, the ways in which they could have discovered your survey

are almost too numerous to mention:

www.pinnion.com

19

Company website

o Embedded survey

o Link to survey

o Pop-up survey

e-Commerce or sales site

o Embedded survey on confirmation page

o Link to survey on confirmation page

o Pop-up survey after sale is complete

Blog

Twitter link

Facebook

o Link to survey

o Embedded into a Facebook app

Email link

Just as we encouraged you to consider the abilities of your survey provider to offer customer

engagement capabilities, we suggest you take a close look at the ways in which surveys can be

distributed through your provider or platform. If you are using a web link to distribute your surveys via

email, Twitter, Facebook, or anywhere else, be sure to test the link on your own smartphone to ensure

that it’s easy to view and complete even on a small screen. After all, if your customer attempts to take

the survey on their smartphone and is frustrated by the experience, what is the chance that he or she

will remember to come back and try again from the home computer? Chances are, the opinions from

that customer are gone for good.

Figure 7 - Pinnion has been designed to help you reach your customers wherever they are

Key Point – The Where of Surveys: Even if you are communicating with customers via email, those emails are being read more-and-more on mobile devices. iPhone surveys and Android surveys will become commonplace as smartphones and tablets begin to replace the PC in many households.

Customer Opinion and Audience Response

20

When?

It probably won’t surprise you to find out that two guys who work at

a survey company think that you should do surveys all the time. Not

literally, of course, as no customer wants to provide you with

feedback every day or even every week. The trick is to let them

know that you are always willing to listen, then give them the

opportunity to share their opinions at a place and time that is

convenient for them. Mobile devices and website embeds make it

possible to offer your customers a feedback mechanism that is quite

visible when it’s needed, yet subtle enough to avoid the annoyance

and spam concerns that come with email surveys.

If you were hoping for a more specific answer than “always,” don’t

worry. A 2004 study actually found that the best time to send an

online survey is Wednesday morning, when it was observed that

half again as many individuals responded than at any other time of

the week (Faught, Green, & Whitten, 2004). On the other hand, a

2001 study found that the specific time of the week may not matter

as much as the amount of time that has passed since the customer’s

last transaction. That’s because customer perceptions vary

throughout the year and are generally higher right after a service

has been used than they are a month later (O'Neill & Palmer, 2001).

Here are some ideas for

scheduling your surveys

over a period of time:

Could you do a monthly

survey series, in which

each survey is tied to an

upcoming holiday or sales

event?

If you are able to

incorporate trivia or other

games into your survey,

then how about a daily

survey series that your

customers can play and

respond to during their

commute?

Or, you might decide to

run one survey for mobile

customers in long

months, another for

website visitors during

short months, and a

Facebook survey that is

segmented by age group

in advance of a major

product launch.

Key Point – The When of Surveys: O’Neill & Palmer found that year-around surveying increases reliability when measuring customer opinion, because doing so will smooth out variable perceptions over time.

www.pinnion.com

21

You have almost limitless opportunities to make your surveys engaging and interactive throughout the

year. Keep that in mind, review your marketing and customer engagement goals, and then consider how

surveys, polls, quizzes, and games can be used on a constant basis to stay connected with your

customers.

Customer Opinion and Audience Response

22

How?

Can you believe it? You’re almost there! Now that we’ve gone over all of the “Ws” for your survey, we

just have the big H left: How? This is where we tie everything together and get your first survey

prepared. Don’t worry: it won’t be your last!

Way back in the Why section, when we considered the reasons for doing a survey in the first place, you

identified a “past experience or upcoming situation” that you wanted to ask your customers about. You

probably have a hunch about the type of feedback that you will receive, so you’re doing the survey in

order to investigate whether your hunch is accurate. This is called “hunch verification,” which

occasionally gets a bad rap. But, when you think about it, this is a pretty close parallel to the scientific

process. Replace “hunch” with “hypothesis” and there you go.

You’ll need specific questions in order to test the hypothesis. How many questions? There is no hard-

and-fast rule, so the best advice is simply to make sure that every question is serving a purpose by

providing actionable data or by increasing engagement. Come up with as many questions as you can

think of or as many as you think you will need, then begin editing them down until you’ve created a

survey full of questions that you know you will need. For example, you may think it would be interesting

to ask each respondent for his or her age range. But, if the decision makers aren’t going to segment the

results by age when they consider the data, then there’s no need to collect that information this time

around.

That said, if you’re having trouble coming up with a list of potential questions, here are some ideas that

may help with your brainstorming:

Ask some questions of your employees, such as:

o “What have you been hearing from our customers lately?”

o “Have you heard any comments from the new customers?”

Find a situation where you can physically observe your customers

Review the company mission statement

o Or, for your own company, write down the biggest, bedrock assumptions on which you

built the business

Are there any financial trends you’d like to understand better?

In a service business, how has the tip money been? Are tips up or down?

If all else fails, start with broad questions

o The question “would you recommend us to your friends?” by itself is extremely

important.

www.pinnion.com

23

You’ll probably come up with plenty of questions once you’ve taken a few of the steps listed above.

Answer options Once you’ve got your questions narrowed down, you’ll want to create answer options for the multiple

choice questions and determine the order in which the questions will be asked. Let’s review some

examples of the various structures that are available for survey questions:

Dichotomous

If this were a quiz, then the answer would be Yes.

“Dichotomous” simply refers to a choice between two

options, such as Yes/No or True/False. These questions are

quick and easy to answer, but their black-or-white nature

doesn’t always provide the level of detail that you may

want or need.

Multiple choice

A multiple choice question presents the respondent with

several answer options. There will be some times when you

want the respondent to choose only one answer option and

others when you will want to allow them to choose multiple

answers together. Your survey software should allow this,

but you may need to include a note such as “Please select

all that apply” or “Please select two options.”

Customer Opinion and Audience Response

24

Rating scales – Likert and Semantic Differential

The Likert scale asks the respondent to indicate the degree

to which they agree or disagree with a statement. You can

have five or seven options and the scale can be “fully

anchored” or “end anchored.” The example here is fully

anchored, in that each number on the scale has a text

description to go along with it (Options 2 and 4 are hidden).

An end anchored scale would only have text for the first

and last options, leaving the weight of the middle numbers

a little more open to interpretation by the respondent.

The same rules for Likert scales apply to Semantic

Differential scales. The difference is that semantic

differential scales don’t ask the respondent to agree or

disagree, they ask for a choice to be made amongst

contrasting options (Options 2 and 3 are hidden).

www.pinnion.com

25

Image select

A multiple choice or semantic differential question can be

set-up as an image select question, where the respondent is

asked to choose from the pictures presented. This can be

more fun than reading and choosing words, while also

conveying information more clearly in some situations.

Keep in mind that images used in a semantic differential

question need to show a clear difference between the

positive and negative options. Be sure to pre-test your

survey so that others can tell you if the pictures are as clear

as you think they are.

Demographic

A demographic question is any question in which you

request personal information about the respondent, such

as age, gender, home address, zip code. Demographic

questions should normally be asked towards the end of

your survey so that your respondent has had time to get

comfortable with the questions that you are asking before

having to answer anything of a personal nature (McDaniel

& Gates, 2008, p. 307). An exception is when demographics

are used to eliminate ineligible respondents. For instance,

if the survey is intended only for adults 25 or older, an age

range question may need to be presented at the very

beginning of the survey.

Customer Opinion and Audience Response

26

Skip logic and branching

As we mentioned in the Who

section, it’s important to

automate your survey so that

each individual only sees

questions that are relevant to

him or her. This can be achieved

by using skip logic to determine

which questions should be shown

to which respondents.

In this example, which is based

on the Multiple Choice question

we showed you on Page 23, it only makes sense to ask

about specific communities on the island of Maui if the

respondent has indicated a desire to travel to that location.

You can see the logic in the screen shot above. Those

individuals whose answers match that logic will see the

question shown to the left.

www.pinnion.com

27

Text piping

The last question that we looked

at included four possible

locations on the island of Maui,

plus one option for those who

aren’t sure where they would

want to stay and one option for

those who plan to stay in another

town. For the next question, we

are going to use skip logic again

and introduce a new concept:

text piping.

Text piping takes an earlier answer and makes it part of the

question. In this screenshot, we use the placeholder

[PIPETEXT] to hold a spot for the answer text. You can see

the result below: the words “I don’t know” have been

carried over from the previous question. “Other” would

appear if that answer had been selected instead. And, the

question would be skipped entirely if neither of these

options were chosen.

You can combine Skip Logic and Text Piping to make surveys

that are very efficient and highly personalized. This helps to

ensure that each question is relevant to each respondent. It

also makes it easier for your customers to provide accurate

data, since there is less room for errors and

misunderstandings.

Note: Please see the Bibliography entries for Flagg, Shapiro, and Walonick to learn more about survey

question formats. Their information was helpful as we developed this section of the book.

Customer Opinion and Audience Response

28

Distribution options Your survey has been written, tested, and refined. You’re ready to present it to your customers and start

collecting their opinions. Uh-oh. How are you going to reach your customers so that they know you are

seeking their input?

If you already communicate with your members or customers online, such as via email, then that same

method may be a good option for sending your survey. However, many brick-and-mortar businesses

may not have any contact information for their customers. Even if you have some customer data, such

as addresses and phone numbers, it’s possible that you won’t have the information you need to

distribute an online survey as quickly as you had hoped. When Barry worked in the electric utility

industry, he heard from utilities who struggled to collect customer opinions online because they had no

way to reach their customers aside from snail mail or the phone.

The good news is that social media is helping to overcome this issue. Even those customers who refused

to share their email address may be following your organization on Facebook or Twitter. And,

announcing your survey through these channels may help you to find new customers or to gain insights

from customers who have not previously shared their opinions with you. Your website can be another

source of opinions, so your survey should be prominently embedded on or linked from your web site.

What’s that? Your organization doesn’t have a website? Or, it’s not on Facebook or Twitter? Then put

this book down and go fix those problems immediately. Customers today expect to find you in online

and in social media as much as a customer two decades ago would have expected to find you in the

Yellow Pages. These outlets are not optional anymore.

You’ll want to consider how your survey provider works on mobile devices

and whether it’s possible to take advantage of the notification system that is

built into many types of smartphones. These notifications can alert the

owner of the phone to all sorts of new items, such as text messages, emails,

sports scores, and weather forecasts. Why not alert them to the fact that

you need their opinion, too? Alerts come in many forms, from silent to loud

and from subtle to intrusive. Avoid annoying your customers by ensuring

that your survey app allows for each individual to adjust notifications to her

or his liking.

Figure 8 - This iPhone notification badge shows that two new Pinnion surveys are available

www.pinnion.com

29

The most common form of distributing an online survey is by providing a simple Web Link. This is an

easy option because the link can easily be pasted into all sorts of places: email, Facebook, Twitter,

newsletters, your web site, blog, press releases, etc. The list goes on and on. The good news is that a

Web Link is a simple way to quickly distribute your survey in as many ways as possible for maximum

distribution. On the downside, it may be difficult or impossible to know much about the individuals who

responded to your survey. And, a web link does very little to build a relationship with respondents –

click, answer, done, forgotten.

That’s why most survey providers offer you at least one other way of getting your survey out into the

world. SurveyMonkey, for example, provides an internal email service. You upload your contact list into

their system and it automatically sends a personalized email to each individual on the list. Instead of a

generic web link, each person’s email contains a unique, trackable link so that you will know who

responded, who didn’t, and when each response arrived.

At Pinnion, we have a concept called Channels for distributing surveys. Your customers can subscribe to

your channel(s) – you might have a generic channel for your organization or several channels designed

to reach different segments – and then be notified whenever a new survey is available in that channel. A

channel can also be

discovered by new individuals

with an interest in the

product or services offered

by your business.

Another downside of web

links is that they take your

respondent away from

whatever it was she was

doing before she decided to

take your survey. If Christine

the consumer is working her

way through a purchase on

your site, the last thing that

you want to do is have her

Figure 9 – Web page embeds are eye-catching and allow the survey to be answered directly, without having to navigate to another website

Customer Opinion and Audience Response

30

click out of that process for any reason. Research has shown that an individual who is interrupted while

proceeding through the flow of a webpage is less likely to complete the purchase or even to return to

the site (Hausman & Siekpe, 2009). You’ve probably experienced this firsthand while surfing the web.

Therefore, try to display your survey in an embed, widget, popup (the friendly kind, not the annoying

kind), or app that will keep your respondents on the same page while they provide their opinions.

Christine’s focus may shift to the survey while she completes it – in fact, we hope that she gives the

survey her complete attention – but she’ll be right back where she started as soon as the process is

complete. What’s more, the user can actually see the introduction to the survey and/or the first

question, which is more likely than a link to draw the user’s eye to your content.

Here are some distribution options to consider when researching survey providers:

Generic web link

o Non-personalized

o Can be pasted into any email or program

o Can be forwarded to other individuals.

Personalized web link

o Includes a tracking code that links the survey response back to the recipient.

o Cannot be forwarded to other individuals.

Mobile web link

o The link itself may not be any different from the ones above, but it should lead to a

“mobile-friendly” interface on small screens.

o Make sure that your survey provider automatically adjusts for proper screen size.

Embed/popup code or widget

o Uses iFrames or Javascript to actually place your survey content into a webpage

Facebook app

o Allows your survey content to appear on your organization’s Facebook page and in your

followers’ News Feeds.

Mobile app

o Displays your surveys within an iPhone or Android app (you may also consider versions

for Windows Phone and/or BlackBerry devices).

o Ideal choice when your organization has multiple surveys open at the same time.

o Apps can notify the individual whenever new surveys are available (although the

individual may choose to turn the notifications off).

Survey site

o Pinnion Channel Pages are dedicated to gathering user opinions.

o These sites pull new individuals to your survey (and your organization).

Reverses the traditional model of pushing surveys only to those individuals who

are already familiar with your organization

www.pinnion.com

31

Conclusion

Let’s take another look at the key points we tried to convey in each section of this book:

Why: Surveys are now a form of content that can provide value back to your customers

Who: Customers and colleagues need well-written surveys for quality feedback and decisions

What: Structured communication creates usable data

Where: All online surveys are now mobile surveys, too.

When: Collecting feedback throughout the year will increase the reliability of your data

How: Combine the right answer options with proper distribution to make a great survey

Most of all, keep it simple. We said at the beginning of the book that surveys should be a snapshot –

they don’t need to cost thousands of dollars and they don’t need to take hours to create or complete.

Keep your goals in mind, remember to always ask “Why” first, and then have fun creating surveys that

inform you while entertaining your customers.

We look forward to hearing about your experiences. Please share your questions and feedback with us:

Pinnion page at Facebook

Bill’s Twitter account

Contact Us page at pinnion.com

Customer Opinion and Audience Response

32

About Us

Bill Leath is the president of Pinnion, a Seattle-based startup with a platform that anyone can use in

order to create multiple-choice questionnaires such as surveys, trivia games, polls, and quizzes. A

lifelong entrepreneur, Bill holds a M.S. in Mechanical Engineering from the University of Washington

and currently resides in Seattle.

Learn more about Bill

Barry Fuchs is a marketing contractor with Pinnion and has authored countless membership surveys

during his career in the association industry. Barry holds a M.S. in Integrated Marketing Communications

from West Virginia University and currently resides in Seattle.

Learn more about Barry

www.pinnion.com

33

Bibliography

comScore, Inc. (2011, December 29). comScore Reports November 2011 U.S. Mobile Subscriber Market

Share. Retrieved January 20, 2012, from Press Release:

http://www.comscore.com/Press_Events/Press_Releases/2011/12/comScore_Reports_Novemb

er_2011_U.S._Mobile_Subscriber_Market_Share

comScore, Inc. (2012, January 9). U.S. Mobile Email Audience Grows by Nearly 20 Million Users in the

Past Year. Retrieved January 20, 2011, from comScore Data Mine:

http://www.comscoredatamine.com/2012/01/u-s-mobile-email-audience-grows-by-nearly-20-

million-users-in-the-past-year/

Faught, K. S., Green, K. W., & Whitten, D. (2004, Spring). DOING SURVEY RESEARCH ON THE INTERNET:

YES, TIMING DOES MATTER. Retrieved January 20, 2012, from The Journal of Computer

Information Systems: ABI/INFORM Global

Flagg, R. (2011, September 28). Online Surveys. Retrieved January 18, 2012, from HowTo.Gov:

http://www.howto.gov/customer-service/collecting-feedback/online-surveys-fact-sheet#test-

survey

Hausman, A. V., & Siekpe, J. S. (2009). The effect of web interface features on consumer online purchase

intentions. Retrieved January 30, 2012, from Journal of Business Research, 62: ScienceDirect

database

IBM. (n.d.). IBM 805 Test Scoring Machine. Retrieved 20 2012, January, from IBM Archives: http://www-

03.ibm.com/ibm/history/exhibits/specialprod1/specialprod1_9.html

Kang, C. (2011, July 11). As smartphones proliferate, some users are cutting the computer cord.

Retrieved September 19, 2011, from The Washington Post:

http://www.washingtonpost.com/business/economy/a-smartphones-proliferate-some-users-

are-cutting-the-computer-cord/2011/07/11/gIQA6ASi9H_story.html?hpid=z3

Kohut, A. (2009, October 14). But What Do the Polls Show? Retrieved January 18, 2012, from

PewResearchCenter Publications: http://pewresearch.org/pubs/1379/polling-history-influence-

policymaking-politics

O'Neill, M., & Palmer, A. (2001). Survey timing and consumer perceptions of service quality: An overview

of empirical evidence. Retrieved January 20, 2012, from Managing Service Quality; 11,3:

ABI/INFORM Global

Peltz, J. (2012, January 8). For some consumers, surveys breed feedback fatigue. Retrieved January 16,

2012, from AP News: The Orange County Register:

http://hosted.ap.org/dynamic/stories/U/US_FEEDBACK_FATIGUE?SITE=CAANR&SECTION=HOM

E&TEMPLATE=DEFAULT

Customer Opinion and Audience Response

34

Shapiro, G. M. (2008). Sample Size. Retrieved January 17, 2012, from Encyclopedia of Survey Research

Methods: SAGE Reference Online

Sheetz. (n.d.). Our Menu. Retrieved January 19, 2011, from Sheetz.com:

http://www.sheetz.com/main/food/menu.cfm

Walonick, D. S. (2004). Survival Statistics. Retrieved May 2, 2010, from StatPac, Inc.:

http://www.statpac.com/surveys/surveys.pdf