survey research 101
DESCRIPTION
TRANSCRIPT
Online Surveys 101
A practical how-to
What is survey research?
Online surveys
Telephone surveysStructured face-to-face interviews
Asking structured questions
Surveys versus other qualitative methodsWhen a you want to know… …you need
What are users’ opinions of the site?
What is the demographic profile of our users?
What kinds of beliefs or attitudes do our users have about our…
-brand-customer service
How can we be sure that design changes changed users opinions of the site?
A survey
How usable is our Web site?
What do our users generally dislike about our Web site?
What kinds of features should we include in our new Web site?
Qualitative
research
method
Is a survey the right research method?
• I know a lot about the topic• I need to summarize the
findings in numerical format• I don’t need a lot of detail and
nuance• I need to generalize the results
to the population• I need to predict the likelihood
of a certain thing happening• I want to measure incremental
change
• I don’t really know much about the topic
• I don’t need to summarize the findings in numerical format
• I need to communicate a lot of detail and nuance
• I’m interested in case study information
• I’m not planning on predicting the likelihood of anything
• I don’t need to measure any change
Yes No
Is it time to do a survey?
Yes
No
Discovered questions and
answers throughQual research?
Clear idea of all possible questions
and answers?
YesNo
Formulated all questions and
possible answers?Do a survey
Do more Qual research
Yes
Pilot test the questions
No
Asking good questions• Mutually exclusive and exhaustive
– No overlap between answers– All possible answers
• No double-barreled questions– Wrong: “What do you think about our response times and customer
service?”– Right: “What is your opinion of our response time?”
• Focus on opinions or beliefs of participants– Wrong: “How usable is our Web site?”– Right: “What are users’ opinions of the site’s look and feel?”
• Close-ended is best– Cuts down on analysis time
• Demographic questions at the end
Sampling and its implications
• You can predict the likelihood of any single person being selected to participate
• Has some randomized element to it
• Allows for you to compare your findings to what you would expect with pure random events
• Allows for prediction of accuracy
• Allows for generalizing to the population at large
• Can use inferential techniques
• You cannot predict the likelihood of any person participating
• Has no random element to it• Cannot be used to predict
future likelihood of anything• Cannot be used to generalize
to the population at large• Still provides some insight,
particularly if you ask everyone in a group the same question
• Does not allow to estimate the accuracy of the results
• Cannot use inferential techniques
Probability SampleNon-probability
Sample
The lay of the land: descriptive statistics
• Summarizes the responses• What is the most “typical” response?
– Mean– Mode– Median
• How much variation is there between responses?– Standard deviation– Inter-quartile range
Predicting behavior: inferential statistics
• Infers or predicts future behavior• Which respondents are more likely to…
– Cross-tabulation
• Which factors most influence– Behavior– Satisfaction– etc
The tools you need
Online Survey Tool
Statistical Analysis Software
The skills you need• Knowledge of the topic• Ability to write good questions• Familiarity with the survey tool• Some basic knowledge of statistics• A framework for writing the report
– Important to write the outline of the final report before you start!
The payoff you’ll get• “More men are dissatisfied with the checkout process.” • “Younger users are much more unhappy with customer
service that older users.”• “Higher-income individuals are the most unlikely to
recommend this site to friends and family.”
What you won’t learn…• Why aren’t people coming to our site?• How can we innovate on our current design?• Why don’t people finish the checkout process?• What are the main characteristics of the people who
don’t come to our site?
Copernicus ConsultingDesign research and strategy
http://www.copernicusconsulting.net