Slide 1

Slide 2 Developing Online Surveys CENTER FOR RESEARCH STRATGIES www.crsllc.org March 2014 Slide 3 Information in this training has been adapted from the following sources: Dillman, D.A., Smyth, J.D., Christian, L.M. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 3 rd Edition. Hoboken. John Wiley & Sons, Inc., 2009. Groves, R.M., Fowler, Jr., F.J., Coupar, M.P., Lepkowski, J.M., Singer, E., Tourangeau, R. Survey Methodology, 2 nd Edition. Hoboken. John Wiley & Sons, Inc., 2009. SurveyMonkey documents (www.surveymonkey.com): Response Rates and Surveying Techniques: Tips to enhance Survey Respondent Participation. Smart Survey Design SurveyMonkey User Manual Slide 4 Overview When should I use a survey? How many people should I survey? How do I construct survey questions? How do I create an electronic (web-based) survey? Slide 5 When to Use a Survey Surveys are used to learn more about a group of people: their background, needs, knowledge, attitudes, or behaviors Surveys can be directed towards clients, training participants, community stakeholders or program staff Slide 6 Do you need to collect primary (original) data? YES if at least one of these is true: There are no or insufficient secondary data available on the intended population group. You need to tailor program efforts to the intended population group. You need direct feedback from the intended population group. NO if: There are sufficient secondary data (previously collected information) available on the intended population group that tell you everything you need to know about your intended population group. There is no need to conduct a survey. Do you want quantitative (numerical) data about your intended population group? NO if: You only want qualitative (narrative) in-depth information or feedback about perceptions, opinions, beliefs and attitudes from a subset of your intended population group. YES if at least one of these is true: You want quantitative facts about your intended population group. You want to collect data from a representative sample of a large group. You want a lower cost option for collecting a large amount of data. You want to make comparisons between your intended population group and other similar groups or the general population. You want to compare differences among your participants before and after implementing an effort or program. There is no need to conduct a survey. Survey Flowsheet Slide 7 Electronic Online Survey Do you want to conduct an electronic survey? Use an electronic survey if you want to: Save time by getting a quicker response Save money by avoiding printing and mailing costs Make the analysis of data easier (e.g. running frequencies, making comparisons) Do not use an electronic survey if the group you want to reach: Limited time (e.g. limited access to email during work hours) Limited or no access to computers or the internet Uncomfortable using computers Slide 8 How Many Surveys Do You Need? Depends on the size of the group you want to reach If your target group includes your clients, try to reach all of them If your target group includes a broader segment of the community, create a sample (e.g. health disparity group) In general, aim to collect 100-150 surveys if possible Slide 9 How To Motivate People to Respond Start with your most general questions and then become more specific (e.g. sensitive topic of sexual health) Start with a question that respondents want to answer Keep respondent burden at a minimum (questions short, as few total questions as possible) Know who your respondents are Explain why the survey is important in a cover letter Provide an incentive (e.g. a prize drawing or a gift card) Slide 10 What to Know About Respondents Are they eligible to participate? What is their literacy level? Are they familiar with taking surveys? What background information do you want to know (e.g., gender, age, race, income level)? Slide 11 Constructing Questions Be consistent in how your questions are worded (i.e., use the same point of reference) Over the past 7 days, how many times did you eat fresh fruit? Over the past 7 days, how many glasses of low-fat milk did you drink? Keep your language clear (simple, concise, familiar, definitive words) Ask one question at a time (no double-barreled questions that contain two components) No: Do you subscribe to or read any magazines? Yes: Do you subscribe to any magazines? Do you read any magazines? Make sure yes means yes and no means no (avoid double-negative questions, such as a yes vote resulting in something not being done) Slide 12 Question Formats Close-ended yes/no or multiple choice (select an option) Do you like to exercise? O Yes O No O Not sure Ranked or ordinal questions (answers in ranking order, better to worse) How important is it to you that your child arrives to school on time? O Very Important O Somewhat ImportantO Not at all important Slide 13 Question Formats Nominal questions (answers with no natural order) What do you like to do in your free time? (Check all that apply) O Read O Watch TV O Go to movies O Exercise Likert scale (frequency) How often do you feel down or depressed? O Never O Sometimes O Always Open-ended (answer in own words) Please describe your first camping experience. Slide 14 I Dont Know and Not Applicable Only include these answer options if it is truly possible for a respondent not to know or for the question not to apply to them Do not include I dont know as an answer option for the question because respondents can guess if they dont know the exact distance. How far is the nearest grocery store from your home? Not applicable would be appropriate if you are asking questions that may not apply to everyone When you saw a specialist in the last year, who did you see? Slide 15 Pilot Test Test your survey with 8-10 people from your target group before administering survey Make the revisions necessary after the pilot test and then administer the survey to the broader intended audience group **If the survey group is small and the pilot testers are in this group, they can take the revised survey and have their responses included Slide 16 Survey Monkey An online platform to design surveys, track and analyze data Slide 17 Slide 18 Analyze Tab Slide 19 Response Summary Slide 20 Individual Responses Slide 21 Browse Responses Slide 22 Filter Responses Slide 23 Cross Tab Reports Slide 24 Data Quality Check for data entry errors. Review Summary of Responses and correct them under the Browse Responses link within the Analyze tab Survey Monkey does not include missing data in its calculation of percentages of respondents who answer each question, so recalculate percentages yourself 100 respondents: 52 male, 40 female, 8 missing Correct: 52% male, 40% female, 8% missing (100 as denominator) Incorrect: 57% male, 43% female (100-8= 92 as denominator) Slide 25 Thank You! This concludes our training. We have a few brief questions to ask you about next steps. Please click on the link below and take 2 minutes to answer. We appreciate it! Developing Online Surveys Evaluation