366_8. survey design things to consider layout time to complete types of questions question...
DESCRIPTION
Survey Design THE RESPONDENT – Does she know what you are asking? – Does she understand the question? – Can she recall the needed information? – Does she have a reason to hide something?TRANSCRIPT
![Page 1: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/1.jpg)
366_8
![Page 2: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/2.jpg)
Survey Design
• Things to consider– Layout– Time to complete– Types of questions– Question wording– Sensitive items
– THE RESPONDENT
![Page 3: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/3.jpg)
Survey Design
• THE RESPONDENT– Does she know what you are asking?– Does she understand the question?– Can she recall the needed information?
– Does she have a reason to hide something?
![Page 4: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/4.jpg)
Survey Design
• THE RESPONDENT– May not be paying close attention
• Make survey easy to read, follow
• Use similar response categories for blocks of questions where possible
• Sensitive items at the end
![Page 5: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/5.jpg)
Survey Design
• THE RESPONDENT– Needs to know why the survey is being done• Why bother to do it?
– Needs to know the responses are anonymous.• Phone vs..• Mail vs..• Internet • Handed to them (face-to face)
![Page 6: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/6.jpg)
Types of Questions
• Open ended– What is the most important issue facing the
country today?
– Can you tell me what it is that makes you like the Democratic Party?
![Page 7: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/7.jpg)
Types of Questions
• Open ended– Valuable to measure information
– Might be biased toward people with more information
![Page 8: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/8.jpg)
Types of Questions
• Open ended– Very labor intensive to enter data
– Must create categories for responses, code into spreadsheet
![Page 9: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/9.jpg)
Types of Questions
• Closed ended– Defined response categories
– Overall, how satisfied are you with how democracy is working in the United States• Very satisfied• Satisfied• Not very satisfied• Not satisfied
![Page 10: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/10.jpg)
Types of Questions
• Closed ended– Do you want a neutral
category?• Agree strongly• Agree• Neither agree nor
disagree• Disagree• Disagree strongly
• Closed ended– Neutral category might
be easy response default• Too little• About right• Too much
• More• Same• Less
![Page 11: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/11.jpg)
Types of Questions
• Closed ended– Are you creating an ordinal (or even interval)
measure?• 1) a very great deal 6) Very little• 2) a great deal 7) None• 3) quite a bit• 4) a fair amount• 5) a little
![Page 12: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/12.jpg)
Types of Questions
• Closed ended– Thermometer scales• On a scale from 0 to 100, with 0 being cool, 50 being
neutral, and 100 being warm, tell me how you feel about [whatever]. Just write a number between 0 and 100 in the space below– [whatever1] _________– [whatever2]__________– [whatever3]__________
![Page 13: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/13.jpg)
Types of Questions
• Closed ended– Forced choice?
• Should response categories include “don’t know”
• Phone vs.. print surveys– If phone, we might not offer DK, but code it
– If print, if no DK option, respondent might skip
![Page 14: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/14.jpg)
Closed Ended Question Blocks
![Page 15: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/15.jpg)
Question Blocks
![Page 16: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/16.jpg)
Layout:
![Page 17: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/17.jpg)
Question Wording
• Be specific– “are you liberal or conservative” vs.:
– When it comes to politics, do you consider yourself liberal or conservative?
![Page 18: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/18.jpg)
Question Wording
• Avoid biasing responses– Leading questions• “Do you favor or oppose dirty coal?”
– Loaded words• “welfare” vs.. Income support for the less affluent
– Double – barreled questions• Should we raise taxes and cut spending to lower the
deficit?
![Page 19: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/19.jpg)
Question Wording
• Avoid biasing responses
– Is Clinton an old-style, tax & spend liberal, or a new style Democrat who who will be careful with the nation’s money? [ABC] vs.:
– Is Clinton a liberal Democrat or a Conservative Democrat (or, see slide 14).
![Page 20: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/20.jpg)
Question Wording
• Sensitive questions– If you must ask, ask indirectly
– Drinking, not voting, rioting
• Occasionally, people drink too much and become [intoxicated]. In the past year, how often have you become [intoxicated] while drinking any kind of alcoholic beverage?
![Page 21: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/21.jpg)
Question Wording• Sensitive questions [Why didn’t you vote?]• In any election, some people are not able to vote because they are sick or busy or have some
other reason, and others do not want to vote. Which best describes your situation in the recent local election in [CITY]
• • Illness or disability (your own, or in your family)• Forgot• Registration problems• Not interested; it doesn't matter• Too busy• Disliked the candidates• Out of town• It was inconvenient
Don’t like the voting rules• Other (VOL)• Don't know (VOL)
![Page 22: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/22.jpg)
More points about the layout
• Begin with brief intro– Class project, anonymity, etc.
• Follow with ‘easy’ substantive question– First we’d like to ask you about...
• Start questions casual, chatty, clear– In general, do you think_______– Thinking about _____ , – How about, _______, Do you think....
![Page 23: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/23.jpg)
More points about the layout
• Most important substantive items next
• Qs that test your main hypotheses
• Demographic questions at end
• Check established surveys for coding– Age – Region of state– Education– Party– Ideology– Religion [ or / and ]– Religious attendance– Family income (last)
![Page 24: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/24.jpg)
More points about the layout
• Whatever layout, include:
• Are you registered to vote?– If no, SKIP to Q X– If Yes, are you registered
to vote here in Whatcom County, or elsewhere?
![Page 25: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/25.jpg)
Before you print
• Are you sure everything is in there?– You have what you need to test your main
hypotheses
– You have questions that measure the attitudes you are interested in
– You have the questions designed to explain variation in the attitudes you are interested in
![Page 26: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/26.jpg)
More points about the layout
• Printing– Single sheet– Back to back
– Booklet
– Font size
![Page 27: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/27.jpg)
Pre-test
• Once DRAFT is completed, give survey to a few friends– How long did it take?– Ask them about it after they take it
• Any items not make sense?
![Page 28: 366_8. Survey Design Things to consider Layout Time to complete Types of questions Question wording Sensitive items THE RESPONDENT](https://reader036.vdocuments.site/reader036/viewer/2022062504/5a4d1b6f7f8b9ab0599b4d38/html5/thumbnails/28.jpg)
Monitor refusals
• Is there a pattern to who is not filling out survey?