SURVEYSSURVEYSSURVEYSSURVEYSCompiled byCompiled byPapia BawaPapia Bawa

What do they do?• Surveys are one of the primary

means of gathering primary evidence.

• They are useful for in-depth research and data development for reports.

Step # 1 in a Survey: Establishing Goals

• The first step in any survey is deciding what you want to learn. The goals of the project determine whom you will survey and what you will ask them. If your goals are unclear, the results will probably be unclear.

Some typical goals include learning more about

• The potential market for a new product or service

• Ratings of current products or services

• Employee attitudes • Customer/patient satisfaction levels

• Reader/viewer/listener opinions • Association member opinions • Opinions about political candidates

or issues • Corporate images

• The more specific you can make your goals, the easier it will be to get usable answers.

Step # 2: Selecting Participants

• There are two main components in determining whom you will interview. The first is deciding what kind of people to interview. (Target group)

• The next thing to decide is how many people you need to interview.

Target Group• If you conduct an employee

attitude survey or an association membership survey, the target group would obviously be the co-workers and supervisors.

However:• If you are trying to determine the likely

success of a product, the target group may be less obvious, since the product is still not in circulation.

• In this case a specific group of people who are most likely to use the product would be an appropriate choice.

• Correctly determining the target population is critical. If you do not interview the right kinds of people, you will not successfully meet your goals.

Determining target group numbers

• Statistically, the larger the sample, the more precisely it reflects the target group.

• You must make a decision about your sample size based on factors such as: time available, budget and necessary degree of precision.

Step # 3: avoiding biased groups

• A biased target group will produce biased results. Totally excluding all bias is almost impossible; however, if you recognize bias exists you can intuitively discount some of the answers. The following list shows some examples of biased samples.

• The consequences of a source of bias depend on the nature of the survey. For example, a survey about Internet products can safely ignore people who do not use the Internet.

Step # 4: Establishing Quota

• A Quota is a sample size for a sub-group. It is sometimes useful to establish quotas to ensure that your sample accurately reflects relevant sub-groups in your target population.

• For example, men and women have somewhat different opinions in many areas. If you want your survey to accurately reflect the general population's opinions, you will want to ensure that the percentage of men and women in your sample reflect their percentages of the general population.

• If you are interviewing users of a particular type of product, you probably want to ensure that users of the different current brands are represented in proportions that approximate the current market share.

Step # 5: Determining Interviewing Methods

• Once you have decided on your sample you must decide on your method of data collection. Each method has advantages and disadvantages.

• Please read the handout in class sessions for more details.

How are they conducted

• In order to conduct a survey we need the following things:

• A survey questionnaire/s• A group of participants• A specific venue/s

Developing Survey Questions

• Broadly we can have two kinds of questions in a survey:

• Open• Closed

Open Questions• An open question is one in which

the respondent is given a wide choice and can give any answer.

Example• "What do you think would make

Purdue a better place for students?"

• The respondent may give you a one word answer or they may talk for 5 or 10 minutes.

Pros and Cons• The advantage of an open question is

that you can potentially get a fuller understanding of what the respondent thinks and why he/she thinks so.

• The biggest problem is that it would be difficult or impossible to quantify (count) these results because each response may be different.

Closed Questions• A closed question is one in which

the person may only answer with a word, a number, or by choosing one answer from a list.

Types of Closed Questions

• Short Q&A• Scale Based• Multiple – choices• Ranking Items

Short Q&A

• These are questions in which people answer the question with a word or number.

Examples: • What is your native language? (one word) • How many children do you have? (number) • Do you own your house? (yes/no)

Scale Based• In these questions, people select

their answer from a scale. You may use a number scale or a word scale.

Example• Scales can be designed based on

numbers or words:• For the following question, you can

give the respondent one of the two different set of options

• How easy is it for you to attend community meetings?

Examples of word and number:

Word Scale:

• Very easy• Easy• So-so• Difficult• Impossible (word

scale)

• Number Scale:• On a scale of 1

(being very easy -5 (being very difficult)

• 1• 2• 3• 4• 5

Multiple choices• In these questions, people choose

their answer from a list you give them. You can ask them to choose just one answer, or all that apply. They can also add a new answer to the list.

Example:• Where do you get your information

about the news?

• The newspaper • The television • Friends • Other

Ranking items:• This is a task of ordering items,

usually from best to worst. Directions for these must be very clear so that respondents don't just check off one item or in other ways get confused.

Example:• Rank the candidates in order of

preference from best (1) to worst (4).

_____ Ben Green _____ Marta Gonzalez _____ Israel Nwidor _____ Jean Martin

7 Principles of Surveys• Be Brief• Be Clear• Be Realistic• Be Even-dimensional• Be Complete• Be Judicious• Be Courteous

Brief:• If a question and its answers are

too long, there is a bias for respondents to answer the last answer presented.

Clear• Use very common, unambiguous terms. • Avoid questions with double meanings.

(e.g., "Have you stopped beating your wife? o Yes o No" )

• If an uncommon term is used, make sure to define it in the survey form.

• Do not use technical or specialty jargon. • Be careful using the word "you"; it can

be singular or plural.

Realistic• What people say and how they respond

doesn't always represent how they will behave; assume natural biases in their responses.

• For the general public, do not rely on hypothetical questions. (e.g., "If such and such, then would you...?")

Even Dimension• Don't combine two questions in

one. (e.g., "Do you shop downtown or do you shop at malls?")

Complete• If multiple alternative responses

are included, they should be mutually exclusive. Also, they should include all possibilities.

• Try to avoid questions that can be answered "I don't know."

Judicious• Avoid leading questions, "loaded"

words, and authoritative statements. • Questions should ask for opinions,

rather than fact. (e.g., Don't ask, "Is downtown a dangerous place at night?" Instead, ask, "Do you think downtown is dangerous at night?")

Courteous• Take respondents and their

answers seriously. • Avoid condescending slang. • Don't present challenging

questions. (e.g., "Why don't you shop downtown more often?")

Combating Biases• It is common for surveys to have

biases. These should be thought of beforehand, and methods found for minimizing their impact. If such biases are too great, the legitimacy of the survey findings could be called into question

Hawthorne Effect: One bias commonly found in

survey research is the Hawthorne Effect, which says that respondents tend to respond differently simply because they have been selected for a survey.

What it means?• Because of the special recognition

which has been given them, it is sometimes found the respondents tend to answer in the way which will most please the researcher. To minimize this bias, the questioner should be as neutral as possible in presenting the survey.

"Self-lifting" bias:•

Closely associated with the Hawthorne Effect, the "self-lifting" bias recognizes respondents want to make themselves appear in a positive light, and will respond accordingly.

How to Deal with this?• This bias can be minimized by

positioning personal questions about respondents at the end of the questionnaire, where they would tend not to affect other, more substantive responses.

The "Habit" bias:•

If given a series of similar questions, respondents will fall into a habit of answering them similarly without considering each on its merit.

What to do?• This bias can be minimized by changing

the format of questions throughout the questionnaire. The format may range from simple "check the box" questions to one-word responses to open-ended responses to completing information on simple graphs and maps. Through these variations, each question is given its own personality, avoiding the "habit" response.

Credits• Don A. Dillman. 1978. Mail and Telephone

Surveys: The Total Design Method. New York City: John Wiley and Sons

• Norman Tyler, AICP • The Civic Participation and Community

Action Sourcebook at www.nelrc.org. • Creative Research Systems• Papia Bawa