STATISTICS FOR MANAGERS

LECTURE 2:

SURVEY DESIGN

COMPONENTS OF SURVEYS

1. Sampling2. Methods of data collection3. Question design4. Interviewing

1. SAMPLING How well a sample represents a

population depend on:1. Sample frame2. Specific selection procedure3. Sample size

1.1. SAMPLE FRAME Sample of people that has a chance

to be selected, given the sampling approach. Classes: Sampling is done from a more or less

complete list of individuals of the population

Sampling is done from a list of people that go somewhere or do something

Sampling is done in two or more stages

1.1. SAMPLE FRAME Characteristic of the sample frame

to be evaluated: Comprehensiveness: a sample can be

representative only of the sample frame. Most sampling leave out a few people. Examples: household based sample exclude people in dormitories, prison and nursing homes.

1.1. SAMPLE FRAME Characteristic (cont.)

Probability of selection.It is not necessary that a sampling scheme give every member in the sample the same chance of selection. However, we should be able to find out the probability of selection for each individual selected. Example.

1.1. SAMPLE FRAME Characteristic (cont.)

Efficiency. In some cases sampling frames include units that are not among those wanted to sample. Examples: elderly using general household and eliminating with no old people; random dialing. Only issue: cost.

1.2. SELECTION OF UNITS One stage sampling:

Simple random sampling Systematic sampling Stratified sampling

Multistage sampling: no way to get at the population directly. Link population with groups.

1.2. SELECTION OF UNITS Simple random sampling: members

of a population are selected one at a time, independent of one another and without replacement. Unless the list is short and all unit are prenumbered drawing from a simple random sample is very laborious.

1.2. SELECTION OF UNITS Systematic sampling. The

researcher determines the number of entries in the list and the number of elements to be selected. For instance list of 8500 people and 100 required. 1 out of 85 should be selected. The process start with a random number.

1.2. SELECTION OF UNITS Systematic sampling (cont.). Then

we take any 85th person in the list. Problem: if the list is ordered following some criteria or has a recurring pattern that will differently affect the sample depending on the random start. Examples: couples club were the male partner is always listed first: only male or female.

1.2. SELECTION OF UNITS Stratified sampling. Generally little is

known about the characteristics of the individuals in the population before the sample. However, we may know some characteristics. Usually stratification involves some regional variable.

MULTISTAGE Examples:

Sampling students from schools Area probability sampling Random digit dialing

Respondent selection: who should be interviewed in the household?

MULTISTAGE If information is easy to report any

adult who is home can answer the question.

If information is specialized the researcher wants to interview the household member most knowledgeable.

MULTISTAGE Group work: Why are registered

unemployed many less than survey unemployed in Spain?

Data

1.3. SAMPLE SIZE Sample size and sampling errors will

be covered in the next lecture.

2. DATA COLLECTION Choice of data collection mode

depend on: Sampling Type of population Available staff Response rate Question form

2. DATA COLLECTION Methods

Personal interview Telephone interview Self-administrated Mail Drop off and pick latter Internet

2.1 Personal interview Advantages

There are some sample design that can be implemented best by personal interview (area probability samples)

Most effective way of enlisting cooperation for most populations

Interviewer can answer respondents questions, explain complex questions, etc.

2.1 Personal interview Advantages (cont)

Multimethod data collection (including visual cues) are feasible.

Confidence building is possible (including reassurances for sensitive material)

Probably longer survey instruments are possible in person than in any other mode

2.1 Personal interview Disadvantages

It is likely to be more costly than the alternatives.

A trained staff of interviewers that is geographically near the sample is needed

Total data collection period is likely to be longer than telephone procedures

2.1 Personal interview Disadvantages (cont)

Some samples (high rise building, high crime areas, elites, students) maybe more accesible by some other mode

2.2 Telephone interview Advantages

Lower unit cost than personal interviews

Random digit dialing (RDD) Better access to certain population Shorter data collection periods Interviewer administration (better than

mail or internet)

2.2 Telephone interview Advantages (cont)

Interviewer staffing and management easier than personal interviews: less staff, not necessary to be near the sample, supervision and quality control potentially better

Likely better response rate from a list sample than mail

2.2 Telephone interview Disadvantages

Sampling limitations (omitting those without telephone).

Non response higher than personal interview.

Questionnaire or measure constraints. Less appropriate for personal or

sensitive questions.

2.3 Self-administrated Advantages

Easy presenting questions requiring visual aid.

Asking questions with long and complex response categories.

Asking batteries of similar questions. Respondent do not have to share

answer with interviewer.

2.3 Self-administrated Disadvantages

Specially careful questionnaire design is needed.

Open questions usually not good idea. Good reading and writing skills by respondent

are needed. The interviewer cannot exercise quality

control Cannot control who answer the questions

2.4 Mail Advantages

Low cost Minimal staff and facilities Provide access to widely dispersed

samples and people difficult to reach by phone or in person.

Respondent have time to give a thoughtful answer, to look up records, etc.

2.4 Mail Disadvantages

Ineffectiveness of mail as way of enlisting cooperation

Disadvantages of not having interviewer involved in data collections

Need for good mailing addresses for sample.

2.5 Drop off-pick up later Advantages

The interviewer can explain the study, answer questions and designate a household respondent.

Response rates tend to be like those of personal interviews.

More opportunity for thoughtful answer and check out the records.

Does not requires trained interw. staff

2.5 Drop off-pick up later Disadvantages

Cost about as much as personal interviews.

Field staff is required (although less trained that for personal interviews).

2.6 Internet surveys Advantages

Low unit cost of data collection Potential high speed of return All the advantages of self-administrated

instruments. Al the advantages of computer assisted

instruments Time for checking out records, etc.

2.6 Internet surveys Disadvantages

Limited to samples of internet users Need for good addresses Challenges of enlisting cooperation Various disadvantages of not having an

interviewer involved in data collection.

Exercises How is the Census conducted? Other special samples:

Trial juries Measures of TV shares Measures of effectiveness of ads.

3. Question design Decide what you want to find out;

this is the most important step in writing a questionnaire.

Always test your questions before taking the survey.

Keep it simple and clear Use specific questions instead of

general ones, if possible.

3. Question design II Relate your questions to the concept

of interest. Decide whether to use open or

closed questions. Report the actual question asked. Avoid questions that prompt or

motivate the respondent to say what you would like to hear.

3. Question design III Use forced-choice rather than agree-

disagree questions. Ask only one concept in each

question. Pay attention to the question order

effect.

3. Question design IV Specific issues:

Contingent valuation Data on the registry of accounting. Trust questions and experiments.