Surveys

Objectives

1. To understand why surveys are popular

2. To describe the types of surveys

3. To discover the advantages and disadvantages of the various types of survey research

4. To gain insight into the factors that determine the choice of particular survey methods

5. To learn the types of error in survey research.

What is a Survey?

Why Survey?

Get lots of structured information from lots of people.

Simplify and standardize data collection. Use statistics to make predictions. Find out about things that can not be

observed. Such as?• Thoughts, Emotions, Intentions, Attitudes

The Need to Know How• Also it is often necessary to understand the process consumers go

through before taking some action. The Need to Know Who

• Information on age, income, occupation, marital status, stage in the family life cycle, education, and other lifestyle factors is necessary to the identification and definition of market segments.

The Need to Know WhyThere generally is a critical need to have some idea about why people do or do not do something.

• Likes, dislikes, attitudes, behaviors, influences

Personal Interview•Door-to-Door (in-home), Mall-Intercept, Purchase-Intercept Technique (PIT), Executive

Telephone-Interview•People vs. Machines•Computer assisted (CATI)

Self-Administered•Mail Survey•Fax•One-time vs. Panels

Computer Direct Computer Interviewing Interactive voice response technology-Automated Telephone Surveys

Survey Methods

Personal Interviews

Personal Interviews

Can arouse and keep interest

Can build rapport and enlist cooperation

Ask complex questions

Can use visual and other aids

Clarify misunderstandings

High degree of flexibility

Probe for more complete answers

Do not need an explicit or current list of households or individuals

Advantages

Personal Interviews

Bias of Interviewer

Response Bias

• Embarrassing/personal questions

Time Requirements

Cost Per Completed Interview Is High.

A trained staff of interviewers that is geographically near the sample is needed.

The total data collection period is likely to be longer than for most procedures.

Disadvantages:

Telephone Interviewing

Selecting telephone numbers• Pre specified list• A directory• Random dialing procedure

• Random digit dialing• Systematic random digit dialing (SRDD)

The introduction When to call Call reports

What are the Important Aspects of Telephone Interviewing?

Telephone Interviewing

Central location, under supervision, at own hours

More interviews can be conducted in a given time

• Travelling time is saved More hours of the day are productive

Repeated call backs at lower cost

Lower administrative costs/ Lower cost per completed interview

Less sample bias

Better access to certain populations Shorter data collection periods.

Advantages:

Telephone Interviewing

Inability to employ visual aids or complex tasks

Can't be longer than 5-10 min. or they get boring

Amount of data that can be collected is relatively less

A capable interviewer essential

Sample bias -- Not all people have phones, or are not listed

Nonresponse associated with RDD sampling is higher than with interviews

Possibly less appropriate for personal or sensitive questions if no prior contact

Limitations and Disadvantages:

Increasing Phone Survey Response

Call at a convenient time (Weekdays 7-9 PM, Sunday afternoon)

Have a nice Pleasant introduction

Emphasize you are not selling anything.

State how long it will take.

Keep the survey short

Respondent Reads Survey

Questions and Records

Answers Without

Assistance

Respondent Reads Survey

Questions and Records

Answers Without

AssistanceDrop Off Survey

Questionnaires are left with respondent to be completed at a later time and

returned to the researcher.

Drop Off SurveyQuestionnaires are left with respondent

to be completed at a later time and returned to the researcher.

Direct Mail SurveyQuestionnaire is distributed to and returned from respondents via the

postal service.

Direct Mail SurveyQuestionnaire is distributed to and returned from respondents via the

postal service.

Mail Panel SurveySelected group of individuals that have

made an advance agreement to participate in a series of direct mail

surveys.

Mail Panel SurveySelected group of individuals that have

made an advance agreement to participate in a series of direct mail

surveys.

Self-Administered Types of Survey Research

Mail Surveys Requires a broad identification of the individuals to be

sampled before data collection begins• Ad Hoc Mail Surveys (cold):

• Questionnaires for a particular project sent to selected names and addresses with no prior contact by the researcher.

• Mail Panels (warm):• Pre-contacted and screened participants who are periodically

sent questionnaires.• A mail panel is a type of longitudinal study. A longitudinal

study in one in which the same respondents are re-sampled over time.

Mail Surveys

Type of Return Envelope Postage Method of Addressing Cover Letter The Questionnaire Length, Layout,

Color, Format Etc Method of Notification Incentive to Be Given

Some Decisions That Need to Be Taken

Mail Surveys

Relatively low cost

Reliable answers as no inhibiting intermediary

Survey answered at respondents discretion

Can be accomplished with minimal staff and facilities.

Provides access to widely dispersed samples.

Respondents have time to give thoughtful answers, look up records, or consult others.

Advantages

Mail Surveys

No control over whom the respondent consults before answering the questions

The identity of the respondent is inadequately controlled

The speed of the response can't be monitored

No control on the order in which the questions are exposed or answered

Especially careful questionnaire design is needed.

Open questions usually are not useful.

Good reading and writing skills are needed by respondents.

The interviewer is not present to exercise quality control with respect to answering all questions, meeting questions objectives, or the quality of answers provided.

Disadvantages

The respondent may not clearly understand the question and no opportunity to clarify

No long questionnaires

Subject to availability of a mailing list

Response rate is generally poor

Number of problems such as obsolescence, omissions, duplications, etc

Ineffective as a way of enlisting cooperation. Need for good mailing addresses for sample.

Disadvantages (Contd.):

Mail Surveys

Perceived amount of work required, and the length of the questionnaire

Intrinsic interest in the topic

Characteristics of the sample

Credibility of the sponsoring organization

Level of induced motivation

Factors Affecting the Response Rate

• Advance postcard or telephone call alerting respondent of survey.

• Follow-up postcard or phone call.

• Monetary incentives (nickel, dime, quarter, half-dollar).

• Premiums (pencil, pen, keychain, coupons, etc.).

• Postage stamps rather than metered envelopes.

• Self-addressed, stamped return envelope.

• Personalized address and well-written cover letter.

• Promise of contributions to favorite charity.

Tactics Employed to Increase Mail Survey Response Rate

• Entry into drawing for prize.

• Emotional appeals.

• Affiliation with institutions or reputed organizations.

• Personally signed cover letter.

• Multiple mailings of the questionnaire.

• Bids for sympathy.

• Offer to share information from the survey.

• Reminder that respondent participated in previous study.

Tactics Employed to Increase Mail Survey Response Rate

The interviewer can explain the study, answer questions, and designate a respondent.

Response rates tend to be like those of personal interview studies.

There is more opportunity to give thoughtful answers and consult records.Costs about as much as personal interviews.A field staff is required.

Drop-off questionnaires

Relatively low cost Can be accomplished with minimal staff and

facilities Provides access to widely dispersed samples. Respondents have time to give thoughtful answers. Local faxes are free. Administrative costs are fixed. It is fast. List management is easy. Can send and receive by computer.

Fax SurveysAdvantages

Higher fixed costs for computer/fax equipment, multiple phone lines.

Cost varies by time on line, time of day, distance, and telephone carrier.

Currently limited to organizational populations. Loss of anonymity.

Fax Surveys

Disadvantages

Internet Samples•Unrestricted

• Open to any Internet user.

•Screened • adjust for unrepresentitiveness of the self-selected respondents by imposing quotas based on some desired sample characteristics

Recruited• to target populations in surveys that require more control of the sample

To realize the importance of the marketing research interviewer.

Internet Surveys

The advantages of interviewer administration (In contrast to mail surveys).

smaller staff needed, High-speed,Instantaneous data accessCost efficient, Automatic data entryMultimedia stimuliEasy to update Ability to reach a lot of people

Internet SurveysAdvantages:

Internet SurveysAdvantages:Potential for longitudinal studiesSurveys can be unobtrusively included with a general sitepre-screening of respondents possibletracking No geographic boundaries supervision and quality control potentially better.

Likely better response rate from a list sample than from mail

use a branching or skip pattern

Disadvantages:

Internet Surveys

•Internet users are not representative of the population as a whole (strong sample bias)

•Strong selection bias for respondents who are not pre-screened

•security/privacy issues

• unrestricted: anyone can complete the questionnaire

• fully self-selecting

E-mail Questionnaires

• The questionnaire is prepared like a simple E-mail message, and is sent to a list of known E-mail addresses. The respondent fills in the answers, and E-mails the form plus replies back to the research organization

Increasing Response: Internet/E-mail Survey

Almost all the actions listed for mail survey should hold for E-Mail/Internet survey with some modifications.

The questionnaire should be fairly short. No need for advance notification Have short, pleasant introduction Monetary incentives not given. But sharing

information would be nice. Reminder e-mail would be useful.

Criteria for Determining Choice Of Particular Survey Methods

Sample Criteria: Ability to reach and get responses from the desired sample (right type of people and “adequate” sample size).

Information Criteria: Ability to get the desired information from respondents.

Need to Expose Respondents to Various Stimuli or Perform Certain Specialized Tasks e.g. Taste tests, product concept and prototype tests, ad tests, card sorts, etc.

Length of Questionnaire Degree of Structure of the Questionnaire

Administrative Criteria: time for data collection and analysis Interviewer control Budget

Personal:

Mail:

Phone:

E-Mail/Internet:.

Factors Determining the Choice of Survey Method.

Use for long, complex questionnaires where respondent is important and budgets are high.

Use when you have a mailing list, somewhat long / complex questionnaire, and budgets are low.

Use to reach a large number of people quickly and you have a short questionnaire that can be easily understood.

Use when target audience is educated, topic interesting, short questionnaire, have e-mail list, representativeness not a major issue

Comparative Evaluation

Personal Phone Mail Sample control Use of physical stimuli Diversity of questions Length Perceived anonymity Potential for interviewer bias Field force control Speed Cost

Advantages & Disadvantages of Survey Methods

Personal Mail/Self Report Telephone

Speed of data collection

Moderate to fast No control over return of questionnaire

Very fast

Geographic flexibility

Limited to moderate

High Good

Respondent cooperation

Excellent except in shopping malls

Moderate –poorly designed questionnaires have poor response rates

Good

Versatility of questioning

Very versatile Highly standardized format

Moderate

Personal Mail/Self Report Telephone

Questionnaire length

Long Varies depending on incentive

Moderate/Short

Respondent misunderstanding

Lowest Highest Moderate

Interviewer influence

High None Moderate

Supervision of interviewers

Moderate Not applicable High

Anonymity of respondent

Low in face-to-face situations

High Moderate

Advantages & Disadvantages of Survey Methods

Personal Mail/Self Report Telephone

Ease of callback or follow-up

Difficult Easy, but takes time

Easy

Cost Highest Lowest Low to moderate

Special features Visual materials may be shown or demonstrated; extended probing possible

Respondents may answer questions at own convenience; has time to reflect on answers

Field and supervision of data collection are simplified; quite adaptable to computer technology

Note: These reflect typical situations. For example, an elaborate mail survey may be far more expensive than a short interview, but this is generally not the case.

Advantages & Disadvantages of Survey Methods

Did you ask the right questions in the right way? Did you ask the right people? Did they tell you the truth, the whole truth, and

nothing but the truth? • Will they? • Can they? Do they have the knowledge, opinions, attitudes, or facts

required. Do they understand the questions?

Structure causes a loss of data richness. Have the Respondents/Interviewers understood and

correctly recorded the responses

Lack of control causes time and response problems.

Potential Problems with SurveysHow accurate are the results?

Overview of the Types of Errors in Survey Research Methods

Random Sampling Error

statistically measured difference between the actual sampled results and the estimated true population results.

error because of chance variation

Nonsampling Error (Systematic Error)

results from mistakes or problems in the research design or from flaws in the execution of the sample design

Causes your data to be misleading or incomplete in some systematic way

If you are aware of the problem you may be able to deal with it.

Survey research errors can be classified as either:

Systematic Sample Design Error Frame Error: The list from which you draw your

sample is not what you think it is—some on it don’t belong and/or some who belong are not on it.

Population Specification Error: incorrect definition of the universe or population from which the sample is to be selected.You left people out of the study.

Selection Error: You include or exclude people in the sample so that it is not random. Use of incomplete or improper sampling procedures or when appropriate selection procedures are not properly followed.

Systematic Measurement Error Surrogate Information Error

• Acquisition of the wrong data because wrong question was asked.

Interviewer Error• interviewer may, consciously or unconsciously, influence

respondents to give untrue or inaccurate answers. Instrument Bias

misunderstood or leading ?s. Processing Error

• sloppy data input Non-response Bias

• differences between the “did” vs. “did not” answer question Response Bias

a failure to tell the truth., consciously or unconsciously

RESEARCH QUESTIONS