Survey Research

Survey data

• Survey data are obtained when individuals respond to questions asked by interviewers or when the individual responds to questions that he has read

• Quantitative and qualitative information

• Census or sample?

Survey Sampling

• Sample Survey– Sample of the population

(e.g., sample of Alberta CEOs)

• Census Survey– Complete population

(e.g., all CEOs in Lethbridge)

Functions

• Descriptive

• Causal (limited function)

• Exploratory (limited function)

Information Provided by Surveys

• respondent’s knowledge of facts – how many times a month do you buy cookies?

• respondent’s attitudes – “chocolate chip is my favorite variety of cookie.”

• May describe processes undertaken by respondent– Stages in the purchase decision, for example

• usually self-report data

• The type of information gathered depends on a survey’s objectives.

• Surveys typically have multiple objectives

Classifications• Method of Communication

– Telephone– Mail– Personal interview

• Degree of Structure– Unstructured– structured

• Degree of Disguise

Limitations:

Structure and disguise

are not clear categories;

most surveys are hybrids

• Temporal classifications– Cross sectional studies– Longitudinal studies

• Trend Studies– Samples general population at each point; complete turnover in who is

actually sampled

• Cohort studies (tracking studies)– Samples from one group over time; e.g., sample of a graduating class

• Panel studies– sample cohort, same specific respondents each sample

Method of Communication

• Media

• Personal Interviews

• Telephone Interviews

• Self administered Questionnaires

Media

• Human interactive

• Electronic interactive: researcher and respondent interact using digital technology

• Non-interactive media

Personal interview• Direct communication between business

and consumers in which interviewers ask respondents questions in face to face situations

• Key advantages:– Opportunities for feedback– Probing for complex answers– Length of interview

Personal interview

• Direct communication between business and consumers in which interviewers ask respondents questions in face to face situations

Key advantages: Personal interview

• Opportunities for feedback

• Probing for complex answers

• Length of interview

• Completion rates

• Ability to use props

• High participation/low refusal rate (especially if using call back)

Key disadvantages:

• Anonymity of respondent

• Interviewer bias

• Interviewer cheating

• cost

Types of personal interviews

• Door to door interviews: – Low refusal rates – Provide a more representative sample– Provide a less representative sample (under-

representation of certain groups)

Types of personal interviews

• Mall intercept: – Lower cost– Provide a larger sample– Useful if targeting particular population– Increased use of visual aids– Higher refusal rate– Provide a less representative sample

Telephone interviews

• Considered to be the primary method of survey research

• Why: – representative samples– Technology– Perception of anonymity

Key advantages: Telephone interview

• Central location - increased quality of supervision

• Computer assisted telephone interviewing - data entry

• Speed

• Cost

• Anonymity- absence of face to face contact

• Less interviewer cheating

• Less researcher bias

• Cooperation???

• Representative samples (RDD)

• Call backs easier

Key disadvantages:

• Cooperation

• Lack of visual media

• Limited duration

Mail and self-administered questionnaires

• A very popular method of data collection

Types of Mail Surveys

– Ad Hoc Mail Surveys:• Questionnaires for a particular project sent to

selected names and addresses with no prior contact by the researcher.

– Mail Panels:• Precontacted and screened participants who are

periodically sent questionnaires.

• A mail panel is a type of longitudinal study. A longitudinal study in one in which the same respondents are resampled over time.

Key advantages: Self-administered

• Geographical flexibility

• Cost

• Respondent convenience - accuracy and reflection

• Anonymity

• Standardized questions - increase validity

Key disadvantages: Self-administered

• Time

• Length (6 pages)

• Interviewers absence

• Response rates

Tactics Employed to Increase Response Rate

• Advance postcard or telephone call alerting respondent of survey

• Follow-up postcard or phone call (coding/keying)

• Monetary incentives (nickel, dime, quarter, half-dollar)

• Premiums (pencil, pen, keychain, etc.)

• Postage stamps rather than metered envelopes

• Self-addressed, stamped return envelope

• Assurances regarding length

• Promise of contributions to favorite charity

• Entry into drawing for prize

• Emotional appeals

• Affiliation with universities or research institutions

• Multiple mailings of the questionnaire

• Bids for sympathy

• Reminder that respondent participated in previous study

• Interesting questions; well laid out

• Ask the right people

• A cover letter

Cover Letter

• introduces who you are• purpose and importance of study• relevance of survey to respondent

– especially if they receive several survey’s a week, need to be convinced survey results matter to them

• why and how they were selected – why their input is crucial– e.g., “As a respected member of the

Trustees, you have a unique perspective on....”)

• big name sponsor, if any– e.g., University of Lethbridge, Heart

Foundation, etc.

• give due date – make reasonable but specific

• contact number if they have any questions – include email these days too

• end by thanking them for completing survey

• explain direct incentive, if any (optional)

Not just the mail...

• At point of service

• Fax surveys (respond by fax; deliver by fax)

• Email surveys

• Computerized questionnaires (computer interactive)

• Internet surveys

Factors Determining Choice of Particular Survey Method

• Sampling Precision Required

• Budget Available

• The Need to Expose the Respondent to Various Stimuli

• Quality of Data Required

• Length of the Questionnaire

• Necessity of Having Respondent Perform Certain Specialized Tasks

• Incident Rate– Percentage of people or households in the

general population that fit the qualifications to be sampled.

• Degree of Structure of the Questionnaire

Error

Types of Error in Survey Research

• Random Sampling Error (Random error)– Error that results from chance variation – Impact can be decreased by increasing sample

size and through statistical estimation (confidence interval) or “rule of thumb”

• Systematic Error (non sampling error)– Error that results for the research design or

execution.

R an d om E rro r

N on resp on se e rro r

D e lib era te F a ls ifica tion U n con sc iou s m is rep resen ta tion

A cq u iescen ce E xtrem ity b ias In te rviewer b ias A u sp ices b ias soc ia l d es irab ility

R esp on se b ias

R esp on d en t e rro r

D ata P rocess in g S am p le se lec tion

In te rviewer ch ea tin g In te rviewer e rro r

A d m in is tra tive e rro r

S ys tem atic E rro r

Tota l E rro r

Types of Systematic Error

1. Administrative Error– Error that results from improper execution.

• Data Processing Error– Quality of data depends on quality of data

entry.– Use of verification procedures can minimize

• Sample Selection Error– Systematic error resulting from improper

sampling techniques either in design or execution.

• Interviewer Error– Data recorded incorrectly (error or selective

perception).

• Interviewer Cheating– Mitigate by random checks

2. Respondent Error– Humans interviewing humans...

• Non-response error– Statistical difference between a survey that

includes only those who responded and a survey that also includes those who failed to respond.

– Non-respondent: person not contacted or who refuses to participate

– Self selection bias: extreme positions represented

• Response bias– Errors that result from tendency to answer in “a

certain direction”.– Conscious or unconscious misrepresentation

• Types:– 1. Deliberate falsification (why?)

• Why would people deliberately falsify data– Appear to be what they are not– Don’t trust confidentiality– Protect– To end the interviewer quicker– “Average man effects”

• Types of response bias continued:– 1. Deliberate falsification– 2. Unconscious misrepresentation

• Reasons for unconscious misrepresentation:– Question format– Question content– Misunderstanding of question leading to biased

answer– Lack of time to consider answer fully– Communication or semantic confusion– other

Types of response bias

• Acquiescence bias: individuals have a tendency to agree or disagree with all questions or to indicate a positive/negative connotation

• Extremity bias: results for response styles varying from person to person; some people tend to use extremes when responding to questions

Types of response bias continued...

• Interviewer bias: Bias in the responses of the subject due to the influence of the interviewer

• Auspices bias: respondents being influenced by the organization conducting the study

• Social desirability bias: caused by respondents’ desire, either consciously or unconsciously to gain prestige or to appear in a different social role