Survey Error:

Focus on Systematic Error

Totalerror

Systematicerror (bias)

Random samplingerror

Total Survey Error: Components

Random Sampling Error

• A statistical fluctuation that occurs because of change variation in the elements selected for the sample

Systematic Error

• Systematic error results from some imperfect aspect of the research design or from a mistake in the execution of the research

Respondent Error

• A classification of sample bias resulting from some respondent action or inaction

• Nonresponse bias

• Response bias

Respondenterror

Nonresponseerror

Responsebias

Respondent Error: Components

Non-response Error

• Non-respondents– Refuse to cooperate– Not-at-homes

• Self-selection bias– Over-represents extreme positions– Under-represents indifference

• Varies by type of interview

Why People Cooperate

• To be supportive/helpful

• Have a social interaction

• Curiosity

• Remuneration

Possible Cover Letter Appeals

Factors Contributing to Refusal

• Fear/anxiety

• Invasion of privacy

• Hostility toward– Survey sponsor– Topic of interview– Interviewer

Response Bias

• A bias that occurs when respondents tend to answer questions with a certain slant that consciously or unconsciously misrepresents the truth

Responsebias

Unconsciousmisrepresentation

Deliberatefalsification

Response Bias: Components

Unwillingness to Respond Accurately

• Invasion of privacy

• Time pressure and fatigue

• Physical/social environment

• Questionnaire-specific issue– Question/response wording– Layout

• Mischievous respondents

Inability to Respond Accurately

• Inarticulate

• Ignorant– Asked about other people– Asked to predict own behavior– Asked question meant for other people

Inability to Respond Accurately

• Faulty memory– Only ask about recent events– Problem with questions about behavior over

time (e.g., frequency)– Telescoping

• Everything happened yesterday

Acquiescence bias

Extremity bias

Interviewer bias

Auspices bias

Social desirability bias

Unconscious Misrepresentation: Components

Acquiescence Bias

• A category of response bias that results because some individuals tend to agree with all questions or to concur with a particular position.

Extremity Bias

• A category of response bias that results because response styles vary from person to person; some individuals tend to use extremes when responding to questions.

Interviewer Bias

• A response bias that occurs because the presence of the interviewer influences answers.

Auspices Bias

• Bias in the responses of subjects caused by the respondents being influenced by the organization conducting the study.

Social Desirability Bias

• Bias in responses caused by respondents’ desire, either conscious or unconscious, to gain prestige or appear in a different social role.

Systematicerror (bias)

Administrativeerror

Respondenterror

Systematic Error: Components

Administrative Error

• Improper administration of the research task

• Blunders• Confusion• Neglect• Omission

Administrative Error

• Interviewer cheating - filling in fake answers or falsifying interviewers

• Data processing error - incorrect data entry, computer programming, or other procedural errors during the analysis stage.

• Sample selection error -improper sample design or sampling procedure execution.

• Interviewer error - field mistakes

Interviewer Error

• Subtle source of cues– Can inadvertently reinforce some responses

• Appearance of incompetence

Interviewer Error

Insufficient or poor probing

Interviewer Error

Fraud

• Why commercial houses validate roughly 20% of completed interviews

Selecting unqualified respondents

Checklist for Unacceptable Questionnaires

• Major portions of questionnaire or key questions left unanswered

• Evidence that respondent did not understand instructions or did not take task seriously

• Missing pages• Respondent not qualified for target

population• Questionnaire returned after cutoff date

Ways to Handle Missing Responses

• Leave blank

• Casewise deletion

• Pairwise deletion

• Mean response

• Imputed response