Introduction to Surveys

Definition of survey

Study involving a subset (sample) of individuals selected from a larger population

where variables are observed or measured at a single point in time

aggregated over all individuals in the sample

to obtain a summary statistic

Observational studies

exposure outcome

outcome exposure

outcome exposure

cohortCause-effect

case-controlEffect-cause

cross-sectional

Extension of cross-sectional design Longitudinal

multiple cross-sectional surveys can estimate incidence if results between multiple surveys differ

true difference? samples very different?

Panel design multiple cross-sectional surveys using exactly same sample

Pseudo-cohort cohort: one age group over time pseudo-cohort: different age groups studied cross-sectionally at one

point in time

Survey To investigate

if taxi drivers are more likely to use seat belts than drivers of private automobiles

To determine the community prevalence of malnutrition measles Tuberculosis HIV

Steps in designing a surveyDefine survey objectivesIdentify study populationIdentify and define study variablesEstablish plan of analysisDetermine sample sizeChoose sampling methodDecide on mode of data collection Design, field test and revise questionnaireTrain interviewers, conduct field workCheck all interview forms for errorEnter, tabulate and analyse the results

Survey objectives

Formulate the objectives of the survey: What are you interested in finding out?

Who do you want to study?

When do you want to do the survey?

What do you expect to learn and why?

Attributes of “SMART” objectives

S pecific

M easurable

A ction oriented

R ealistic

T ime frame

Example 1: Objectives

What? What are the knowledge, attitudes and practices (KAP) to HIV/AIDSWho? of adults > 18 years of ageWhere? in MwananyamalaWhen? in year 2006Why? prior to the start of a new health education programme

Example 2: Objectives

To measure and compare the prevalence of smoking among MOH and Ministry of Agriculture employees in Tabora

To measure and compare the proportion of non-smokers in MOH and MOA exposed to tobacco smoke in their working environment and to determine the extent of the exposure

Steps in designing a surveyDefine survey objectives Identify study populationIdentify and define study variablesEstablish plan of analysisDetermine sample sizeChoose sampling methodDecide on mode of data collection Design, field test and revise questionnaireTrain interviewers, conduct field workCheck all interview forms for errorEnter, tabulate and analyse the results

(Target-) Population

Sample (study population)

Study population

Study population Objectives target population Study population must be representative of

target population Time: seasonality, day of the week, time of the day Place: urban, rural Persons: age, sex, other characteristics

A non-representative sample will produce biased results (validity )

Steps in designing a surveyDefine survey objectives Identify study population Identify and define study variablesEstablish plan of analysisDetermine sample sizeChoose sampling methodDecide on mode of data collection Design, field test and revise questionnaireTrain interviewers, conduct field workCheck all interview forms for errorEnter, tabulate and analyse the results

Study variables

Identification as many as necessary but as few as possible each must “pay its own way” if of no use in analysis, do not include variable

Requirements clear definition method of measuring should be:

repeatable valid

Identify study variables

Objective: To measure and compare the prevalence of smoking among MOH and MOA employees in Tabora

Variables: Smokers

in MOH in MOA

Definition: A smoker is someone who, at the time of the survey, smokes

cigarettes or any other tobacco products either daily or occasionally. daily: smokes any tobacco product 1 / day occassionally: smokes, but not every day

Steps in designing a surveyDefine survey objectives Identify study population Identify and define study variables Establish plan of analysisDetermine sample sizeChoose sampling methodDecide on mode of data collection Design, field test and revise questionnaireTrain interviewers, conduct field workCheck all interview forms for errorEnter, tabulate and analyse the results

Establish plan of analysis (POA)

How will the key variable(s) be measured to meet the objectives? Proportion, mean, ratio, ... Level of measurement influences

questionnaire design type of statistical analysis

What additional variables will need to be measured?

How will the results be used in the analysis (dummy tables)?

Plan of analysisTo measure and compare the prevalence of smoking among MOH and MOA staff in Tabora

Variables obtained from all study participants

Number (%) of persons smoking - daily - occasionally

Number (%) of persons smoking cigarettes, pipefuls of tobacco, or cigars / cigarillos Number (%) of nonsmokers who are

- neversmokers - exsmokers (and time since quitting)

Variable obtained from smokers Average number of cigarettes, pipefuls of tobacco, cigars / cigarillos smoked per day or week

Baseline characteristic of study population in UNHCR and UN -------------------------------------------------------------------------------------------------------- MOH MOA N = X N = Y -------------------------------------------------------------------------------------------------------- number (percent) Age*

30 years or younger 31 - 45 years 46 years or older

Female Work status

Staff member Short Term Professional Short Term Consultant Others

Dummy table - describe sample

Prevalence of smoking among staff of UNHCR and UN ------------------------------------------------------------------------------------------------------ MOH MOA N = X N = Y ------------------------------------------------------------------------------------------------------ number, percent Smokers

smoking daily smoking occasionally total n = x n = y

Nonsmokers

neversmokers exsmokers

quit 1 month ago quit 2-6 months ago

Dummy table - describe variables

Steps in designing a surveyDefine survey objectives Identify study population Identify and define study variables Establish plan of analysis Determine sample sizeChoose sampling methodDecide on mode of data collection Design, field test and revise questionnaireTrain interviewers, conduct field workCheck all interview forms for errorEnter, tabulate and analyse the results

Estimating sample size• Indicate expected frequency of factor of interest• Decide on desired precision of the estimate• Decide on acceptable risk that estimate will fall

outside its real population value• Adjust for estimated design effect• Adjust for expected response rate• (Adjust for population size, if sample > 10% of target

population)

Methods used in probability samples

Simple random sampling Systematic sampling Stratified sampling Cluster sampling Multi-stage sampling

Steps in designing a surveyDefine survey objectives Identify study population Identify and define study variables Establish plan of analysis Determine sample size Choose sampling method Decide on mode of data collection Design, field test and revise questionnaireTrain interviewers, conduct field workCheck all interview forms for errorEnter, tabulate and analyse the results

Method of data collection

Personal interview Telephone Self administered

Questionnaire Mail Computer direct E-mail

What method to choose?

Nature of questions What is the research question to be

addressed? Convenience

Which method is most readily available? Cost

How much money is available?

Personal interviews “face-to-face” Advantages

population coverage can clarify issues, allows complex questions longer interviews tolerated (2-3 hours) usually high response rate (~90%)

Disadvantages expensive slow intrusive requires interviewing skills interviewer bias

Self-administered questionnaires(mail or hand distributed)

Advantages cheap no interviewer bias long list of response categories possible sensitive questions

Disadvantages slow does not allow clarification poor response rate (particularly in populations of lower education and

literacy level) population coverage? who completes questionnaire?? sequence of questions cannot be enforced

Telephone surveys “Averages” advantages and disadvantages of personal and

mail surveys Advantages

relatively fast sampling through random digit dialling

Disadvantages need telephone problem with answering machines, mobile telephones no one at home during the day. Limits calling time to a window of 6-

9 p.m. interview time < 20 minutes

Summary interviewing methods

Steps in designing a surveyDefine survey objectives Identify study population Identify and define study variables Establish plan of analysis Determine sample size Choose sampling method Decide on mode of data collection Design, field test and revise questionnaireTrain interviewers, conduct field workCheck all interview forms for errorEnter, tabulate and analyse the results

Preparing for data analysis

Remember “GIGO” principle “Garbage in - garbage out” The computer cannot make data better than

they are Coding Data entry Data cleaning and editing

Data analysis

Type of data analysis is dependent on the of outcome variable Continuous variables Categorical variables

Points about survey data analysis

Target population in a survey often finite Most software packages assume:

infinite population sizes sampling with replacement both conditions often not upheld in surveys

Software specially developed for surveys SUDAAN (Survey data analysis) Stata Epi Info CSurvey

Ethical issues Informed consent

purpose of study explained? consent given by individual

to be included? Confidentiality

an individual should not be identifiable Service provision

No survey without service (?)

Response rate Dependent on

length and mode of interview educational level perceived threat of questions “warm” versus “cold” interviews

warm = interviewee has prior knowledge about survey Non-responders are almost always different from

responders To reduce non-response bias

follow up non-responders at least once

Use of results

Biggest fault in survey design: Results are not disseminated or used.

Why was the suryey carried out in the first place?