Fayssal M Farahat, MD, MSc, PhD

Consultant and Assist Professor, Public Health

Infection Prevention and Control, KAMC

King Saud bin AbdulAziz University for Health

Sciences (KSAU-HS)

Data Collection Methods and

Questionnaire Design

Validity of the results depends on

the quality of these instruments

• Questionnaires

• Interviews

Open-ended

– Free to answer with fewer limits

– Understand a concept as respondents express it

– Can be used as basis for more structured items

in a later phase

Useful when it is important to hear what

respondents have to say in their own words

Closed-ended

– Quicker

– Easier to answer

– Easier to tabulate and analyze

– Useful in multi-item scales designed to produce

a single score

Closed-ended

• Lead respondents in certain directions

• Do not allow them to express their own, potentially unique, answers

Closed-ended May not include an answer that is most appropriate for a particular respondent

• Pretest using an open-ended version of

the questions.

• When answers do not include all possible

options, include an option such as: – “other (please clarify) or

– none of the above.

• When a single response is desired, possible

responses should be mutually exclusive (the

categories should not overlap).

Formatting

• On the cover, describe: – Purpose of the study

– How the data will be used

– How to fill out

• Provide an example

• Similar information at the beginning of an interview as part of obtaining consent.

Formatting

• Headings or short descriptive statements

for grouped questions concerning major

subject areas

Formatting

• Emotionally neutral questions

• More sensitive questions

• Personal characteristics such as income

Formatting

• If the format differs from that of other

questions on the instrument, instructions

must indicate clearly how to respond

VISUAL DESIGN

• As simple as possible (complex design)

• Plenty of space (neat format)

• Longer is better than crowded

• Enough space for answering

• Large font size and high contrast (black on white)

Wording

• Every word can influence the validity and

reproducibility of the responses.

• The objective is to:

– Encourage accurate and honest responses

without embarrassing or offending the

respondent.

Simplicity

• Use simple, common words.

• Use simple grammatical structure.

• Avoid technical terms.

• Over-the-counter medications.

• Drugs you can buy without a doctor’s

prescription.

Scales and Scores

• Difficult to assess concepts like quality of

life from single question

• Dis-advantage of multi-items they produce

results that are difficult to understand (e.g.,

quality of life= 46.3)

• Likert scales are commonly used to

quantify attitudes, behaviors, and domains

of health-related quality of life.

• For each item, circle the one number that best

represents your opinion:

Item Strongly

agree

agree neutral disagree Strongly

disagree

•Smoking in public

places should be illegal

•Advertisements for

cigarettes should be

banned

•Public funds should be

spent for antismoking

campaigns

Likert scale

Scales and Scores

Make a list of variables

• List of variables

• Role of each item (predictor, outcome,

potential confounder)

Collect existing measures

• Assemble a file of questions or

instruments that are available for

measuring each variable.

Save time and allow results to be

compared with those of other

studies.

Collect existing measures

• Existing instruments may not be entirely

appropriate for the question or the

population or they may be too long.

• It is ideal to use existing instruments

without modification.

Collect existing measures

• If too long, check for shorter versions.

• If some of the items are inappropriate

(different cultural group), you may delete,

change or add few items.

Collect existing measures

• Deleting items:

– change the meaning of scores.

– diminish its reproducibility or its sensitivity to

detect changes.

may be possible to delete

sections or subscales

Collect existing measures

• When no adequate

measure can be found.

• Use good judgment and basic

principles of writing good

questions.

FOCUS GROUP /

INTERVIEWS

Compose a draft

• The first draft should be formatted just as

final questionnaire would be.

Revise

• Read carefully.

• Answer each question as if you are a

respondent.

• Try to imagine possible ways to

misinterpret questions.

• Identify words or phrases that might be

confusing or misunderstood or complex.

• Include colleagues and experts in

questionnaire design review.

• ONLY Questions that are essential to

answering the main research question.

• Might be useful to sketching out the final

tables.

• If in doubt, usually best to leave it out.

Shorten the set of instruments for

the study

Pretest

• To refine, and time the instrument.

• Whether each question produces an

adequate range of responses.

• To test the validity and reproducibility of

the instrument.

Validate

• Starts with choosing questions that have

face validity, a subjective judgment that

the items are assessing the characteristics

that the investigator wants to assess.

Validate

• Accuracy can be compared with gold

standard measurements of the condition of

interest.

• The process of validating new instrument

is time consuming and expensive, and

worthwhile only if existing instruments are

inadequate for the research question or

population to be studied.

Validate

Questionnaires

• Uniform way to administer simple

questions.

• Less expensive.

• Do not require much time.

Interviews

• Better for complicated questions that

require explanation or guidance.

• You make sure that all answers are

completed.

• Necessary when participants have

variable abilities to read and understand

questions

• Can be standardized but still will be

administered a little different each time.

Costly, time consuming.

Influenced by the interviewers.

Interviews

Qs / Is

• Susceptible to errors because of imperfect

memory.

• Can be affected by the respondent’s

tendency to give socially acceptable

answers.

When both methods are feasible, compare

between cost-effectiveness and complexity of

the questions to be answered.

Interviewing

• Standardizing the interview procedure

from one interview to the next is the key to

maximizing reproducibility.

• Uniform wording.

• Uniform nonverbal signals.

• Uniform tone of the voice.

• Write the interview in language similar to

common speech.

• Follow up respondent’s answers to

encourage to give an appropriate answer=

probing. • How many cups of coffee do you drink on a typical day.

• Answers: I am not sure, it is different from day to day

Interviewing

Do the best you can,

Tell me approximately how many

you drink on a typical day?

Questionnaire

• In person

• Mail (reach broader population but less to

be returned).

• Email (immediate response, easily enter

into database).

• Web site (can produce very clean data,

automatically checked for missing and out-

of-range values, errors are pointed to the

responders.