questionnaire design. qualities of a good questionnaire should be client-friendly: it should capture...

QUESTIONNAIRE DESIGNQUESTIONNAIRE DESIGN

QUALITIES OF A GOOD QUALITIES OF A GOOD QUESTIONNAIREQUESTIONNAIRE

• Should be client-friendly:It should capture the information required to meet the study tasks in as accurate a form as possible

• Should be respondent-friendly:It should be easy to answerIt should not capture info that is not needed since fatigue is a very real factor pulling down quality

• Should be interviewer-friendly

• Should be analysis-friendly

SCREENING QUESTIONNAIRES, SCREENING QUESTIONNAIRES, RQS, LISTING QUESTIONNAIRESRQS, LISTING QUESTIONNAIRES

• Screening and RQs are just aimed at finding whether a particular respondent is eligible or not.

• Listing questionnaires are used when penetration is being estimated, and sampling is correspondingly that much better controlled.

• The difference between screening and RQ is just whether the main interview is happening later or right away

KINDS OF QUESTIONSKINDS OF QUESTIONS

THREE KINDS OF QUESTIONSTHREE KINDS OF QUESTIONS

• Behaviour questions

• Opinion questions

• Classification questions

BEHAVIOUR QUESTIONSBEHAVIOUR QUESTIONS

• Record facts, not opinions

• Can generally be assumed to be accurate

Important exceptions are:

- when memory is stretched too far“the last 20 times you ate / drank anything other

than at regular meals, what did you eat / drink?”

- when the subject is sensitive e.g. money, personal habits, etc.

BEHAVIOUR QUESTIONS…Contd.BEHAVIOUR QUESTIONS…Contd.

Examples

• Have you ever….?• Do you ever….?• When did you last…?• Which do you do most often….?• Who does it….?• How many…?• In what way do you do it…?

OPINION QUESTIONSOPINION QUESTIONS

• Not such an easy area

• Attitude measurement has spawned a lot of literature and conflicting views

• Addressed more in section “Scaling”

• Examples:- What do you think of….?- Why do you….?- Do you agree or disagree….?- How do you rate…?- Which is the best (or worst) for….?

CLASSIFICATION QUESTIONSCLASSIFICATION QUESTIONS

• Help to understand market in terms of demographic segments

• Examples are:

– Monthly household income– Monthly household expenses– Occupation– education– No. of children at home– SEC– etc.

INTERVIEWER FRIENDLYINTERVIEWER FRIENDLY

SOME PRINCIPLES…..SOME PRINCIPLES…..

• Plenty of space for recording answers to open - ended questions

• Plenty of space for recording translations for those responses

• Clear flow instructions (GOTO…, CONTINUE, ASK ALL, ASK ONLY IF….OTHERWISE)

• Whenever there is a “SHOW CARD…” question, print the card option on the questionnaire. This will enable checking later if there is a doubt about the way the question has been administered.

SOME PRINCIPLES…..(Contd.)SOME PRINCIPLES…..(Contd.)

• The questionnaire brief note is a crucial aid to quality fieldwork. The purpose is to explain the way a question is to be ahead, the way probing is to be done, the way responses are to be recorded. For example,

Q Statements Brands1 2 3 4 5

12345 Associations OR Ratings on 5-pt

scale

…finish one statement for all the brands, and then move on to the next statement.

ANALYSIS - FRIENDLYANALYSIS - FRIENDLY

SOME PRINCIPLES (Discuss…)SOME PRINCIPLES (Discuss…)

Example: TOM SpontAided

1(1) 1(2) 1(3)

2(1) 2(2) 2(3) 3(1) 3(2)

3(3)

ANALYSIS - FRIENDLYANALYSIS - FRIENDLY

KNOWLEDGE OF RESPONDENTKNOWLEDGE OF RESPONDENT

• Is an issue when doing B2B research (does any one person know all about the decision), children’s product research (does the mother answer on behalf of the child)

• Is also an issue when doing customer satisfaction research. Different people within the respondent organisation / household can answer about different aspects - purchase process, product in use, service.

ORDER OF QUESTIONSORDER OF QUESTIONS

• Clearly falls more into the ‘art’ part than the ‘science’ part, but there are some general principles that are proven to work.

• By and large, classification questions, unless required for screening purposes, towards the end of the questionnaire.

• It is a good idea to start with easy and straight forward questions, go on to more difficult questions, and then finish with easy questions

• This means that we normally start with awareness questions, go on to behaviour questions such as products and brands used, bought, etc., then go on to opinion questions such as rating of brands, and finish with classification questions and media habits.

WORDING AND TYPE OF QUESTIONSWORDING AND TYPE OF QUESTIONS

• It is a rule-of-thumb that the very factual questions should be short, while the questions that require opinion answers tend to get better answers if the wording is not very abrupt. This explains why you will often see brand rating questions having a rather long introduction. The longer wording serves to explain and to make the questions less threatening.

WORDING AND TYPE OF WORDING AND TYPE OF QUESTIONS…Contd.QUESTIONS…Contd.

• Avoid ambiguous words such as ‘frequently’. What is frequent to X may be infrequent to Y.

• Avoid double-barreled statements (e.g.) “The economy is down and the PM is not doing enough to help”. If this is subjected to an agree/disagree question, and the respondent disagrees with the first and agrees with the second part of the question, then he will not know how to answer

• Be specific (e.g.) It is better to ask “Have you traveled by air in the past one year”than “have you traveled by air in the last year”

• Avoid jargon: this brand is meant for the ‘discerning consumer’. Hardly anyone knows what that means.


• Avoid slang and colloquialism: They may not be valid throughout the country or across all SEC classes.

• Avoid Bias: What actually is this?Anything that leads the respondent towards one option i.e., anything that interferes with the respondent’s freedom to choose the option onhis / her own.

Example:Would you agree that brand X is very good? ….. NO

What would you say about brand X?


Very good : 1Quite good : 2Neither good nor bad : 3Quite bad : 4Very bad : 5

• Try and avoid negatively worded questions. They confuse the respondent.


• For most classifications questions, use the standard wording. Always write the wording for the classification question. The list of standard wordings that can be readily used is given below.

• Standard wording- HW- CWE- SEC- MHI

The housewife is the person in the household who is incharge of all purchases for the kitchen

The chief wage earner is the person who contributes the maximum money towards the household expenses

TRANSLATIONTRANSLATION

• Try and think in Hindi/Bengali/Malayalam/Tamil etc

• Try this exercise: Get the exact equivalent in the local language of….- premium brand- discerning consumer- brand preference- ad gives a you a warm feeling- cool brand of jeans

• Always check questionnaire translation

TRANSLATIONTRANSLATION

• Be sensitive to the fact that field sometimes uses vernacular words even for those English words that are commonly understood e.g. technology, computer etc.

• Try and get a back translation for each questionnaire

OPEN-ENDED VS CLOSE-ENDEDOPEN-ENDED VS CLOSE-ENDED

• The real problem with open-ended questions is that we do not get high quality probing

• Probing needs to be of two kinds: further and deeper

• Example:Q : How likely are you to purchase this brand ?5 point scale

Q : TO THOSE WITH NEGATIVE INTENTION :Why do you say that you won’t buy this brand ?

OPEN-ENDED VS CLOSE-OPEN-ENDED VS CLOSE-ENDED….Contd.ENDED….Contd.

R1 : Because I am not interested

R2 : Because I do not like feature A

R3 : Depends on price R1 Probe R2 Probe R3 Probe------------ ------------- ----------Why are you not Is there any other

Build into questioninterested ? reason ?

Assuming the price is acceptable

to you,or is going to

be Rs.X.

PILOTINGPILOTING

• Always pilot test the questionnaire

• Pilots should be done with specified target group

• An interviewer should do the pilot

• The researcher should sit through the pilot

• What should the pilot check?- does the respondent understand the questions?- are the instructions clear to the interviewer?- are the recording formats convenient and efficient?- how long is the interview taking?- options for close ended questions

SCALINGSCALING

KINDS OF SCALESKINDS OF SCALES

• NominalNon metric

• Ordinal

• IntervalMetric

• Ratio

KINDS OF SCALES…..Contd.KINDS OF SCALES…..Contd.

• Nominal : Mode

• Ordinal : Median, Mode

• Interval : Mean, Median, Mode

• Ratio : Mean, Median, Mode


• NominalThe scale points simply identify the responsee.g., Male : 1 Mumbai : 1

Female : 2 Chennai : 2Delhi : 3

• OrdinalThe scale points indicate rank-orderinge.g., Freshers : 1

Sophomore : 2Final years : 3

No indication on amount of difference


• IntervalThe scale points tell us how far apart the responses are:e.g., temperature scale - Celsius, FarenheitThe difference between scale points are equal

• RatioInternal scale + absolute zeroe.g., Kelvin scale of temperature

HeightsWeightsQuantity consumed etc

SCALES NORMALLY USEDSCALES NORMALLY USED

• Verbal rating scales

• Ranking scales

• Agree-Disagree scales

• Association scales

• Numerical rating scales- Marks out of 10- Points from 1 to 10These are useful in comparative product test research. They are to be avoided in monadic measurement studies.

VERBAL RATING SCALESVERBAL RATING SCALES• Examples are

1. How satisfied are you with X?Totally dissatisfied 1Substantially dissatisfied 2Somewhat dissatisfied 3Neither satisfied nor dissatisfied 4Somewhat satisfied 5Substantially satisfied 6Totally satisfied 7

2. How likely are you to buy ABC?Definitely will not buy 1Probably will not buy 2May or may not buy 3Probably will buy 4Definitely will buy 5

VERBAL RATING SCALES….Contd.VERBAL RATING SCALES….Contd.

3. How would you rate the strength of the fragrance of this perfume?

Much too strong for my liking 1Somewhat too strong for my liking 2Just right 3Somewhat too weak for my liking 4Much too weak for my liking 5

4. What would you say about the taste of this biscuit?Excellent 1Very good 2Quite good 3Fair 4Bad 5Very bad 6


5. Which statement on this card best describes your opinion about statement?

Disagree strongly 1

Disagree somewhat 2

Neither agree nor disagree 3

Somewhat agree 4

Strongly agree 5

• Note that we generally start with the negative point in the scale. This is because of a belief that respondents are less likely to feel compelled to be polite.

• Also note that all the scales are balanced except for scale no. 4. The reason for scale 4 not being balanced is the belief that respondents tend to give positive - sounding answers even if they don’t really feel positive about the brand (This is perhaps an Indian phenomenon)


Having more positive points than negative in the scale can compensate for this by forcing the respondents to choose a lower positive when they want to be polite.

questionnaire design. qualities of a good questionnaire should be client-friendly: it should capture...

Documents