questionnaire design and validation

Karim ElghanamFaculty of Medicine, Menoufia

University [email protected]

g September 15, 2015

Questionnaire Development& Validation

2

What’s the Q.?

Is a formalized set of questions to obtain certain information

from certain respondents.

Questionnaire Development

3

Developing a good questionnaire is NOT EASY!

• It takes time, time, and more time.

• You may end up writing multiple drafts.

• It helps to work with others when developing a questionnaire.


4

Some Definitions Before Starting


5

Types of Q.?

1- Self-administrated• Computer

assisted and Web-based

• Paper and pencil2- Interviewer-administrated• Face to face• Telephone


6

• Self-administered questionnaire

Advantages: Cheap and easy to administer Preserves confidentiality Completed at any time No influence by interviewer Allow for instant data coding


7

• Self-administered questionnaire

Disadvantages: Low response rate Questions can be

misunderstood No control by interviewer


8

• Interview-administered Q.

Advantages: Participation of illiterate

people Clarification of ambiguity Quick answers


9

• Interview-administered Q. Disadvantages:

Interviewer bias Needs more resources Only short questionnaires possible

especially on telephone Difficult for sensitive issues


11

1- Formulations of idea (Conceptualization)


Clear Idea Clear objectives ( primary and

secondary objectives ) You will be Strict to them.

Home Message

“Ask what you want to know, not something else”

(Bradburn, 2004)

13

2- Literature Review


Asking others for their own questionnaires !

14

3- First draft


Ideas Roadmap and the begin to change theses idea to questions

Idea Question/s

15

First Draft


16

4- Condensed Literature Review..!


Another draft more specific than the first one

17

Another Draft


18

5- Detect .. ?


1. Study design 2. Population 3. Sample size 4. Type of the Q. will you

use ?

19

6- Design Questionnaire


20

Types of questions ?

1- Open-ended question

2- Closed-ended question

3- Contingency questions.


21

1- Closed questions

You ask the respondent to choose, among a possible number of answers, the response that most closely represents his/her viewpoint.


22

Advantages of closed question

• The respondent is restricted to a certain set of responses,

• They are easy to answer ( no hesitations ),

• Less time consuming,• Easy to code, and Questionnaire Development

23

You will force the respondent to choose between specific answers … BIAS

You offer answers that otherwise would not have come to his mind,

They don’t allow for creativity and generating ideas,

Disadvantages of closed question


24

They do not permit the respondent to qualify the chosen response or express a more complex or subtle meaning,

There is may a tendency for the respondent to tick systematically either the first or last answer,

May Select what may be considered as the most socially desirable response,



25

To answer all items in a list in the same way



26

2- Open-ended questions

Free-response questions are not followed by any choices and the respondent must answer by supplying a response, usually by entering a number, a word, or a short text.


27

Advantages of open-ended They allow respondents to express

their ideas spontaneously in their own language.

They are less likely to guide the

answer than closed questions .

They can add new information when there is very little existing information available about a topic.


28

They may be difficult to answer

They require effort and time from the respondent,

They require the development of a system of coded categories with which to classify the responses..!

Can’t be coded !

Disadvantages of open-ended


29

They require the respondent to have some degree of writing ability

Disadvantages of open-ended


30

3- Contingency questions

Is a special case of a closed-ended question because it applies only to a

subgroup of respondents…!


31 Questionnaire Development

Types of closed questions ?

Scales ?

32

6- Design Questionnaire


33 Questionnaire Development

Wording of the question

Logic flow of the questions

Simple language Mother language is the

best Use concise and clear

wordsAnd Avoid …..

34

Avoid


35

1- Double-barreled questions:

Single questions that ask for two things and therefore require two

answers. 1. Do you have your own table or your own room to do

your homework?2. Do you think it is good idea for children to study

geography and history in primary school?Questionnaire Development

36

2- Double negatives

Either in questions or answers

1.All the following aren’t used in the TTT of … Except ? :D


37

3- Avoid overlapping response categories

Under 20 120-30 2 30-40 340-50 4 50-60 560 … or more

How old are you?

Under 20 121-30 2 31-40 341-50 4 51-60 561 … or more


38

4- Leading questions

Would you say that you are not in favour of school on Saturday morning??


39

5- Overload the respondent’s memory

Would you say that you are not in favour of school on Saturday morning??


40

6- Long questions

Generally, it is recommended to hold questions to 25 words or less.

If a longer sentence is used then it should be broken up so that there will be

several shorter sentences.


41

Avoid 7- Hypothetical questions • Prediction of behavior and future

• “Would you use this resource in your class if it were available..?”

• People are generally poor predictors.


42

A new Draft


Testing

44

1.Informal tests2.Experts review3.Small-scale tests (pre test) 4.Functional or technical testing5.Large-scale pilots and trials

Testing

45

1.Informal tests

Testing

46

A new Draft


47

1. …..2. Experts review

Testing

48

A new Draft


49

Pre-pilot

Focusing group

Pre-testing the questionnaire


50

Aims

Pre-testing the questionnaire


51

a. Is each item producing the kind of information needed? b. What role is the item going to play in the proposed analysis? c. Are the questions meaningful to the respondents? d. Are respondents easily able to understand the items? e. Can respondents use the response format for each item?


52

g. Was the question order logical and did the interview flow smoothly? h. Did some parts of the questionnaire arouse suspicion? I. Did other parts of the questionnaire seem repetitive or boring? j. Were interviewers able to read the questions without difficulty? k. Were respondents able to follow all instructions? l. Was the questionnaire too long?


53

Select a sample similar in socio-economic background and geographic location to the one that will be used in the main study.

This sample will not be included in the final survey.

Not to be a convenience sample.

Character Of this group:


54

A new Draft


55

1.……2.……3. ……

4.Functional or technical testing

Testing

56

A new Draft


57

1. …….2. …….3. …….4. …….

5.Large-scale pilots and trials

Testing

58

A new Draft


59

Reliability and validity

Validity: Concerns the degree to which a question measures what it was intended to measure and not something else.• Content (or face) validity • Empirical (or predictive) validity• Concurrent validity Questionnaire Development

60

Reliability: concerns the consistency of a measure. That is, the tendency to obtain the same results if the measure was to be repeated by using the same subjects under the same conditions.

Kappa coefficient !


61

Covering letters and introductory paragraphs

You must explain the objectives of the survey,

In an interview, one of the tasks of the interviewer is to persuade the

respondent to co-operate.

In a self-administered questionnaire, the covering letter is the only

instrument for overcoming resistance. Questionnaire Development

62

Identify the organization conducting the

study.

Explain the purpose of the study.

Assure the respondent that information

provided will be managed in a strictly

confidential manner.

Covering letter Items


63

Explain WHY it is important that the respondent should complete the questionnaire.

Brief detail on how the respondent was selected (for example, ‘Your name was randomly selected ....’).

Expression of appreciation for the respondent’s help.



64

Estimate of questionnaire completion time.

Provide the name and contact numbers of the Principal Researcher(PI).



[email protected]

.edu.eg

01068950053

FB: Karim Elghanam

Contact Me 65

Hope Happiness finds you when you don’t Expect to find it .

mailto:[email protected]

mailto:[email protected]

questionnaire design and validation

Science