CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED

QUESTIONS

CHAPTER 5: CONSTRUCTING OPEN- AND CLOSED-ENDED

QUESTIONSDamon BurtonDamon Burton

University of IdahoUniversity of Idaho

What is the difference between open- and closed-ended questions?

QUESTION-TYPE DEFINITIONS

•Open-Ended Questions – are questions in which the choice of responses are not specified and respondents must generate their own responses.

•Close-Ended Questions – are items in which responses choices are specified and respondents must choose one of

the available alternatives.

TYPES OF OPEN-ENDED QUESTIONS

• Descriptive Questions – in which respondents are asked to provide in-depth information on the topic of the question.

• Numerical Response Questions – respondents must provide numerical data such as dates, frequency, monetary value, count, amount or scalar value.

• List-of-Items Questions – respondents must provide a list such as grocery stores they frequent, brands of clothing, or major purchases they’d like to make.

We’re going to discuss a number of different question types and show you how to best write each type of question.

NUMERICAL RESPONSE GUIDELINES

1.Ask for the specific unit desired in the question stem (5.1).

2.Provide answer spaces that are sized appropriately for the response task (5.2).

3.Provide unit labels with the answer spaces (5.1).

4.Specificity enhances response accuracy.

5.1 NUMBER BOX QUESTIONS

5.2 IMPACT OF ANSWER SPACES ON RESPONSES

DESCRIPTION & ELABORATION GUIDELINES

1.Provide extra motivation to respond (5.3).

2.Provide adequate space for respondents to completely answer the question (5.3).

3.Use scrollable boxes on web surveys (5.3).

4.Consider programming probes to open-ended responses in internet surveys (5.4).

5.3 DESCRIPTION & ELABORATION QUESTIONS

= 4/DAY

5.4 PROBING OPEN-ENDED QUESTIONS

You’ll probably use more closed- than open-ended questions, so here are guidelines for how to write good items.

CLOSED-ENDED GUIDELINES

1.State both positive and negative sides in the question stem when asking either/or questions (5.5).

2.Develop lists of answer categories that include all reasonable possible answers (5.6).

3.Develop lists of answer categories that are mutually

exclusive questions (5.7).

5.5 INCLUDE POSITIVE & NEGATIVE SIDES

= 4/DAY

5.6 MUTUALLY EXCLUSIVE

CATEGORIES

= 4/DAY

5.7 ORDERING OF RESPONSES

= 4/DAY

CLOSED-ENDED GUIDELINES

4. Maintain spacing between answer categories that is consistent with measurement intent (Figure 5.8).

5. Ask respondents to rank only a few items at once rather than a long list (Figure 5.9).

• Ranking questions are difficult to understand and complete correctly.

• Only have respondents rank top 3-4.

• Pair all possible options and calculate overall rankings during analysis.

5.8 SPACING RESPONSE OPTIONS

EVENLY

= 4/DAY

5.9 CLOSED-ENDED WITH UNORDERED

RESPONSES

= 4/DAY

CLOSED-ENDED GUIDELINES

6. Avoid bias from unequal comparisons (Figure 5.10).

7. Randomize response options if there is concern about order effects (Figure 5.11).

8.Use forced-choice questions instead of check-all-that-apply questions (Figure 5.11).

5.10 AVOIDING UNEQUAL COMPARISON

BIAS

= 4/DAY

5.11 UNORDERED RESPONSE OPTIONS

= 4/DAY

5.12 COMPARISON OF RESPONSE RATES

= 4/DAY

CLOSED-ENDED GUIDELINES

9. Use differently shaped answer spaces (circles vs squares) to help respondents distinguish between single- and multiple-answer questions (Figure 5.13).

5.13 QUESTION TYPE & SHAPES OF ANSWER

SPACES

= 4/DAY

How do the type of analyses you plan to do influence your response choices?

ORDINAL CLOSED-ENDED GUIDELINES

10. Choose appropriate scale length of 4-5 categories (Figure 5.14).

• Typically you must have a minimum of 5 choices on a Likert scale to assume interval data and utilize inferential statistics.

• Even versus odd-number scales are based on whether you want to allow respondents

to choose a neutral response.

5.14 SCALER QUESTIONS AND RESPONSE

OPTIONS

= 4/DAY

ORDINAL CLOSED-ENDED GUIDELINES

11. Choose direct or construct-specific labels to improve understanding (Figure 5.15).

12. Provide scales that approximate the actual distribution of the characteristic in the population. • Typically respondents assume that the

midpoint response category represents the middle of the distribution in the population.

How does the reason for conducting the survey influence response options (e.g., academic research versus consumer satisfaction)?

ORDINAL CLOSED-ENDED GUIDELINES

13. Provide balanced scales where categories are relatively equal distance apart conceptually (Figure 5.16).

14. Consider how verbally labeling and visually displaying all response categories may influence answers (Figure 5.17). • Dillman suggests that most surveyors prefer fully

labeled scales that range from 2-5 choices.

• Conversely, researchers tend to prefer numerical scales of 5-9-point Likert scales (see examples).

5.16 EVENLY-SPACE CONCEPTUAL CATEGORIES

= 4/DAY

5.17 VERBAL VERSUS NUMERICAL LABELS

= 4/DAY

ORDINAL CLOSED-ENDED GUIDELINES

15. Carefully evaluate the use of numeric labels and their impact on measurement (see examples).

• Numbers reinforce the interval nature of data.

• Dillman contends that numbers add to processing time, but this depends on whether questions use same response categories or change categories from question to question.

• Number consistently across questions.

ORDINAL CLOSED-ENDED GUIDELINES

16. Align response options vertically in one column or horizontally in one row and strive for equal distance between

categories (Figure 15.18).17. Place nonsubstantive options at the

end of the scale and separate from substantitive options (Figure 15.19).

• Be very careful about using “don’t know” or “no opinion” categories.