Items and Scales

Measurement Part 2

Objectives

• Characteristics of questionnaire items• Distinguish between single-item measures versus

composite scales, “batteries of items.”• Qualities of measurements

– Validity

– Sensitivity

– Reliability and error

• Examples of scales used in marketing research

Clarifications

• The term questionnaire item is used to denote a single question on a survey, corresponding to a single column in a dataset.

• Scales typically denote sets of questions which become mathematical combinations of survey items.

Scaled-Response Format Questions

• Labeled: All positions, especially mid-point and intermediate positions for the respondent are marked, or bear “labels”

• Unlabeled: No intermediate points are labeled, sometimes positions bear only numbers.

Likert item: Labeled People should shop at local merchants even though the prices may be significantly higher.

1 . . . . . . 2 . . . . . . 3 . . . . . . 4 . . . . . . 5

Stronglydisagree

NeutralDisagreeStrongly

agreeAgree

Numeric differential item: Unlabeled Paying higher prices at local merchants...

1 . . . . 2 . . . . 3 . . . . 4 . . . . 5 Showsintelligence

Showsignorance

Adding too many labels

1 . . . . 2 . . . . 3 . . . . 4 . . . . 5 . . . . 6 . . . . 7

Stronglydisagree

NeutralDisagreeStrongly

agreeAgreeDisagreesomewhat

Agreesomewhat

1 . . . . 2 . . . . 3 . . . . 4 . . . . 5 . . . . 6 . . . . 7

Never IrregularlyRarely AlwaysRegularlySeldom Often

Single-items adequate for measurement?

• Suppose an instructor had single-question exams?• Suppose the ACT (or GMAT) had only 5 possible

scores (similar to A,B,C,D,F grades)?• Suppose the ACT had only 4 questions, 1 each for

mathematics, English, reading comprehension, and science?

These are the issues behind marketing research use of multiple-item scales.

Composite, or Multiple-Item Scales

• Capture the sensitivity to the continuous nature of many subtle differences in between consumers.

• Simultaneously address concerns of:– Accuracy: Just as a 25-question exam can ask

questions evaluating the many topics covered in a course, multiple-item scale can ask the many subtle aspects that underlie a consumer attitude or behavior.

– Consistency: Though we do not administer the same surveys repeatedly to the same consumers, we strive to develop scales that consumers would provide consistent responses to over time.

• All relate to larger issue of measurement error.

Semantic Differential Scale

Indicate your impression of shopping at Dillard’s by checking the box corresponding to your opinion for each pair of descriptions.

Good valueUnfamiliar brandsDistinctive fashionLow qualityHelpful staff

Poor valueBest brandsNo fashionHigh qualityNo staff

Brand Personality: CompetenceHow do the following terms describe your view of the Cingular brand?

Reliable

Secure

Intelligent

Successful

Technical

Confident

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Brand Personality: ExcitementHow do the following terms describe your view of the Cingular brand?

Trendy

Exciting

Imaginative

Up-to-date

Cool

Contemporary

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

Not at all descriptive __:__:__:__:__Extremely descriptive

VALIDITY AND RELIABILITY OF MEASUREMENTS

• Validity: Accuracy– Conceptual Issue– Content/Face Validity– Predictive/Criterion Validity

• Reliability: Consistency– Proportion of statistical error– Test-Retest– Inter-Item Correlation

• Increasing the number of questions (items) can increase both the validity and reliability of a scale.

Reliability

• Conceptually, reliability is the extent that item/scale is free from error.

• Unfortunately, it can only be measured statistically, via split-half and inter-item correlations.

Validity of an item/scale

• Qualitative aspects, assessed by examining the text in the actual items

• Content, or Face Validity, does the text in the questionnaire item(s) correspond to a constitutive definition of the concept being measured?

• “I have bought “Joy” every time I have purchased dishwashing liquid in the last two years.

• “I will buy “Joy” whenever I receive a cents-off coupon.

• “I really notice the television ads for “Joy” dishwashing liquid.

• “I have a bottle of “Joy” in my kitchen cabinet.

• “I plan to purchase “Joy” on my next trip to the grocery store.”

Formative versus Reflective Items

• Formative items may have low inter-item correlation—reliability tests are inappropriate, items may be independent.

“Check the boxes indicating the type of services provided by your current wireless carrier:

“Indicate the importance you place on the following cell-phone services:

Formative Scale Items: Satisfaction

1. “My last flight on Southwest departed on-time.2. “Southwest did not lose my luggage on my last

trip.3. “Southwest has competitive fares.”4. “Southwest ticketing personnel are polite.5. “Southwest has friendly reservation operators.6. “Southwest has ample leg-room for me in coach

seating.7. “I have not been “bumped” from a Southwest

flight in the last two years.”

Additional items related to satisfaction

1. “I know it’s not the airline’s fault when a flight is cancelled.

2. “It upsets me to know others on the same flight have paid a lower price for their seat.

3. “An airline could always be on-time if they made that their priority.

4. “The two-item restriction on carry-on luggage is insensitive to the needs of today’s passengers.”

Reflective Items: Materialism

• “I admire people who own expensive homes, cars, and clothes.

• “Some of the most important achievements in life include acquiring material possessions.

• “I don’t place much emphasis on the amount of material objects people own as a sign of success.*

• The things I own say a lot about how well I’m doing in life.

• I don’t pay much attention to the material objects other people own.*

* Reverse coded

Formative and Reflective Items

• Formative items: Can be combined to measure the multiple aspects of a construct, though not necessary that respondents answer each item similarly.

• Reflective items: Measures a single trait and respondents should answer each item similarly.

Please check the items in your kitchen:

• Electric frying pan• Food processor• Spring-form pan• 9” cake pan• Flour sifter• Hand-held electric

mixer• Measuring cup

• 9” frying pan• 12”+ frying pan• 2-quart sauce pan• Rotary blender• Pressure cooker• Cookie sheet• Set of mixing bowls