existing and new approaches for analysing data from check all that apply questions

Existing and new approaches for the analysis of CATA data

Michael Meyners Procter & Gamble B. Thomas Carr Carr Consulting John C. Castura Compusense Inc.

know if products are different?

characterize products?

know how attributes are correlated?

know what drives consumer liking?

focus reformulation efforts?

With CATA data, can we…

Are 2 products different?

Sign Test (exact)

McNemar’s Test (approximation)

Selected Not Selected

Selected a b

Not Selected c d

Product B

Pro

du

ct A

Are 3 (or more) products different?

Cochran’s Q Test (approximation)

extends McNemar’s Test

Testing strategy

Attributes Products Test 1 All 1 2

Cochran’s Q Test Sign Test

Significant?

Testing strategy

Attributes Products Test All All 1 All 1 2

? Global Test ? Cochran’s Q Test Sign Test

Significant?

Significant?

Global Test

Test Statistic: Sum of 𝑄 statistics

Approximate Test

Chi-square distribution formed using property of additivity

Problem: null distribution incorrect

(chi-square distributions are not independent)

Global Test

Solution:

Randomization test (quasi-exact):

null distribution is valid because it is based on repeated and appropriate reshufflings of observed data

Testing strategy

Attributes Products Test All All 1 All 1 2

Randomization Test Cochran’s Q Test Sign Test

Significant?

Significant?

1 2 3 … Ideal Total

Fresh 89 70 71 … 142 642

Warm 6 7 4 … 90 135

Crusty 24 4 32 … 47 193

Soft 76 95 75 93 570

Sweet 102 18 30 85 466

… … … … … ... …

Total 2169 1144 1539 … 2579

Contingency table

Bar charts

Problem: This is a mess.

We need a visual summary of this data.

Bar charts

Correspondence Analysis

Reduces dimensionality of contingency tables (Think: PCA for ordinary data)

Attributes

Pro

du

cts


Could use…

Chi-square distances

Sensitive to attributes with low counts

Hellinger distances

Low counts not a problem


Sweet

2

4

3

1

5 6

Molasses

Exciting

Tempting

Satisfying

Tasteful Fullness Comfort

Dense

Firm

Moist Warm

Homemade

Fresh Chewy Traditional

Soft Springy Wheat

Brown

Healthy

Nutritious

Crusty

Texture

Coarse

Sesame

Rustic Nutty

Crunchy

Grainy Seeds

Component 1 (79.2%)

Co

mp

on

ent

2 (

11

.9%

)

Component 1 (79.2%)

2 3

1

5 6 Traditional

Soft Springy Brown

Nutty

Crunchy Grainy

Seeds

4


Can only look at 2 (or 3) dimensions at once Problem: True relationships between products and attributes might be obscured.

Multidimensional Alignment (MDA)

Solution: Look at each product individually.

Positive association with these attributes

Negative association with these attributes

φ-coefficient

Measures correlation between 2 binary attributes

ClassicalMDS on matrix of 𝟏 − 𝝋 distances

Fresh

Warm Crusty

Soft

Moist

Nutty

Tempting Sweet

Dense

Texture

Wheat

Exciting

Crunchy

Homemade

Seeds

Satisfying

Coarse

Nutritious

Molasses

Tasteful Rustic

Traditional

Springy

Sesame

Firm

Fullness

Healthy

Grainy

Comfort

Brown

Chewy

Penalty-Lift Analysis

Problem: What is the relationship between what consumers perceive and their hedonic response? Solution: Look at elicitations of attributes and association with liking

change in liking (attribute elicited vs. not elicited)

Comparison with Ideal

Problem: How to modify a product to delight consumers? Solution 1: Close gap between product and ideal (accounting for natural variability!)

Difference in elicitation rates

Product – Ideal

Penalty Analysis

Problem: How to modify a product to delight consumers? Solution 2: Focus on the most relevant attributes for consumers’ hedonic responses

% consumers

Higher Relevance for Many

Consumers

+2

+1

0

10 30 50 40 20

Penalty analysis based on “must have” attributes

Liki

ng

Fresh Warm

Crusty

Soft

Moist

Chewy

Nutty

Tempting

Sweet

Grainy

Dense

Texture

Healthy

Exciting

Crunchy

Homemade

Brown

Seeds

Satisfying

Coarse

Fullness

Nutritious

Molasses

Tasteful

Rustic Traditional

Springy

Sesame

Wheat

Comfort







Yes!

Thank you for your attention!

For more information see the following publication and references therein:

Meyners, M., Castura, J.C., Carr, B.T. (2013). Existing and new approaches for the analysis of CATA data. Food Quality and Preference, 30, 309-319.

B. Thomas Carr Carr Consulting

Michael Meyners Procter & Gamble

John C. Castura Compusense Inc.

existing and new approaches for analysing data from check all that apply questions

Science