Cooperative Respondents,Enlightened Clients,

and Other Research Myths

Cooperative Respondents,Enlightened Clients,

and Other Research Myths

CMAG, March 13th, 2008

Respondent Cooperation(or more specifically, lack there of)Respondent Cooperation

(or more specifically, lack there of)

CASRO – 10/02 CASRO – 10/02 Respondent Cooperation Summit – 11/06Respondent Cooperation Summit – 11/06 Quirk’s – 1/07Quirk’s – 1/07

CMAG Challenge – 11/07CMAG Challenge – 11/07

We get that there’s a respondent We get that there’s a respondent cooperation issue. cooperation issue.

Don’t just revisit the problem. Don’t just revisit the problem. Help us find some kind of a solution?Help us find some kind of a solution?

Great Ideas We Won’t Discuss Great Ideas We Won’t Discuss

Improved Sampling Procedures Improved Sampling Procedures

Better IncentivesBetter Incentives

Proprietary PanelsProprietary Panels

Reinstituting “Non-Response Work”Reinstituting “Non-Response Work”

More Engaging AdministrationMore Engaging Administration

A Novel Thought A Novel Thought

Can We Create a Win for Respondents by:Can We Create a Win for Respondents by:

encouraging their cooperation with more engaging encouraging their cooperation with more engaging surveys?surveys?

Might This Also Create a Win for Clients:Might This Also Create a Win for Clients:

because more engaging surveys deliver more because more engaging surveys deliver more enlightening results?enlightening results?

One Ecosystem – Three SpeciesOne Ecosystem – Three Species

Clients, respondents, and the research community are inextricably entwined.

A win/win model will benefit both clients and respondents but we’re the biggest winners.

It is therefore not surprising that the problem is ours to resolve.

Respondents – the supply of information

Clients – the demand for information

Market Research - the brokers of information

RespondentsHow Do We Abuse Them?

RespondentsHow Do We Abuse Them?

The Seven Deadly Sins of Market Research Design

1.Client Directed Methodology2.Lack of Focus (Omnibus Studies)3. Inappropriate Targeting4. Insulting Intelligence5. Inept Construction6.Exposed Game 7.Brutal Length

Applying the “Groucho Marx” test

Recognizing their priorities regarding surveys Understanding that they like to please

Providing interesting construction Delivering tolerable length surveys

Volunteering as “Guinea Pig #1”

RespondentsHow Might We Do Better?

RespondentsHow Might We Do Better?

Folks who: need to know a lot tend to overestimate what they know …

about their business often overestimate what they know …

about market research are native leaders

ABOVE ALL: Clients are decision makers

ClientsWho are They?

ClientsWho are They?

Clients want insight— not just information.

““Don’t drown me in data – just Don’t drown me in data – just tell me what I need to know.” tell me what I need to know.”

Delivering such insight is an Delivering such insight is an easy promise to make but not so easy promise to make but not so easy to keep. easy to keep.

ClientsWhat Do They Seek?

ClientsWhat Do They Seek?

Insight, like beauty, is in the eye of the beholder …

and the beholder is your client.

Not new?

Not needed?

Not insight!

So What is Insight?So What is Insight?

And the first fundamental requirement for generating Client Insight is?

Cooperative Respondents!

Designing to Insight Clientswithout Inciting RespondentsDesigning to Insight Clients

without Inciting Respondents

Ever been asked any of these questions?

1. How come my tracker seldom gives me anything new or useful?

2. What actions can I take with the tracker feedback regarding price perceptions?

3. Speaking of price, is there any reliable methodology that can help me price my product?

4. How about ways to better evaluate programs?

In the time remaining, let’s see how many of these issues we can address with approaches meeting our new specifications:

Considerate of the respondent yet Insightful for the client.

Conventional ShortcomingsConventional Shortcomings More respect for longitudinal consistency

than current relevancy Limited argument for attribute refreshing Measurement calcification (MBOs) Surveys are brutally long and generally dull NOTHING CHANGES And lots more

Relevant Space Methodology Relevant Space Methodology

Brand Tracking System

A Win/Win Example

Brand Tracking System

A Win/Win Example

Respondent-levelRelevant Space

Att

ribu

tes

Respondent selects attributesto demonstrate relevance

Brands

Most relevant brandsare then determined

RELEVANTRELEVANTSPACE:SPACE:

Providing a handful of ratings on brands you know and issues you care about.

COMPREHENSIVECOMPREHENSIVEMATRIX:MATRIX:

Att

ribu

tes

Respondent rates all brandson all attributes

Brands

Providing hundreds of ratings on brands and issues of variable interest.

Respondent Respect Relevant Space Methodology

Respondent Respect Relevant Space Methodology

Client InsightKnowing What Matters

Client InsightKnowing What Matters

Insights

1. Knowledgeable respondents “know what matters”

2. Experiential attributes play key role in developed markets

3. Emotive attributes serve as surrogate for experience elsewhere

4. The less you know, the more that brand matters

5. Developed markets can guide emerging markets

Client InsightAttribute Importance to Brand Choosers

Client InsightAttribute Importance to Brand Choosers

Cisco Choosers

Competitive Choosers

= Cisco leads all other brands in that purchase attribute

= Cisco is part of the leadership group for that attribute

= Cisco neither leads nor trails in that attribute

= Cisco is part of the trailership group for that attribute

= Cisco trails all other brands in that attribute

15 17 13 8 112

16 18 1213 20 1419

Many Ease Issues

Many Expertise Issues

16

02

23

11

22

17

04

06

28

21

16

20

07

25

03

1008

12

27

01

18

2419

26

05

14

09

13

15

Derived Importance

Sta

ted

Imp

ort

ance

Key DriversKey DriversHigh Stated/High Derived

Hidden PrioritiesHidden PrioritiesLow Stated/High Derived

Extra CreditExtra CreditLow Stated/Low Derived

Minimum RequirementsMinimum RequirementsHigh Stated/Low Derived

Client InsightStated & Derived Importance

Client InsightStated & Derived Importance

The Pricing ConundrumWhy It’s a Frustrating AttributeThe Pricing ConundrumWhy It’s a Frustrating Attribute

Interesting commentary from the past:Can’t you tell me something other than price?Don’t worry about price. It’s not a driver!

It’s a tough attribute to capture:We treat it like “just another attribute” when in reality it’s comparative to the entire set of benefit attributes.Even when treating it like “just another attribute”, it’s tough to properly communicate.

Maybe a more robust approach will make it easier for respondents to answer and more powerful for us to analyze for our clients?

The Pricing ConundrumA Different Approach

The Pricing ConundrumA Different Approach

Prices You Normally Pay

Products You Usually Seek

Premium Average Bargain

Premium 9 8 7

Average 6 5 4

Bargain 3 2 1

First we look to understand the purchasing philosophy of an organization:

Then we have respondents classify brands by how they fit.

Traditionally we have a two dimensional analysis as shown earlier – brands by attribute.

This process recasts price from “negative benefit” to comparative variable - and allows for triangulation – the confluence of benefits, price, and brand.

The Pricing DilemmaThe Toughest Assignment of All

The Pricing DilemmaThe Toughest Assignment of All

Pricing research is always dicey.

Market dynamics make today’s wisdom, tomorrow’s folly.

We design a game then get gamed.

Prediction trumps predilection.

It’s behavior we’re really after.

The Pricing DilemmaApplying Controlled Experiments

The Pricing DilemmaApplying Controlled Experiments

Here’s what a controlled experiment typically looks like:

PRICES Panel A Panel B Panel C Panel D

Comp 1 69 69 69 69

Comp 2 75 75 75 75

Comp 3 59 59 59 59

Comp 4 48 48 48 48

Comp 5 62 62 62 62

Comp 6 69 69 69 69

Your Brand 65 70 75 80

The four panels are randomly equivalent and the ONLY difference between them is the price of your product.

The respondents task is to review the alternatives and then allocate future purchases across the set of brands.

The Pricing DilemmaApplying Controlled Experiments

The Pricing DilemmaApplying Controlled Experiments

Results are simultaneously simple and profound. Demand curve based upon price. With financial information and strategic assumptions provides a sound platform for price establishment.But still no “slam dunk”

15 17 13 8 112

16 18 1213 20 141916

6570

7580

0

5

10

15

20

25

Ch

oo

ser

Sh

are

Price Points

oncept Selection:

Are we sayingthe right things?

op

y

Develo

pm

en

t:A

re w

e s

ayin

gth

ing

s rig

ht?

omponent Testing:

Do the executions work individually?

am

paig

n

Eff

ecti

ven

ess:

Do t

he e

xecu

tion

sw

ork

collecti

vely

?

If we recognize the creative process to be iterative rather than sequential, then the right place to enter

the conversation is – wherever the conversation happens to be right now.

Ad TestingWhat Does That Mean?

Ad TestingWhat Does That Mean?

In the marketing world, In the marketing world, communications is the most costly and important component communications is the most costly and important component … … that routinely escapes rigorous testing. that routinely escapes rigorous testing.

Controlled Experiments Feasibility via Layering

Controlled Experiments Feasibility via Layering

One Dimensional

TV A

600

22.2

TV B

600

29.8

TV C

600

27.6

TV D

600

21.0

TV E

600

26.3

Two Dimensional

TV A TV B TV C TV D TV E

Print A

Print B

Print C

200

200

200

22.2

200

200

200

29.8

200

200

200

27.6

200

200

200

21.0

200

200

200

26.3

27.4

25.4

23.0

Creative design makes surveys easier for respondents, not harder.

ConclusionConclusion

Respecting Respondents; Enlightening Clients

Respecting Respondents; Enlightening Clients

1. Excellent research incorporates the needs of both clients and respondents.

2. Give respondents boring surveys and they will reward you with boring results.

3. Exhaustion and boredom are the enemies of insight. Treat respondents with the respect and courtesy they deserve.

4. Insight, like beauty, is in the eye of the beholder, and the beholder is your client. Not new, not needed, not insight! Insight is hand crafted. Precisely, procreated.

5. Information close to a decision illuminates – which greatly enriches the potential for insight. True insight is a gamble; illumination, our ”table stakes” promise.

6. Your ultimate contribution is as a “decision coach”. This means knowing your client, knowing your client’s business, and knowing your client’s critical decisions.

7. Penetrating insight is an uncommon outcome of blunt inquiry.

8. If you aren’t fond of searching through haystacks, practice designing needles.

9. Expect an insight to often be controversial or contentious. If it’s nothing more than confirmation of “conventional wisdom”, it’s not much of an insight.

10. The best surveys capture what customers have to say—not just what clients want or expect to hear. Further, the former is more rewarding to both parties.

Directions Research, Inc.1-888-651-2990

www.directionsresearch.com

cgoodwin@directionsresearch.comdmurphy@directionsresearch.com

Cincinnati’s Flatiron Building is the headquarters of Directions Research, Inc. Constructed circa 1900, two years prior to the Flatiron Building in New York, this building originally served as the local home for PPG Industries.

Completely gutted by a fire in August 1903, the building served primarily as a warehouse until a major renovation and rededication in 1987.

An uncommon setting An uncommon setting An uncommon contributionAn uncommon contribution

Directions Research Inc.

401 East Court Street, Suite 200Cincinnati, Ohio 45202

513 651-2990 (main)513 719-2192 (fax)

www.directionsresearch.com