Research RebootedWhy Most Market Research Is Broken,

And How Lean Can Help Fix It

what’s wrong with research?

the value is broken.

$30,000,000,000+

time.it’s in a box. and it slips.

information request

research RFP

study design

recruitment

fielding

Analysis

Brand/product team

research & insights

research company

recruiter/panel

moderators

research company

The CEO’s Kids

“I keep a lot of kids around me and ask them questions and peek around the corner when they think I'm not listening. My master plan was I had a lot of kids. I have six kids. Three girls that are six, a 15-year-old, 18-year-old and 22-year-old. I have the perfect research team in my house."

http://acupofteawithphd.wordpress.com/2013/06/19/live-from-cannes-cannes-lions-2013-so-farpart-4-its-diddy/

the design is broken.

The fear of small numbersManagers worry about the signal-to-noise ratio.

The smaller the sample, the more we worry about noise (wrong people, inaccuracies or untruths, groupthink, etc.)

So they compensate with bigger sample sizes.

But more numbers make more noise.

over targeting leads us astray

“[M]ost companies with a practiced discipline of listening to their best customers and identifying new products that promise greater profitability and growth are rarely able to build a case for investing in disruptive technologies until it is too late.”

- Clay Christensen, The Innovator’s Dilemma

research is often designed to confirm biases.when it doesn’t, it’s painful.

our customer or our design target?

research ignores switchers, rejectors, fringe & new users

“Generally, disruptive technologies underperform established products in mainstream markets. But they have other features that a few fringe (and generally new) customers value.”

- Clay Christensen, The Innovator’s Dilemma

the process is broken.

garbage in, garbage out

“The research manager’s job is to manage the hell out of the research.”

- Michelle Crawford

quant über alles

packing 10 lbs into a 5 lb bag

30 minute surveys

2 hour focus groups

10 page discussion guides

Too many questions, too much data, not enough understanding

stockpiling data

trapping insight in powerpoint decks

It's really hard to design products by focus groups. A lot of times,

people don't know what they want until you show it to them.

- Steve Jobs

you ≠ steve jobs.

research ≠ designResearch is the process by which we

understand problems.

Design is the process by which we solve problems.

how lean can help fix research.

what is a lean company for?

1. To specify customer value.

2. To identify & implement the actions that create that value.

3. To remove anything that doesn’t create value.

4. To analyze the results (and repeat).

why does feedback & customer pull matter?

Feedback and testing (a.k.a., research) help teams settle questions and make decisions based on evidence not preferences.

Continuous feedback helps teams see progress.

Go and see.This is the new intimacy.

how do we do lean research?

the owner has to change

empower the research manager

We have to transform the research manager from a project manager into a Scrum master.

Their role is to remove obstacles to progress, and to facilitate communication.

Their responsibility to the team is to facilitate hypothesis testing, and to create opportunities to challenge assumptions.

validation

the tools have to change

use right-sized tools

Use the tools closest to hand

Talk to people where it’s relevant or convenient

i.e., NOT HERE

talk to them here.

the whole team can do this.

http://giffconstable.com/2010/07/12-tips-for-early-customer-development-interviews/

the flow has to change

Do less, more oftenLean research is about continuous

customer, partner & team feedback.

set people up for success

the output has to change.

data doesn’t “tell” you what to do.

nothing beats a conversation.

To make research meaningful to the whole team, the whole team needs:

Investment

Access

Accountability

forget the horse’s mouth

Don’t count on users to make your decisions for you - that’s not their job.

Don’t let the report be the end of your learning process - it’s just a way to memorialize what you’ve learned and provide a reference when you encounter similar issues again.

information should be reusable, not disposable.

watch this space.and thanks, this was fun.