1 research techniques for uncovering end-user needs london, october 20 th frankfurt, october 22 nd

1

Research Techniques forResearch Techniques forUncovering End-User NeedsUncovering End-User Needs

London, October 20London, October 20th th

Frankfurt, October 22Frankfurt, October 22ndnd

2

Who we are

Beacon Technology Partners LLC is a full-service primary market research firm dedicated to B-to-B marketers (primarily technology)

– Founded in 1996– Offers a full-range of primary market research services, including:

• Survey research (including telephone, Web-based, mail and hybrid methodologies)

• Statistical modeling (including correspondence analysis or perceptual mapping, regression modeling, conjoint analysis, market segmentation, and other statistical tools and methods)

• Qualitative research (focus groups – both in-person and on-line - in-depth interviews and small group studies)

3

Why this topic?

• Information managers are asking (or being asked) – Who are their users/customers/prospects? – What these users/customers/prospects need or want?– How satisfied are these users/customers/prospects with the

tools and services they currently use?

• Information managers are having to answer these questions– Anecdotally, or– Via do-it-yourself (DIY) techniques, or– Collaboratively with others in cross-functional projects, or– Thru partnering with 3rd party firms and service providers

4

Strategic Research Process

ReportingConsultation

Statistical Modeling

AnalysisData Collection

QuestionnaireDesign

Programming and Fielding Tabulation

Interviews &Focus Groups

Project Development

Consultation

Presentations Whitepapers

Webinars In-person

Reporting

5

Strategic Research – Planning and Consultation

Project Definition

Establishing the critical objectives – determining the upfront strategy – is a critical step in any strategic research initiative.

It’s here where the problems to be resolved are isolated and examined and the appropriate strategies are formulated.

During the planning and consultation period, you should ascertain:

• Insights to be gleaned• Scope – geographic, linguistic, budgetary, timeline• Suitable methodologies and experimental design• Sample architecture – who needs to be heard from, how

easily can they be reached, how much data is required for sound analysis

• Best analytic tools and models employ

6

Strategic Planning – Data Collection

Questionnaire Design/Development, Programming and Tabulation

Industry standarddata collection and tabulation

Analysts craft questionnaires tailored specifically to their organization’s needs. Project managers program and field the questionnaire, compile and tabulate responses in a comprehensive report that clients study to answer their questions. Survey research packages differ markedly with regard to functional capabilities. Rather than employing cookie cutter survey platforms, Beacon uses programmers skilled in Java and Perl to customize its online questionnaires. This allows analysts to deploy innovative questionnaire designs and randomization algorithms to provide the best data for its modeling.

7

Strategic Research – Data Collection

Qualitative techniques are often best for answering “why?” questions

Qualitative Methodologies

Qualitative research techniques - such as focus groups, “mini” groups or one-on-one in-depth interviews – can be employed quickly and cost effectively for projects that face geographical or time constraints.

Beacon uses qualitative techniques for:

• Product concept evaluation• Customer needs• Pricing • Customer loyalty• Segmentation and persona development

and many other topics.

In Depth Interviews (IDIs) Diads, Triads, Mini-Groups

Face to face Focus Groups Webcam Focus Groups

8

Strategic Research - Questions

Field and Tab What do you need to know?

What are our competitive advantages?

How effective is our marketing?

What do our customers want?How loyal are our customers?

Who should we target?

How should we communicate our

advantages?

What does the futurehold?

9

• Problem to be solved– Need to differentiate users/customers/prospects/patrons

• Who are your “best” customers?• What do they want or need?• What drives or motivates them?• How can they be reached effectively?

– Audience subgroups can be hard to identify and target

Strategic Research - Analysis


10

• Case #1 (June 2012)– Online publishing platform provider– Wanted to understand customer motivations across multiple job

functions/responsibilities• Case #2 (October 2010)

– Software supplier of auction software (similar to eBay) for non-profits– Needed to understand user motivations and psychographics– Had basic contact information (basic) linked to click stream bidding

behavior



11


Field and TabWho should we target?

Segmentation (case #2) •Utilizes multiple quantitative research methods (e.g. survey research, data mining, cluster analysis etc.)•Forms rigorous and identifiable depictions of distinct audiences that can be found and targeted with specific products, promotions and pricing.

Answer:Persona and segmentation development

Personas (case #1) •Depicts “at-a-glance” archetypes•Delves into demography, motivations, skills, experiences, behaviors, attitudes, goals and aspirations.•Employs qualitative and ethnographic techniques.

12

• Case #3 – Voice of the Customer– Reprints Desk (July 2010)– Needed to measure

• The efficiency/effectiveness of its operations• Its competitive advantages/disadvantages• Customer loyalty – and isolate which customers were at risk

– Had contact and other information (e.g. tenure, transaction data, sales contacts, etc.)


How loyal are our customers?

13


Field and TabHow loyal are our customers?

Loyalty matrix

Expectation Analysis

Voice of the customer projects are unique

•Sample architecture need to include current (both new and long term) and past customers

•If possible, include data already in hand about each customer

•Attributes need to be actionable

•Scales need to be re-balanced to correct for positive skew in ratings

•Need to employ derived rather than stated importance

14



Reprints Desk performance metrics – expectation analysis – July 2010

Based on 10-pt. scaleRanked in descending order of derived importance

3%

2%

10%

23%

38%

14%

26%

20%

16%

61%

59%

84%

64%

80%

84%

16%

Provides completeinformation about order

status, availability, or pricingvia its website

Website provides ampleresources

Has an easy-to-navigatewebsite

Provides clear informationabout order status,

availability, or pricing via itswebsite

Clear and easy to readdocumentation

Keeps us informed aboutchanges to their products

and solutions8.358.35

MEAN(10-pt. scale)

MEAN(10-pt. scale)

Exceeds Exceeds expectationsexpectations

Meets Meets expectationsexpectations

Falls short of Falls short of expectationsexpectations

8.458.45

7.697.69

8.568.56

7.707.70

7.357.35

15



Reprints Desk performance metrics – loyalty matrix – July 2010

What this shows:

The loyalty matrix is a critical tool which leads to corrective action measures. It separates performance attributes into those which merit attention, can be downplayed or upgraded, or safely ignored.

Greater Concern:Focus improvements –

Competitive vulnerability (relative)

•Competitive pricing•Reducing total cost of ownership •Complete solution•Effectively balancing workloads

Competitive Advantage:Maintain or improve performance –

competitive advantage

•Customer service, support, communication, empathetic understanding and response•Utility of web site (navigation, product updates, ease of use)•Add capacity•Captures all bibliographic data

Less Concern:Reduced urgency

•Completeness of Web-based order status, availability or pricing

Maintain:Maintain or reduce investment

•Delivery service•Services are easy to use, easy to manage, easy to configure

16

• Problem to be solved– Customers/prospects don’t know what they want in the way of new

products….….but they do know about their pain points and challenges

– Customers/prospects can’t adequately articulate how to solve their problems….….but they can react to suggested solutions


What do our customers want…

17

Solution: require respondents to make choices – just like they do in real life



Not at all important

Somewhat unimportant

Neither important

nor unimportant

Somewhat important

Extremely important

Protection from loss with upside earning potential 1 2 3 4 5

Tax deferral 1 2 3 4 5

Higher potential returns than CDs 1 2 3 4 5

Lifetime income option 1 2 3 4 5

Flexible fixed indexed-linking options 1 2 3 4 5

Why fixed indexed annuities may be appropriate in an IRA or qualified plan 1 2 3 4 5

15. When you consider [TOPIC], how important are each of the following features or benefits? (Please select ONE best response for each feature/benefit.) 15. When you consider [TOPIC], how important are each of the following features or benefits? (Please select ONE best response for each feature/benefit.)

Problem of “stated importance” - scaled questions can be overly artificial or inadequate

18

• Case #4 (June 2011)– Reprints Desk – Needed to test new service concept (“Bibliogo”) for STM lit acquisition– Wanted reaction among four segments (scientists/researchers, corporate librarians, brand/product managers,

compliance)• Case #5 (Sept 2013)

– Digital signage software supplier– Wanted to develop “next generation” features/functionality that would be desired by its current and potential users

• Case #6 (June 2014)– Supplier of semiconductor IP cores – Wanted to prioritize customer needs/challenges



19


Field and TabWhat do our customers want….

Answer #4: Focus groups w/concept test – June 2011

Bibliogo is a web application for secure workgroup collaboration around journals, citations, and bibliographies (lists of citations).•Users create their own content by adding RSS feeds (via blogs, news websites or literature search engines or by journal title or product brand name), entering citations one at a time, or through bulk citation import. •This content can be tagged, searched, shared, approved and annotated within the Bibliogo application; and it is all “alertable” by authenticated RSS feed or by email. •This enables users to easily follow updates by search terms, journal, product brand name, company name, therapeutic area, and more. •All alerting is managed within Bibliogo rather than via multiple publisher websites or platforms. •Regardless of how alerts are received, each alert item arrives embedded with article tools. Users can quickly obtain full-text copies of articles in any quantity, legally send citations or articles internally or externally, request copyright permissions, request price quotes for bulk article reprints, and much more. •Any user can become a bibliography owner simply by sharing a personal bibliography with one or more colleagues or other collaborators. The bibliography then becomes, in effect, a bibliographic workgroup. •Bibliography owners can set permissions for each citation in the bibliography. Admin rights and permissions can be assigned at the individual and workgroup level. Sharing can be made open or more narrowly controlled by limiting access to users with emails from approved domains (e.g., [email protected]) and/or by IP address.

mailto:[email protected]

20



Benefit laddering – June 2011

Eliminate the need to use multiple systems

Simplify bibliographic information management

Simplify creation and distribution of SDIs (Selective Dissemination of Information)

Improve your typical or routine activities discovering and accessing citations and articles

Simplify regulatory-compliant article usage

Simplify copyright-compliant article usage

Simplify literature monitoring

Discover what colleagues are working on

Simplify the ability to share what I am working on

Save time accessing full-text literature

More importantMore important

Less importantLess important

21



Answer for Case #5: Kano modeling

Excitement

Performance

Basic

Needs Fulfilled

Customer Satisfaction

Customer Dissatisfaction

Needs Not Fulfilled

Indifference

How to read this chart:

Kano analysis classifies attributes into 4 categories:

•Basic attributes are “must haves” – absence dissatisfies but presence doesn’t add to satisfaction

•Performance attributes increase satisfaction and their absence decreases satisfaction

•Excitement attributes are “extras” that delight or add to satisfaction but are not expected

•Indifferent attributes make little impact on satisfaction

22

Basic Performance Excitement


What do our customers want….What do our customers want….

23



Answer for Case #6: Maximum difference (MaxDiff) scaling/modeling– Each respondent presented with x tasks (depending on sample size and attributes), which consisted

of a random combination of any four of the attributes being tested– Respondent chooses between each set of attributes (most important and least important) that is

more descriptive or appropriate (e.g. the type of vendor with whom they prefer to do business)– Respondent choices are summarized as utilities

1 Offers IP that is easy to use 7Offers IP that is easy to integrate into your designs

2 Has high performance IP 8 Has a track record as a technology leader

3Has a flexible pricing or licensing model for their interconnect IP

9 Is a trusted supplier of IP

4 Offers responsive support 10 Is an innovative supplier

5Produces IP that is of highest quality and reliability

11Enthusiastically seeks solutions on behalf of its customers

6 Offers the widest breadth of interconnect IP 12Offers a credible future technology roadmap for its next generation IP

24



Average Utility

Represents over 40% of overall

utility

*Attribute utilities scores are relative rather than absolute; even the lowest-rated attribute may have a positive impact (albeit less important) on supplier selection.

25

• Problem to be solved– Customers have difficulty assigning value to goods and services – “Price” has multiple meanings and connotations

• Case #7 (November 2013)– Major American newspaper– Wanted to test market acceptance (including features and pricing) of new mobile app

• Case #8 (March 2012)– Digital library/database provider– Wanted to test new e-book concept and subscription pricing model



… and what should we charge?

26


Field and TabWhat do our customers want and what should we charge?

Answer: Choice based conjoint (CBC) Each respondent presented with 12 tasks which consists of mobile app subscription package

options or attributes compared one to another in various combinations

Respondent choose between three sets of attribute groupings and a “None” option that describe the type of subscription options they prefer to have

Respondent choices summarized as part-worths or utilities

27


Field and TabWhat do our customers want and what should we charge?P

ote

nti

al M

on

thly

Rev

enu

eP

ote

nti

al M

on

thly

Rev

enu

e

Subscription Price (per month)

What this chart shows:

This estimates total monthly revenue for the mobile app by examining likely demand at various price levels between $.99/month to $6.99/month.

The upper estimate assumes that all likely app purchase will actual buy a subscription, while the lower estimate uses a rule of thumb that only half of those who say they are likely purchasers will in fact actually follow through and purchase.

Notice that as prices rise, revenue also increases (evidence of demand inelasticity, previously noted). But also note that revenue growth tends to slow and begin to flatten as prices are increased, evidence of a demand “elbow” and greater price sensitivity. PRICE POINT

App purchasers

Conservative rule of thumb

Price elasticity

HH App purchasers

Potential monthly

revenue - App

purchasers

HH Conservative rule of thumb

Potential monthly

revenue - Conservative rule of thumb

$0.99 97.2% 48.6% 54,268 $53,700 27,134 $26,900

$2.99 93.5% 46.8% -0.02 52,188 $156,000 26,094 $78,000

$4.99 85.1% 42.6% -0.13 47,518 $237,100 23,759 $118,600

$6.99 73.5% 36.7% -0.34 41,008 $286,700 20,504 $143,300

Share of Preference App purchasersConservative rule of

thumb

28


Field and TabWhat do our customers want and what should we charge?

With the addition of two With the addition of two chapters and with a 90-day chapters and with a 90-day expiration to the base product – expiration to the base product –

subscribers value the added subscribers value the added content downloading options at content downloading options at roughly $2 per month roughly $2 per month regardless of subscription regardless of subscription statusstatus

$42.99: Base product: One e-book plus three chapters - 30-day expiration

+ $.81: One e-book plus five chapters - 30-day expiration

Library: + $.81 Bookshelf: + $.70 Cancelled: + $1.01

+ $1.24: One e-book plus three chapters - 90-day expiration

Library: + $1.25 Bookshelf: + $1.23 Cancelled: + $1.08

Current subscription pricesCurrent subscription prices

29


Field and TabWhat do our customers want and what should we charge?S

ha

re o

f P

refe

ren

ce

Sh

are

of

Pre

fere

nc

e

Bookshelf / CancelledBookshelf / CancelledSubscribersSubscribers

Current Library Current Library SubscribersSubscribers

Price per monthPrice per month

Elasticity @ increase of $1.60 = 7.3

Elasticity @ increase of $3 = 14.6

Demand for Concept 1 (one e-book, 3 additional chapters, no accrual - 30 day expiration) at Varying Price Points

0

10

20

30

40

50

60

70

$42.99 $44.59 $45.99 $47.49 $48.99 $50.49 $51.99

Sh

are

of

Pre

fere

nc

eS

ha

re o

f P

refe

ren

ce

Bookshelf / CancelledBookshelf / CancelledSubscribersSubscribers

Current Library Current Library SubscribersSubscribers

Price per monthPrice per month

Elasticity @ increase of $1.60 = 7.3

Elasticity @ increase of $3 = 14.6

Demand for Concept 1 (one e-book, 3 additional chapters, no accrual - 30 day expiration) at Varying Price Points

30

So where do you go from here?

• Do primary research

• If you do it yourself– Follow the correct process (including stablishing

objectives and designing survey architecture)– Be careful with questionnaire wording, scales and models– Know what responses you want or are looking for before

you ask questions

• Partner with an independent research consultancy

Contact: Jim Warrick Email: [email protected]: (978) 248-0797Web site: www.beacontech.com

Contact: Jim Warrick Email: [email protected]: (978) 248-0797Web site: www.beacontech.com

WeWe’’re here to help youre here to help you


http://www.beacontech.com/


http://www.beacontech.com/

1 research techniques for uncovering end-user needs london, october 20 th frankfurt, october 22 nd

Documents

strategic research planning

strategic research initiative

strategic research ques

survey research packages

primary market research

mini groups

best data

information managers