Research Process (with a focus on Problem

Formulation)

Jeremy Kees, Ph.D.

Review from last week…

• Marketing research is the marketer’s link to understanding the consumer and the external environment

• The main purpose of marketing research is to inform decisions

– Is a source of a competitive advantage for many successful firms

• Every research project is different– It often takes many research projects over

many years to really understand a phenomenon

Ongoing “PharmFirm” Case Study

• Will be used as an example for many of the concepts we discuss

• Large-scale, multi-study research project for a major pharma company in the Philadelphia area

• Key research questions focused on “fair balance” of a online medical resource targeted at physicians– Proactive study in anticipation of pushback

from FDA

Stages in theResearch Process(ResearchersPerspective)

Determine Research Design

Analyze and Interpret the Data

Design Sample and Collect Data

Formulate Problem

Design Data Collection Method and Forms

Prepare the Research Report

Problem Formulation

• Differentiate between a decision problem and a research problem

• Try to get beyond an “info request”Decision Problems Research Problems

Develop package for a new product Evaluate alternative package designs

Increase store traffic Measure current image of the store

Increase market penetration through the opening of new stores

Evaluate prospective locations

Decide which merchandise will be available for purchase over the Internet

Determine consumers’ confidence in purchasing different categories of products unseen

Research Process: Problem Formulation• Perhaps the most important step in the research

process analysis• A well-defined study begins with a clearly

defined objective“The formulation of a problem is often more essential than its

solution” - Albert Einstein

• The problem is rarely clear-cut• Slight variations in research questions can lead to

substantial changes in the research process (so be careful)

– Drives decisions related to research design, measurement, sampling, etc.

Problem Formulation

• Differentiate between a decision problem and a research problem

• Try to get beyond an “info request”

Decision Problems Research Problems

Develop package for a new product Evaluate alternative package designs

Increase store traffic Measure current image of the store

Increase market penetration through the opening of new stores

Evaluate prospective locations

Decide which merchandise will be available for purchase over the Internet

Determine consumers’ confidence in purchasing different categories of products unseen

Research Process: Problem FormulationStage in the Process

Typical Questions

Formulate problem • What is the purpose of the study (i.e., to solve a problem? Identify an opportunity?)

• Is additional background information necessary?

• What are specific research questions and what information is needed to make the decision?

• How will the information be utilized?

• Has a decision already been made?

• Should research be conducted?

Research Process: Problem Formulation

PharmFirm• Longest stage in the process

– ~4 months

– Talked with LOTS of people

• Over-arching problem– Need evidence that a medical information

website is ‘fairly balanced’• Unclear what fair balance means in an computer

mediated environment

• Anticipated issues from FDA

Research Process: Problem FormulationPharmFirm• Overarching Research Objective

– provide an empirical test of perceived benefit and risk information (i.e., fair balance) presented on the medical information website

Research Process: Problem FormulationPharmFirm• More specific study objectives

– Phase I: Develop a metric for the construct of fair balance

– Phase II: Test the perceived fair balance of the website

• Different versions

• Versus print marketing materials

– Phase III: Test the perceived fair balance of the website versus other websites

Research Process: Problem Formulation

PharmFirm

• 4 separate studies, each with its own very specific objectives

• Expected Decisions– Positive results = move forward

confidently– Negative or mixed results = adapt

website

Research Process: Determine Research Design

• Dictated by the problem or research question

• Exploratory Research

• Descriptive Research

• Causal Research

Research Process: Determine Research DesignStage in the Process

Typical Questions

Determine research design

• How much is already known?

• Can a hypothesis be formulated?

• What types of questions need to be answered?

• What type of study will best address the research questions?

Research Process: Determine Research Design• PharmFirm research questions call for….

1. Some pretesting to develop and refine the metric

2. Experiments1. To test versions of the website with different

presentations of risk information

2. To test the website versus print marketing materials (a context where fair balance is well-defined)

3. To test the website versus other existing online resources that are deemed “fairly balanced”

Research Process: Design Data Collection & Forms

• Secondary research – not necessary

• Survey Research– Lots of ways to collect data

• Measurement– Constructs / Variables

Research Process: Design Data Collection & FormsStage in the Process

Typical Questions

Determine data collection method and forms

Can existing data be used to advantage?

What is to be measured? How?

What is the source of the data?

Can objective answers be obtained by asking people?

How should people be questioned?

Should the questionnaires be administered in person, over the phone, or through the mail?

Should electronic or mechanical means be used to make the observations?

Should structure or unstructured items be used to collect the data?

Should the purpose of the study be made known to the respondents?

Should rating scales be used in the questionnaire?

Research Process: Design Data Collection & Forms• PharmFirm data collection requires….

1. Scale development and testing1. Define construct

2. Develop valid and reliable measures

2. Stimuli development1. Website mock-ups

3. Measurement instrument (i.e., survey)1. Manipulation checks

2. Dependent variables

3. Individual difference variables

4. Demographic/categorical information

Research Process: Design Sample and Collect Data• Determine your “target market”

• Determine necessary sample size– Dictated by study design

• Sampling Methodology

• Logistics

• Cost

• Ethics

Research Process: Design Sample and Collect DataStage in the Process

Typical Questions

Design sample and collect the data

• Who is the target population?

• Is a list of population elements available?

• Is a sample necessary?

• Is a probability sample desirable?

• How large should the sample be?

• How should the sample be selected?

• Who will gather the data?

• How much supervision is needed?

• What operational procedures will be followed?

• What methods will be used to ensure the quality of the data collected?

• PharmFirm sample call for….1. ~1000 physicians from three specialty

groups• Outsource!!

2. Data collected online• Sample is too geographically dispersed and time

sensitive to collect data in person

• Qualtrics research software

Research Process: Design Sample and Collect Data

Research Process: Analyze and Interpret the Data

• Edit / clean / code the data

• Analyze

• Interpret

Research Process: Analyze and Interpret the DataStage in the Process

Typical Questions

Analyze and interpret the data

• Who will handle the editing of the data?

• How will the data be coded?

• Who will supervise the coding and keypunching?

• What tabulations / statistical tests / analysis techniques will be used?

• PharmFirm analysis calls for….1. Factor analysis to develop metric

• Dictated by study design

2. Multivariate Analysis of Covariance (MANCOVA) to analyze experimental data• Dictated by study design

Research Process: Analyze and Interpret the Data

Research Process: Prepare Research Report

• Know your audience

• Sell your findings

Research Process: Prepare Research ReportStage in the Process

Typical Questions

Prepare the research report

• Who will read the report?

• What is their technical level of sophistication?

• Are managerial recommendations called for?

• What will be the format of the written report?

• Is an oral report necessary?

• How should the oral report be structured?

• PharmFirm report out calls for….1. Four separate presentations to the client

2. Final comprehensive report outlining purpose, research questions, design, sample, results, etc.

3. Compilation of data in easy-to-use format• Measurement definitions

• Coding schemes

Research Process: Prepare Research Report

Form Groups and…

Contrast the process we have just talked about with that of Andreasen

Problem Formulation (as discussed in Andreasen)• “Actionable marketing research”

– Identify the intended action FIRST; avoid “these are the things that would be nice to know”

• Exceptions: basic, exploratory research that lays the groundwork for future investigation and methodological research

– Close and consistent collaboration between researcher and client (decision maker)

Problem Formulation (as discussed in Andreasen)“Backward Approach”

1. How will the research results be implemented? (i.e., what action will be taken)

2. Determine what information will make up the final report

3. Specify analyses that will need to be done to “fill in the blanks” in the report

4. Determine what kind (format) of data is needed to run the particular analyses (simple as possible)

5. Does this data already exist?6. If primary data is needed, design sampling strategy

and instruments7. Collect data8. Fill in the blanks from above

Problem Formulation (as discussed in Andreasen)• Try to avoid “interesting” or “nice to know”

questions at all costs• Lay out the action alternatives on the front end• What key information in the final report will be

used to make decisions one way or another?• Present the client with hypothetical scenarios to

ensure that the information is actionable– This often leads to more questions that the client

hadn’t thought of previously– It also helps the client:

• commit to the research• deepen their understanding of the nature of the research• minimize surprises• recognize limitations

Andreasan (1985) Take-Aways

• This “backward” process helps us avoid research that does not lead to action (or directly benefit decision making)

• The emphasis on close collaboration with the client helps to ensure maximum usefulness and minimum surprises

TEAM ASSIGNMENT #1

• Discuss some issues that you deal with at your workplace that can be answered with primary research

1. Discuss the issue broadly (i.e., provide some context)

2. Develop several clear, concise research questions

3. Develop hypotheses (and null hypotheses)

4. Discuss the decisions that would be made depending on the different outcomes of the research

Hints….

• Try to acutely identify the problem at hand (and avoid “nice to know” questions)– What information is necessary in order to make a

decision?

• Determine what course(s) of action will result from your findings– If we find A, what will you do?– What about if we find B?– Make all alternative courses of action explicit