Integrated Qual/Quant: A Research and Innovation Powerhouse

Merging qualitative with quantitative to form one study, one session,

one seamless approach

By Kimberly Struyk, Principal SVP QualQuant Signals Phone: 201-665-6467 // Kimberly@qualquantsignals.com

2 | P a g e

EXECUTIVE SUMMARY

______________________________________________________________________________________________________

An evident market shift since the economic downturn - i.e., shrinking budgets, shrinking teams and shrinking timelines - forces marketing, research and innovation individuals alike to re-evaluate measurement strategies. Until now, research typically falls into one of two areas: qualitative or quantitative methodologies. Both have limitations that often fall short of insights that are generalizable and predictable of market success, increasing type 1 and type 2 errors (false positives).

In a quest to extract deeper learning about each methodology and an alternative to both (i.e., integrated qualitative with quantitative), we will discover that integration of the two provides the most powerful methodology on several fronts. This paper discusses the key points of difference across methodologies throughout the research lifecycle (questionnaire design, data collection, reporting and analysis) to make a thorough comparison of each. Through this comparison, we hope to demonstrate that the alternative of integrated qual and quant is best suited for today’s environment where “consumer empowerment and education” should be at the center of marketers, research and innovation teams. And for those reading this, consider the type of insight received when returning to the basic principles of human exchange by listening, observing and inviting comments rather than forcing them as often seen with typical quant surveys.

3 | P a g e

INTRODUCING AN ALTERNATIVE RESEARCH METHODOLOGY: INTEGRATED QUAL/QUANT

______________________________________________________________________________________________________

In the fields of research and innovation, QualQuant Signals often sees a trade-off being made between using qual or quant research within organizations. Since our team has worked in both areas over the years, many of us have also won and lost projects based on which side of the road we were traveling at that point in time. Unless in the very beginning stages of product design or exploration, an argument often can be made for the use of either type of research design. Sufficient documentation exists in terms of the opposing sides on when, why and how to use the research. However, QualQuant Signals found little to no evidence where the benefits of each method were directly compared and contrasted against one another to truly find out what is the right fit for the business question at hand. Therefore, the nature of this paper is to do just that and to provide a sense of the most powerful method for execution. As the innovation marketplace continues to evolve, it is a worthwhile conversation to have and to think about today. Before we begin to directly compare and contrast the approaches, a brief list about the typical uses for qual and quant and where it has already been applied and explored in terms of pros and cons is located below. Do note, the aim of this paper is to move beyond these findings to explore deeper what we would get if we combined the two (qual and quant) into one study, one session, one approach.

In a quest to extract deeper learning about each methodology and an alternative to both (integrated qual and quant), we will discover that integration of the two provides the most powerful methodology on several fronts. This is because it weaves together the strong points of each aforementioned methodology, providing:

A more flexible and impactful line of questioning

Quantified Qual expressions to bring forward the power of expression and leverage the strength of numbers

Consensus building to prioritize the most impactful insights

A deeper connection with research participants that leads to more profound & reliable insights

Qualitative Methodology Quantitative Methodology

Observations Survey questionnaires

Interviews Pre/Post tests

Focus groups Existing databases

Non-statistical (methods vary) Statistical analysis

Source: http://www.xavier.edu/library/help/qualitative_quantitative.pdf

4 | P a g e

Features of Approach

Qual Quant Integrated QualQuant

Group StudiedSmaller & not representative of customer base

Larger & representative of customer base

Larger & representative of customer base

View of Customer Behavior

Dynamic, situational & personal. Regular & predictableDynamic, situational, personal &predictable

Leverages Collaboration No No Yes

View of CustomerWide-angle lens; examines the breadth & depth of a single customer.

Narrow-angle lens; examines breadth & depth of customer preferences.

Wide & narrow angle lens; merging together offers both views of customers.

Nature of Reality Multiple realities; subjective.Single reality; objective.

Multiple realities;Making the subjective, objective

Allows for Free Expression

Yes No Yes

Data Analysis Non-StatisticalFindings are conclusive & are representative of customer base.

Findings are conclusive & most representative of customer base.

Quantifies Free Expression

Yes (Most Popular) NoYes with more explained variables (i.e. Most Popular & Most Relevant & Most Unique)

Final InsightsNarrative insights with contextual description & direct quotations.

Statistically reliable quantified results with limited customer explanations.

Both narrative & statistically reliable results.

The greatest amount of predictability and generalization over other methodologies due to the contextual learning that is immediately deemed as relevant or irrelevant to other participants

Integrated qual and quant is applicable to any stage of the research or innovation process and

covers many verticals which is unique because both qual and quant alone do not encompass

this type of flexibility in the design. Further, in 2004, then Procter &Gamble CEO A.G. Lafley,

the pointed out that, “The period we live in is more than ever a period of change, where we

need to reinvent our models and the way we market to consumers.” That has never been more

relevant than now considering the high penetration of social media, day-to-day behaviors that

are shaped around mobile usage, and the growth in consumer expectations for ongoing

empowerment. And the integrated approach focuses on “returning to the basic principles of

human exchange by listening, observing and inviting comments rather than forcing them”

(Flores & Whitting, 2007), as often seen with typical quant survey. To prove all of this, let’s

begin by taking apart the nuances of each approach.

METHODOLOGY COMPARISON ______________________________________________________________________________ This paper will discuss key points of difference throughout the research lifecycle (questionnaire design, data collection, reporting and analysis) to make a thorough comparison across the three methodologies. Exploring at this high level what each design brings to the table aids in selecting a better quality and faster turnaround for insights and ideas that can be leveraged along the entire innovation and/or research process. The table below provides a quick overview of the benefits and drawbacks of each.

Summary of Key Differences across Methodologies

5 | P a g e

Note- Of course, the topic of qualitative research and quantitative techniques is complex with richer overview available but it is outside the

scope of this paper. For more information on these topics, please reference “Research Design: Qualitative, Quantitative, and Mixed Methods

Approaches” by John W. Creswell.

Qual and Quant research do have different uses which is why they are generally conducted separately from one another. However, with technology at the backbone of today’s organizations, it may serve them to work towards connecting the strengths of both techniques, as we will make a case for throughout this paper.

QUAL/QUANT PROVIDES FLEXIBLE AND IMPACTFUL LINE OF QUESTIONING ______________________________________________________________________________________________________

For the purposes of this paper, our organization will stick to a similar topic (i.e. improvements for a website) in order to directly compare and contrast approaches. But before looking at those results, it is important to briefly discuss how some of the approaches inherently have more built-in flexibility when designing a line of questioning. Below are some example questions that would be used per each approach using the example of improving a website.

Qualitative Question (open-ended) What else would you like to see on this website and why?

Quantitative Question (closed-ended) Which one of the following features would you like to see on this website? (Ability to find and print recipes, Ability to share recipes and see others' recipes, Ability to rate recipes, Feedback section, Question and answer section, Videos demonstrating recipes, Photos displaying recipes)

Qual/Quant Question (open-ended until collaboration begins) Thinking about all of the features and content currently available on this website, is there anything else you would like to see added in the future? Specifically, do you feel there is anything missing with respect to the product information or that would interest you when researching which brand to purchase? Please tell us everything that comes to mind, being as specific as possible in your response.

As shown in the demonstration above, when working with a qualitative approach, the line of questioning has more flexibility and encourages deeper probing to occur through the use of follow-up questions where participants will explain the reason behind what they are suggesting. In contrast, the quantitative method requires a more rigid discipline. Multiple topics are not explored simultaneously and survey participants must respond to predetermined response items. In this case, it’s sometimes difficult to understand why certain topics are being chosen over others which often leads to the need for more research, lengthening the decision making process. On the flipside, combining qual and quant allows for the greatest flexibility. It is acceptable for the approach to ask or discuss many topics within one setting as illustrated above; for example,

6 | P a g e

a type of psychological probing is used to get the creative juices flowing and onto paper. The uniqueness of this approach is that ideas and insights are freely expressed and not limited to any one area. This has the ability to make it easier to understand the most important or meaningful topics as well as pulling out the supporting details of that topic or an afterthought to the original idea. Overall, when using the integrated qual/quant line of questioning, we have been better able to feed the innovation process to improve existing concepts or create brand new products or customer experiences because the true thoughts are exposed and explained in a way where teams know exactly how to take action.

QUAL/QUANT IS ABOUT MORE THAN COUNTING POPULARITY ______________________________________________________________________________________________________

The key difference across the three approaches is mainly in the results: how the results are formed and received. While all approaches do quantify results, there are various ways to quantify:

Qual— Counting up all common mentions and quantifying into common themes

Quant— Counting the amount of times a specific item is checked or agreed upon

Qual/Quant—Counting up commonalities but only after going through the exercise of selecting specific items that are important, rating the strength or relevance of those items and then organizing each item by lowest to highest priorities

Interestingly, we found when running a side-by-side comparison of quantified results (illustration below), the outcome can vary dramatically, as only one mention (photos and visuals) aligned and is comparable. The below illustration depicts the outcome of this comparison:

So, which one is more accurate? Simply counting results, whether themes or agreements, does not garner the same outcome as when involving a multi-dimensional approach to

Question: What would you do to improve this website?

Qual Responses Quant Responses Qual/Quant Responses

Add Coupons Coupons/Samples Loyalty program

Include some Videos Ability to print recipes Diversified product use

More Pictures Photos of recipes Detailed visuals of full meals

Better search bar Meal planner Healthier recipes &

nutritional content

Add Ingredients Cooking & educational info Videos on how to make

recipes

7 | P a g e

quantification. A multi-dimensional approach (selecting, rating, counting) brings a stronger level of discrimination between items which accounts for the difference in results. What we’ve learned working extensively across all methodologies is that counting the most popular items is not enough. Without understanding what discriminates each item from one another, organizations may take a wrong turn in deciding where to go next.

THE POWER OF QUAL/QUANT STRATEGY DEVELOPMENT ______________________________________________________________________________________________________

When applying the results to a real world scenario and executing on them, it seems different strategies can be formed based on the research approach that was utilized. Let’s take a closer look into what an organization can take away from asking “How do we improve our existing website?”

Strategy Development from Qualitative Results If the team acted upon the Qual insights, they would put most of their efforts into

couponing, creating a heavily discounted brand and investing in designing, creating and

launching videos on the website which requires significant investments in time and capital

for a website feature that has the risk of being underutilized.

Strategy Development from Quantitative Results On the contrary, if the team relied on the Quant feedback, again a strategy would have been placed toward promotions, followed by adding a few simple website features (i.e., print feature, photos, meal planning, cooking info). The implementation of videos would have been omitted from this case.

Strategy Development from Qual/Quant Results The key was to think about promotions as a longer term strategy (loyalty program), as suggested from the Qual/Quant approach. Additionally, content was suggested for the website that will help customers use more products in the future, indicating the website is expected to cross-sell. Strategy development might look like this: Implement more product suggestions, visuals and videos of full meals (not just the recipe for one item). In this case, instead of draining resources and capital, the team can focus on strategies that have the ability to create new revenue.

At face value, the results may not look much different when comparing across Qual, Quant or

Qual/Quant techniques, but after analyzing the results, it becomes clear the overall strategy will

differ. This leads us to a discussion about which results are in fact most reliable and predictable

of success.

8 | P a g e

QUAL/QUANT PREDICTABILITY STEMS FROM A BROADER INSIGHTS-GATHERING

PROCESS ______________________________________________________________________________________________________

So is one methodology truly predictable or more powerful over another? Quant is always considered more predictable by nature because a larger population is able to provide insight as to what the majority thinks about a particular topic. But remember, somewhere along the way, most likely the qual approach was included (before the quant study) to gather ideas and topics that should be further researched and quantified. Again, there is a discomfort with the fact that these two, which both have unique value in and of themselves, must always be thought of and conducted separately. Based on the work done with integrated qual/quant, we challenge this for many reasons… QUANTIFIED EXPRESSIONS ARE MORE POWERFUL The ability to quantify free expressions does two very important things: It brings forward expectations while also illuminating the strength of those expectations which is by far the most powerful combination. Further, free expression provides an open floor for participants to bring their own thoughts and does not set up an environment where one must agree or conform to someone else’s thinking. This removes the “group think” bias that is often seen with qual research. Lastly, free expression uncovers conscious and unconscious thoughts about the product and services, ultimately pulling out more content and unmet needs than quant research. MAKING A DEEPER CONNECTION LEADS TO MORE RELIABLE INSIGHTS Free expression (qual insights) along with collaboration typically leads to a more engaged

participant. As participants are more engaged, they become more open and honest during the

research and innovation process. In addition, when forging these type of deeper connections

with participants, honesty is at an all-time high and therefore there is context or explanation

surrounding the insights. All of this gives stronger reliability to the insight and true action-

ability to the findings as was evident when reviewing the responses we received from each

methodology. Again, qual/quant provided the platform for better strategy development over

the others.

9 | P a g e

GREATEST AMOUNT OF PREDICTABILITY AND GENERALIZATION Better strategy development does depend on the integrated qual/quant process to perform

with a higher “hit rate” than other methodologies; or in layman’s terms, have the capability to

accurately determine the idea or insight that will be met with the highest market acceptance.

To ensure a better “hit rate,” a

predictive algorithm was designed to

power the integrated qual/quant

method. Several characteristics were

required for an effective algorithm

with the most important key criterion

reducing type 1 and type 2 errors

(false positives). Upon testing the

algorithm (Misclassification of

Minimization) it resulted in increased

What would you do to improve this website?

(Integrated Qual/Quant Responses)

Quantified Qual Insight/Themes

Actual responses

Healthier recipes &nutritional content

“I think that the nutritional content is very important for the recipes

(especially the sodium, protein and calorie counts, as well as the vitamin

content).”

“I need to see ingredients and nutrition data. Ideally I'd like to see all the

information on the packaging. Often times the manufacturer's website is the

only source of that information.”

“Certain recipes of the Latin & Hispanic cultures are not the healthiest, due

to the use or overuse of ingredients such as lard/animal fats, sodium, sugar

or MSG. Whenever possible the health benefits of using products should be

spotlighted.”Actual responses

Videos on how to make recipes

“I believe that the videos have great potential but needs more variety

among its recipe videos. I would like to see some videos featuring desserts.”

“It would be helpful to include a video of recipes on your website to guide

inexperienced cooks like myself.”

“More videos that correspond with the recipes so you can watch someone

make it.”

Diversified product use “Tips for incorporating products into different ethnic recipes and those

recipes could be on the website.”

“Provide recipes that are from other parts of the Caribbean, not just the

Hispanic countries.”

“How to use these products in common American cuisine.”

10 | P a g e

statistical reliability and confidence in the selection of concepts, ideas, new market

opportunities, product/service optimization, etc.

Overall, the algorithm reduces the risk of wrong selection and is stable enough to achieve this

feat with as low as 50 participants and as high as a thousand participants. Allowing for this type

of generalization and predictability is what further differentiates integrated qual/quant from

qual or quant alone since these findings often do not go the length to forecast market success

like the integrated version of qual/quant.

CONSENSUS BUILDING PRIORITIZES THE MOST IMPACTFUL INSIGHTS The “majority rules” rule still applies when implementing a qual/quant methodology but only when we know more information than “I agree,” “I like it” or “I vote yes.” Consensus building happens once free expressions (qual) go through the quantification process (quant) using a collaborative session. QualQuant Signals has found that collaboration and team effort is the best way to get to the consensus because thoughts and ideas undergo a refinement process to be the “best they can be.” The illustration to the right

explains this in more

detail. While only one

individual may have a

unique idea, collaborating

with others brought out the power of that unique idea. When simply counting the most

popular mentions, other elements appeared to be of higher importance but when free

expression (idea generation) is quantified by presenting it to a larger audience during

collaboration (idea evaluation & prioritization), many appeared to have the same unmet need.

Without collaboration, this idea may have been passed over by an organization as a viable

solution for an unmet need.

11 | P a g e

INTEGRATED QUAL/QUANT: A REVOLUTIONARY TURN ______________________________________________________________________________________________________ Not everyone will come up with the next big idea. But most likely, many will know it when they see it. That is exactly where integrated qual/quant focuses. With shrinking timelines, shrinking resources and shrinking budgets, it is critical to choose a well-rounded methodology that can accomplish even more than traditional techniques, all while guaranteeing the same, if not more, confidence in the results. This seems to make sense because open innovation can leverage an integrated qual/quant

approach that provides a business advantage, as it seamlessly stitches together various stages

of the R&D innovation process. And all of the elements that an integrated qual/quant approach

capitalizes on fit into this scenario: free expression, uncovering expectations, product evolution

through collaboration and finally consensus building. This approach finds concrete action steps

that sustain, revive or grow organizations in a smart and efficient manner with the highest

probability for in-market success. If more organizations would deploy an integrated approach,

it may spark a revolutionary turn for the research and innovation marketplace.

ABOUT QUALQUANT SIGNALS ______________________________________________________________________________________________________

QualQuant Signals is a game-changing innovation insights company that scientifically uncovers

& amplifies (evaluates and prioritizes) predictive ideas & insights that WORK through our

patented customer QualQuant collaboration process.

Our team, who brings innovation to innovation, strives to think differently, delivering well

beyond the status quo. Sitting at the intersection of consumers’ expectations and client

objectives, we increase innovation effectiveness, carve out new market opportunities, identify

unmet needs, help develop, test and improve concepts, and help revive brands by facilitating

innovative collaboration with customers from the outside in or the inside out.

The prowess of collective human intelligence combined with our patented qual/quant

algorithm sets our clients on a decision-making path: a path with concrete direction, solidified

priorities, minimized risk and building market success.

And that is what we define as predictive insights that WORK!

12 | P a g e

REFERENCES ______________________________________________________________________________________________________ 1) Florès, L., Moskowitz H., (2003), From weak signals to successful product development:

using advanced research technology for consumer driven innovation, ESOMAR Technovate,

Cannes.

2) Florès, L., Whiting, M., (2007), What can innovation learn from Biology? Mother Nature

shows the way to developing new ideas and products.

3) Kearon J., (2005), Speeding up innovation and squeezing more insights, ESOMAR Innovate

Conference.

4) Rugen, B., (2011), Taking a qual-quant approach to design research, Quirks.

5) Christensen, L., Johnson, B., Lictman, M., (2008), Qualitative versus Quantitative research.

Retrieved October 12, 2012 from www.xavier.edu