brian fine and con menictas advanced quant - 2011
TRANSCRIPT
![Page 1: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/1.jpg)
Event sponsored by Affinnova All copyright owned by The Future Place and the presenters of the material For more informa=on about Affinnova visit h>p://www. affinnova.com/
For more informa=on about NewMR events visit newmr.org
Advanced Quant Techniques July 14, 2011
The Superiority of Panel Research A Fast Food Choice Modelling Example
Brian Fine, Australia Online Research and Con Menictas, Synovate
![Page 2: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/2.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Superiority of Panel Research A Fast Food Choice Modelling Example
Brian Fine, CEO, Australia Online Research Con Menictas, Decision Systems Director, Synovate
![Page 3: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/3.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Introduction • We are seeing a global initiative to ensure quality
standards of online panels.
• Online panels provide both valid real world replication and cost effectiveness.
• Only online panels offer effective administration of the latest modelling methods to take place e.g. choice modelling.
![Page 4: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/4.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Introduction 2 • We present a fast food choice modelling case study
using online panel research.
• The study illustrates the use of online panel data for a fast food industry market participant in Australia.
• The client wanted to be able to simulate real market impact by changing a number of product factors.
![Page 5: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/5.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Introduction 3 • Specifically, the client wanted to understand
□ Optimal pricing
□ Optimal meal bundles
□ Preference differences between lunch and dinner
trade
□ Retention and churn forecasts based on simulations
of price and profit scenarios
![Page 6: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/6.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Benefits of Online Panels: The Sample
• In Australia internet penetration is 80+%.
• Unconstrained by geography.
• Respondents can “log on” at any location.
• Respondents therefore find it easier to participate in marketing research via online than previously.
![Page 7: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/7.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Benefits of Online Panels: Suitability to Designs
• Online research is well suited to experimental designs used for choice experiments.
• Experimental designs call for intricate and complex representations of factors and factor levels e.g.
Attributes AlternativeA AlternativeB AlternativeC
A 1 3 2
B 2 1 3
C 3 2 1
![Page 8: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/8.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Benefits of Online Panels: Visual Tasks and Data
• Online research provides a visual medium where the researcher can simplify the complexity of combinations presented to respondents.
• As respondents click through the choice tasks they can easily see what changes.
• Collecting data is automated as the respondent completes each task.
![Page 9: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/9.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Benefits of Online Panels: Opportunity for Weighting
• Respondents can belong up to 10+ panels for any number of reasons, one of which is an additional income source.
• The degree of panel membership a respondent is classified into accounts for panel composition bias in demographics, behaviours and attitudes (Fine et al., 2006).
• In order to remove panel composition bias to tap into the large respondent panel base (n=400,000+), a non-parametric weighting scheme is applied that simultaneously removes the bias whilst weighting to the population (Fine et al., 2007)
![Page 10: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/10.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
SAMPLE • The panel we used contained over 400,000 respondents.
• Randomisation of strata fulfilment provided an un-biased solicitation process.
• The client’s customers were screened for recent purchase; meal occasion; demographics; in-restaurant or drive-through purchase.
![Page 11: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/11.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
EXPERIMENTAL DESIGN • Premised on an experimental design allowing for a
systematic combinatorial mixing of alternatives, attributes and levels (Street and Burgess 2007).
• Respondents were provided blocks of 16 tasks based on a fractional factorial design.
![Page 12: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/12.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
VALIDITY
• To ensure realism, we included each respondent’s actual last meal purchase and the real market price at the time of purchase
• We used this information in each choice task to force the respondent into a real market trade-off scenario
• This made the experiment a realistic decision process as would occur in the real market
![Page 13: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/13.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
Scenario Number: Example of the choice tasks
You last purchased Price you most likely paidMeal Typek $24.50
Menu items in scenario Potential new menu pricesMealA $25.70 MealB $27.60MealC $25.70MealD $17.30
When you last visited <Client>, if the menu item's prices were as shown, would you have chosen the same mealcomponent you did then, or would you have made a differentchoice?
○ Same as last purchase○ MealA○ MealB○ MealC○ MealD
![Page 14: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/14.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
USING STATED AND REVEALED PREFERENCE DATA
• As mentioned, we asked respondents for their last meal purchased.
• This enabled the combining of stated data (what respondents chose) to revealed data (what they had done in the past – their last meal purchase
• The process allows for simulation of real market behaviour
![Page 15: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/15.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
SAMPLE SEGMENTS
Lunch DinnerSelf a c
Family b d
Occasion Segment
Purchase Segment
![Page 16: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/16.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
WEIGHTING TO REMOVE SAMPLE AND PANEL COMPOSITION BIAS • Respondents who register on online panels can belong to >1
panel. • Fine et al. (2006) found differences in demographics,
attitudes and behaviour, due to panel composition. • A non-parametric weighting scheme using CART was
developed to model main effects and n-way interactions to simultaneously remove panel bias and weight to population.
![Page 17: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/17.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
EXAMPLE OF REMOVING PANEL BIAS FOR PRIVATE HEALTH INSURANCE
Private Health Insurance 1 2 3-4 5-7 7+ Total
Unweighted 52.86 49.54 42.24 38.99 36.96 45.47
Demographic Weighting 65.44 62.26 54.25 51.33 48.11 58.35
CART Weighting 56.95 55.84 47.41 45.00 40.19 50.82
Popn 51.00
Panel composition
20
40
1
![Page 18: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/18.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment Attributes
Un-weighted β
Demographic Weight β
Cart Weight β
β0 -3.733 -4.762 -4.429 β1 -0.258 -0.190 0.054 β2 -0.146 0.075 0.487 β3 1.773 2.342 1.792 β4 0.170 -0.220 -0.190 β5 0.047 0.116 0.033 β6 0.031 0.052 0.071 β7 -0.043 0.094 0.085 β8 -0.443 -0.470 -0.587 β9 -0.079 -0.147 -0.199
β10 -0.239 -0.298 -0.450
legend: *p<0.05; **p<0.01; ***p<0.001
![Page 19: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/19.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
Fit Statistics
Un-weighted
Demographic Weight
Cart Weight
McF’s ρ2 0.1019 0.1251 0.1422 LL -1,178 -1,212 -952 AIC 2,444 2,512 1,991 BIC 2,653 2,724 2,193
COMPARING MODEL FIT BETWEEN DEMOGRAPHIC AND CART WEIGHTING
![Page 20: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/20.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
ANALYSIS • McFadden’s (1974) (MNL) used to derive the utilities for
each menu item and associated attributes and levels
Meals Price and Profit
y = -4680.2x2 + 214120x - 1E+06
$1,260,000$1,280,000$1,300,000$1,320,000$1,340,000$1,360,000
$21.50 $22.50 $23.50 $24.50 $25.50 $26.50 $27.50
![Page 21: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/21.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
DECISION SUPPORT SYSTEM (DSS) • Revenue and Profit Simulator
Profit Optimization based on 100,000 customers matching selection criteria
Meal Price
Meal Cost Revenue Cost Meal
Profit Profit Bound
Customer Numbers
Meal 1 24.5 7.56 587,043 181,145 405,898 2,000,000 23,961
Meal 2 28.5 10.61 911,123 339.193 571,929 2,000,000 31,969
Meal 3 28.5 10.63 228,079 85,069 143,010 2,000,000 8,003
Meal 4 15.75 4.55 293,734 84,856 208,878 2,000,000 18,650
Total 2,019,979 690,264 1,329,715 82,583
Gain 0 0 0 0
![Page 22: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/22.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
DECISION SUPPORT SYSTEM (DSS) • Price change facility
Meal 1 Scenario $24.25 Base $0.00
Meal 2 Scenario $26.60 Base $-‐0.26
Meal 1 Scenario $24.15 Base $0.14
Meal 2 Scenario $17.30 Base $-‐0.19
![Page 23: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/23.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
DECISION SUPPORT SYSTEM (DSS) • Covariates
Visit Client
Customer ProfileAge group
Last Month Visit On a trip
Purchased At
Gender
![Page 24: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/24.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
DECISION SUPPORT SYSTEM (DSS) • Market Share Statistics
23.96%
31.97%
8.00%
18.65% 17.42%
Meal 1 Meal 2 Meal 3 Meal 4 Meal 5
![Page 25: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/25.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
The Case Study: A Fast Food Discrete Choice Experiment
DECISION SUPPORT SYSTEM (DSS) • Market Share meal scenario simulation
Market Share Simulation Scenario % Current % Change %
Meal 1 8.0 8.0 0.0
Meal 2 31.0 33.0 -2.0
Meal 3 22.4 26.4 4.0
Meal 4 20.6 21.1 -0.50
Not visit 15.4 15.4 0.0
![Page 26: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/26.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
![Page 27: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/27.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Conclusion • Online panel can easily control for:
□ complex data requirements regarding for choice modelling methods;
□ easy to understand visual tasks for respondents; and
□ the ability to tailor representations of past behaviour for each respondents, such as last purchase, on the fly.
![Page 28: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/28.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Conclusion 2 • Online research can easily match data from past
behaviours (revealed preference) to data from the choice experiment (stated preference)
• This feature allows for realism in predicting preferences and behaviour
• In online research the key to enhancing prediction even higher is to ensure optimal weighting of the sample corrects for panel bias.
• A CART weighting procedure does this by accounting for all effects
![Page 29: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/29.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Conclusion 3 • The latest developments in experimental design (Street
and Burgess 2007) can easily be incorporated via online research.
• The complexity of factorial designs coupled with version and strata fulfilment requirements are only made possible with online research data collection.
• Using optimally efficient experimental designs means that the researcher can optimise parameter estimates.
![Page 30: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/30.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Conclusion 4
• The aid of a Decision Support System (DSS) is the key to presenting choice modelling results to the client.
• DSSs make research results both intuitive and easy to understand
• DSSs can also extend the original simulation with tools such as profit simulators based on real consumer behaviour data.
![Page 31: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/31.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Conclusion 5 • Online research methods therefore provide for a richer
and more flexible experimental and data collection methods, when compared to CATI and CAPI.
• The ability for a researcher to get closer to the consumer’s decision process we believe will be increasingly sought after.
• We hope to have demonstrated the value of on-line panels when used for complex research.
![Page 32: Brian fine and con menictas advanced quant - 2011](https://reader030.vdocuments.site/reader030/viewer/2022032700/55d1f37ebb61ebe6468b461a/html5/thumbnails/32.jpg)
Brian Fine, Australia Online Research and Con Menictas, Synovate, Australia NewMR Advanced Quant Techniques, July 14, 2011
Q & A
Con Menictas Synovate
Brian Fine Australia Online Research
Australia Online Research Synovate www.australiaonlineresearch.com www.synovate.com [email protected] [email protected]
Sue York The Future Place