Download - #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research
![Page 1: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/1.jpg)
Indicators and Forecasts
David Rothschild, PhD
August 1, 2013
![Page 2: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/2.jpg)
Mean Absolute Error: 2.78
Median Absolute Error: 2.14 Feb 16, 2012
![Page 3: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/3.jpg)
Data
• Fundamental (politics): past
election results, incumbency,
presidential approval ratings,
economic indicators,
ideological indicators,
biographical information
• Social media: Twitter,
• Other online: search,
page-views, comments
• Polls
• Prediction Markets
• Experts
Passive Data Active Data
![Page 4: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/4.jpg)
Why do we create
Indictors &
Forecasts?
![Page 5: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/5.jpg)
Why Forecasting: Efficiency
Business Efficiency:
Election Spending: $6 billion in 2012
Similar Methods and Uses:
political economy, marketing,
economic indicators, finance, public
policy, business outcomes, etc.,
![Page 6: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/6.jpg)
Why Forecasting: Research
How/Why:
Not just the outcome, but how/why
the outcome ultimately occurs.
![Page 7: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/7.jpg)
Why Forecasting: Necessary
Technology:
Methods almost unchanged for 75+
years, but will be totally different in
5-10 years
Old technology is getting more
expensive
New technology is getting more
efficient
![Page 8: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/8.jpg)
What is the Goal?
![Page 9: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/9.jpg)
Gather information analyze it, and
aggregate that information into
indicators of upcoming events.
Relevant
Timely
Accurate
Economically Efficient
Raw Data -> Indicators
![Page 10: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/10.jpg)
Relevant?
![Page 11: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/11.jpg)
Relevant? (Oct 28)
![Page 12: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/12.jpg)
Relevant? (Oct 28)
![Page 13: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/13.jpg)
Obama expected
to get 51% of vote.
Relevant? (Oct 28)
![Page 14: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/14.jpg)
Obama 80% likely
to win
Electoral College.
Relevant? (Oct 28)
![Page 15: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/15.jpg)
Relevant? (Oct 28)
Romney up
by 4 in latest
Gallup poll
of likely
voters
Obama 80%
likely to win
Electoral
College
![Page 16: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/16.jpg)
Why I do not care about
economic indicator
forecasts
released the night before.
Timely?
![Page 17: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/17.jpg)
Efficiency
Early: more resources left to allocate
Often: always updated
Research
Early: capture more of campaign
Often: granular
Timely?
![Page 18: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/18.jpg)
Accurate?
Supporting Actress Nate Silver David Rothschild
Anne Hathaway 67.1% 99.5%
Sally Field 13.4% 0.4%
Helen Hunt 11.1% 0.1%
Amy Adams 8.4% 0.0%
Jacki Weaver 0.0% 0.0%
Supporting Actor Nate Silver David Rothschild
Tommy Lee Jones 35.4% 44.1%
Christoph Waltz 23.8% 40.4%
Robert De Niro 6.4% 13.6%
Philip Seymour Hoffman 24.1% 1.5%
Alan Arkin 10.3% 0.4%
![Page 19: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/19.jpg)
Error
Calibration
Out-of-sample
Accurate?
![Page 20: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/20.jpg)
Cost Effective?
Original Screenplay Nate Silver David Rothschild
Django Unchained 52.0%
Zero Dark Thirty 27.4%
Amour 20.2%
Moonrise Kingdom 0.4%
Flight 0.0%
Sound Mixing Nate Silver David Rothschild
Les Miserables 97.4%
Skyfall 1.5%
Life of Pi 0.6%
Argo 0.3%
Lincoln 0.2%
![Page 21: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/21.jpg)
New Questions
New Answers
Cost Effective?
![Page 22: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/22.jpg)
Data
![Page 23: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/23.jpg)
Data
• Fundamental (politics): past
election results, incumbency,
presidential approval ratings,
economic indicators,
ideological indicators,
biographical information
• Social media: Twitter,
• Other online: search,
page-views, comments
• Polls
• Prediction Markets
• Experts
Passive Data Active Data
![Page 24: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/24.jpg)
Fundamental Data
![Page 25: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/25.jpg)
Polling &
Prediction Markets
![Page 26: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/26.jpg)
GOP Primary
![Page 27: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/27.jpg)
Three 2012 Debates
![Page 28: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/28.jpg)
Social Media Data
![Page 29: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/29.jpg)
![Page 30: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/30.jpg)
Social Media Data
![Page 31: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/31.jpg)
Social Media Data
![Page 32: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/32.jpg)
Next Generation
Polling and
Prediction Games
![Page 33: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/33.jpg)
Next Generation
Non-Random / Non-Representative Users
Incentivize self-selected users w/ high info
New questions (graphical interfaces)
New aggregation methods/market makers
Incentive structures for truthful
participation
Accurate for new answers and domains
New types of questions: relevant & timely
New domains: cost effective
![Page 34: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/34.jpg)
Xbox Daily Poll
Between 3 and 5 questions rotated on a
daily basis.
Over 350k answered at least once,
providing demos.
Over 750k polls taken in total.
30k+ completed 5 or more polls.
10k+ completed 10 or more polls.
5k+ completed 15 or more polls.
![Page 35: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/35.jpg)
Predicting the winner of a state’s electoral college
Both correct
217 races
(63%)
Both wrong
45 races (13%)
Intent correct
20 races (24%)
Expectations
correct
63 races
(76%)
Disagree
83 races
(24%)
All Races Where the
methods disagree
Voter Intentions: in 239 / 345 races = 69%
Voter Expectation: in 279 / 345 races = 81%
Difference in proportion: in proportions: z=3.52***
![Page 36: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/36.jpg)
![Page 37: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/37.jpg)
Full Distributions
![Page 38: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/38.jpg)
Switches by Prior Support
![Page 39: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/39.jpg)
Overall Shift
Shift in Likelihood
of Taking Poll/Vote
(65%) Other to
Romney
(75%)
Obama to
Romney
(25%)
Shift in Support
(35%)
Total Shift Shift in Support
![Page 40: #MITXData "The Impending Transformation of Market Research" presented by Microsoft Research](https://reader034.vdocuments.site/reader034/viewer/2022042714/54bd7e884a795948698b45e3/html5/thumbnails/40.jpg)
Real-Time Polling