from the big bang to the new economy, a journey in making sense of big data
DESCRIPTION
TRANSCRIPT
![Page 1: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/1.jpg)
…
![Page 2: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/2.jpg)
How old are you?
![Page 3: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/3.jpg)
FROM THE BIG BANG TO THE NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATAPatrick DeglonDirector of Engineering, Analytics Area Tech [email protected]/pdeglon
![Page 4: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/4.jpg)
4FROM THE BIG BANG TO ECOMMERCE,
A JOURNEY IN MAKING SENSE OF BIG DATA
from the Big Bang…
![Page 5: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/5.jpg)
5Image: CERN
15 billions years
5 billions years
1 billion years
300,000 years
2 min
0.0000000001 sec
10-34 sec = 0.0…001 sec (34 zeros)
10-43 sec = 0.0…001 sec (43 zeros)
![Page 6: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/6.jpg)
During 1996-2002, worked at CERN (the European Laboratory for Particle Physics) for my MS and PhD at the University of Geneva
6
Geneva Switzerland
Image: CERN
17 miles underground tunnelfor the LEP & LHC accelerator
Source: CERN
Mont Blanc
![Page 7: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/7.jpg)
7Image: CERN Source: CERN
![Page 8: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/8.jpg)
8
Tape robotSource: CERN
PAW – Physics Analysis WorkstationSource: Wikipedia
Data collection & analysis was done in Fortran. Advance
analysis/statistics was done through PAW. [1996-2002]
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 9: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/9.jpg)
Example of a particle collision
9FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 10: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/10.jpg)
Solving the puzzle… which particles go together?
10
?
A
B
CD
1. AB + CD?2. AC + BD?3. AD + BC?
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 11: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/11.jpg)
Solution: Big Data infrastructure enables large scale computational such as combine all possibilities (cross-product)
11
Statistical Noise
Signal(particle resonance)
Source: http://www.atlas.ch/news/2011/ATLAS-discovers-its-first-new-particle.html
Schematic View CERN Example(discovery of a new particle bb)
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 12: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/12.jpg)
12
Size of the electron?
01
23
45
6
R < 5.1 x 10-19 m ***
*** Patrick Deglon, Etude de la diffusion Bhabha avec le détecteur L3 au LEP, Th. phys. Genève, 2002; Sc. 3332
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 13: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/13.jpg)
13
Extra dimension?
MS > 1.1 TeV ***
e-
e+
e+
e-
our universe in 4 dimensions
extra dimension
*** Patrick Deglon, Etude de la diffusion Bhabha avec le détecteur L3 au LEP, Th. phys. Genève, 2002; Sc. 3332
graviton
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 14: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/14.jpg)
14FROM THE BIG BANG TO ECOMMERCE,
A JOURNEY IN MAKING SENSE OF BIG DATA
… to the New Economy
![Page 15: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/15.jpg)
15
What my friends think I do
What my mum thinks I do
What the BU thinks I do
What I think I do What the BU wants me to do
What I really do
Source: Pierre Donzier
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 16: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/16.jpg)
16
Example #1
KPI reporting & Impact Measurement
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 17: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/17.jpg)
Measuring impact of initiatives
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
Aug 1st Sep 1st Oct 1st
Number of listings
2012
2011B
A
C
Pre/Post analysis illustrative example (Simulation)
D
Impact of the initiative
pre post
Initiativelaunched
• Used to measure the impact of an initiative in a full market or a market segment
• Randomized Test/Control group methodology is a golden standard in research
A/B test illustrative example (Simulation)
0
50
100
150
200
250
300
350
400
450
Aug 1st Sep 1st Oct 1st
Number of purchases
Impact of the initiative
Initiativelaunched
control group
test group
17FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 18: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/18.jpg)
18
Key Performance Indicators
Motorola Factory# Shipments
Distribution Channels# Sales
First Usage# Activations
Simplified Business Flow
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 19: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/19.jpg)
19
Data Flow
Motorola Factory# Shipments
Distribution Channels# Sales
First Usage# Activations
Google BigQuery
MotorolaCloud
Insights
...
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 20: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/20.jpg)
20
Google Spreadsheet as a Reporting Engine
Spreadsheet
BigQuery
HTML body in sheet
App Script
Scheduler
Distribution Group
Charts
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 21: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/21.jpg)
21
Google Spreadsheet as a Reporting Engine
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 22: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/22.jpg)
22
Analytics Portal
App Engine
Data Source:Big Query
Data Source:Google Analytics
iFrame Source:Tableau Server
iFrame Source:Google Documents
Data Source:Spreadsheet & CSV
ReportMeta Data:
Google Drive
(Text file with JSON)
Report Meta Data:Datastore
(Report copy & usage tracking)
Users Access Control:
Google Users + Drive Sharing
Google Drive
Android App
Web portal
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 23: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/23.jpg)
23
42…so what?
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
Answer to the Ultimate Question of Life, The Universe, and Everything
![Page 24: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/24.jpg)
24
Campaign MeasurementCampaigns
• Campaign Id• Campaign Name• Time range• Set of Countries• Set of Products
KPI
• Date• Country• Product• KPI[]
X
Trend
• Campaign Id• Date• Total of KPI[]
Summary
• Campaign Id• Campaign Name• Impact Measurement[]• Statistical Error[]
Time Series
Analysis
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 25: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/25.jpg)
25
Campaign Measurement
Campaign Window
Don’t include weekday cycle in
your volatility measurement
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 26: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/26.jpg)
26
Campaign Measurement
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
Define Campaign
Run Campaign
Measure Impacts
Drive Insights
![Page 27: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/27.jpg)
27
Example #2
Internet Marketing
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 28: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/28.jpg)
Case study: Online Search
Natural/OrganicSearch (free)
Paid Search
28FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 29: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/29.jpg)
X days
2 purchases
missing
X days
Y days
all purchasesare incremental1 purchase is
uncorrelated
Y days
Jan 1st Feb 1st
$ $ $ $ $ $ $
click
$ $ $
Behavioral purchasesUncorrelated to Marketing
clickMar 1st
$
Influence purchaseCorrelated to Marketing
Customer behaviors and Internet Marketing Investment
Which customer purchases are influenced by Marketing?
29FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 30: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/30.jpg)
Remember this physics problem?
30
?
A
B
CD
1. AB + CD?2. AC + BD?3. AD + BC?
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 31: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/31.jpg)
Solution: Big Data infrastructure enables large scale computational such as combine all possibilities (cross-product)
31
Statistical Noise
Signal(particle resonance)
Source: http://www.atlas.ch/news/2011/ATLAS-discovers-its-first-new-particle.html
Schematic View
Combine correlated events and uncorrelated events produce a system with a statistical noise (which is simple enough to extract) and the researched signal
CERN Example(discovery of a new particle bb)
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 32: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/32.jpg)
Marketing incrementality
(correlated purchases) Level of
behavioral purchases
Positive LatencyPurchase after Click (potential causality)
Behavior & Internet Marketing impact
Level of behavioral purchases
0 2 4 6 8 10 12 14
Latency (days)
Number of events (pairs click-purchase)
Negative LatencyPurchase before Click (no causality)
Behavior only
-14 -12 -10 -8 -6 -4 -2
User clicks on an ad-banner at time=0
User makes a purchase X days later
Latency time for each pair click - purchase
32FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 33: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/33.jpg)
33
Sales ROI Channel A 8% +20%Channel B 5% -10%Channel C 1% +10%
Method 1• Reduce spend on channel B• Invest in channel A• When prioritizing, ignore
channel C
Sales ROI Channel A 7% -20%Channel B 6% +30%Channel C 12% +60%
Method 2• Reduce spend on channel A• Invest heavily on channel C• Marketing counts actually for
25% of the site
<>
… So what?
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 34: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/34.jpg)
Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment, Thomas Blake, Chris Nosko, Steven Tadelis
Case study: Online Search
34FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 35: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/35.jpg)
Case study: Online Search
Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment, Thomas Blake, Chris Nosko, Steven Tadelis 35
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 36: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/36.jpg)
Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment, Thomas Blake, Chris Nosko, Steven Tadelis
Case study: Online Search
36FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 37: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/37.jpg)
Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment, Thomas Blake, Chris Nosko, Steven Tadelis
Case study: Online Search
37FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 38: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/38.jpg)
38
So, what’s next?Marketing 101
Don’t Do Marketing Do Marketing
No Purchase
PurchaseL L
D DC
C?
?
Cost
Direct Return
Incr Return
Rule #1: Never, ever, spend money unless you really-really have to
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 39: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/39.jpg)
So, what’s next?
Investment (costs)
Output Cost
Return (Revenues)
ProfitMax SalesNo Profit
Total ROI = 0
Max Profit
DReturn = DInvestmenti.e. marginal ROI = 0Rule #2: If you have to spend, you spend
to the point of marginal return=0
39FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 40: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/40.jpg)
SpendBucket i
SpendBucket 0
(most profitable)
SpendBucket N
(leastprofitable)
…
…
Marginal Return Chart
CumulativeCost
ROI
CurrentSpend Level
Area/initiatives/segment withnegative profitability
Cost reduction opportunity!
Point of marginal
return = 0(maximum profit)
In depth Analysis require to validate
high ROI
40FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
![Page 41: From the Big Bang to the New Economy, a journey in making sense of Big Data](https://reader035.vdocuments.site/reader035/viewer/2022070302/54733a04b4af9fa90a8b515c/html5/thumbnails/41.jpg)
CERN vs New Economy
41
CERN
• Write kilometers long Fortran code
New Economy
• Write miles long SQL code• Analysis can run for many hours… before a
batch robot error• Queries can run for many hours… before a
spool space error
• Study billions of collision data • Study billions of customer data• Great depth of data structure & complexity • Great depth of data structure & complexity• Know your local expert for question – but try
to find the solution by yourself… much quicker
• Know your local expert for question – but try to find the solution by yourself… much quicker
• Remove “bad runs” (unclean data batch) • Remove “wackos” (non material transactions)
• Transform a complex system into insights • Transform a complex system into insights• Communicate findings to conferences • Communicate recommendation to business
review• Strong competitive landscape (4 distinct
experiments competing to the first to publish, or publish better results)
• Strong competitive landscape
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA