a journey in making sense of big data (part 1: big bang)
DESCRIPTION
From the big bang to the new economy, a journey in making sense of big data (part 1: Big Bang)TRANSCRIPT
FROM THE BIG BANG TO THE NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATAPatrick [email protected]
linkd.in/pdeglon
2FROM THE BIG BANG TO ECOMMERCE,
A JOURNEY IN MAKING SENSE OF BIG DATA
from the Big Bang…
3Image: CERN
15 billions years
5 billions years
1 billion years
300,000 years
2 min
0.0000000001 sec
10-34 sec = 0.0…001 sec (34 zeros)
10-43 sec = 0.0…001 sec (43 zeros)
During 1996-2002, worked at CERN (the European Laboratory for Particle Physics) for my MS and PhD at the University of Geneva
4
Geneva Switzerland
Image: CERN
17 miles underground tunnelfor the LEP & LHC accelerator
Source: CERN
Mont Blanc
5Image: CERN Source: CERN
6
Tape robotSource: CERN
PAW – Physics Analysis WorkstationSource: Wikipedia
Data collection & analysis was done in Fortran. Advance
analysis/statistics was done through PAW. [1996-2002]
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
Example of a particle collision
7FROM THE BIG BANG TO NEW ECONOMY,
A JOURNEY IN MAKING SENSE OF BIG DATA
Solving the puzzle… which particles go together?
8
?
A
B
CD
1. AB + CD?2. AC + BD?3. AD + BC?
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
Solution: Big Data infrastructure enables large scale computational such as combine all possibilities (cross-product)
9
Statistical Noise
Signal(particle resonance)
Source: http://www.atlas.ch/news/2011/ATLAS-discovers-its-first-new-particle.html
Schematic View CERN Example(discovery of a new particle bb)
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
10
Size of the electron?
01
23
45
6
R < 5.1 x 10-19 m ***
*** Patrick Deglon, Etude de la diffusion Bhabha avec le détecteur L3 au LEP, Th. phys. Genève, 2002; Sc. 3332
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
11
Extra dimension?
MS > 1.1 TeV ***
e-
e+
e+
e-
our universe in 4 dimensions
extra dimension
*** Patrick Deglon, Etude de la diffusion Bhabha avec le détecteur L3 au LEP, Th. phys. Genève, 2002; Sc. 3332
graviton
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
12FROM THE BIG BANG TO ECOMMERCE,
A JOURNEY IN MAKING SENSE OF BIG DATA
… to the New Economy
13
What my friends think I do
What my mum thinks I do
What the BU thinks I do
What I think I do What the BU wants me to do
What I really do
Source: Pierre Donzier
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
14
Example #1
KPI reporting & Impact Measurement
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
15
Example #2
Internet Marketing
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
16
Example #3
Creating Value through the organization
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA
CERN vs New Economy
17
CERN
• Write kilometers long Fortran code
New Economy
• Write miles long SQL code• Analysis can run for many hours… before a
batch robot error• Queries can run for many hours… before a
spool space error
• Study billions of collision data • Study billions of customer data
• Great depth of data structure & complexity • Great depth of data structure & complexity
• Know your local expert for question – but try to find the solution by yourself… much quicker
• Know your local expert for question – but try to find the solution by yourself… much quicker
• Remove “bad runs” (unclean data batch) • Remove “wackos” (non material transactions)
• Transform a complex system into insights • Transform a complex system into insights
• Communicate findings to conferences • Communicate recommendation to business review
• Strong competitive landscape (4 distinct experiments competing to the first to publish, or publish better results)
• Strong competitive landscape
FROM THE BIG BANG TO NEW ECONOMY, A JOURNEY IN MAKING SENSE OF BIG DATA