becoming (big) data driven -...
TRANSCRIPT
![Page 1: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/1.jpg)
Becoming (Big) Data Driven
![Page 2: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/2.jpg)
Geert Van Landeghem
3
Big Data Consultant@DataCrunchers
From Vision to Implementation
References
![Page 3: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/3.jpg)
13
The Ubiquity of Data Opportunities
Continuous investments in business infrastructure
Virtually every aspect of business can be measured
More and more External data is available
More interest into getting information out of data
Data mining versus data science
![Page 4: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/4.jpg)
13
From Business Problems to Data Mining
![Page 5: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/5.jpg)
13
Data mining tasks
Classification
Regression
Clustering
Profiling
Similarity matching
Co-occurence grouping
Data reduction
Link prediction
Casual modeling
![Page 6: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/6.jpg)
Data Science Context
7
![Page 7: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/7.jpg)
2
How DDD affects performance
The more data driven a firm is, the more productive it is!
One standard deviation higher on the DDD scale is associated with a
4%–6% increase in productivity.
DDD also is correlated with higher return on assets, return on equity, asset
utilization, and market value, and the relationship seems to be causal
Source: Study by Economist Erik Brynjolfsson and his colleagues from MIT (2011)
![Page 8: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/8.jpg)
2
Data Driven marketeers outperform their competition
Source: The State of Marketing 2013, IBM’s Global Survey of Marketeers
![Page 9: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/9.jpg)
Many leaders are data-driven
4
![Page 10: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/10.jpg)
What does it mean to be data-driven?
The culture and capabilities that
operationalize capturing, processing, and
utilizing data to make timely decisions
that improve products, customer
experiences, operational efficiency and
competiveness.
2
![Page 11: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/11.jpg)
Data driven hurdles
Disparate Data Sets
Data owned by different silos within the org
Lack of funding
Lack of time/resources
Legacy operating rhythm
2
![Page 12: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/12.jpg)
Embarking on your data driven journey
Transformation will not happen overnight
Support of C-Level (top-down)
Requires cross-functional collaboration
Combination of people, process, tools
Cultural champion is needed
2
![Page 13: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/13.jpg)
How does big data fit in?
Big data refers to data that is too big to fit
on a single server, too unstructured to fit
into a row-and-column database,
or too continuously flowing
to fit into a static data warehouse
6
![Page 14: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/14.jpg)
Big Data Technical Drivers
7
![Page 15: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/15.jpg)
Big Data Business Drivers
Do More (Analytics) with Less (Costs)
8
![Page 16: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/16.jpg)
Big Data Driven Roadmap
13
![Page 17: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/17.jpg)
14
Source: Bill Schmerzo (EMC)
![Page 18: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/18.jpg)
How to build the 360° view?
Two-way data integrations across
multi-channels
Data is always updated and
accessible for use
2
![Page 19: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/19.jpg)
The Enterprise Data Hub
![Page 20: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/20.jpg)
Enterprise Data Hub Definition
Contains all enterprise data
Internal & External
Processes all data in „near real-time” to generate views
Historic & Flowing
Views
Answers to business questions
2
![Page 21: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/21.jpg)
A Common Start Situation
16
![Page 22: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/22.jpg)
Our Vision #1
Focus on Data not on Derived Data
17
![Page 23: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/23.jpg)
Our Vision #2
Data is immutable
18
![Page 24: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/24.jpg)
Our Vision #3
Query = function (all data)
19
![Page 25: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/25.jpg)
Conceptually
20
![Page 26: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/26.jpg)
21
Lambda architecture
![Page 27: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/27.jpg)
View generation
- Use machine learning
Recommendations
Clustering
Classification
Regression
R
…
- SQL Like (Hive)
- Data Flow Programming (Pig)
25
![Page 28: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/28.jpg)
13
References
![Page 29: Becoming (Big) Data Driven - download.minoc.comdownload.minoc.com/...bmit_bigdata...datacrunchers.pdf · Becoming (Big) Data Driven . Geert Van Landeghem 3 ... The more data driven](https://reader034.vdocuments.site/reader034/viewer/2022042809/5f9220f28d33ca07a712e6fb/html5/thumbnails/29.jpg)
28