building new business models through big data dec 06 2012
TRANSCRIPT
Aki Balogh
www.linkedin.com/in/akibalogh
Creating New Business Models with Big Data & Analytics
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 2
Agenda
1. What is Driving Big Data?
2. What is Big Data?
3. What is Analytics?
4. What can you do with Big Data & Analytics?
5. Example Architectures
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 3
Where is Big Data today?
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 4
What is driving Big Data?
1.Rising volumes of data
2.Falling cost of data
management tools
3.Rising number of Data
Scientists
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 5
#1: Data volumes are growing
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 6
#2: Data management tools like Hadoop are driving down cost
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 7
#3: Data Science as a discipline is growing
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 8
What is Big Data?
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 9
Big Data is Turning data into insights to drive decision-making
Source: Allen (1999)
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 10
A Simple Framework: 3 Vs of Big Data
• Volume
• Variety
• Velocity
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 11
#1: Volume
Source: Christopher Bingham, Crimson Hexagon. “Better Algorithms from Bigger Data.”
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 12
Data can dramatically change the way marketers gain customer intelligence and
measure campaign effectiveness.
1. CRM Data + Web Data = Improve lead quality scoring
2. Call-Center Data + Web data = Better analyze calls you should avoid
3. Past Purchase Data + Web Data = Segment customers based on past buying
behavior and target them on your website
4. Campaign Data + Web Data = Understand multi-touch attribution and
optimize your campaign mix
5. Social Media Data + Web Data = Measure traffic to your website from social
media campaigns
Source: “Why Web Analytics is Not Enough.” Quantivo. (Paraphrased)
#2: Variety
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 13
#3: Velocity
Source: Guavus Reflex Platform. http://www.guavus.com/#/solutions/guavus-platform/
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 14
What is Analytics?
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 15
Five Common Analytics Objectives
Classify
•Clustering
•Unsupervised and supervised machine learning
•Fraud analytics
Trend
•Time-series analysis
Optimize
•Find the optimal outcome of an objective function (min/max)
Predict
•Predict the outcome of a single event
Simulate
•Explore the consequences of different choices to help drive decision-
making
•Open-ended: Scenario planning, DSS
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 16
What can you do with Big Data & Analytics?
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 17
What does Big Data Analytics require?
Data: data availability + storage + integration + data management tools
+
Analytics: analytic formulas + statistical integrity + analytic applications
+
Interpretation: business problem + domain expertise + visualization +
decision-making
This typically requires a team of people with different skillsets.
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 18
What can you do with Big Data & Analytics?
1. New revenue models
Ex: Rapleaf scraping the web, collecting contact information and selling full datasets
2. New user experiences
Ex: Gmail recommendations for people to CC: on your email
3. Cost optimization (i.e. deliver same product or service at less cost)
Ex: Give your financial advisors tools to help automate your investment decisions
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 19
Example Architectures
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 20
Combining Big Data and the Enterprise Data Warehouse
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 21
Database Types and Examples
Database Type Example Database Usage
SQL Row DBMS MySQL, PostgreSQL Real-time transactions on SQL data
SQL Column DBMS Vertica, InfiniDB Real-time analytics on SQL data
SQL In-Memory MemSQL Real-time transactions
Document-store MongoDB JSON data
Graph Database Neo4j Social network connections
Hadoop Hive, Hadapt, Accumulo
Unstructured and semi-structured data
Complex Event Processing
Storm Real-time events
Math Package R Analytic libraries
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 22
Redis
Source: http://redis.io/presentation/Redis_Cluster.pdf
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 23
MongoDB
Source: http://www.slideshare.net/PhilippeJulio/big-data-architecture
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 24
HDFS
Source: http://gigaom.com/cloud/what-it-really-means-when-someone-says-hadoop/
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 25
Hadoop + R
Source: http://blog.revolutionanalytics.com/2011/09/slides-and-replay-from-r-and-hadoop-webinar.html
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 26
Stream Processing + Column DBMS
Source: Guavus Reflex Platform. http://www.guavus.com/#/solutions/guavus-platform/
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 27
EDW + HDFS + NoSQL + CEP (Simplified)
Source: http://www.oracle.com/technetwork/topics/entarch/articles/oea-big-data-guide-1522052.pdf
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 28
EDW + Hadoop + Reporting
Source: http://www.oracle.com/technetwork/topics/entarch/articles/oea-big-data-guide-1522052.pdf
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 29
EDW + Data Science Sandboxes + CEP
Source: Big Data Analytics: Profiling the Use of Analytical Platforms in User Organizations (SAS)
© D
iam
ond M
anagem
ent
& T
echnolo
gy C
onsultants
, In
c.
More resources: www.GenerationAnalytics.com | 30
Appendix: 451 Group Big Data Landscape
Source: http://blogs.the451group.com/information_management/files/2012/11/DB_landscape.jpg