building new business models through big data dec 06 2012

Aki Balogh

www.linkedin.com/in/akibalogh

Creating New Business Models with Big Data & Analytics

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 2

Agenda

1. What is Driving Big Data?

2. What is Big Data?

3. What is Analytics?

4. What can you do with Big Data & Analytics?

5. Example Architectures

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 3

Where is Big Data today?

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 4

What is driving Big Data?

1.Rising volumes of data

2.Falling cost of data

management tools

3.Rising number of Data

Scientists

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 5

#1: Data volumes are growing

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 6

#2: Data management tools like Hadoop are driving down cost

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 7

#3: Data Science as a discipline is growing

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 8

What is Big Data?

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 9

Big Data is Turning data into insights to drive decision-making

Source: Allen (1999)

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 10

A Simple Framework: 3 Vs of Big Data

• Volume

• Variety

• Velocity

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 11

#1: Volume

Source: Christopher Bingham, Crimson Hexagon. “Better Algorithms from Bigger Data.”

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 12

Data can dramatically change the way marketers gain customer intelligence and

measure campaign effectiveness.

1. CRM Data + Web Data = Improve lead quality scoring

2. Call-Center Data + Web data = Better analyze calls you should avoid

3. Past Purchase Data + Web Data = Segment customers based on past buying

behavior and target them on your website

4. Campaign Data + Web Data = Understand multi-touch attribution and

optimize your campaign mix

5. Social Media Data + Web Data = Measure traffic to your website from social

media campaigns

Source: “Why Web Analytics is Not Enough.” Quantivo. (Paraphrased)

#2: Variety

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 13

#3: Velocity

Source: Guavus Reflex Platform. http://www.guavus.com/#/solutions/guavus-platform/

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 14

What is Analytics?

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 15

Five Common Analytics Objectives

Classify

•Clustering

•Unsupervised and supervised machine learning

•Fraud analytics

Trend

•Time-series analysis

Optimize

•Find the optimal outcome of an objective function (min/max)

Predict

•Predict the outcome of a single event

Simulate

•Explore the consequences of different choices to help drive decision-

making

•Open-ended: Scenario planning, DSS

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 16

What can you do with Big Data & Analytics?

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 17

What does Big Data Analytics require?

Data: data availability + storage + integration + data management tools

+

Analytics: analytic formulas + statistical integrity + analytic applications

+

Interpretation: business problem + domain expertise + visualization +

decision-making

This typically requires a team of people with different skillsets.

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 18

What can you do with Big Data & Analytics?

1. New revenue models

Ex: Rapleaf scraping the web, collecting contact information and selling full datasets

2. New user experiences

Ex: Gmail recommendations for people to CC: on your email

3. Cost optimization (i.e. deliver same product or service at less cost)

Ex: Give your financial advisors tools to help automate your investment decisions

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 19

Example Architectures

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 20

Combining Big Data and the Enterprise Data Warehouse

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 21

Database Types and Examples

Database Type Example Database Usage

SQL Row DBMS MySQL, PostgreSQL Real-time transactions on SQL data

SQL Column DBMS Vertica, InfiniDB Real-time analytics on SQL data

SQL In-Memory MemSQL Real-time transactions

Document-store MongoDB JSON data

Graph Database Neo4j Social network connections

Hadoop Hive, Hadapt, Accumulo

Unstructured and semi-structured data

Complex Event Processing

Storm Real-time events

Math Package R Analytic libraries

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 22

Redis

Source: http://redis.io/presentation/Redis_Cluster.pdf

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 23

MongoDB

Source: http://www.slideshare.net/PhilippeJulio/big-data-architecture

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 24

HDFS

Source: http://gigaom.com/cloud/what-it-really-means-when-someone-says-hadoop/

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 25

Hadoop + R

Source: http://blog.revolutionanalytics.com/2011/09/slides-and-replay-from-r-and-hadoop-webinar.html

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 26

Stream Processing + Column DBMS

Source: Guavus Reflex Platform. http://www.guavus.com/#/solutions/guavus-platform/

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 27

EDW + HDFS + NoSQL + CEP (Simplified)

Source: http://www.oracle.com/technetwork/topics/entarch/articles/oea-big-data-guide-1522052.pdf

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 28

EDW + Hadoop + Reporting

Source: http://www.oracle.com/technetwork/topics/entarch/articles/oea-big-data-guide-1522052.pdf

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 29

EDW + Data Science Sandboxes + CEP

Source: Big Data Analytics: Profiling the Use of Analytical Platforms in User Organizations (SAS)

© D

iam

ond M

anagem

ent

& T

echnolo

gy C

onsultants

, In

c.

More resources: www.GenerationAnalytics.com | 30

Appendix: 451 Group Big Data Landscape

Source: http://blogs.the451group.com/information_management/files/2012/11/DB_landscape.jpg