What Does Big Data Really Mean for your Business?

Leslie Hawthorn Director of Developer Relations

leslie.hawthorn@elasticsearch.com @lhawthorn

Even our small data has “Big Data”

Every day we create more…

User Generated Machine Generated

•  Tweets •  Blogs •  Books •  Notes •  Emails •  Comments

•  Logs •  Events •  Statistics •  Alerts •  Metrics

And it’s growing…

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

2010 2016

Exab

ytes

Year

machine social voip enterprise

But there is a gap

Storing Data

Actionable Insight

Data From Any Source

Instantly Analyze

Actionable Insights

The promise of “Big Data”

Massive effort in cleaning the data.

Takes weeks to analyze.

Insights come too

late.

The reality

“I’m a data janitor.” Senior Director of Data Science, major Apache Hadoop distro

Why?

Balancing Structure and Scale

SQL   Rela)onships   Structure   Can  it  Scale?  

SQL   Yes   Yes   No  

NoSQL   No   Yes   Yes  

Hadoop   No   No   Yes  

We got excited about Scale, but now suffer for lack of Structure.

Which means: We can store a lot of data, but it takes a ton of work to

search, analyze and get insights out of it.

Hadoop Also Brings: Intensive Engineering and a Large Upfront Investment

Where we’re going

Your Big Data pipeline

•  Incredibly crowded landscape in both the open source and proprietary worlds

•  Most big data tools were built for people with Massive Data

•  Tools now focus on making working with both structured & unstructured data easier

Why open source? http://w

ww.flickr.com

/photos/16462767@N00/3984507322/

flexibility

Data From Any Source

Instantly Store, Search

& Analyze

Actionable Insights

Elasticsearch: Search, Analytics, Logging & Visualization

Logstash Elasticsearch Kibana

The ELK Stack

Data Democratization

Big Data for our business

•  Measuring our community health metrics

Demo time….

[see appendix for screenshots from demo]

Be a data superhero, not a janitor!

Questions?

Leslie Hawthorn Director of Developer Relations

leslie.hawthorn@elasticsearch.com @lhawthorn

Appendix – Demo Slides