how to build a real time analytics enterprise with open source
DESCRIPTION
Given the ease of scaling from zero to millions of users with very little capital investment and availability of managed infrastructure, building startups or enterprise grade products has become easier, faster and cheaper then ever before. We will go over what it would take to build an enterprise big data real time analytics system, with insights onto architecture, open source and closed source available software, support requirements and alternatives around managed services. We will explore the use of open source tools (using cloud providers such as AWS, Google and Microsoft) to build these big data applications. From zero to enterprise will be the theme of this meetup, come join us to learn, explore and contribute.TRANSCRIPT
Building enterprise analy0cs from Open Source
Lecole Cole, Founder & CEO 1
• My name is Lecole Cole • Worked in data analysis for +15 years • TwiEer: @lecole • Email: [email protected] • Company Skydera Inc. • Projects: Chartleaf
(2)
A little about me.
Interes0ng Packages
• Real-‐0me Stream – AWS Kinesis – TwiEer Storm – Hadoop v2?
(#)
• Analy0cs package – Apache Mahout – R-‐project – Pandas (Python) – pyBrain (Python) – Custom Python
• Visualiza0on – D3.js – R-‐project
Interes0ng Packages
• Batch Processors – Hadoop V1 – EMR (AWS)
• NoSQL: – MongoDB – DynamoDB (AWS) – BigTable (Google) – Cassandra
(#)
Example Stack
• Compute: – Google Compute
• Database – MySQL – BigTable
• Analy0cs – Hadoop
(#)
• Language: – Java applica0on for Hadoop
• Data access – Apache Pig – Apache Hive
Example Stack
• Compute: – AWS EC2
• Database – MySQL – DynamoDB
• Data warehouse – Redshib
(#)
• Analy0cs Batch – EMR
• Analy0cs Real-‐0me: – AWS Kinesis
Screen Shots
(#)
Screen Shots
(#)
Screen Shots
(#)
Screen Shots
(#)
Screen Shots
(#)
Screen Shots
(#)