big data prototyping in aws cloud
TRANSCRIPT
Big Data Prototyping in AWS CloudSamuel YeeAWS Certified Solutions Architect
Minimum Viable Product (MVP)
Why Cloud Computing?• Rapid prototyping for Minimum Viable Product (MVP)
• No need to worry about what-to-buy, what-if-buy-wrong etc.• Scale up and out if demands pick up• Tear down at minimal cost if no demand or project failure
• Deploy anytime and everywhere• Worldwide infrastructure• Flexible, elastic, agile
• No high upfront costs• Pay-as-you-use model• Can start instantly without much cost justifications
Typical Big Data Use Cases
RetrospectiveAnalysis and reporting
Here-and-NowReal-time processing
and dashboards
PredictionsMachine learning and
smart applications
Typical Big Data Pipeline
Ingest
Process
Store
Analyse (BI)
Data Answer
Word Count Example
Ingest
Process
Store
Analyse (BI)
Data Answer
Which one to buy?
Demo• Spin your new Hadoop cluster in AWS within 15 minutes• Terminate your Hadoop cluster instantly when you don’t need it
Data Types Ingest Process Store Analyse (BI)
Flat Files
Database Data
StreamingData
Hadoop-as-a-Service
File Objects
Database Objects
NoSQL Data Store
Desktop Apps
Enterprise Apps
Open Source
What’s Next?• Measure the outcomes of your data projects
• No need – Change yourself or educate your users/bosses• Fail – Try to understand the needs of your users/bosses better. Deliver faster!• Success – Congratulation!
• Should I continue in AWS cloud?• Do I need it 24x7, anywhere?• Can I move my system and network admins to host my Big Data apps?• Do I have the resource, time & expertise to deal with the intricate details of
Hadoop cluster configurations and troubleshooting?• How do I secure my big data infrastructure?
• If answer is still no, go back to the Big Data Landscape chart. But at least you can keep your users/bosses happy and engaged with your MVP in cloud.