architecting big data solutions in the cloud
TRANSCRIPT
![Page 1: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/1.jpg)
![Page 2: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/2.jpg)
Session Objectives And Takeaways
![Page 3: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/3.jpg)
Lambda Architecture
http://lambda-architecture.net/
1.All data entering the system is dispatched to both the
batch layer and the speed layer for processing.
2.The batch layer has two functions: (i) managing the
master dataset (an immutable, append-only set of raw
data), and (ii) to pre-compute the batch views.
3.The serving layer indexes the batch views so that they
can be queried in low-latency, ad-hoc way.
4.The speed layer compensates for the high latency of
updates to the serving layer and deals with recent data
only.
5.Any incoming query can be answered by merging
results from batch views and real-time views.
![Page 4: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/4.jpg)
Lambda Architecture
![Page 5: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/5.jpg)
Linux
Windows
What is HDInsight
![Page 6: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/6.jpg)
C# Java.NET
HDInsight clusters on Azure
![Page 7: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/7.jpg)
HDInsight clusters on Azure
![Page 8: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/8.jpg)
What is HBase
![Page 9: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/9.jpg)
Order No Customer Name Customer Phone Company Name Company Address
12012015 Mostafa 101-232-2345 Microsoft Redmond, WA
Customer Company
Order No Customer
Name
Customer
Phone
Company Name Company
Address
12012015 Mostafa 101-232-2345 Microsoft Redmond, WA
![Page 10: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/10.jpg)
Create
Select
Update
Select
What is HBase
![Page 11: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/11.jpg)
data warehouse system
What is Hive
![Page 12: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/12.jpg)
![Page 13: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/13.jpg)
distributed fault-tolerant open-source
analytics solutions
templates
What is Apache Storm
![Page 14: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/14.jpg)
Topologies
topology
Stream
Tuple
Spout
Bolt streams tuples streams
Apache Storm Components
![Page 15: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/15.jpg)
![Page 16: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/16.jpg)
100x 10x
What is Apache Spark
![Page 17: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/17.jpg)
![Page 18: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/18.jpg)
complexities of ingesting and storing all of your data batch streaming interactive analytics
Azure Data Lake (ADL)
![Page 19: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/19.jpg)
Azure Data Lake (ADL)
![Page 20: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/20.jpg)
Azure Data Lake Analytics
![Page 21: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/21.jpg)
![Page 22: Architecting big data solutions in the cloud](https://reader033.vdocuments.site/reader033/viewer/2022042517/58b888b11a28ab44078b7a13/html5/thumbnails/22.jpg)