presentation title goes heredownload.microsoft.com/download/c/f/f/cff0a653-6cd6-4e52-b97… ·...
TRANSCRIPT
![Page 1: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/1.jpg)
HDInsight-微软Azure云中的Hadoop大数据解决方案概览及案例分享 孙巍 资深项目经理 微软亚太研发集团
课程代码DBI-B308
![Page 2: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/2.jpg)
Hadoop?
![Page 3: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/3.jpg)
![Page 4: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/4.jpg)
Apache 开源项目
高扩展性分布式文件系统 (HDFS)
分布式数据处理框架
![Page 5: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/5.jpg)
![Page 6: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/6.jpg)
Microsoft解决方案
![Page 7: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/7.jpg)
![Page 8: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/8.jpg)
Hadoop 2.2 and 2.4
80% data compression with ORC
Hadoop
on
Windows
Hive 100x Query Speed Up
30,000+ code line contributions
HDFS in Cloud
(Azure)
10,000+ engineering hours
Committers to Hadoop
![Page 9: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/9.jpg)
![Page 10: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/10.jpg)
Hadoop 2.0
![Page 11: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/11.jpg)
Data Node Data Node Data Node Data Node
Task Tracker Task Tracker Task Tracker Task Tracker
Name Node
Job Tracker
HMaster Coordination
Region Server Region Server Region Server Region Server
![Page 12: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/12.jpg)
![Page 13: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/13.jpg)
Stream processin
g
Search and query
Data analytics (Excel)
Web/thick client
dashboards
Devices to take action
RabbitMQ /
ActiveMQ
![Page 14: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/14.jpg)
![Page 15: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/15.jpg)
![Page 16: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/16.jpg)
![Page 17: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/17.jpg)
![Page 18: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/18.jpg)
![Page 19: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/19.jpg)
![Page 20: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/20.jpg)
HDInsight on Hadoop 2.2 April 2014
HDInsight on Hadoop 1.1.2 Oct 2013
HDInsight on Hadoop 2.4 June 2014
O/S Upgrades
O/S Patching
![Page 21: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/21.jpg)
$£€¥
![Page 22: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/22.jpg)
![Page 23: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/23.jpg)
![Page 24: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/24.jpg)
Cloud
![Page 25: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/25.jpg)
案例
![Page 26: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/26.jpg)
![Page 27: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/27.jpg)
![Page 28: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/28.jpg)
课后提醒
![Page 29: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/29.jpg)
https://channel9.msdn.com/Events/Ignite/Microsoft-Ignite-China-2015
http://aka.ms/IgniteChina2015
![Page 30: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/30.jpg)
![Page 31: Presentation title goes heredownload.microsoft.com/download/C/F/F/CFF0A653-6CD6-4E52-B97… · Hadoop 2.2 and 2.4 80% data compression with ORC Hadoop on Windows Hive 100x Query Speed](https://reader034.vdocuments.site/reader034/viewer/2022042308/5ed444d14c96ec4df575db7c/html5/thumbnails/31.jpg)