Why Hadoop and SQL just want to be friendsSimon Elliston Ball
@sireb
ETL
OLTP
Archive
EDWETL
ETL
OLTP
Archive
EDWETL
ETL
OLTP
Archive
EDWETL
ETL
More dataShorter windowsWider queries
ETL
OLTP
Archive
EDWETL Sqoop
PigHive
OozieFalcon
ELT
ETL
OLTP
Archive
EDWETL
Less structured
Sqoop
ELT: saving the T for later2012-01-06 09:22:27 W3SVC1273337584 RD00155D360166 10.211.146.27 GET /ustensiles - 80 Test0001 94.245.127.11 HTTP/1.1 Mozilla/5.0+(compatible;+MSIE+9.0;+Windows+NT+6.1;+WOW64;+Trident/5.0) __RequestVerificationToken_Lw__=KLZ1dz1Aa4o2UdwJVwr0JhzSwmmSHmID9i/gutMvQkZWX9Q4QDktFHHiBhF8mSd6Cg5oIEeUpy/KNF7VLRFkrqN28raL8PfNuv0IfuKXxgl5s+uZpcvfGE6Olfsu7uNLg2bWwLZkrqXjv9cpRGaiXelmaM8=;+.ASPXAUTH=D5796612E924B60496C115914CC8F93239E99EEF4B3D6ED74BDD5C8C38D8C115D3021AB7F3B06E563EDE612BFBCBBE756803C85DECFACCA080E890C5DA6B4CA00A51792D812C93101F648505133C9E2C10779FA3E5AC19EE5E2B7E130C72C18F6309AEB736ABD06C87A7D636976A20534833E20160EC04B6B6617B378845AE627979EE54 http://site.supersimple.fr/Users/Account/LogOn?ReturnUrl=%2Fustensiles site.supersimple.fr 200 0 0 7136 849 1249
Schema on write:
ELT: saving the T for later
ParseModel Store Query
● Keep going back to the drawing board● Reprocessing all the data
Schema on read:
ELT: saving the T for later
● Only model what you need● Agile Data Modelling● Don’t move the data
QueryStore Model Parse
Cost per TB...
Come for the cheap storage...
The Data Lake
https://www.flickr.com/photos/msvg/5891279010
...stay for the analytics
Machine learning librariesRecommendation systemsBatch Big Data
Summary
Hadoop can:● Improve your ETL processing● Help you with unstructured data● Save you money
Thank you!Simon Elliston Ball
@sireb
