unseating the giants
DESCRIPTION
Unseating the Giants. Monte Zweben CEO, Splice Machine October 16, 2014. The Big Squeeze. Data growing much faster than IT budgets. Source: 2013 IBM Briefing Book. Source: Gartner, Worldwide IT, Spending forecast, 3Q13 Update. Traditional RDBMSs Giants Overwhelmed… - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/1.jpg)
Unseating the Giants
Monte ZwebenCEO, Splice Machine
October 16, 2014
![Page 2: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/2.jpg)
2
The Big SqueezeData growing much faster than IT budgets
Source: 2013 IBM Briefing Book
Source: Gartner, Worldwide IT, Spending forecast, 3Q13 Update
![Page 3: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/3.jpg)
Traditional RDBMSs Giants Overwhelmed…Scale-up becoming cost-prohibitive
Splice Machine | Proprietary & Confidential
![Page 4: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/4.jpg)
4
Scale-Out: The Future of DatabasesDramatic improvement in price/performance
Scale Up(Increase server size)
Scale Out(More small servers)
vs.$ $ $ $ $ $
![Page 5: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/5.jpg)
5
Unseating the Giants
vs.
Scale-Up Giants
Scale-Out Challengers
![Page 6: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/6.jpg)
6
Scale-Out Example #1
![Page 7: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/7.jpg)
New application chooses NoSQL
Splice Machine | Proprietary & Confidential
![Page 8: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/8.jpg)
8
ADVERTISER ROCKET FUEL
145RTB advertisingsupply partners
21,103,424Websites
19BnDaily impressions
###MM WW CONSUMERS91,999 DEVICES
![Page 9: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/9.jpg)
Rocket Fuel: New Application
AdExchange
Rocket Fuel Platform
Auto Optimization
Real-Time Bidding
Publishers
Exchanges
Ad networks
Advertisers
Data Providers
![Page 10: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/10.jpg)
10
$2.38965$0.6782$1.7234
$0.09$1.78964$1.6782$1.7234$0.809$2.421.25
$2.11$1.26
$2.178$2.056$0.809$2.421.25
$2.11$1.26$2.78$1.56
$1.809$2.421.25
$2.11$1.26$2.78$0.56$2.421.25
$2.11$1.26$2.78
$0.756$0.809$2.421.25
$2.11$1.26$2.78
$1.256$1.809$2.421.25
$2.11$1.26$2.78
$0.586$2.009
1.25$2.11$1.26$2.78$1.56
$0.00
[ + ][ + ]
Site/PageGeo/WeatherTime of DayBrand AffinityUser
![Page 11: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/11.jpg)
![Page 12: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/12.jpg)
12
Hourly
Refresh
RTB
EXCHANGES
Bid Servers
HB
AS
E
User Profile Store
Ad ServersPixel Servers
Direct Publishers
H D F S
Master
Slaves
ETL
Bidder Logs Ad Server Logs
LikelihoodScores & Bid Value
Master Database
Apollo Ad-hoc Analytics Tools
Campaign Framework
Response Prediction
Models
Eligible Ads
ExchangePublishers
Apollo Reporting & Campaign Tools
Bid Call
Tag for Selected Ad & Bid
Rocket Fuel Ad Tag
Ad Creative Tag
Response Prediction
Models
UserLookup
UserLookup
& Update
Evaluate Ads
Load Balancer
Load Balancer
Hourly
Refresh
Ad-Rejection
1
2
3
45
6
7
8
9
![Page 13: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/13.jpg)
13Splice Machine Proprietary and Confidential
HBase: Proven Scale-Out
Auto-sharding Scales with commodity hardware Cost-effective from GBs to PBs
High availability thru failover and replication
LSM-trees
![Page 14: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/14.jpg)
14
Rocket Fuel: Results
Facebook likes
Searches on Google
Bid Requests Considered by Rocketfuel
5 B
6 B
45 B
Requests per day
World class request velocity on over 10 PBs of data
![Page 15: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/15.jpg)
15
Scale-Out Example #2
![Page 16: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/16.jpg)
Web application replaces Oracle
Splice Machine | Proprietary & Confidential
![Page 17: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/17.jpg)
17
Before Architecture: OracleOracle too expensive, too slow, and too difficult to scale and modify
Metadata Storage
Shutterfly Website
Photo File Storage
UploaderApp
Consumers
![Page 18: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/18.jpg)
18
After Architecture: MongoDBFlexibility and scalability of NoSQL ideal for simple web app
Metadata Storage
Shutterfly Website
Photo File Storage
UploaderApp
Consumers
![Page 19: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/19.jpg)
19
MongoDB ArchitectureDocument data model sharded across commodity servers
![Page 20: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/20.jpg)
20
MongoDB: Compelling Results vs. Oracle
⅕ costwith commodity scale out
9x fasterthrough parallelized queries
Increased agilitywith flexible schema and “shard on demand”
![Page 21: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/21.jpg)
21
Scale-Out Example #3
![Page 22: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/22.jpg)
Splice Machine | Proprietary & Confidential
Existing OLTP & OLAP Apps Replace Oracle
![Page 23: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/23.jpg)
23
Before Architecture: Oracle RACOracle RAC too expensive and too slow, with queries up to ½ hour
• Operational Reports for Campaign Performance• Ad Hoc Audience
Segmentation
Social Feeds
Web/eCommerce Clickstreams
ETL
1st Party/CRM Data
3rd Party Data (e.g., Axciom)
POS Data
Email Marketing
Data Quality
![Page 24: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/24.jpg)
24
After Architecture: Hadoop RDBMSRDBMS functionality with proven scale-out from Hadoop
• Operational Reports for Campaign Performance• Ad Hoc Audience
Segmentation
Social Feeds
Web/eCommerce Clickstreams
ETL
1st Party/CRM Data
3rd Party Data (e.g., Axciom)
POS Data
Email Marketing
Data Quality
![Page 25: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/25.jpg)
25
Hadoop RDBMS: Best of Both Worlds
Scale-out on commodity servers Proven to 100s of petabytes Efficiently handle sparse data Extensive ecosystem
RDBMS ANSI SQL Real-time, concurrent updates ACID transactions ODBC/JDBC support
Hadoop
![Page 26: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/26.jpg)
26
Distributed, Parallelized Query Execution
Parallelized computation across clusterMoves computation to the dataUtilizes HBase co-processorsNo MapReduce
HBase Co-Processor
HBase Server Memory Space
L EG EN D
![Page 27: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/27.jpg)
27
Hadoop RDBMS: Compelling Results vs. Oracle
¼ costwith commodity scale out
3-7x fasterthrough parallelized queries
10-20x price/perfwith no application, BI or ETL rewrites
![Page 28: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/28.jpg)
28
Scale-Up vs. Scale-Out
Scale-Up: Top Reasons1. Willing to pay for engineered systems
2. Lots of custom code (e.g., PL/SQL)
3. Proven reliability
4. Avoid risk of newer technologies
5. Less migration required
Scale-Out: Top Reasons6. Reduce costs by 4x-10x
7. Increase performance by 3x-10x
8. Ease of scalability
9. Support for flexible schemas
10.Huge ecosystem of open source tools
![Page 29: Unseating the Giants](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813214550346895d986dd0/html5/thumbnails/29.jpg)
29
Unseating the Giants: Why is it different this time?It’s not just technology – user requirements fundamentally changed
Seismic User Shift• Budgets flat• Massive increase in data:
- Volume- Velocity- Variety
• No longer acceptable to throw data away
Disruptive Tech: Scale-Out
• Leverage commodity H/W• Reduce costs by 4-5x• Increase perf by 5-10x• Increase agility