newsql (vsnosqloroldsql)
TRANSCRIPT
![Page 1: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/1.jpg)
the NewSQL database you’ll never outgrow
NewSQL
(vs NoSQL or OldSQL)
Michael Stonebraker, CTOVoltDB, Inc.
![Page 2: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/2.jpg)
VoltDB 2
How Has OLTP Changed in 25 years
Professional terminal operator has been dis-‐intermediated (by the web)+ Sends volume through the roof
TransacNons originate from PDAs+ Sends volume through the roof
![Page 3: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/3.jpg)
VoltDB 3
How Has OLTP Changed in 25 years
Most OLTP can fit in main memory+ 1 Terabyte is a reasonably big OLTP data base
+ And fits in a modest 32 node cluster with 32 gigs/node
Nobody will send a message to a user inside atransacNon
+ Aunt Martha may have gone to lunch
![Page 4: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/4.jpg)
VoltDB 4
How Has OLTP Changed in 25 years
In 1985, 1,000 transacNons per second was consideredan incredible stretch goal!!!!+ HPTS (1985)
Now the goal is 2 – 4 orders of magnitude higher
![Page 5: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/5.jpg)
VoltDB 5 5VoltDB 5
New OLTP
You need to ingest a firehose in realNme
You need to perform high volumeOLTP
You o[en need real-‐Nme analyNcs
![Page 6: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/6.jpg)
VoltDB 6
SoluNon OpNons
OldSQL (the RDBMS elephants)
NoSQL (the 75 or so companies that suggestabandoning both SQL and ACID)
NewSQL (the companies that keep SQL and ACID, butwith a different architecture than the elephants)
![Page 7: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/7.jpg)
VoltDB 7
The Elephants (Unless You Squint)
Disk-‐based
Drank the Mohan koolaid (Aries)
Listened to Mike Carey (dynamic record-‐level locking)
AcNve-‐passive replicaNon
MulN-‐threaded
![Page 8: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/8.jpg)
VoltDB 8
Reality Check
TPC-‐C CPU cycles
On the Shore DBMS prototype
Elephants should be similar
![Page 9: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/9.jpg)
VoltDB 9
The Elephants
Are slow because they spend all of their Nme onoverhead!!!+ Not on useful work
Would have to completely re-‐architect their legacycode to do becer
Non-‐starter on New OLTP
![Page 10: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/10.jpg)
VoltDB 10
NoSQL
Give up SQL+ InteresNng to note that Cassandra and Mongo are moving to(yup) SQL
Give up ACID+ If you need ACID, this is a decision to tear your hair out by doingit in user code
+ Can you guarantee you won’t need
ACID tomorrow?
![Page 11: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/11.jpg)
VoltDB 11
What’s More…..
Much of the Old OLTP overhead is sNll there+MulN-‐threaded
+ Disk-‐based
+ Lighter weight locking
+ Some have WALs
![Page 12: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/12.jpg)
VoltDB 12
To Go a Lot Faster You Have to……
Focus on overhead+ Becer B-‐trees affects only 4% of the path length
Get rid of ALL major sources of overhead+Main memory deployment – gets rid of buffer pool
— Leaving other 75% of overhead intact
— i.e. win is 25%
![Page 13: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/13.jpg)
VoltDB 13
NewSQL
Keep SQL
Keep ACID
Go fast using some non-‐tradiNonal architecture
Examples: VoltDB, NuoDB (NimbusDB), Clustrix, Akiban
![Page 14: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/14.jpg)
VoltDB 14
NewSQL Example -‐-‐ VoltDB
Shared nothing architecture+ Sharded
+ Hash or range parNNoning
Main memory DBMS
Single threaded+Main memory divided (staNcally right now) per core
+ No shared data structures
Open source+ Cheaper sales model (no 4-‐legged sales teams)
![Page 15: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/15.jpg)
VoltDB 15
NewSQL Example -‐-‐ VoltDB
Stored procedure interface+ OLTP updates clearly use this model
+ Compiled on the fly for ad-‐hoc queries
All tables replicated (K-‐safety)+ IdenNcally sharded
+ LAN for now – we are just starNng to think about WANreplicaNon
+ AcNve-‐acNve replicaNon with failover/failback (HA built-‐in)
+ ACID!!!!
![Page 16: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/16.jpg)
VoltDB 16
NewSQL Example: VoltDB
Single node transacNons+ Run to compleNon at the correct node (for each replica)
+ In Nmestamp order (no locking)
MulN-‐node transacNons+Moving to speculaNve execuNon – with backout if necessary
+ There are a bunch of other ideas that we might try also
![Page 17: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/17.jpg)
VoltDB 17
VoltDb Summary
No buffer pool
No locking (no Mike Carey)
No WAL (no Mohan)
No threading overhead
![Page 18: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/18.jpg)
VoltDB 18
Fast!!!!!!
60 X an elephant on TPC-‐C
8 X Cassandra
1 X Memcached (using key-‐value stored procedurelayer)
![Page 19: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/19.jpg)
VoltDB 19
Scalable!!!
Linearly with the number of cores in a node+ No shared data structures
Linearly with the number of nodes
Up to the biggest cluster (30 X 8) we could get ourhands on+ Ran 3.4M simple xacts/sec.
![Page 20: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/20.jpg)
VoltDB 20
What About Cluster-‐wide Failures?
How?+ Power failure with no UPS
+ Bohr bugs
+ User errors (RM *)
Can’t cause a data loss……
![Page 21: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/21.jpg)
VoltDB 21
What Does VoltDB Do?
TransacNon consistent checkpoinNng (currentlyconNnuous & asynchronous)+ cheap
Stored procedure log (#4, “diet coke”)+ Group commit
![Page 22: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/22.jpg)
VoltDB 22
What Does VoltDB Do?
Restore latest checkpoint & re-‐execute storedprocedures a[er checkpoint
Compared this to Aries (paper almost ready)+ Run-‐Nme overhead way way way smaller
+ Recovery Nme longer, since we don’t currently checkpointindexes
+With indexes, it would be about the same……
Summary: Mohan lite…..
![Page 23: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/23.jpg)
VoltDB 23
VoltOne – For the Low End
Single node system
With command logging
No replicaNon
No HA
![Page 24: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/24.jpg)
VoltDB 24
What if your DB is Too Big?
If OLTP data gets stale+ Age out to a companion data warehouse
+ Or HDFS or ….
+ Here now
If 90/10 rule is true+ Use “semanNc caching”
+ Subset of the rows are in memory. TransacNon is stalled whileneeded data is loaded, then it is run
+ Coming soon
![Page 25: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/25.jpg)
VoltDB 25
Shipping V 2.0
A dozen or so producNon customers+ That we know about…..
A couple thousand downloads a month
Current VoltDB Status
![Page 26: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/26.jpg)
VoltDB 26
WAN replicaNon
SemanNc caching
Compression (messages, data, command log)
More SQL
On-‐the-‐fly reprovisioning
Next Steps
![Page 27: NewSQL (vsNoSQLorOldSQL)](https://reader035.vdocuments.site/reader035/viewer/2022081622/613d59b4736caf36b75c4aaf/html5/thumbnails/27.jpg)
VoltDB 27
Disk vs main memory
Dynamic locking vs something else
AcNve-‐passive vs acNve-‐acNve
Data log vs stored procedure log
Slow vs fast
OldSQL vs NewSQL