anti-caching in main memory database systems
DESCRIPTION
Anti-Caching in Main Memory Database Systems. Justin DeBrabant Brown University [email protected]. A bit of history…. 1974 – System R. query optimization recovery transaction serialization lots of locks. Application. Buffer Pool. Primary Storage. Change is Good. - PowerPoint PPT PresentationTRANSCRIPT
A bit of history…
1974 – System R
• query optimization • recovery • transaction serialization– lots of locks
Application
Primary Storage
Buffer Pool
Change is Good
1970 1973 1976 1979 1982 1985 1988 1991 1994 1997 2000 2003 2006 2009 20121E+00
1E+02
1E+04
1E+06
1E+08
1E+10
USD
($)
source: http://www.archivebuilders.com/whitepapers/22045p.pdf
Great, that’s what the buffer pool is for...right?
OLTP Through the Looking Glass, and What We Found ThereSIGMOD ‘08, pp. 981-992, 2008.
31%
31%
26%
12%
Buffer Pool
Locking
Recovery
Real Work
What to do with all this memory?
Application
Consistent?Updates?Writes?
Parallel Main MemoryTransaction Processing System
H-Store: A High-Performance, DistributedMain Memory Transaction Processing SystemVLDB 2008.
YCSB, Update-Heavy, data < memory
<>
Anti-Caching in H-Store
• Memory becomes primary storage for “hot” data
• “Cold” data is evicted to disk in blocks, fetched when requested by a transaction
• Still no locks/latches
Application
Anti-Cache
Primary Storage
YCSB, Update-Heavy, data > memory
15x
+ Anti-Caching > + Buffer
Pool
The New Traditional Wisdom
Future Work
• Alternative eviction strategies• Larger-than-memory queries• New hardware –flash, persistent memory
The Team
StanZdonik(Brown)
MikeStonebraker(MIT)
StephenTu(MIT)
AndyPavlo(Brown)
JustinDeBrabant(Brown)
Anti-Caching: A New Approach toDatabase Management System ArchitectureIn Preparation.
Questions?