recipe : converting concurrent dram indexes to persistent...
TRANSCRIPT
![Page 1: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/1.jpg)
RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes
Se Kwon Lee, Jayashree Mohan, Sanidhya Kashyap*,
Taesoo Kim, Vijay Chidambaram
1*On the job market
![Page 2: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/2.jpg)
Persistent Memory (PM)
2
Intel Optane DC Persistent Memory
• New storage class memory technology
• Performance similar to DRAM
• Non-volatile & high-capacity
• Up-to 6TB on a single machine
![Page 3: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/3.jpg)
3
• PM has high capacity and low latency• 6TB on a single machine → 100 billion 64-byte key-value pairs
• Indexing data on PM is crucial for efficient data access
Indexing on PM
![Page 4: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/4.jpg)
PM Indexes need to achieve three goals simultaneously
PM Indexes
4
PM
Crash
ConsistencyConcurrency
Cache
Efficiency
![Page 5: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/5.jpg)
PM
• Cache Efficiency• Persistent memory is attached to the memory bus
• 3x higher latency than DRAM → More cache-sensitive
PM Indexes
5
Crash
ConsistencyConcurrency
Cache
Efficiency
![Page 6: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/6.jpg)
PM
• Concurrency• High concurrency is necessary for scalability on any modern
multicore platform
PM Indexes
6
Crash
ConsistencyConcurrency
Cache
Efficiency
![Page 7: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/7.jpg)
PM
• Crash Consistency• CPU cache is still volatile
• Arbitrarily-evicted cache lines → Persistence reordering
PM Indexes
7
Crash
ConsistencyConcurrency
Cache
Efficiency
![Page 8: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/8.jpg)
logcommit
Core
Cache
• Crash Consistency• CPU cache is still volatile
• Arbitrarily-evicted cache lines → Persistence reordering
PM Indexes
8
Volatile
Persistent
Program order
write (log);
write (commit);
① ②
commitlog
![Page 9: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/9.jpg)
commit
Core
Cache
• Crash Consistency• CPU cache is still volatile
• Arbitrarily-evicted cache lines → Persistence reordering
PM Indexes
9
Volatile
Persistent
Persistence reordering
write (log);
write (commit);log Reordered
commit
![Page 10: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/10.jpg)
commit
commit
Core
Cache
• Crash Consistency• CPU cache is still volatile
• Arbitrarily-evicted cache lines → Persistence reordering
PM Indexes
10
Volatile
Persistent
Persistence reordering
write (log);
write (commit);log
commit Inconsistency
Crash
![Page 11: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/11.jpg)
Core
Cache
• Crash Consistency• CPU cache is still volatile
• Arbitrarily-evicted cache lines → Persistence reordering
• Flush: persist writes to PM
• Fence: ensure one write prior another to be persisted first
PM Indexes
11
Volatile
Persistent commitlog
Consistent persistence ordering
write (log)
flush (log)
fence ()
write (commit)
flush (commit)
fence ()
![Page 12: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/12.jpg)
Challenge in building PM indexes
12
Correct
Concurrency
Correct
Crash
Consistency
Consistent
Data
Conflict Crash
Correctness condition: return previouslyinserted data without data loss or corruption
![Page 13: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/13.jpg)
Bug
Bug
Challenge in building PM indexes
13
Concurrency and crash consistency interact witheach other, a bug in either can lead to data loss
Bug
Concurrency
Bug
Crash
Consistency
Data
Loss
Conflict Crash
![Page 14: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/14.jpg)
• We found bugs in FAST&FAIR [FAST’18] and CCEH [FAST’19]
• FAST&FAIR: Concurrent PM-based B+Tree
• One bug in concurrency mechanism
• Two bugs in recovery mechanism
• CCEH: Concurrent PM-based dynamic hash table
• One bug in concurrency mechanism
• One bug in recovery mechanism
Bug in Concurrent PM Index
14
![Page 15: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/15.jpg)
15
How can we reduce the effort involved in building concurrent, crash-consistent PM
indexes?
Answer: We can convert concurrent DRAM indexes to PM indexes with low effort
Insight: Isolation and Crash Consistency are similar
![Page 16: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/16.jpg)
16
How can we reduce the effort involved in building concurrent, crash-consistent PM
indexes?
Approach: Convert concurrent DRAM indexes to PM indexes with low effort
Insight: Isolation and Crash Consistency are similar
![Page 17: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/17.jpg)
• Already designed for cache efficiency and concurrency
DRAM Index
17
Concurrency
Cache
Efficiency
T-Tree
CSS-
Tree CSB+
Tree
BD-
Tree
FAST
Bw-
Tree
ART
HOT
Mass
tree
1986
CLHT
2010 2019
Ca
ch
e E
ffic
ien
cy
Concurr
ency
Time
![Page 18: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/18.jpg)
18
Crash
DRAM Index
Volatile
Concurrency
Cache
Efficiency
DRAM Index
Concurrency
Cache
Efficiency
Crash
Vulnerable
DRAM Index
![Page 19: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/19.jpg)
Challenge in Conversion
19
• Require minimal changes to DRAM index• Without modifying the original design principles of DRAM index
DRAM Index
Volatile
Concurrency
Cache
Efficiency
PM Index
Concurrency
Cache
Efficiency
Crash
Consistency
Conversion
![Page 20: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/20.jpg)
201. Steven Pelley et al., Memory Persistency, ISCA’14
• Similar semantics between isolation and consistency1
• Isolation• Return consistent data while multiple active threads are running
• Crash consistency• Return consistent data even after a crash happens at any point
Insight for Conversion
![Page 21: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/21.jpg)
• Similar semantics between isolation and consistency1
• Isolation• Return consistent data while multiple active threads are running
• Crash consistency• Return consistent data even after a crash happens at any point
211. Steven Pelley et al., Memory Persistency, ISCA’14
Insight for Conversion
Approach: reuse mechanisms for isolation in DRAM
indexes to obtain crash consistency
![Page 22: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/22.jpg)
RECIPE
• Principled approach to convert DRAM indexes into PM indexes
• Case study of changing five popular DRAM indexes
• Conversion involves different data structures such as Hash
Tables, B+ Trees, and Radix Trees
• Conversion required modifying <= 200 LOC
• Up-to 5.2x better performance in multi-threaded evaluation
22
RECIPE
https://github.com/utsaslab/RECIPE
![Page 23: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/23.jpg)
• Overall Intuition
• Conversion Conditions
• Conversion Example: Masstree
• Assumptions & Limitations
• Evaluation
Outline
23
![Page 24: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/24.jpg)
• Overall Intuition
• Conversion Conditions
• Conversion Example: Masstree
• Assumptions & Limitations
• Evaluation
Outline
24
![Page 25: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/25.jpg)
Overall Intuition for Conversion
25
• Blocking algorithms• Use explicit locks to prevent the conflicts of threads to shared
data
• Non-blocking algorithms• Use well-defined invariants and ordering constraints without
locks
• Employed by most high-performance DRAM indexes
![Page 26: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/26.jpg)
Overall Intuition for Conversion
26
• Non-blocking algorithms• Readers Detect and Tolerate inconsistencies
• E.g., Ignore duplicated keys
InconsistencyReader
Tolerate
Detect
![Page 27: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/27.jpg)
Overall Intuition for Conversion
27
• Non-blocking algorithms• Readers Detect and Tolerate inconsistencies
• E.g., Ignore duplicated keys
• Writers also Detect, but Fix inconsistencies
• E.g., Eliminate duplicated keys
WriterDetect Fix
ConsistentInconsistency
![Page 28: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/28.jpg)
Overall Intuition for Conversion
28
• Non-blocking algorithms• Readers Detect and Tolerate inconsistencies
• Writers also Detect, but Fix inconsistencies
• Helping mechanism1 ≈ Crash Recovery2
• Such indexes are *inherently* crash consistent
WriterDetect Fix
ConsistentInconsistency
1. Keren Censor-Hillel et al., Help!, PODC’15
2. Ryan Berryhill et al., Robust shared objects for non-volatile main memory, OPODIS’15
![Page 29: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/29.jpg)
• Not all DRAM indexes can be converted with low effort
• Exploit inherent crash recovery in the index
• Provide specific conditions that must hold for a DRAM index to be converted
• Provide a matching conversion actions for each condition
29
![Page 30: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/30.jpg)
• Overall Intuition
• Conversion Conditions
• Conversion Example: Masstree
• Assumptions & Limitations
• Evaluation
Outline
30
![Page 31: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/31.jpg)
• Condition 1: Updates via Single Atomic Store
• Condition 2: Writers fix inconsistencies
• Condition 3: Writers don’t fix inconsistencies
• Conditions are not exhaustive!
Three Conversion Conditions
31
![Page 32: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/32.jpg)
• Condition 1: Updates via Single Atomic Store
• Condition 2: Writers fix inconsistencies
• Condition 3: Writers don’t fix inconsistencies
Three Conversion Conditions
32
![Page 33: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/33.jpg)
Condition 1: Updates via Single Atomic Store
33
• Non-blocking readers, (Non-blocking or Blocking) writers
• Updates become visible to other threads via single atomic commit store
Atomic Store
Crash
Step 1 Step 2 Step N… Step 1 Step 2 Step N…
Invisible Visible
![Page 34: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/34.jpg)
Condition 1: Updates via Single Atomic Store
34
• Updates become visible to other threads via single atomic commit store
• Conversion: Add flushes after each store and bind final atomic store using fences
Atomic Store
Step 1 Step 2 Step N… Step 1 Step 2 Step N…
Flushes
Flush
FenceFence
![Page 35: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/35.jpg)
• Condition 1: Updates via Single Atomic Store
• Condition 2: Writers fix inconsistencies
• Condition 3: Writers don’t fix inconsistencies
Three Conversion Conditions
35
![Page 36: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/36.jpg)
Condition 2: Writers fix inconsistencies
36
• Non-blocking readers and writers (don’t hold locks)
• Readers & Writers → Detect ( ), Tolerate ( ), Fix ( )
Step 1 Step 2 Step 3
Update
Writer 1
A sequence of ordered deterministic steps
Commit
Step
![Page 37: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/37.jpg)
Step 3
Condition 2: Writers fix inconsistencies
37
• Non-blocking readers and writers (don’t hold locks)
• Readers & Writers → Detect ( ), Tolerate ( ), Fix ( )
Step 1 Step 2Writer 1
Reader
Detect
Commit
Step
Tolerate
![Page 38: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/38.jpg)
Condition 2: Writers fix inconsistencies
38
• Non-blocking readers and writers (don’t hold locks)
• Readers & Writers → Detect ( ), Tolerate ( ), Fix ( )
Step 1 Step 2Writer 1
Writer 2Step 3
Fix
Detect
Commit
Step
![Page 39: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/39.jpg)
PM
Condition 2: Writers fix inconsistencies
39
• Readers & Writers → Detect ( ), Tolerate ( ), Fix ( )• Inherently crash recoverable
Step 1 Step 2 Step 3Writer 1Commit
Step
![Page 40: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/40.jpg)
PM
Condition 2: Writers fix inconsistencies
40
• Readers & Writers → Detect ( ), Tolerate ( ), Fix ( )• Inherently crash recoverable
Step 1 Step 2 Step 3Writer 1
Crash
Commit
Step
![Page 41: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/41.jpg)
PM
Condition 2: Writers fix inconsistencies
41
• Readers & Writers → Detect ( ), Tolerate ( ), Fix ( )• Inherently crash recoverable
Step 1 Step 2Thread Step 3
Recover
Commit
Step
![Page 42: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/42.jpg)
PM
Condition 2: Writers fix inconsistencies
42
• Readers & Writers → Detect ( ), Tolerate ( ), Fix ( )• Inherently crash recoverable
• Conversion: Adding flushes and fences after each store and specific loads
Step 2Thread Step 3
Flushes Flushes Flushes Flushes
Fence Fence Fence
Step 1Commit
Step
![Page 43: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/43.jpg)
• Condition 1: Updates via Single Atomic Store
• Condition 2: Writers fix inconsistencies
• Condition 3: Writers don’t fix inconsistencies
Three Conversion Conditions
43
![Page 44: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/44.jpg)
• Non-blocking readers, Blocking writers (hold locks)
• Readers & Writers → Detect ( ), Tolerate ( ), Fix (X)
Condition 3: Writers don’t fix inconsistencies
44
Step 1 Step 2 Step 3Writer 1
Step 1 Step 2 Step 3Writer 2
Update
A sequence of ordered deterministic steps
UpdateCommit
Step
Commit
Step
A sequence of ordered deterministic steps
![Page 45: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/45.jpg)
• Non-blocking readers, Blocking writers (hold locks)
• Readers & Writers → Detect ( ), Tolerate ( ), Fix (X)
Condition 3: Writers don’t fix inconsistencies
45
Step 1 Step 2 Step 3Writer 1
Step 1 Step 2 Step 3Writer 2
Commit
Step
Commit
Step
![Page 46: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/46.jpg)
• Non-blocking readers, Blocking writers (hold locks)
• Readers & Writers → Detect ( ), Tolerate ( ), Fix (X)
Condition 3: Writers don’t fix inconsistencies
46
Step 1 Step 2Writer 1
Step 1 Step 2 Step 3Writer 2
Crash
Commit
Step
Failure
![Page 47: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/47.jpg)
Writer 2
• Readers & Writers → Detect ( ), Tolerate ( ), Fix ( )
• Conversion: Add helping mechanism• Reuse existing algorithm handling each step
Condition 3: Writers don’t fix inconsistencies
47
Step 1 Step 2 Step 3Thread
Step 1 Step 2
Detect
FixCommit
Step
![Page 48: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/48.jpg)
• Overall Intuition
• Conversion Conditions
• Conversion Example: Masstree
• Assumptions & Limitations
• Evaluation
Outline
48
![Page 49: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/49.jpg)
• Example: B-link Tree (Masstree)
Conversion of Masstree
49
10
1 10 15 25
30
30
…
…
High Key
High Key
![Page 50: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/50.jpg)
• Example: B-link Tree (Masstree)
Conversion of Masstree
50
Insert 26
1510
1 10
30
30
…
15 25
1. installing new sibling
2. insert middle key to parent
![Page 51: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/51.jpg)
• Example: B-link Tree (Masstree)
Conversion of Masstree
51
Insert 26
10
1 10
30
30
…
15 25
Intermediate state
![Page 52: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/52.jpg)
• Example: B-link Tree (Masstree)
Conversion of Masstree
52
Insert 26
10
1 10
30
30
…
Lookup 25
15 25
![Page 53: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/53.jpg)
• Example: B-link Tree (Masstree)
Conversion of Masstree
53
Insert 26
10
1 10
30
30
…
Lookup 25
15 25
Detect
(25 > 15)
>>>>
Tolerate
![Page 54: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/54.jpg)
• Example: B-link Tree (Masstree)
Conversion of Masstree
54
10
1 10
30
30
…
15 25
Crash Permanent
Inconsistency
![Page 55: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/55.jpg)
• Example: B-link Tree (Masstree)• Add helping mechanism to resume split
Conversion of Masstree
55
10
1 10
30
30
…
15 25
Insert 30
Detect
15
Resume split (recovery)
![Page 56: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/56.jpg)
Conversion Results of Five DRAM Indexes
56
DRAM Index DS Type
CLHT (Cache-Line Hash Table) [ASPLOS’15] Hash table
HOT (Height Optimized Trie) [SIGMOD’18] Trie
BwTree [ICDE’13] B+Tree
ART (Adaptive Radix Tree) [ICDE’13] Radix Tree
Masstree [Eurosys’12] Hybrid (B+Tree & Trie)
![Page 57: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/57.jpg)
Conversion Results of Five DRAM Indexes
57
DRAM Index PM Index Condition
CLHT P-CLHT #1
HOT P-HOT #1
BwTree P-BwTree #1, #2
ART P-ART #1, #3
Masstree P-Masstree #1, #3
• We produce the P-* family of PM indexes
![Page 58: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/58.jpg)
• Overall Intuition
• Conversion Conditions
• Conversion Example: Masstree
• Assumptions & Limitations
• Evaluation
Outline
58
![Page 59: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/59.jpg)
• Assume garbage collection in memory allocator
• Assume locks are volatile or re-initialized after a crash
• Provide low level of isolation: Read Uncommitted
• RECIPE applies only to individual data structures
Assumptions & Limitations
59
![Page 60: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/60.jpg)
• Overall Intuition
• Conversion Conditions
• Conversion Example: Masstree
• Assumptions & Limitations
• Evaluation
Outline
60
![Page 61: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/61.jpg)
Evaluation
61
• How much effort is involved in converting indexes?
• What is the performance of converted indexes?
• Are the converted indexes crash consistent?
![Page 62: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/62.jpg)
Evaluation
62
• How much effort is involved in converting indexes?
• What is the performance of converted indexes?
• Are the converted indexes crash consistent?
![Page 63: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/63.jpg)
Evaluation
63
• How much effort is involved in converting indexes?
• What is the performance of converted indexes?
![Page 64: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/64.jpg)
Modified Lines of Code
64
• Conversion for all indexes → <= 200 LoC changes
RECIPE-converted IndexesLines of Code
Index Core Modified
P-CLHT 2.8K 30 (1%)
P-HOT 2K 38 (2%)
P-BwTree 5.2K 85 (1.6%)
P-ART 1.5K 52 (3.4%)
P-Masstree 2.2K 200 (9%)
![Page 65: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/65.jpg)
Modified Lines of Code
65
• Conversion for all indexes → <= 200 LoC changes
RECIPE-converted IndexesLines of Code
Index Core Modified
P-CLHT 2.8K 30 (1%)
P-HOT 2K 38 (2%)
P-BwTree 5.2K 85 (1.6%)
P-ART 1.5K 52 (3.4%)
P-Masstree 2.2K 200 (9%)
Conversion for all indexes: <= 200 LoC changes
<= 9% from core code base
![Page 66: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/66.jpg)
Evaluation
66
• How much effort is involved in converting indexes?
• What is the performance of converted indexes?
![Page 67: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/67.jpg)
• 2-socket 96-core machine with 32MB LLC
• 768 GB Intel Optane DC PMM, 378 GB DRAM
• YCSB with 16 threads
• Ordered/Unordered indexes, Integer/String keys
Performance Evaluation
67
Load Workload A Workload B Workload C Workload E
Insertion 100%Insertion 50%
Point Lookup 50%
Insertion 5%
Point Lookup 95%
Point Lookup
100%
Insertion 5%
Range Scan 95%
![Page 68: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/68.jpg)
Ordered Index
68
• Support both point and range operations
• P-HOT
• Persistent Height-Optimized Trie converted by RECIPE
• FAST & FAIR [FAST’18]
• Hand-crafted PM-based concurrent B+Tree
![Page 69: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/69.jpg)
Ordered Index
69
• P-HOT produced by RECIPE conversion
• P-HOT performs up-to 5.2x better in point operations
• Cache-efficient designs of P-HOT → Low cache misses
0123456
Load A B C E
No
rma
lize
d th
rou
gh
pu
t FAST&FAIR P-HOT
![Page 70: RECIPE : Converting Concurrent DRAM Indexes to Persistent ...vijay/papers/sosp19-recipe-slides.pdf · RECIPE : Converting Concurrent DRAM Indexes to Persistent-Memory Indexes Se Kwon](https://reader034.vdocuments.site/reader034/viewer/2022042310/5ed8903f6714ca7f476828c3/html5/thumbnails/70.jpg)
RECIPE
• Principled approach to convert concurrent DRAM indexes into PM indexes
• Case study of changing five DRAM indexes
• Evaluations with YCSB show RECIPE indexes have better performance than hand-crafted PM indexes
• Try our indexes: https://github.com/utsaslab/RECIPE
70
RECIPE