![Page 1: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/1.jpg)
Andy Pavlo // Carnegie Mellon University // Spring 2016
Lecture #14 – Logging & Recovery (Alternative Methods)
DATABASE SYSTEMS
15-721
![Page 2: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/2.jpg)
CMU 15-721 (Spring 2016)
TODAY ’S AGENDA
Course Announcements Physical Logging Clarification Command Logging In-Memory Checkpoints Shared Memory Restarts
2
![Page 3: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/3.jpg)
CMU 15-721 (Spring 2016)
COURSE ANNOUNCEMENTS
Project #2 is now due March 9th @ 11:59pm
Project #3 proposals are still due March 14th
No Mandatory Reading for March 2nd
3
![Page 4: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/4.jpg)
CMU 15-721 (Spring 2016)
GRADE BREAKDOWN
Reading Reviews (10%) Project #1 (10%) Project #2 (25%) Project #3 (45%) Final Exam (10%) Extra Credit (+10%)
4
![Page 5: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/5.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888
![Page 6: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/6.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888
![Page 7: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/7.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888
UPDATE employees SET salary = salary * 1.10
![Page 8: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/8.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888
UPDATE employees SET salary = salary * 1.10
![Page 9: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/9.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888
UPDATE employees SET salary = salary * 1.10
$110
$732
![Page 10: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/10.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
UPDATE employees SET salary = salary * 1.10
$110
$732
![Page 11: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/11.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
UPDATE employees SET salary = salary * 1.10
$110
$732
![Page 12: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/12.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888 $900
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
UPDATE employees SET salary = salary * 1.10
$110
$732
![Page 13: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/13.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888 $900
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
UPDATE employees SET salary = salary * 1.10
$110
$732
![Page 14: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/14.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888 $900
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
UPDATE employees SET salary = salary * 1.10
$110
$732
![Page 15: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/15.jpg)
CMU 15-721 (Spring 2016)
Logical Log
LOGICAL LOGGING EXAMPLE
5
UPDATE employees SET salary = salary * 1.10
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
NAME SALARY
O.D.B. $100
El-P $666
Joy $888 $900
UPDATE employees SET salary = 900 WHERE name = ‘Joy’
UPDATE employees SET salary = salary * 1.10
$110
$732
X
![Page 16: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/16.jpg)
CMU 15-721 (Spring 2016)
SILOR – ARCHITECTURE
6
Epoch Thread
epoch=100
![Page 17: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/17.jpg)
CMU 15-721 (Spring 2016)
SILOR – ARCHITECTURE
6
Epoch Thread
epoch=100
![Page 18: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/18.jpg)
CMU 15-721 (Spring 2016)
SILOR – ARCHITECTURE
6
Epoch Thread
P epoch=100
![Page 19: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/19.jpg)
CMU 15-721 (Spring 2016)
SILOR – ARCHITECTURE
6
Epoch Thread
P epoch=200
![Page 20: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/20.jpg)
CMU 15-721 (Spring 2016)
SILOR – ARCHITECTURE
6
Epoch Thread
P
epoch=200
epoch=200
epoch=200
pepoch=200
epoch=200
![Page 21: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/21.jpg)
CMU 15-721 (Spring 2016)
SILOR – ARCHITECTURE
6
Epoch Thread
P
epoch=200
epoch=200
epoch=200
pepoch=200
epoch=200
![Page 22: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/22.jpg)
CMU 15-721 (Spring 2016)
SILOR – ARCHITECTURE
6
Epoch Thread
P
epoch=200
epoch=200
epoch=200
pepoch=200
epoch=200
Txn #101
![Page 23: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/23.jpg)
CMU 15-721 (Spring 2016)
LOGGING SCHEMES
Physical Logging → Record the changes made to a specific record in the
database. → Slower for execution, faster for recovery.
Logical Logging → Record the high-level operations executed by txns. → Faster for execution, slower for recovery.
7
![Page 24: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/24.jpg)
CMU 15-721 (Spring 2016)
OBSERVATION
Node failures in OLTP databases are rare. → OLTP databases are not that big. → They don’t need to run on hundreds of machines.
It’s better to optimize the system for runtime operations rather than failure cases.
8
![Page 25: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/25.jpg)
CMU 15-721 (Spring 2016)
COMMAND LOGGING
Logical logging scheme where the DBMS only records the stored procedure invocation → Stored Procedure Name → Input Parameters → Additional safety checks
Command Logging = Transaction Logging
9
RETHINKING MAIN MEMORY OLTP RECOVERY ICDE 2014
![Page 26: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/26.jpg)
CMU 15-721 (Spring 2016)
DETERMINISTIC CONCURRENCY CONTROL
For a given state of the database, the execution of a serial schedule will always put the database in the same new state if: → The order of txns (or their queries) is defined before
they start executing. → The txn logic is deterministic.
10
![Page 27: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/27.jpg)
CMU 15-721 (Spring 2016)
DETERMINISTIC CONCURRENCY CONTROL
For a given state of the database, the execution of a serial schedule will always put the database in the same new state if: → The order of txns (or their queries) is defined before
they start executing. → The txn logic is deterministic.
10
A=100
![Page 28: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/28.jpg)
CMU 15-721 (Spring 2016)
DETERMINISTIC CONCURRENCY CONTROL
For a given state of the database, the execution of a serial schedule will always put the database in the same new state if: → The order of txns (or their queries) is defined before
they start executing. → The txn logic is deterministic.
10
A=100
A = A + 1 Txn #1
A = A × 3 Txn #2
A = A - 5 Txn #3
![Page 29: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/29.jpg)
CMU 15-721 (Spring 2016)
DETERMINISTIC CONCURRENCY CONTROL
For a given state of the database, the execution of a serial schedule will always put the database in the same new state if: → The order of txns (or their queries) is defined before
they start executing. → The txn logic is deterministic.
10
A=100
A = A + 1 Txn #1
A = A × 3 Txn #2
A = A - 5 Txn #3
A=298
![Page 30: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/30.jpg)
CMU 15-721 (Spring 2016)
DETERMINISTIC CONCURRENCY CONTROL
For a given state of the database, the execution of a serial schedule will always put the database in the same new state if: → The order of txns (or their queries) is defined before
they start executing. → The txn logic is deterministic.
10
A=100
A = A + 1 Txn #1
A = A × 3 Txn #2
A = A - 5 Txn #3
A = A × NOW()
![Page 31: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/31.jpg)
CMU 15-721 (Spring 2016)
DETERMINISTIC CONCURRENCY CONTROL
For a given state of the database, the execution of a serial schedule will always put the database in the same new state if: → The order of txns (or their queries) is defined before
they start executing. → The txn logic is deterministic.
10
A=100
A = A + 1 Txn #1
A = A × 3 Txn #2
A = A - 5 Txn #3
A = A × NOW() X
![Page 32: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/32.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
![Page 33: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/33.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Partitions
![Page 34: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/34.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Partitions
Single-threaded Execution Engines
![Page 35: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/35.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Procedure Name Input Params
![Page 36: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/36.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Procedure Name Input Params run(phoneNum, contestantId, currentTime) {
result = execute(VoteCount, phoneNum); if (result > MAX_VOTES) { return (ERROR); } execute(InsertVote, phoneNum, contestantId, currentTime); return (SUCCESS); }
VoteCount: SELECT COUNT(*) FROM votes WHERE phone_num = ?;
InsertVote: INSERT INTO votes VALUES (?, ?, ?);
![Page 37: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/37.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Procedure Name Input Params run(phoneNum, contestantId, currentTime) {
result = execute(VoteCount, phoneNum); if (result > MAX_VOTES) { return (ERROR); } execute(InsertVote, phoneNum, contestantId, currentTime); return (SUCCESS); }
VoteCount: SELECT COUNT(*) FROM votes WHERE phone_num = ?;
InsertVote: INSERT INTO votes VALUES (?, ?, ?);
![Page 38: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/38.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Procedure Name Input Params run(phoneNum, contestantId, currentTime) {
result = execute(VoteCount, phoneNum); if (result > MAX_VOTES) { return (ERROR); } execute(InsertVote, phoneNum, contestantId, currentTime); return (SUCCESS); }
VoteCount: SELECT COUNT(*) FROM votes WHERE phone_num = ?;
InsertVote: INSERT INTO votes VALUES (?, ?, ?);
![Page 39: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/39.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Procedure Name Input Params run(phoneNum, contestantId, currentTime) {
result = execute(VoteCount, phoneNum); if (result > MAX_VOTES) { return (ERROR); } execute(InsertVote, phoneNum, contestantId, currentTime); return (SUCCESS); }
VoteCount: SELECT COUNT(*) FROM votes WHERE phone_num = ?;
InsertVote: INSERT INTO votes VALUES (?, ?, ?);
![Page 40: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/40.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
![Page 41: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/41.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Command Log TxnId
Procedure Name Input Params
![Page 42: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/42.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
![Page 43: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/43.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
![Page 44: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/44.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – ARCHITECTURE
11
Snapshots
![Page 45: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/45.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – LOGGING PROTOCOL
The DBMS logs the txn command before it starts executing once a txn has been assigned its serial order.
The node with the txn’s “base partition” is responsible for writing the log record. → Remote partitions do not log anything. → Replica nodes have to log just like their master.
12
![Page 46: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/46.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – CONSISTENT CHECKPOINTS
A special txn starts a checkpoint and switches the DBMS into copy-on-write mode. → Changes are no longer made in-place to tables. → The DBMS tracks whether a tuple has been inserted,
deleted, or modified since the checkpoint started.
A separate thread scans the tables and writes tuples out to the snapshot on disk. → Ignore anything changed after checkpoint. → Clean up old versions as it goes along.
13
![Page 47: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/47.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – RECOVERY PROTOCOL
The DBMS loads in the last complete checkpoint from disk.
Nodes then re-execute all of the txns in the log that arrived after the checkpoint started. → The amount of time elapsed since the last checkpoint
in the log determines how long recovery will take. → Txns that are aborted the first still have to be
executed.
14
![Page 48: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/48.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – REPLICATION
Executing a deterministic txn on the multiple copies of the same database in the same order provides strongly consistent replicas. → DBMS does not need to use Two-Phase Commit
15
Master Replica
![Page 49: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/49.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – REPLICATION
Executing a deterministic txn on the multiple copies of the same database in the same order provides strongly consistent replicas. → DBMS does not need to use Two-Phase Commit
15
Master Replica
Procedure Name Input Params
![Page 50: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/50.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – REPLICATION
Executing a deterministic txn on the multiple copies of the same database in the same order provides strongly consistent replicas. → DBMS does not need to use Two-Phase Commit
15
Master Replica
Procedure Name Input Params
TxnId Procedure Name
Input Params
![Page 51: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/51.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – REPLICATION
Executing a deterministic txn on the multiple copies of the same database in the same order provides strongly consistent replicas. → DBMS does not need to use Two-Phase Commit
15
Master Replica
Procedure Name Input Params
![Page 52: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/52.jpg)
CMU 15-721 (Spring 2016)
VOLTDB – REPLICATION
Executing a deterministic txn on the multiple copies of the same database in the same order provides strongly consistent replicas. → DBMS does not need to use Two-Phase Commit
15
Master Replica
Procedure Name Input Params OK
![Page 53: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/53.jpg)
CMU 15-721 (Spring 2016)
PROBLEMS WITH COMMAND LOGGING
If the log contains multi-node txns, then if one node goes down and there are no more replicas, then the entire DBMS has to restart.
16
![Page 54: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/54.jpg)
CMU 15-721 (Spring 2016)
PROBLEMS WITH COMMAND LOGGING
If the log contains multi-node txns, then if one node goes down and there are no more replicas, then the entire DBMS has to restart.
16
Partition #1 Partition #2
X ← SELECT X FROM P2 if (X == true) { Y ← UPDATE P2 SET Y = Y+1 } else { Y ← UPDATE P3 SET Y = Y+1 } return (Y) Partition #3
![Page 55: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/55.jpg)
CMU 15-721 (Spring 2016)
PROBLEMS WITH COMMAND LOGGING
If the log contains multi-node txns, then if one node goes down and there are no more replicas, then the entire DBMS has to restart.
16
Partition #1 Partition #2
X ← SELECT X FROM P2 if (X == true) { Y ← UPDATE P2 SET Y = Y+1 } else { Y ← UPDATE P3 SET Y = Y+1 } return (Y) Partition #3
![Page 56: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/56.jpg)
CMU 15-721 (Spring 2016)
PROBLEMS WITH COMMAND LOGGING
If the log contains multi-node txns, then if one node goes down and there are no more replicas, then the entire DBMS has to restart.
16
Partition #1 Partition #2
X ← SELECT X FROM P2 if (X == true) { Y ← UPDATE P2 SET Y = Y+1 } else { Y ← UPDATE P3 SET Y = Y+1 } return (Y) Partition #3
![Page 57: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/57.jpg)
CMU 15-721 (Spring 2016)
PROBLEMS WITH COMMAND LOGGING
If the log contains multi-node txns, then if one node goes down and there are no more replicas, then the entire DBMS has to restart.
16
Partition #1 Partition #2
X ← SELECT X FROM P2 if (X == true) { Y ← UPDATE P2 SET Y = Y+1 } else { Y ← UPDATE P3 SET Y = Y+1 } return (Y) Partition #3 X
??? ???
![Page 58: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/58.jpg)
CMU 15-721 (Spring 2016)
IN-MEMORY CHECKPOINTS
There are different approaches for how the DBMS can create a new checkpoint for an in-memory database. → The choice of approach in a DBMS is tightly coupled
with its concurrency control scheme.
The checkpoint thread scans each table and writes out data asynchronously to disk. → If the DBMS provides access to each table’s heap then
the thread completely ignores indexes.
17
![Page 59: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/59.jpg)
CMU 15-721 (Spring 2016)
IN-MEMORY CHECKPOINTS
Approach #1: Naïve Snapshots
Approach #2: Copy-on-Update Snapshots
Approach #3: Wait-Free ZigZag
Approach #4: Wait-Free PingPong
18
FAST CHECKPOINT RECOVERY ALGORITHMS FOR FREQUENTLY CONSISTENT APPLICATIONS SIGMOD 2011
![Page 60: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/60.jpg)
CMU 15-721 (Spring 2016)
NAÏVE SNAPSHOT
Create a consistent copy of the entire database in a new location in memory and then write the contents to disk. → The DBMS blocks all txns during the checkpoint.
The copying does not need to be explicit if you fork the DBMS process. → Checkpoint is consistent if there are not active txns. → Otherwise, use the in-memory undo log to roll back
txns in the child process.
19
![Page 61: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/61.jpg)
CMU 15-721 (Spring 2016)
COPY-ON-UPDATE SNAPSHOT
During the checkpoint, txns create new copies of data instead of overwriting it. → Copies can be at different granularities (block, tuple)
The checkpoint thread then skips anything that was created after it started. → Old data is pruned after it has been written to disk
20
![Page 62: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/62.jpg)
CMU 15-721 (Spring 2016)
OBSERVATION
Txns have to wait for the checkpoint thread when using naïve snapshots.
Txns may have to wait to acquire latches held by the checkpoint thread under copy-on-update
21
![Page 63: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/63.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
Maintain two copies of the entire database → Each txn write only updates one copy.
Use two BitMaps to keep track of what copy a txn should read/write from per tuple. → Avoid the overhead of having to create copies on the
fly as in the copy-on-update approach.
22
![Page 64: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/64.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
0
![Page 65: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/65.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
0
![Page 66: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/66.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
0
![Page 67: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/67.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
0
![Page 68: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/68.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
Checkpoint Thread
0
![Page 69: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/69.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
Checkpoint Thread
0
0
0
0
0
0
0
![Page 70: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/70.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
Checkpoint Thread
0
0
0
0
0
0
0
![Page 71: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/71.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
0
![Page 72: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/72.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
0
Txn Writes
![Page 73: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/73.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
Txn Writes
![Page 74: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/74.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
Txn Writes
![Page 75: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/75.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
![Page 76: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/76.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
Checkpoint Thread
![Page 77: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/77.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
0
0
0
Checkpoint Thread
![Page 78: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/78.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
0
0
0
Checkpoint Thread
1
0
1
0
0
1
![Page 79: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/79.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
0
0
0
![Page 80: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/80.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
0
0
0
![Page 81: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/81.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
0
0
0
![Page 82: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/82.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE Z IGZAG
23
Copy #1
5
9
7
2
4
3
Copy #2
5
9
7
2
4
3
0
0
0
0
0
Read BitMap
1
1
1
1
1
1
Write BitMap
6
1
9 0
1
1
1
0
0
0
3
8
0
1
![Page 83: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/83.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
Trade extra memory + CPU to avoid pauses at the end of the checkpoint. Maintain two copies of the entire database at all times plus extra space for a shadow copy. → Pointer indicates which copy is the current master. → At the end of the checkpoint, swap these pointers.
24
![Page 84: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/84.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master: Copy #2
![Page 85: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/85.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master: Copy #2
![Page 86: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/86.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master: Copy #2
![Page 87: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/87.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Checkpoint Thread Master: Copy #2
![Page 88: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/88.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Checkpoint Thread Master:
Txn Writes
Copy #2
![Page 89: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/89.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Checkpoint Thread Master:
6
1
9
6
1
9
1
1
1
Txn Writes
Copy #2
![Page 90: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/90.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Checkpoint Thread Master:
6
1
9
6
1
9
1
1
1
Txn Writes
Copy #2
![Page 91: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/91.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Checkpoint Thread Master:
6
1
9
6
1
9
1
1
1
-
-
-
-
-
-
0
0
0
0
0
0 Txn Writes
Copy #2
![Page 92: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/92.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master:
6
1
9
6
1
9
1
1
1
-
-
-
-
-
-
0
0
0
0
0
0
Copy #2
![Page 93: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/93.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master:
6
1
9
6
1
9
1
1
1
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
![Page 94: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/94.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master:
6
1
9
6
1
9
1
1
1
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3 Checkpoint Thread
![Page 95: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/95.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master:
6
1
9
6
1
9
1
1
1
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3 Checkpoint Thread
![Page 96: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/96.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master:
6
1
9
6
1
9
1
1
1
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3 Checkpoint Thread
![Page 97: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/97.jpg)
CMU 15-721 (Spring 2016)
WAIT-FREE PINGPONG
25
Copy #1
5
9
7
2
4
3
Copy #2
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3
5
9
7
2
4
3
1
1
1
1
1
1
Master:
6
1
9
6
1
9
1
1
1
-
-
-
-
-
-
0
0
0
0
0
0
Copy #3 Checkpoint Thread
![Page 98: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/98.jpg)
CMU 15-721 (Spring 2016)
CHECKPOINT IMPLEMENTATIONS
Bulk State Copying → Pause txn execution to take a snapshot.
Locking → Use latches to isolate the checkpoint thread from the
worker threads if they operate on shared regions.
Bulk Bit-Map Reset: → If DBMS uses BitMap to track dirty regions, it must
perform a bulk reset at the start of a new checkpoint.
Memory Usage: → To avoid synchronous writes, the method may need
to allocate additional memory for data copies.
26
![Page 99: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/99.jpg)
CMU 15-721 (Spring 2016)
IN-MEMORY CHECKPOINTS
27
Bulk Copying Locking
Bulk Bit-Map Reset
Memory Usage
Naïve Snapshot Yes No No 2x
Copy-on-Update No Yes Yes 2x
Wait-Free ZigZag No No Yes 2x
Wait-Free Ping-Pong No No No 3x
![Page 100: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/100.jpg)
CMU 15-721 (Spring 2016)
OBSERVATION
Not all DBMS restarts are due to crashes. → Updating OS libraries → Hardware upgrades/fixes → Updating DBMS software
Need a way to be able to quickly restart the DBMS without having to re-read the entire database from disk again.
28
![Page 101: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/101.jpg)
CMU 15-721 (Spring 2016)
OBSERVATION
Not all DBMS restarts are due to crashes. → Updating OS libraries → Hardware upgrades/fixes → Updating DBMS software
Need a way to be able to quickly restart the DBMS without having to re-read the entire database from disk again.
28
![Page 102: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/102.jpg)
CMU 15-721 (Spring 2016)
FACEBOOK SCUBA
Distributed, in-memory DBMS for time-series event analysis and anomaly detection.
Heterogeneous architecture → Leaf Nodes: Execute scans/filters on in-memory data → Aggregator Nodes: Combine results from leaf nodes
29
![Page 103: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/103.jpg)
CMU 15-721 (Spring 2016)
FACEBOOK SCUBA – ARCHITECTURE
30
Leaf Node Leaf Node Leaf Node Leaf Node
Aggregate Node
Aggregate Node
Aggregate Node
![Page 104: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/104.jpg)
CMU 15-721 (Spring 2016)
FAST RESTARTS
Decouple the in-memory database lifetime from the process lifetime.
By storing the database shared memory, the DBMS process can restart and the memory contents will survive.
31
![Page 105: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/105.jpg)
CMU 15-721 (Spring 2016)
SHARED MEMORY RESTARTS
Approach #1: Shared Memory Heaps → All data is allocated in SM during normal operations. → Have to use a custom allocator to subdivide memory
segments for thread safety and scalability.
Approach #2: Copy on Shutdown → All data is allocated in local memory during normal
operations. → On shutdown, copy data from heap to SM.
32
![Page 106: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/106.jpg)
CMU 15-721 (Spring 2016)
SCUBA – FAST RESTARTS
When the admin initiates restart command, the leaf node halts ingesting updates.
DBMS starts copying data from heap memory to shared memory. → Delete blocks in heap once they are in SM.
Once snapshot finishes, the DBMS restarts. → On start up, check to see whether the there is a valid
database in SM to copy into its heap. → Otherwise, the DBMS restarts from disk.
33
![Page 107: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/107.jpg)
CMU 15-721 (Spring 2016)
PARTING THOUGHTS
Logical logging is faster at runtime but difficult to implement recovery.
I think that copy-on-update checkpoints are the way to go especially if you are using MVCC
Shared memory does have some use after all…
34
![Page 108: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/108.jpg)
CMU 15-721 (Spring 2016)
SEMESTER PROGRESS
35
Concurrency Control Storage Models Indexes Scheduling & Execution Join Algorithms Logging & Recovery
Compression Query Optimization Vectorization Scan Sharing JIT Compilation Mat. Views NVM / HTM
![Page 109: CMU SCS 15-721 :: Logging & Recovery (Alternative Methods) · 2016-09-01 · CMU 15-721 (Spring 2016) VOLTDB – REPLICATION . Executing a deterministic txn on the multiple copies](https://reader033.vdocuments.site/reader033/viewer/2022050422/5f91998742527b1ee417e177/html5/thumbnails/109.jpg)
CMU 15-721 (Spring 2016)
NEXT CLASS
Project #3 Topics Extra Credit Project #2 is now due March 9th @ 11:59pm Project #3 proposals are still due March 14th No Mandatory Reading for March 2nd
36