Download - Bringing Transactional Guarantees to MongoDB
![Page 2: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/2.jpg)
Agenda
•ACID Transactions
•Compensating Transactions
•Code Example
•Today and Planned
![Page 3: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/3.jpg)
Transactions with RDBS
Update balance and create an order atomically
id username item size
0 0 Stumpjumper L
id username email voucher
0 paul.robinson paul… 3000
Invoices
Users
id username item size
0 0 Stumpjumper L
1 0 Zesty L
id username email voucher
0 paul.robinson paul… 200
Invoices
Users
![Page 4: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/4.jpg)
{ id: "<ObjectID1>", username: "paul.robinson", email: "[email protected]" voucher: 3000, invoices: { {"Stumpjumper", "L"}, } }
Transactions with Document Stores
Update balance and create an order atomically
{ id: "<ObjectID1>", username: "paul.robinson", email: "[email protected]" voucher: 200, invoices: { {"Stumpjumper", "L"}, {"Zesty", "L"} } }
![Page 5: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/5.jpg)
But, sometimes this isn’t possible…
![Page 6: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/6.jpg)
Change Multiple Documents{ user: ‘Paul’ balance: 1000 }
{ user: ‘Fred’ balance: 0 }
{ user: ‘Paul’ balance: 700 }
{ user: ‘Fred’ balance: 300 }
E.g. Money Transfer, audited delete, integration
![Page 7: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/7.jpg)
Why doesn’t MongoDB support multi-document transactions?
![Page 8: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/8.jpg)
Scaling MongoDB
Router (mongos)
Shard
Master
D1
Slave 1
D1
Slave 2
D1
Shard
Master
D2
Slave 1
D2
Slave 2
D2
![Page 9: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/9.jpg)
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
D1
![Page 10: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/10.jpg)
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
![Page 11: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/11.jpg)
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
Client D1?
![Page 12: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/12.jpg)
Client
Multi-Document Transactions
ClientD2
Shard
D1Router
(mongos)Shard
D2
Client D1?
![Page 13: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/13.jpg)
Client
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
Client D1?
![Page 14: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/14.jpg)
D1Client
Multi-Document Transactions
Client
Shard
D1Router
(mongos)Shard
D2
Done
ClientD1
![Page 15: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/15.jpg)
(Multi-document) ACID doesn’t scale …
![Page 16: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/16.jpg)
Don’t throw out transactions altogether!
![Page 17: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/17.jpg)
Extended Transactions
• Umbrella term
• Alternatives to ACID
Guarantees
ACIDNone Extended
![Page 18: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/18.jpg)
ACID Vs Compensating Transactions
• ACID 1. Lock each resource 2. Commit/rollback each resource !
• Compensating 1. Commit each resource 2. Confirm/compensate each resource
![Page 19: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/19.jpg)
Compensating Transactions
• Eventually consistent
• Relaxed isolation
• Can move locks to application logic
• Support Long running tasks
![Page 20: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/20.jpg)
Transaction OptionsAtomic Atomic Batch Traditional
ACIDCompensating!Transactions
Single Doc ✓ ✓ ✓ ✓Multi Doc O ✓ ✓ ✓Sharding ✓ O O ✓
Multi stores O O ✓ ✓ACID ✓ ✓ ✓ O
App Unaware ✓ ✓ ✓ O
![Page 21: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/21.jpg)
Existing Patterns
• Compensating Transaction
• Just a pattern
• Needs implementing
• Recovery is hard!
MongoDB
Application (incl transaction and recovery code)
![Page 22: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/22.jpg)
Narayana’s Solution in WildFly 8
• Middleware
• Transaction Manager driven
• Annotation-based API
• Based on Sagas
MongoDB
Narayana
Application
![Page 23: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/23.jpg)
Protocol Details
![Page 24: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/24.jpg)
Starting State{ user: ‘A’ bal: 1000 }
{ user: ‘B’ bal: 0 }
![Page 25: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/25.jpg)
Log Compensation Handler 1{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
<txid> pending <comp1>
![Page 26: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/26.jpg)
Update Document 1{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
<txid> pending <comp1>
<txid> pending <comp1>
![Page 27: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/27.jpg)
Log Compensation Handler 2{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
![Page 28: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/28.jpg)
Update Document 2{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
![Page 29: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/29.jpg)
Update Transaction Log{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
![Page 30: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/30.jpg)
Confirm Document 1{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
![Page 31: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/31.jpg)
Confirm Document 2{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
![Page 32: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/32.jpg)
Completed{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 1000 }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 0 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> pending <comp1>
<txid> pending <comp1>
<txid> pending <comp1> <comp2>
<txid> pending <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 tx: <txid> }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 }
{ user: ‘B’ bal: 300 tx: <txid> }
<txid> committing <comp1> <comp2>
{ user: ‘A’ bal: 700 }
{ user: ‘B’ bal: 300 !}
![Page 33: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/33.jpg)
Code Example
![Page 34: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/34.jpg)
Money Transfer
• Each user has a balance
• Move money between users
• Update both documents atomically
![Page 35: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/35.jpg)
public class BankingService { ! @Inject AccountManager accountManager; ! @Compensatable public void transferMoney(String fromAccount, String toAccount, Integer amount) { ! accountManager.debitAccount(fromAccount, amount); accountManager.creditAccount(toAccount, amount); }
Service
![Page 36: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/36.jpg)
public class AccountManager { ! @Inject AccountDAO accountDAO; @Inject private CompensationManager compensationManager; @Inject CreditData creditData; ! @TxCompensate(UndoCredit.class) public void creditAccount(String account, Integer amount) { ! //High value transfers (over 500) are not allowed with this service if (amount > 500) compensationManager.setCompensateOnly() && return; creditData.setToAccount(account); creditData.setAmount(amount); accountDAO.incrementBalance(account, amount); } …
Account Manager (credit part)
![Page 37: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/37.jpg)
… @Inject DebitData debitData; ! @TxCompensate(UndoDebit.class) public void debitAccount(String account, Integer amount) { ! debitData.setFromAccount(account); debitData.setAmount(amount); ! accountDAO.incrementBalance(account, -1 * amount); } }
Account Manager (debit part)
![Page 38: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/38.jpg)
@CompensationScoped public class CreditData implements Serializable { ! private String toAccount; private Integer amount; … } !@CompensationScoped public class DebitData implements Serializable { ! private String fromAccount; private Integer amount; … }
Compensation Data
![Page 39: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/39.jpg)
public class UndoCredit implements CompensationHandler { ! @Inject CreditData creditData; @Inject AccountDAO accountDAO; ! public void compensate() { ! if (creditData.getToAccount() != null) { accountDAO.incrementBalance( creditData.getToAccount(), -1 * creditData.getAmount()); } } }
Compensation Handler (Credit)
![Page 40: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/40.jpg)
What we have today
• Compensating-Transactions API
• MongoDB Example
• Other Quickstarts
• Blog post series
• Ships with WildFly 8.Final+
![Page 41: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/41.jpg)
What’s Planned
• MongoDB RM
• Compensating state Recovery
• Throughput/Scalability study
• RDBMS integration
• Other language support
![Page 42: Bringing Transactional Guarantees to MongoDB](https://reader036.vdocuments.site/reader036/viewer/2022062514/5597079a1a28ab9a2f8b4610/html5/thumbnails/42.jpg)
Getting Started• Explore the quickstarts http://github.com/jbosstm/quickstart/
• Give feedback (forum) http://community.jboss.org/en/jbosstm
• Track issues http://issues.jboss.org/browse/JBTM
• Subscribe to Blog http://jbossts.blogspot.co.uk/
• Contact me [email protected], @pfrobinson