concurrency control in mongodb 3.0
TRANSCRIPT
Concurrency Control in MongoDB 3.0
Kaloian ManassievSenior Engineer at MongoDB
[email protected]@kaloianm
2
Audience
• Operations engineers• Application developers• Third-party storage engine developers• Anybody who is curious
3
What is concurrency control?
4
What is concurrency control?
• Locking• Data consistency
$100 + $100 = $200 (not $100 )• MVCC
5
Collection Collection
6
Collection Collection
Storage Engine API
7
What you will learn
• Top level concurrency control–Instance, database and collection
• How top level cooperates with the storage engine
8
What you will NOT learn
• WiredTiger internals–Separate session on this
9
Talk outline
• MongoDB concurrency control• Multiple-granularity locking• WiredTiger vs MMAP V1• Questions
10
MongoDB 2.0 (and before)
THETOP LOCK
R WR OK NOW NO NO
11
MongoDB 2.2
Lock per database
db1.coll.insert({a:1}) db4.coll.find({c:5})
12
MongoDB 2.2
THETOP LOCK
+ intents
13
TOP
db1.coll.insert({a:1}) db4.coll.find({c:5})
IX IS
X S
14
TOP
db.coll.insert({a:1}) db.coll.find({c:5})
IX IS
X S
15
Why intents?
• Get rid of the global lock?
• Use read (shared) lock?
16
TOP
db1.coll.insert({a:1}) db4.coll.find({c:5})
S S
X S
17
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
S S
X
18
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
IX S
X
19
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
IX S
XS S S S
20
TOP
db.fsyncLock({lock:1})
S
S S S S
db1.coll.find({a:1})
IS
S
21
Intents
• “Intention” to access one or more children of an item–Compatible with other intents–Need to acquire lock further down–No overhead
22
Locks
• “Lock” an item for particular access–Item being locked–All items below it
23
Intents compatibility
IS IX S XIS OK OK OK NOIX OK OK NO NOS OK NO OK NOX NO NO NO NO
24
MongoDB 3.0
TOP
25
Multiple-granularity locking
TOP
26
Lock Manager
• Ensures the locking protocol is obeyed
• Very low overhead
• Testable
27
Lock Manager
• Improved fairness
• Support for locking policies
• Lock statistics gathering
28
TOP
1
db.coll1.update({a:1}) db.coll1.update({a:4})
IX IX
IX IX
IX IX
X X
29
Storage Engine API
TOP
30
Storage Engine API
TOP
Storage Engine API
31
TOP
db.coll1.update({a:1}) db.coll1.update({a:4})
IX IX
IX IX
IX IX
WiredTiger MVCC
32
TOP
db.coll1.update({a:1}) db.coll1.drop()
IX IX
IX X
IX
WiredTiger MVCC
33
How does MMAP V1 work?
• Supports collection concurrency
• Takes lock on the collection–Instead of intent
34
TOP
db.coll1.update({a:1}) db.coll2.update({a:4})
IX IX
IX IX
X X
MMAP V1
35
TOP
db.coll1.update({a:1}) db.coll1.find({a:4})
IX IS
IX IS
X S
MMAP V1
36
WiredTiger
• Intents: Global, Database, Collection
• Document-level concurrency left to the storage engine
37
MMAP V1
• Intents: Global, Database
• Locks: Collection
38
Conclusion
• Storage engines have direct control over concurrency–Completely decoupled from top-level locking
–Enabled document-level control
39
Conclusion
• Retrofitted MMAP V1 to use multi-granularity locks–Support for collection-level locking–Got rid of the global lock usage for journaling
QUESTIONS?
EXTRA SLIDES
42
43
44
45
46
47
48
49