Mongo performance

Stuff that mattersMonitoring

TuningConclusion

vladimir.malyk@gmail.com

Mongo performance

Stuff that matters

Mongo Pain. Write lock

Multiple readers / Single-writer lock( http://en.wikipedia.org/wiki/Readers-writer_lock)

ver < 2.0 - has a process-wide write lock

2.1.1+ per-database locking( https://jira.mongodb.org/browse/SERVER-4328 )

per-collection locking( https://jira.mongodb.org/browse/SERVER-1240 )

Mongo Pain. mmap

http://docs.mongodb.org/manual/faq/storage/

Page faults: will occur if you’re attempting to access part of a memory-mapped file that isn’t in memory.( http://docs.mongodb.org/manual/reference/glossary/#term-page-fault )

Performance: page fault can take around 40,000 times longer than a nonfaulting memory operation

ver 2.0+, this is addressed by detecting the likelihood of a page fault and releasing the lock before faulting - YIELD operation;

SSD is a doctor's stuff in this case;

Mongo Atomicity

Write operations are atomic on the level of a single document: no single write operation can atomically affect more than one document or more than one collection. (http://docs.mongodb.org/manual/tutorial/isolate-sequence-of-operations/)

hint: isolates a write operation that affects multiple documents from other write operations (http://docs.mongodb.org/manual/reference/operator/isolated/)

http://docs.mongodb.org/manual/reference/command/findAndModify/

Mongo performance

Monitoring

Monitoring: HTTP Consolehttp://docs.mongodb.org/ecosystem/tools/http-interf

aces/#http-console

http://dbhost.net:28017/

Replica Set Admin UI, Oplog status

clients

DBTOP

write lock % time in write lock, by 4 sec periods (form ~12% to ~6%)

Log - By default the slow operation threshold is 100 millis

Monitoring: SSH Console

netstat -n | wc -l

ps aux | grep mongodb

cat /proc/{pid}/limits | grep "Max open files"

ulimit -n 65000 (http://www.andrewrollins.com/2010/10/20/mongodb-open-file-limit/)

Monitoring: Custom

http://sj.malyk/admin/MongoProfileLog/

milestone marks: do fake update before and after app execution

http://docs.mongodb.org/manual/tutorial/manage-the-database-profiler/

http://docs.mongodb.org/manual/reference/database-profiler/

https://gist.github.com/vladimir-malyk/5047107

https://gist.github.com/kgorman/995a3aa5b35e92e5ab57

Mongo performance

Tuning

Tuning: Heavy tricks

degradation - don't write to an unimportant collections;

partitioning - move an unimportant collections to a separate replicaset.

Tuning: Indexes

* Create Indexes to Support Your Queries

* Use Compound Indexes to Support Several Different Queries

* -10% write performance for each additional index (http://www.slideshare.net/mongodb/mongodb-performance-tuning)

* Create Indexes that Support Covered Queries

* Use Indexes to Sort Query Results

* http://blog.mongolab.com/2012/06/cardinal-ins/

* First, fields on which you will query for exact values.

* Second, fields on which you will sort.

* Finally, fields on which you will query for a range of val.

* Ensure Indexes Fit RAM

* Creating index on a big collection takes a lot of time with write lock

Tuning: Common tricks

http://blog.serverdensity.com/mongodb-schema-design-pitfalls/

* Avoid growing documents (“moved” in system.profile)

* Use field modifiers: Instead of sending a whole new document to update an existing one, you can set or remove specific fields

* Preallocate documents: preallocate the document with placeholder values, then use the $set field modifier to change the actual value later

* Field names take up space: http://blog.serverdensity.com/on-shortened-field-names-in-mongodb/

* Consider using _id for your own purposes

Tuning: Explain

http://docs.mongodb.org/manual/reference/explain/

* http://docs.mongodb.org/manual/reference/explain/#explain.cursor

* http://docs.mongodb.org/manual/reference/explain/#explain.net

* http://docs.mongodb.org/manual/reference/explain/#explain.nscanned

* http://docs.mongodb.org/manual/reference/explain/#explain.n

* http://docs.mongodb.org/manual/reference/explain/#explain.scanAndOrder

* http://docs.mongodb.org/manual/reference/explain/#explain.indexOnly

* http://docs.mongodb.org/manual/reference/explain/#explain.nYields

* http://docs.mongodb.org/manual/reference/explain/#explain.allPlans

Mongo performance

Conclusion

Mongo: Tips and tricks

* deal with db disk fragmentation:

* manual defragmentation (via replica resync);

* http://docs.mongodb.org/manual/reference/command/compact/

* Can't take a write lock while out of disk space:

* you need some free space - db defrag will help you;

Mongo: Concurrency

http://docs.mongodb.org/manual/faq/concurrency/

http://docs.mongodb.org/manual/faq/concurrency/#how-do-i-see-the-status-of-locks-on-my-mongod-instances

http://docs.mongodb.org/manual/faq/concurrency/#which-operations-lock-the-database

http://docs.mongodb.org/manual/faq/concurrency/#which-administrative-commands-lock-the-database

* db.auth() - lock the database but only hold the lock for a very short time

http://blog.serverdensity.com/goodbye-global-lock-mongodb-2-0-vs-2-2/

why it's a good idea to upgrade your mongodb

Mongo: Some links

http://www.slideshare.net/mongodb/mongodb-performance-tuning

http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html