2010 mongo berlin-scaling
TRANSCRIPT
![Page 1: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/1.jpg)
Eliot Horowitz@eliothorowitzMongoBerlin
October 4, 2010
Scaling with MongoDB
![Page 2: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/2.jpg)
Scaling
•Storage needs only go up•Operations/sec only go up•Complexity only goes up
![Page 3: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/3.jpg)
Scaling by Optimization
•Schema Design•Index Design•Hardware Configuration
![Page 4: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/4.jpg)
Horizontal Scaling
•Vertical scaling is limited•Hard to scale vertically in the cloud•Can scale wider than higher
![Page 5: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/5.jpg)
Schema
•Modeling the same data in different ways can change performance by orders of magnitude
•Very often performance problems can be solved by changing Schema
![Page 6: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/6.jpg)
Embedding•Great for read performance•One seek to load entire object•One roundtrip to database•Writes can be slow if adding to
objects all the time•Should you embed comments?
![Page 7: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/7.jpg)
Indexes
•Index common queries•Make sure there aren’t duplicates:
(A) and (A,B) aren’t needed•Right-balance indexes keep
working set small
![Page 8: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/8.jpg)
RAM Requirements
•Understand working set•What percentage of your data has
to fit in RAM?•How do you figure this out?
![Page 9: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/9.jpg)
Hardware
•Disk performance•How many drives•What about ec2?•Network performance
![Page 10: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/10.jpg)
Read Scaling•One master at any time•Programmer determines if read hits
master or a slave•Pro: easy to setup, can scale reads very
well•Con: reads are inconsistent on a slave •Writes don’t scale•Good for read heavy applications
![Page 11: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/11.jpg)
One Master, Many Slaves
•Custom Master/Slave setup•Have as many slaves as you want•Can put them local to appservers•Good for 90+% read heavy
applications (Wikipedia)
![Page 12: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/12.jpg)
Replica Sets•High Availability Cluster•One master at any time, up to 6 slaves•A slave automatically promoted to
master if failure•Drivers [will] support auto routing of
reads to slaves if programmer allows•Good for applications that need high
write availability but mostly reads (Commenting System)
![Page 13: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/13.jpg)
•Many masters, even more slaves•Can scale reads and writes in two
dimensions•Add slaves for inconsistent read
scaling and redundancy•Add Shards for write and data size
scaling
Sharding
![Page 14: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/14.jpg)
Common Setup•Typical setup is 3 shards with 3
servers per shard: 3 masters, 6 slaves•One massive collection, dozen non-
sharded•Can add sharding later to an existing
replica set with no down time•Can have sharded and non-sharded
collections
![Page 15: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/15.jpg)
Use Cases
•Millions of user profiles•User activity stream•Photos•Logging
![Page 16: 2010 mongo berlin-scaling](https://reader031.vdocuments.site/reader031/viewer/2022030309/58f2f1b61a28aba23d8b4589/html5/thumbnails/16.jpg)
Download MongoDBhttp://www.mongodb.org
and let us know what you think@eliothorowitz @mongodb
10gen is hiring!http://www.10gen.com/jobs