mongodb replica,shard cluster 中央大學電算中心 楊素秋 2014-05-05
TRANSCRIPT
![Page 1: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/1.jpg)
MongoDB Replica,Shard Cluster
中央大學電算中心 楊素秋2014-05-05
![Page 2: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/2.jpg)
OUTLINE
• 1. MongoDB Replica• 2. Deploy a Replica Set• 3. Sharing Cluster• 4. Deploy a Sharded Cluster• 5. Conclusion
![Page 3: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/3.jpg)
1. MongoDB Replica
• Provides redundancy– protects a database from• loss of a single server
• Increases data availability– recover from• hardware failure• service interruptions
![Page 4: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/4.jpg)
![Page 5: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/5.jpg)
![Page 6: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/6.jpg)
![Page 7: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/7.jpg)
2. Deploy a Replica Set
• 安裝 / 啟動 mongoDB on each hosts – http://docs.mongodb.org/manual/tutorial/install-mongodb-on-red-hat-centos-or-
fedora-linux/
• 設定 /etc/mongod.conf• service mongod restart
• 在 primary host (140.115.2.32)– rs.initiate()– rs.add(“140.115.2.32:27017”)– rs.add(“140.115.2.31:27017”)– rs.addArb(“140.115.2.34:27017”)
![Page 8: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/8.jpg)
![Page 9: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/9.jpg)
![Page 10: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/10.jpg)
![Page 11: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/11.jpg)
• Priority – cfg = rs.conf()– cfg.members[0].priority = 2.0 – cfg.members[1].priority = 0.5– rs.reconfig(cfg)
2. Deploy a Replica Set (cont.)
![Page 12: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/12.jpg)
• Slave Node – MongoDBManager.java
2. Deploy a Replica Set (cont.)
public static synchronized DB getDB() throws Exception { if(mongo == null) { mongo = new Mongo(); mongo.slaveOk(); } return mongo.getDB("fdns"); } public static synchronized Mongo getMongo() throws Exception { if(mongo == null) { mongo = new Mongo(); mongo.slaveOk(); } return mongo; }
![Page 13: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/13.jpg)
• Slave Node – mongo shell
• use fdns
• rs.slaveOk()• show collections
– mongo shell• db.collectionName.remove() // FAIL• db.collectionName.drop()
2. Deploy a Replica Set (cont.)
![Page 14: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/14.jpg)
3. Sharing Cluster
• Single machine challenges– High query rates• exhaust CPU capacity
– Larger data sets• exceed the storage capacity
• Referances– http://docs.mongodb.org/manual/core/sharding-introduc
tion/– http://docs.mongodb.org/manual/core/sharded-cluster-c
omponents/
![Page 15: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/15.jpg)
![Page 16: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/16.jpg)
3. Sharing Cluster(cont.)
• Sharded Cluster Components– Shards • holds a subset of a collection’s data• a single mongod instance, or a replica set
– Config Servers• a mongod instance
– holds metadata about the cluster– metadata maps chunks to shards
![Page 17: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/17.jpg)
3. Sharing Cluster (cont.)
– Routing Instances • a mongos instance
– routes the reads and writes from applications to the shards
• Applications do not access the shards directly
![Page 18: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/18.jpg)
3. Sharing Cluster (cont.)
• Vertical scaling– adds more CPU and storage resources to increase capacity
• horizontal scaling (Sharding)• divides the data set• distributes data over multiple servers(shards)– Each shard is an independent database– shards make up a single logical database
![Page 19: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/19.jpg)
![Page 20: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/20.jpg)
• Range based partitioning– MongoDB divides the data set into ranges• determined by the shard key values to provide
![Page 21: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/21.jpg)
• Hash Based Sharding• MongoDB computes a hash of a field’s value– uses these hashes to create chunks.
![Page 22: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/22.jpg)
4. Deploy a Shard Cluster
– http://docs.mongodb.org/manual/tutorial/deploy-shard-cluster/
![Page 23: MongoDB Replica,Shard Cluster 中央大學電算中心 楊素秋 2014-05-05](https://reader033.vdocuments.site/reader033/viewer/2022061516/56649ea35503460f94ba7fba/html5/thumbnails/23.jpg)
5. Conclusion
• Replica Set **– Primary : service– Secondary : Data Mining
• Apache Mahout: cluster, classification• Mining flooding, attacks traffic
– Arbiter
• Sharing Cluster– Load balance– Scalability