Getting Started with Replica Set

Why Replication?

• How many have faced node failures?• How many have been woken up from sleep to do• a fail-over(s)?• How many have experienced issues due to

network latency?• Different uses for data

Normal processing Simple analytics

Replicas Lifecycle

Replica Set – Initialize

Replica Set – Failure

Replica Set – Failover

Replica Set – Recovery

Replica Set – Recovered

Replication

Communication Test Step 1. Start all instants Step 2. All members of a replica set must be able to connect to every

other member of the set to support replication.E.g. given replica set with three members running on three different

machines host1 : 27017host2 : 27017host3 : 27017Check from host1 ./mongo --host host2IP - - port ./mongo --host host3IP - - portSimilar way one by one check from host2 and host3.

Step 3. Start all mongod instance by issuing following command mongod –host 10.1.1.61 --port 27017 --replSet rs0 mongod –host 10.1.1.62 --port 27017 --replSet rs0 mongod –host 10.1.1.63 --port 27017 --replSet rs0Here rs0 is name of replicaSet name.

Step 4. open mongo shell and connect to the first mongod instance ./mongo 10.1.1.61 –port 27017

Step 5. Use rs.initiate() to initiate a replica set consisting of the current member and using the default configuration:

rs.initiate() Step 6. Display the current replica configuration rs.conf()Step 7. Add two members to the replica set by issuing a sequence of

commands similar to the following.rs.add(“10.1.1.62:27017")rs.add(“10.1.1.63:27017")

Step 8. Check the status of your replica set at any time with the rs.status() operation.

Step 9. Using Configuration filecreate mongodb.conf file by issuing following commands vi /etc.mongodb.conf with following details port = 27017bind_ip = 10.1.1.61dbpath = /data/dbfork = true ( not working in windows) replSet = rs0

Step 10. Start mongod by following operation ./mongod --config /etc/mongodb.config

Add Members to a Replica SetRequirements

1. An active replica set.2. A new MongoDB system capable of supporting your dataset, accessible

by the active replica set through the network.3. Deploy MongoDB new instance, specifying name of replica set. 4. Open mongo shell and connect to replica set’s primary. If you don’t know

which member is primary, then issue following commands in mongo shell, db.isMaster()

5. In the mongo shell, issue the following command to add the new member to the replica set.rs.add(“10.1.1.61:27017")

6. Confirm new member is instance of replica set’s.

Replica set Maintenance and Administration

1. No downtime2. Rolling Upgrade maintenance

Start with secondaryPrimary Last

Sharding Shard: A single replica set that stores some portion of total sharded

cluster data set.Shard Key : In the sharded collection, shard key is the field that MongoDB

uses to distribute the document among the member of sharded clusters.

Feature of sharding 1. Range-base Data Partitioning

MongoDB distribute document among shards base on the shard key.Each chunk is block of document with value that fall within specific range.

2. Automatic Data Volume Distributionsharding system automatic balance data across cluster without in intervention from application layer. Effective automatic sharding depend on well chosen sharding key.

3. Transparent Query RoutingSharding is completely transparent to the application layer,

because all connection goes through mongos.4. Horizontal Capacity

A typical sharded cluster consist of, • 3 config server that store metadata. Metadata map chunks to

shard.• More than one replica set or mongod instances. These are the

shard.• A number of lightweight process, called mongos.

Sharding

When to use sharding 1. your data set approaches or exceeds the storage capacity of a single

node in your system.2. The size of your system’s active working set will soon exceed the capacity

of the maximum amount of RAM for your system.3. your system has a large amount of write activity, a single MongoDB

instance cannot write data fast enough to meet demand, and all other approaches have not reduced contention.if these attribute are not in your system, sharding will add additional complexity to your system without providing much benefits.

Sharding Requirements 1. Three config sever.

For development and testing purposes you may deploy a cluster with a single configuration server process, but always use exactly three config servers for redundancy and safety in production.

2. Two or more shardeach shard consist of one or more mongod. Typically each shard is a replica sets.

3. One or more mongos instance.4. Shard key

“Shard keys” refer to the field that exists in every document in a collection that MongoDB uses to distribute documents among the shards.

Setting Up ShardingStep 1. Starting the Servers

create a config server database directory by issuing following command,

$mkdir –p /configsvr/configThe config server needs to be started first, because mongos uses

it to get its configuration.$./mongod - -dbpath /configsvr/config - - port 10000

Step 2. Starting mongos serversRouting servers don’t even need a data directory, but they need

to know where the config server is:$./mongos --port 20000 --configdb localhost:10000Shard administration is always done through a mongos.

Step 3. Adding shardA shard is just MongoDB instance (or replica set). Start three MongoDB instances using following operations,

$./mongod –dbpath /data/sard1 - - port 10001$./mongod –dbpath /data/sard2 - - port 10002$./mongod –dbpath /data/sard3 - - port 10003

Now connect monos process using mongo by issuing following commands $ ./mongo localhost:20000/adminMake sure that you are connected to mongos not to mongod.Now you can add shard by following commands,>db.runCommands ( { addShard : “localhost:10001”, allowLocal : true } )Out put > {

"added" : "localhost:10000","ok" : true}

The "allowLocal" key is necessary only if you are running the shard on localhost.

Step 4. similar way add remain shards

Sharding Data1. MongoDB won’t just distribute every piece of data you’ve ever stored:

you have to explicitly turn sharding on at both the database and collection levels.Create database “shardDB”, first we enable sharding for database blog> db.runCommands ( { “enablesharding” : “shardDB” } )

2. Once you’ve enabled sharding on the database level, you can shard a collection by running the shardcollection command:> db.runCommands ( { “shardcollection” : “shardDB.blog” ,

“key” : { “_id” : 1} } ) 3. User define sharding key

you must create index on sharding key if it is not a “_id” field.

Many MongosYou can also run as many mongos processes as you want. One recommended setup is to run a mongos process for every application server. That way, each application server can talk mongos locally. If server goes down, no will be trying to talk with mongos.

A Sturdy ShardIn production, production each shard should be a replica set. That way individual server can fail without bringing whole shard. To add replica set as a shard, pass its name and a seed to the addshard command.> db.runCommand ( { "addshard" : "foo/10.1.1.61:27017“ } )

Sharding Administration 1. Sharding information mostly stored on config server, which can be

accessed by any mongos process.2. Connect to mongos process to access config database. Switch to config

DB issue following commands, > db.getCollectionNames()it shows all collection name.You can find list of shards in shards collection> db.shards.find()each shard assigning unique human readable id.

3. DatabasesThe databases collection contain a list of databases that exist on shards and information about that.>db.databases.find()

> db.databases.find(){ "_id" : "admin", "partitioned" : false, "primary" : "config" }{ "_id" : "foo", "partitioned" : false, "primary" : "shard1" }{ "_id" : "x", "partitioned" : false, "primary" : "shard0" }{

"_id" : "test","partitioned" : true,"primary" : "shard0","sharded" : {"test.foo" : {"key" : {“id" : 1},"unique" : false

}}}

4. Chunks chunks information stored in chunks collection. You can actually see how your data has been divided up across the cluster.> db.chunks.find()

{"_id" : "test.foo-x_MinKey","lastmod" : { "t" : 1276636243000, "i" : 1 },"ns" : "test.foo",

"min" : {"x" : { $minKey : 1 }},

"max" : {"x" : { $maxKey : 1 }}

"shard" : "shard0"}

Sharding Commands1. The printing sharding status give a quick summary of sharded collection.

>db.printShardingStatus()2. Removing a shard

Shards can be removed from a cluster with the removeshard command. Removeshard drains all of the chunks on a given shard to the other shards.>db.runCommand({"removeshard" : "localhost:10000"});As the shard is drained, removeshard will give you the status of how much remains on the shard.>db.runCommand({"removeshard" : "localhost:10000"});

when the shard has finished draining, removeshard shows that the shard has been successfully removed.