mongosv 2011 - mongodb on amazon ec2
DESCRIPTION
Slides from theTRANSCRIPT
Amazon Web Services
Jared Rosoff @forjared
Agenda
• Single node– Instance sizing – Storage configuration – OS configuration
• Replica Set – Availability Zones – Disaster Recovery – Multi Data-Center
• Sharded Cluster
INSTANCES, STORAGE AND SINGLE NODE CONFIGURATION
Instance Sizing
32-bit = Don’t Use
32-bit = Don’t Use
Huge
Big
Small
Arbiter / Config
OS• Amazon OS now an option• Turn off atime• Raise file descriptor limits
cat >> /etc/security/limits.conf << EOF* hard nofile 65536* soft nofile 65536EOF
• DO NOT use large VM pages• Use ext4, xfs• Use RAID
– RAID10 on MongoD– RAID1 on ConfigbDB
• Warning! Known problems with Ubuntu 10.04 & EBS– https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853– https://bugzilla.kernel.org/show_bug.cgi?id=16991
A MongoDB Data Node
• 64-bit instance• More RAM == better • Run EXT4 or XFS file system• Turn off atime & diratime• EBS volumes in RAID10
mongod
EBS – RAID10
This is your typical database node. Used for actual data storage. We’ll use this icon to represent this
type of node throughout this prezo.
A MongoDB Config Server
• 64-bit instance– Micro is fine
• EBS volumes in RAID1 mongoc
EBS – RAID1
Used to store configuration data about sharded deployments. Deployed as a set of 3 servers. Relatively low load and storage requirements.
A MongoDB Arbiter
• Micro is fine • No storage requirements• Must be separate node from rest
of replica set
mongod --arbiter
Used to store configuration data about sharded deployments. Deployed as a set of 3 servers. Relatively low load and storage requirements.
REPLICA SET DESIGNS
Single Region Replica SetRegion
Availability Zone A Availability Zone B Availability Zone C
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
Region 2
Disaster Recovery SiteRegion 1
Availability Zone A Availability Zone B Availability Zone A
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
HiddenEBS – RAID10
Region 3
Multi Data Center
Availability Zone A
mongod
EBS – RAID10
Region 2
Availability Zone A
mongod
Priority 1EBS – RAID10
Region 1
Availability Zone A
mongod
EBS – RAID10
SHARDED CLUSTERS IN EC2
Single Region Sharded ClusterRegion
Availability Zone A Availability Zone B Availability Zone C
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongoc
EBS – RAID1
mongoc
EBS – RAID1
mongoc
EBS – RAID1
Region 2
Sharded with Disaster Recovery SiteRegion 1
Availability Zone A Availability Zone B Availability Zone A
mongod
EBS – RAID10
mongod
EBS – RAID10
mongodhidden
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongodhidden
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongodhidden
EBS – RAID10
mongoc
EBS – RAID1
mongoc
EBS – RAID1
mongoc
EBS – RAID1
Region 3
Multi Region Sharded
Availability Zone A
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongoc
EBS – RAID1
Region 2
Availability Zone A
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongoc
EBS – RAID1
Region 1
Availability Zone A
mongod
EBS – RAID10
mongod
EBS – RAID10
mongod
EBS – RAID10
mongoc
EBS – RAID1
SECURITY
Security GroupsComponent Default PortsMongoD 27017, 28017 (Status page)MongoD Shard Server 27018MongoS 27017ConfigDB 27019
• All nodes listen on all interfaces unless –bind_ip is specified
• Specify alternate port with –port • Don’t mix names & IP’s. Names preferred (not ec2s)• Use security groups as access control lists • (e.g. allow 27017 from app servers, but not
everybody)
Region 1
Provisioning
Group: mongodbGroup: app-servers
$ ec2-create-group mongodb --description "security group for mongodb nodes”$ ec2-create-group app-servers --description "security group for app servers
## Allow mongodb nodes to talk to each other $ ec2-authorize mongodb –o mongodb –u <your ec2 account id>
## Allow app servers to talk to mongodb nodes $ ec2-authorize mongodb –o app-servers –u <your ec2 account id>
## Allow SSH access to MongoDB nodes $ ec2-authorize mongodb –p 22
## Allow remote MongoDB nodes to talk to cluster$ ec2-authorize mongodb –s 1.1.1.1 –p 27017$ ec2-authorize mongodb –s 2.2.2.2 –p 27017
Region 2
1.1.1.1
2.2.2.2
Questions?
• I am– Jared Rosoff ([email protected])
• Try it out! – http://www.mongodb.org/downloads– http://www.mongodb.org/display/DOCS/Amazon
+EC2