Build, Deploy and Maintain MongoDB on OpenStack-based SUSE® CloudA Detailed Technical Guide

Technical Guide

Guidewww.suse.com

Big Data

Table of Contents page

About This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Step 1: Configuring SUSE Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Step 2: Deploying SUSE Cloud . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Step 3: Building the Application Image . . . . . . . . . . . . . . . . . . . . . . . . . . 5Step 4: Deploying Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Documentation ConventionsThe following typographical conventions are used in this manual:

Bold text represents things you should watch for, buttons you should click, text or options that you should select or text you should enter into a GUI .

Option > Option > Option represents a chain of items selected from a menu .

BOLD_UPPERCASE_ITALIC text represents a “lab variable” that you replace with another value .

bold monospace text represents commands that you type at a command line .

Note | Important | Warning indicates something important to take note of or watch out for .

2

Big Data Technical GuideBuild, Deploy and Maintain MongoDB on OpenStack-based SUSE Cloud

In this guide you will follow step-by-step instructions to:

DeploySUSEManagerandconfigureittoprovidepatchesfortheSUSEinfrastructureandfortheMongoDBapplication

DeployandconfigureaSUSECloudOpenStack-basedprivatecloudtohostaMongoDBappliance

CreateaMongoDBapplianceusingSUSEStudioanddeploytoSUSECloud

About MongoDBMongoDB is an open source, cross-platform database and the leading NoSQL database . An increasing number of businesses are using MongoDB to create new types of applications, improve cus-tomer experience, accelerate time to market and reduce costs . As an agile database, MongoDB allows schemas to change quickly as applications evolve, while still providing the functionality develop-ers expect from traditional databases, such as secondary indexes, a full query language and strict consistency . MongoDB is built for scalability, performance and high availability, scaling from single server deployments to large, complex multi-site architectures . By leveraging in-memory computing, MongoDB provides high per-formance for both reads and writes . MongoDB’s native replication

and automated failover enable enterprise-grade reliability and operational flexibility .

MongoDB is also used by many businesses in big data applica-tions for storing and using huge amounts of structured and un-structured data in a single data store . Native analytics facilitate the delivery of actionable insights in real time, and horizontal scaling allows organizations to support massive data volumes and high ingestion rates .

About SUSE ManagerSUSE Manager automates Linux server management, allowing you to provision, configure and maintain your servers faster and more accurately . SUSE Manager monitors the health of each Linux server from a single console so you can identify server perfor-mance issues before they impact your business . For the large clusters often required in Hadoop deployments, SUSE Manager allows you to comprehensively manage your Linux servers across physical, virtual and cloud environments while improving data center efficiency . SUSE Manager automates the provisioning of new systems and allows you to create customized repositories for the delivery of either operating system packages or RPM-based applications and content .

About This GuideThis guide provides detailed instructions for using SUSE® Manager, SUSE Studio™ and SUSE Cloud to create, configure and deploy a MongoDB appliance instance to a SUSE Cloud OpenStack-based private cloud. While we use MongoDB as an example here, this guide may also be considered a template for building, deploying and maintaining other applications in a SUSE Cloud OpenStack environment.

3www.suse.com

About SUSE CloudSUSE Cloud is the original enterprise-ready OpenStack distribu-tion for building Infrastructure-as-a-Service private clouds . SUSE Cloud includes full support for OpenStack Orchestration (Heat) and OpenStack Telemetry (Ceilometer) and is the first enterprise OpenStack distribution that automates high availability configura-tion and deployment to ensure a seamless installation experience and the continuous delivery of critical cloud services . And SUSE Cloud lets you optimize performance and licensing costs by let-ting you deploy an OpenStack-based private cloud that supports KVM, Xen, Microsoft Hyper-V and VMware vSphere environments .

About SUSE StudioSUSE Studio is a web application for building and testing physical, virtual or cloud-based applications in a web browser . In addition to supporting Amazon EC2 and Windows Azure formats, SUSE Studio includes SUSE Cloud/OpenStack/KVM as one of the pre-defined application image formats, so that you can easily create and build cloud images for private and public clouds . With the SUSE Studio WebHooks feature, all your SUSE Cloud images built in SUSE Studio are automatically and immediately imported into your SUSE Cloud instance . SUSE Studio also provides a test-drive mode that launches your image inside a KVM virtual machine and tracks changes . This allows you to boot, test, reconfigure and save the changes without having to relaunch, a huge time saver in the development cycle . If you want to share your creation with the world, place it in SUSE Gallery, a browseable and searchable pub-lic listing of thousands of appliances . There’s also an onsite ver-sion of SUSE Studio that provides you more security and control, with a private gallery of appliances that can be used for factory installations, gold masters or starting points for other appliances .

Step 1: Configuring SUSE ManagerObjectives:Task I: Verify Repos Are in SyncTask II: Add the Application RepositoryTask III: Add the Application ChannelTask IV: Create an Activation KeyTask V: Create a Bootstrap Script

Special Instructions and Notes: AfunctionalSUSEManagerinstallationisassumed. Connectivitytotherepositoryisarequirementforthe

outlinedsyncmethod.

Task I: Verify Repos Are in SyncSUSE Studio images are created using a baseline or the latest available patches . The SUSE Manager channel should be up to date as well .

1. At the SUSE Manager shell, execute mgr-ncc-sync . This process may take several minutes as it downloads all the latest packages .

2. Execute mgr-ncc-sync -l and verify the SLES11SP3 repositories are enabled . You may verify this as well from within the SUSE Manager UI .

3. If the SLES11SP3 repos are not set up, please add them via mgr-ncc-sync --add-product .

Task II: Add the Application RepositorySUSE Manager will periodically download new versions of the pack ages and enable the administrator to patch the server on demand .

1. Login to the SUSE Manager Web interface; browse to Channels, Manage Software Channels, Manage Repositories, create new repository:

Repository Label: databasesrepo Repository URL: http://download.opensuse.org/

repositories/server:/database/SLE_11_SP3/

Deselect Has Signed Metadata?

Note: The provided MongoDB package is not supported . For Enterprise MongoDB Support please contact MongoDB .com .

4

Big Data Technical GuideBuild, Deploy and Maintain MongoDB on OpenStack-based SUSE Cloud

Task III: Add the Application Channel Next create and assign the repository to a new channel .

1. Login to the SUSE Manager Web interface; browse to Channels, Manage Software Channels, create new channel:

Channel Name: databases Channel Label: databases Channel Summary: Databases Channel

Click Create Channel .

2. Browse to the Repositories tab, select databasesrepo and click Update Repositories .

Note: Visit the Sync tab to Sync Now the repositories .

Task IV: Create an Activation KeyActivation keys link systems to their corresponding repositories, configuration, etc .

1. Login to the SUSE Manager Web interface; browse to Systems, Activation Keys, create new key:

Description: database servers key Key: SLESDATABASES Base Channel: SLES11-SP3-Pool for x86_64 Add-On Entitlements: Virtualization, Provisioning

Click Create Activation Key. .

2. Browse to the Child Channels tab, holding Ctrl select databases, SUSE-Manager-Tools and, if required, the Updates channel and click Update Key .

3. Switch to the Packages tab and add the osad, rhncfg- actions, rhncfg packages, one per line, and then click Update Key .

Note: Add the rhnmd package to enable monitoring .

Task V: Create a Bootstrap Script Bootstrap scripts handle the installation of the necessary com-ponents and system registration .

1. SSH into the SUSE Manager server . cd /srv/www/htdocs/pub/bootstrap cp bootstrap.sh bootstrap-databases.sh

2. Edit the bootstrap-databases.sh file and comment out line 71, from

exit 1 to #exit1

3. Continue editing bootstrap-databases.sh and input our activation key by modifying the parameter ACTIVATION_KEYS=1-SLESDATABASES .

4. Change the value of the parameter FULLY_UPDATE_THIS_BOX= from 1 to 0 .

Note: Keep the value as 1 if the registered systems should fully update on registration .

Step 2: Deploying SUSE CloudNext, you will install and configure SUSE Cloud based on OpenStack .

Objectives:Option I: Manual InstallationOption II: Appliance InstallTask I: Deploy the OpenStack ComponentsTask II: Prepare Your Project

Option I: Manual InstallationSUSE Cloud is easy to deploy when the instructions are carefully followed .

1. The main documentation is located at: www.suse.com/documentation/suse-cloud3/singlehtml/book_cloud_deploy/book_cloud_deploy.html

2. Contact your SUSE sales executive should you need any assistance .

5www.suse.com

Option II: Appliance Install Both a bare and a package populated appliance are available to ease the main install process .

1. Fill out the Cloud Deployment Questionnaire: www.suse.com/communities/conversations/files/2014/ 03/SUSECloudDGQuestionaire1.pdf

2. Download one of the SUSE Cloud Appliances:

Bare: https://susestudio.com/a/Mrr6vv/suse-cloud-3-admin

Embedded: https://susestudio.com/a/Mrr6vv/suse-cloud- 3-admin-embedded

3. Follow the Appliance deployment guide: www.suse.com/communities/conversations/files/2014/ 03/SUSE-Cloud-AA-Guide.pdf

(Guides also provided under “SUSE Cloud Admin Appliance Guide”)

Task I: Deploy the OpenStack ComponentsSUSE Cloud is easy to deploy when the instructions are carefully followed .

1. Follow the Deploying OpenStack Services guide: www.suse.com/documentation/suse-cloud3/singlehtml/book_cloud_deploy/book_cloud_deploy.html#cha.depl.ostack

2. Login to the SUSE Cloud/OpenStack dashboard .

Task II: Prepare Your Project1. OpenStack provides access control and a Firewall-as-a-

Service among many components . Create a keypair to access the instances and open the ports necessary for MongoDB .

2. At the SUSE Cloud dashboard, select your current project and click Access & Security . Then Edit Rules for the default group . You may create a specific group for MongoDB instances .

3. Click Add Rule and add the following rules: SSH Custom TCP Rule – Ingress – Port Range – 27017 to 27019

Note: Add the ALL ICMP rule should you need to ping the instance .

4. At the SUSE Cloud dashboard, select your current project and click Access & Security. Then choose the Keypair tab . Create Keypair or Import Keypair to use this key for future SSH access into spin-up instances . You will use myKeypair later .

Step 3: Building the Application ImageIn this step you create a platform for the application .

Objectives:Task I: Create the ApplianceTask II: Add the ApplicationTask III: Modifying the Appliance

Special Instructions and Notes: SUSEStudioisprovidedvia:www.susestudio.com AnEnterpriseversionisavailablewhichincludesadditional

features.

Task I: Create the ApplianceUsing SUSE Studio, create the platform appliance .

1. Login to SUSE Studio .

2. Click Create new appliance.

3. Under SUSE Linux Enterprise 11 SP3 select Just enough OS (JeOS) or Server .

4. Select your architecture; use 64-bit .

5. Name your appliance Cloud MongoDB .

6. Click Create appliance .

7. Additionally, search for xfsprogs and click +add .

6

Big Data Technical GuideBuild, Deploy and Maintain MongoDB on OpenStack-based SUSE Cloud

Task II: Add the ApplicationAdd a community repo and MongoDB . While this example uses MongoDB, the community repo contains many other popular databases .

1. Select the Software tab .

2. Click Add repositories; click Import new repository; then Add a repository via URL .

Repository name: ‘MongoDB’ Location (URL): http://download.opensuse.org/

repositories/server:/database/SLE_11_SP3/

3. Click Add repository .

4. Return to the Software tab .

5. Once the Repository is added, input mongodb at the Search for software field under the Software tab .

6. Click the +add button left of the mongodb package .

Task III: Modifying the AppliancePersonalize the appliance as needed .

1. Save Appendix A as mongodb.conf and add it as a file under the Files tab; click Upload file . . . and point to your new mongodb .conf . Please ensure the Directory is /etc/ by selecting the recenly uploaded mongodb .conf and clicking Move/Rename .

2. Continue by selecting the Configuration tab, and under the General section add the necessary users . Personalize the logo and background as needed .

3. Unselect Enable firewall .

4. Select the Appliance section under the Configuration tab and check the SUSE Cloud, Integrate with SUSE Cloud/OpenStack box .

5. For SUSE Manager integration, select the Integrate with SUSE Manager box .

Server hostname or IP: ‘HostnameOrIP’ Bootstrap script name: bootstrap-databases.sh

6. Select the Build tab and choose the Default format as SUSE Cloud/OpenStack/KVM . Click Build .

Note: 1. ‘HostnameOrIP’ is the actual resolvable name or IP address of the SUSE Manager server; 2. Further modi-fications may be performed for the appliance but are not covered in the scope of this document .

Step 4: Deploying ImagesNow you will import the SUSE Studio images into SUSE Cloud .

Objectives:Task I: Import the ImageTask II: Deploy an InstanceTask III: Assign a Floating/Public Address to the InstanceTask IV: Attach a VolumeTask V: Configure MongoDB to Use the Volume as dbpath

Special Instructions and Notes: NetworkaccessorinternetconnectivitytotheSUSEStudio

serverispreferred. AnoptiontomanuallyuploadtheimagesintoGlanceisalso

available.

Task I: Import the ImageLogin to SUSE Studio and SUSE Cloud to import the image .

1. Browse in the Build tab of SUSE Studio within your Cloud MongoDB appliance . Right click the Download link and select the equivalent to Copy Link Location .

2. At the SUSE Cloud dashboard, select your current project and click Images & Snapshots . At the upper right-hand corner click Create Image .

3. Input all relevant information for our image, as

Name: MongoDB SLES11SP3 Format: QCOW2 – QEMU Emulator

Image Source may be Image Location should you input a URL directly from SUSE Studio (as copied in step 1) or Image File to directly upload a downloaded image .

Select the Public box to share the image across Projects/tenants .

4. Verify as the image Status changes between Saving to Active .

7www.suse.com

Task II: Deploy an InstanceThe final goal is to deploy a MongoDB instance .

1. At the SUSE Cloud dashboard, select your current project and click Images & Snapshots . Click Launch besides the MongoDB SLES11SP3 image . Then input

Instance Name: mongo1 Flavor: m1.medium

Switch to the Access & Security tab and choose your Keypair .

Switch to the Networking tab and click the + sign besides the fixed network and then click Launch .

2. Verify as the mongo1 instance starts .

Task III: Assign a Floating/Public Address to the InstanceNow make the instance accessible from outside the private inter-nal SUSE Cloud network .

1. At the SUSE Cloud dashboard, select your current project and click Instances . At the Actions section besides the mongo1 instance, click More and select Associate Floating IP .

Click the + sign to add an IP Address from the Floating Pool; click Allocate IP and then Associate .

2. Allow the assignment to take place and verify that the IP Address field now contains the new address as well .

3. Ping the address if necessary .

Task IV: Attach a VolumeInstances run on ephemeral storage; attach a volume to keep away from any disaster .

1. At the SUSE Cloud dashboard, select your current project and click Volumes .

Click Create Volume at the upper right corner and input the volume details .

Volume Name: mongo1 Size (GB): 6

Click Create Volume .

2. After the image is created, click Edit Attachments under the Actions section beside the mongo1 volume .

3. Select the mongo1 instance and input /dev/vdb as Device Name .

4. Click Attach Volume . Notice the Attached To section change .

Task V: Configure MongoDB to Use the Volume as dbpathAs the volume is recognized on the fly by the kernel, a file system and mount point are necessary .

1. Access the mongo1 instance . Use ssh or the VNC console .ssh -i myKeypair.pem root@mongo1

2. Execute fdisk -l or dmesg to identify the new disk .

3. Format the new disk via mkfs.xfs /dev/vdb .

4. Create a mount point for the mongo volume, mkdir -p /var/lib/mongodb . This path may already exist and is referred to by dbpath = /etc/mongodb.conf .

5. Create an fstab entry and mount the volume .

echo ‘/dev/vdb /var/lib/mongodb xfs defaults,auto,noatime,noexec 0 0’ >> /etc/fstab

mount -a

6. Edit /etc/mongodb.conf and validate the dbpath param-eter against the mounted volume .

7. Grant the mongodb user access to write into the dbpath . chown mongodb:mongodb /var/lib/mongodb

8. Set for the mongodb service to start on boot . Do this after the volume is created as the initial service start creates the mongodb system files .

chkconfig mongodb on service mongodb start

9. Enter the mongodb shell by executing mongo . Further configuration may be needed within /etc/mongodb.conf to access the shell remotely .

8

Big Data Technical GuideBuild, Deploy and Maintain MongoDB on OpenStack-based SUSE Cloud

10.  The Replica Set needs to be initialized and a list of nodes configured on each node . Assuming 2 different instances have been started (repeat Task II to this point), we will create a replica set configuration and initialize it .

Note: Please replace IPs .

config = { _id: “rs0”, members:[ { _id : 0, host : “192.168.123.71:27017”}, { _id : 1, host : “192.168.123.72:27017”} ]};

rs.initiate(config);rs.status();

11.  Success would appear as

“info” : “Config now saved locally . Should come online in about a minute .”,

“ok” : 1

and an exit and re-entering mongodb a few moments later to show

rs0:PRIMARY> and rs0:SECONDARY>

Appendix AMongodb conf

# /etc/mongodb.conf# mongod server with replication enabledport = 27017dbpath = /var/lib/mongodbfork = truereplSet = rs0logpath = /var/log/mongodb/mongodb.loglogappend = yes

www.suse.com

Contact your local SUSE Solutions Provider, or call SUSE at:

1 800 796 3700 U .S ./Canada1 801 861 4500 Worldwide

SUSEMaxfeldstrasse 590409 NurembergGermany

264-000007-001 | 08/14 | © 2014 SUSE LLC. All rights reserved. SUSE and the SUSE logo are registered trademarks, and SUSE Studio is a

trademark of SUSE LLC in the United States and other countries. All third-party trademarks are the property of their respective owners.