tr-4680 enterprise data apps aas with robin systems and ... · 1 introduction data drives today’s...

Technical Report

Enterprise Data Apps as a Service Using Robin Systems and the NetApp Data Fabric Partha Seetala and Tushar Doshi, Robin Systems

Santosh Rao, Karthikeyan Nagalingam, and Pete Schay, NetApp

April 2018 | TR-4680

In partnership with

2 Enterprise Data Apps as a Service Using Robin Systems and the NetApp Data Fabric

© 2018 NetApp, Inc. All Rights Reserved.

TABLE OF CONTENTS

1 Introduction ........................................................................................................................................... 4

2 Robin Systems Overview ..................................................................................................................... 4

2.1 Installing Robin Systems Software ..................................................................................................................5

2.2 Registering IP Pools .......................................................................................................................................5

2.3 Registering NetApp ONTAP Storage Arrays ...................................................................................................6

2.4 Upload Application Bundles ............................................................................................................................7

3 NetApp Overview .................................................................................................................................. 8

3.1 NetApp All Flash FAS .....................................................................................................................................8

4 Solution Overview ................................................................................................................................ 9

4.1 Deploy Application in One Click ......................................................................................................................9

4.2 Snapshot Application Clusters in One Click .................................................................................................. 13

4.3 Backup and Restore an Application Cluster in One Click ............................................................................. 15

4.4 Clone an Application Cluster with One Click ................................................................................................. 16

4.5 Patching and Upgrading an Application Cluster with One Click .................................................................... 17

4.6 Scaling Cluster Resources with One Click .................................................................................................... 19

4.7 Scale Out Cluster in One Click ...................................................................................................................... 19

5 Solution Validation ............................................................................................................................. 20

5.1 MongoDB Cluster Configuration ................................................................................................................... 21

5.2 Performance Benchmarks............................................................................................................................. 21

1.1. Setup for YCSB ............................................................................................................................................. 21

5.3 MongoDB Cluster Configuration for YCSB.................................................................................................... 22

5.4 YCSB Results ............................................................................................................................................... 22

6 Summary ............................................................................................................................................. 24

Where to Find Additional Information .................................................................................................... 24

Version History ......................................................................................................................................... 24

LIST OF TABLES

Table 1) MongoDB cluster configuration details. .......................................................................................................... 21

LIST OF FIGURES

Figure 1) The Robin Systems and NetApp advantage. ..................................................................................................4

Figure 2) Solution capabilities. .......................................................................................................................................5

Figure 3) Robin IP pool assignment. ..............................................................................................................................6



Figure 4) Setting up OnCommand API Services. ...........................................................................................................6

Figure 5) Add New ONTAP Storage Array workflow. .....................................................................................................7

Figure 6) Deploy MongoDB. ...........................................................................................................................................9

Figure 7) Relationship between the MongoDB data volume and the ONTAP objects that are created by Robin. ........ 10

Figure 8) Customize a Hortonworks cluster. ................................................................................................................. 11

Figure 9) Customize a multiple-data center Cassandra cluster. ................................................................................... 12

Figure 10) Customize the deployment of a Splunk cluster. .......................................................................................... 13

Figure 11) Intuitive interface to configure snapshot creation and retention. ................................................................. 14

Figure 12) Configure a backup schedule. ..................................................................................................................... 15

Figure 13) Clone a MongoDB cluster with one click. .................................................................................................... 16

Figure 14) Update and patch a MongoDB cluster in one click. ..................................................................................... 18

Figure 15) MongoDB cluster resources can be dynamically adjusted with one click. ................................................... 19

Figure 16) MongoDB services can be scaled out in one click. ..................................................................................... 20

Figure 17) Test setup for validating the Robin and NetApp solution for MongoDB. ...................................................... 21

Figure 18) Test setup for running YCSB....................................................................................................................... 22

Figure 19) YCSB workload A: 50% reads and 50% writes. .......................................................................................... 23

Figure 20) YCSB workload B: 95% reads and 5% writes. ............................................................................................ 23

Figure 21) YCSB workload C: 100% reads. ................................................................................................................. 24



1 Introduction

Data drives today’s business decisions, and organizations rely on quick access to various big data,

NoSQL, and database tools to store, analyze, and manage their exploding volume of data. Projects that

involve data-intensive applications are often delayed because of the complexity of deploying and

managing these tools. This problem is made worse by the inflexibility of adjusting compute, storage, and

network infrastructure to meet an organization’s changing needs.

This technical report describes how your organization can radically simplify its big data, NoSQL, and

database environments. It explains how to create a self-service platform that integrates the data

management strengths of NetApp® ONTAP® software and the Robin Systems application-aware cloud

management platform.

This joint solution creates an app-store-like user experience that takes an application-centric approach to

achieve one-click deployment of big data, NoSQL, and database applications. Postdeployment lifecycle

management operations for entire applications, such as snapshots, clones, upgrades, scaling, quality-of-

service (QoS) management, backups, and restores, become one-click operations. The application context

allows you to avoid worrying about the underlying compute, storage, and networking infrastructure

components. Figure 1 presents an overview.

Figure 1) The Robin Systems and NetApp advantage.

2 Robin Systems Overview

Robin Systems produces a software solution that enables a true application-centric approach to

deployment and management of the lifecycle of big data, NoSQL, and database applications. The Robin

Systems solution is composed of four core components:

• Containers (both Docker and Linux)

• NetApp ONTAP storage that is programmatically integrated into the solution

• Integrated networking

• A built-in, application-centric workflow manager

This solution provides one-click simplicity for deployment and management of the lifecycle of complex

distributed big data, NoSQL, and database applications. With this solution, you can deploy, snapshot,

clone, patch, upgrade, backup, restore, scale, control QoS, and much more. All these processes can be

initiated with a single mouse click regardless of the size and complexity of the application. Figure 2

presents an overview of the capabilities of the solution.

http://robinsystems.com/



Figure 2) Solution capabilities.

2.1 Install Robin Systems Software

Robin Systems software is a single file installer that can be installed on any x86_64 server that runs the

Red Hat version or the CentOS 7.x version of Linux. The destination server can be bare metal. Or a

virtual machine (ESXi, Hyper-V, Kernel-Based Virtual Machine [KVM], or Xen) can be hosted on the

premises in a private data center or in the public cloud (Amazon Web Services [AWS], Azure, or Google).

Installation is simple, and within 15 minutes, users can start using the Robin Systems platform to deploy

big data, NoSQL, and database applications.

To install Robin Systems software, complete the following steps:

1. Create a new Robin cluster by logging in to the system and running the following command:

FirstNode> ./robin-installer.sh new

2. Add other nodes to the Robin cluster.

OtherNodes> ./robin-installer.sh join <IP-Address-Of-FirstNode>

The Robin installer discovers the compute and networking resources that are offered by the node on

which the product is installed. After the cluster is formed, the resources appear as a pool of CPU cores,

DRAM, storage disks, and network subnets that can be used to spin up applications.

Before launching an application, Robin needs to know what IP addresses it can assign to the containers

that run the applications. It must also know what storage to use for an application.

2.2 Register IP Pools

IP addresses that Robin can assign to applications are registered by using the Add IP Pool workflow, in

which you specify a range of IP address and the subnet to which these addresses belong. Figure 3 shows

40 IP addresses in the range of 10.56.160.41 to 10.56.160.80 within the subnet 10.56.160.0/24 that are

registered for use by Robin. You can register multiple IP pools. During the application creation workflow,

you can select one of the registered IP pools from which Robin assigns IP addresses for the containers

that are created to run applications. Figure 3 depicts the IP pool assignment window.



Figure 3) Robin IP pool assignment.

2.3 Register NetApp ONTAP Storage Arrays

Robin allocates storage volumes for applications to store their data by programming NetApp ONTAP

storage arrays. To facilitate this programmable creation of storage volumes, the Robin configuration

management server communicates by using REST APIs with NetApp OnCommand® API Services, which

in turn manage ONTAP storage arrays (see Figure 4). Storage administrators can set up OnCommand

API Services on a separate VM by following the detailed instructions on the NetApp Support site. Security

policies and the storage virtual machines (SVMs, previously called Vservers) on the storage array that

Robin uses are also configured in OnCommand API Services.

Figure 4) Setting up OnCommand API Services.

After OnCommand API Services is configured, the ONTAP storage arrays that Robin can use to store the

application’s data are registered by using the Add New ONTAP Storage Array workflow. You must enter

the SVM name and API server endpoint details in the window that is depicted in Figure 5. Robin

automatically discovers the aggregates that are configured on the SVM along with their characteristics.

During the application creation workflow, you can select one of the registered storage arrays, and Robin

automatically creates and configures the storage volumes, LUNs, QoS groups, and so on, on the selected

storage array.

https://mysupport.netapp.com/NOW/cgi-bin/software/?product=OnCommand+API+Services&platform=Linux



Figure 5) Add New ONTAP Storage Array workflow.

2.4 Upload Application Bundles

Robin’s goal is to provide an app-store experience for deploying any application, with special focus on the

needs of complex, distributed, data-heavy big data, NoSQL, and database applications.

Applications are deployed inside containers on the Robin platform. Both Docker and Linux containers are

supported. To make one-click deployment easy for any application, a Robin system must know some

basic information about an application:

• Services that an application offers. For example, the services that MongoDB provides include ConfigServer, QueryRouter, Shards, and ReplicaSets.

• The Docker or Linux container images for each of these services. The images can be vendor-published or custom Docker images that are hosted on Docker Hub or on any private Docker registry.

• Any application-specific placement constraints (optional). For example, you can specify affinity and antiaffinity policies for services.

This information is written down in a simple YAML file called a manifest file. The manifest file, along with

an icon for the app and any optional application-specific hooks, is packaged into a single TAR file and is

uploaded into the Robin platform. This package is called an application bundle in Robin.

The Robin installer comes prepackaged with the following ready-to-use application bundles:

• Big data:

Hortonworks

Cloudera

Spark

Elasticsearch

ELK

Splunk

• NoSQL:

MongoDB

Cassandra

CouchDB



• Databases:

Oracle

Oracle RAC

SAP HANA

Microsoft SQL Server

IBM Db2

PostgreSQL

MySQL

MariaDB

With these bundles, you can quickly onboard any application or database that runs on Linux.

After Robin is uploaded, it presents an app-store-like view that enables you to deploy these applications

from the UI with one click, with a single command from the CLI, or with a single REST call.

3 NetApp Overview

The Data Fabric is the NetApp vision for the future of data management. The NetApp Data Fabric

seamlessly connects different data management environments across disparate clouds into a cohesive,

integrated whole. The NetApp Data Fabric helps organizations maintain control and choose the way that

they manage, secure, protect, and access their data across the hybrid cloud, no matter where it is.

Although the Data Fabric evolves constantly, your organization can start taking advantage of it today by

using NetApp technologies that enable data management and seamless data movement across the

hybrid cloud. For more information about the Data Fabric powered by NetApp, see WP-7218, NetApp

Data Fabric Architecture Fundamentals: Building a Data Fabric Today.

3.1 NetApp AFF

NetApp AFF is an enterprise-grade all-flash array that offers powerful benefits:

• High performance at scale

• Inline deduplication and compression

• Modular scale-out

• Best-in-class data management

• Deep application integration

These capabilities, coupled with MongoDB, can help you build a highly scalable, high-performing, and

cost-efficient analytics and scale-out database solution.

The key benefits of AFF also include the following capabilities:

• Accelerate databases with 4 to 12 times higher IOPS and 20 times faster response, powered by NetApp ONTAP FlashEssentials.

• Reduce solid-state drive (SSD) storage loads fivefold to tenfold on average by using data reduction technologies.

• Scale out to 24 storage nodes in a cluster and move data nondisruptively between flash and HDD tiers.

• Safeguard your data with an integrated data protection suite that has a starting cost as low as $25,000.

• Set up your system easily.

For more information about AFF, see the NetApp AFF overview on NetApp.com.

https://www.netapp.com/us/media/wp-7218.pdf

https://www.netapp.com/us/media/wp-7218.pdf

https://www.netapp.com/us/products/storage-systems/all-flash-array/aff-a-series.aspx



4 Solution Overview

The rest of this report uses MongoDB as a sample application to demonstrate how you can use one click

to deploy and manage any big data, NoSQL, or database application.

4.1 Deploy an Application with One Click

To create a new MongoDB cluster, log in to the Robin UI, navigate to the applications bundle page, and

click the MongoDB application bundle. To allow customization of the application, Robin renders the

following wizard by parsing the manifest file in the MongoDB application bundle.

You can customize the MongoDB application by specifying just the resource requirements, without

worrying about where they are allocated from, as shown in Figure 6.

Figure 6) Deploy MongoDB.



If you click Save as Template, the MongoDB deployment configuration is stored for later reuse. If you

click Provision Application, a new MongoDB cluster is spun up. The following four provisioning steps are

automated by Robin, making them invisible to the end user:

1. Robin generates a placement plan that determines the best compute resources for the different instances of MongoDB components. Attributes such as system utilization, isolation policies, licensing restrictions, and application-specific affinity and antiaffinity properties are honored when assigning hosts for the various components.

2. ONTAP storage arrays are programmed to allocate storage volumes based on your required specifications. This feature includes honoring media type (HDD, SSD), QoS policies, isolation policies, data protection and management policies, and so on.

3. An integrated Open vSwitch is programmed to create network bridges, and it assigns an IP address to each component.

4. Hook scripts that are written in Python, Perl, Bash, or Ansible are executed to perform any application-specific customization.

Figure 7) Relationship between the MongoDB data volume and the ONTAP objects that are created by Robin.

Figure 8 depicts an example of a Hortonworks cluster for which Robin allows you to customize compute,

memory, and storage resources. The Placement Rules section of the UI allows you to configure how the

various services are brought up to confirm that they comply with Hortonworks built-in node-aware and

rack-aware fault domains. As Figure 6 illustrates, Robin presents a familiar interface for all applications. In

this interface, you specify the resource and scale requirements of an application without having to worry

about the assembly of the underlying infrastructure components (servers, network, and storage).



Figure 8) Customize a Hortonworks cluster.

Figure 9 depicts the deployment of a multiple-data center Cassandra cluster on Robin and NetApp. You

can configure advanced placement rules so that seed and member services are paired correctly when

they are deployed across different data centers.



Figure 9) Customize a multiple-data center Cassandra cluster.

Figure 10 depicts the deployment of a Splunk cluster on Robin and NetApp.



Figure 10) Customize the deployment of a Splunk cluster.

4.2 Create a Snapshot of Application Clusters with One Click

Storage-only snapshots have existed for over two decades in the industry. When an application is

distributed or uses multiple storage volumes across one or more shards or partitions, orchestration of

storage-only snapshots across all of them can be tedious, error-prone, and inconsistent. Also, storage-

only snapshots are unaware of the application configuration and its topology, which means that restoring,

cloning, or backing up snapshots is cumbersome and is prone to misconfiguration.



For the first time, Robin Systems allows you to snapshot an entire application with one click from the UI,

with a single CLI command, or programmatically through one single REST call. An application snapshot

captures:

• The application topology and its configuration

• A snapshot of all data volumes

• All placement and QoS policies

After deployment, an application can be triggered on demand or can be automated by using a schedule,

as shown in Figure 11.

Figure 11) Intuitive interface to configure snapshot creation and retention.

Application-Consistent Snapshots for Distributed Applications

Creating a snapshot of an application allows you to restore or to clone that application. You don’t have to

restore individual storage volumes and remap them to their respective partitions or shards. Robin

Systems application snapshots are fully consistent. Therefore, for a distributed database such as

MongoDB, the snapshot confirms that I/O is quiesced across all volumes and across all shards and their

replica sets before the snapshot is captured.

How Application-Consistent Snapshots Work with Robin Systems and ONTAP Integration

Triggering an application snapshot from the Robin Systems UI, from the CLI, or from REST APIs causes

the following actions to be performed in the background:

1. The Robin Systems snapshot orchestrator uses a two-phase protocol. In the first phase, called the presnapshot phase, an I/O quiesce operation is triggered across all shards. This step is divided into two parts:

a. An application-specific hook is invoked to flush any in-core state to the I/O subsystem.

b. The file system’s page cache flusher is invoked to flush dirty data from the page cache to the block-layer subsystem.

2. The Robin Configuration Manager (RCM) logically groups all storage volumes across all shards and replica sets into a transaction group. It then invokes ONTAP APIs to create Snapshot copies of volumes. Consistency is confirmed across all volumes in the transaction group. The NetApp ONTAP WAFL® file system uses redirect-on-write (ROW) technology, which means that Snapshot copies are created almost instantaneously. ONTAP returns Snapshot IDs for each volume in the transaction group.



3. In the second phase, called the post Snapshot phase, RCM captures the: (a) application topology and configuration, (b) the Snapshot IDs in the transaction group, and (c) all placement and QoS policies into a Snapshot metadata object. This metadata object is given a user-assigned name along with an optional user-assigned description. The metadata object is persisted within the Robin Systems RCM database.

4. The post Snapshot phase is completed by invoking any application-specific, post Snapshot hook script.

4.3 Backup and Restore an Application Cluster with One Click

You can back up an application cluster by configuring a backup schedule in Robin. Figure 12 shows how

you can configure the schedule with one click.

Figure 12) Configure a backup schedule.

Robin uses NetApp SnapVault® technology to back up Snapshot copies from the primary storage array to

a secondary storage array. The input requires only the target secondary storage array and the number of

copies of the hourly, daily, weekly, and monthly Snapshot copies that must be retained on the secondary

storage array.

Figure 7 presents an example for protecting the volume mounted at /data/db for MongoDB shard

shard01 and for creating a FlexVol volume named M1_shard01_data. A data protection volume

named M1_shard01_data_backup of the same size is created on the target backup storage array. A

NetApp SnapMirror® relationship between M1_shard01_data (on the source or primary storage array)

and M1_shard01_data_backup (on the target or backup storage array) is also set up.

Before enabling a backup schedule, the SVMs or clusters that are used as a source or a target must

already be peered. Robin automatically initializes SnapMirror so that the baseline transfer is performed

initially. After that baseline transfer, all backups are incremental. Because Robin creates application-level

consistency with backups, it manages the backup job as one transactional unit. In other words, if the

transfer of Snapshot copies across all volumes and multiple shards and services fails, then Robin marks

that backup job as faulted and purges any partial Snapshot transfers from the target storage array.

Restore an Application Cluster from Backup

All snapshots that are created by Robin for an application cluster are available through a snapshot

catalog as shown in Figure 13. The entire application cluster is restored from a snapshot simply by

clicking the Restore to This Point option from the actions menu. Restoring an application includes re-



creating the application cluster topology and configuration settings along with the data volumes as of the

time when the snapshot was taken.

When performing a restore operation, NetApp SnapRestore® technology is used to restore volumes if the

snapshot is available on the source storage array. If snapshots have been transferred to the target

storage array and have been purged from the source, then SnapRestore is used to restore from the target

storage array. These details are automatically handled by the Robin Systems software.

4.4 Clone an Application Cluster with One Click

Cloning an application cluster is useful for running test/dev reports, for user acceptance testing, and for

testing patches and upgrades. Typically, creating clones of an application cluster requires significant

manual or scripting work to coordinate cloning storage volumes from their snapshots that are hosted in a

SAN or DAS environment. After cloning storage volumes, you must orchestrate (manually or with scripts)

the bring-up of another application cluster, for example, MongoDB, that matches the topology and

configuration of the source MongoDB cluster being cloned.

With the integration of Robin Systems and NetApp ONTAP technology, you can easily clone an

application cluster from a previously taken snapshot. You can clone with a one-click operation from the UI

(Figure 13), with a single CLI command, or programmatically by using a single REST call. Cloning creates

a fully functional application cluster, for example, MongoDB, that can be used immediately.

Cloning with Robin Systems goes beyond just cloning the storage volumes. An application (in this case,

the MongoDB cluster) is cloned, resulting in the spinning up of a new MongoDB cluster that matches the

configuration and topology of the cluster when the snapshot was taken. The cloned MongoDB cluster

receives a new network and cluster identity, but the data is the same as when the snapshot was taken.

Clones are fully writable, and any new updates that are made to the data are preserved in their own

independent storage volumes. The data is shared from the snapshot-copied data volumes, but any new

updates and writes that are made from the clone are preserved in their own write stream.

Figure 13) Clone a MongoDB cluster with one click.

Cloning an application requires multiple steps that Robin software automates and hides from the end

user. Broadly, the steps that Robin executes are as follows:

1. Data volumes across multiple shards are cloned from their respective snapshots.

2. The topology and configuration settings of the application at the time the snapshot was taken are retrieved.

3. The cloned volumes are mapped to their respective shards and replica sets.



4. New sets of containers are spun up, matching the topology of the MongoDB cluster.

5. New IP addresses are allocated to each container.

6. Where applicable, application-specific hooks are executed to modify the application configuration with the new IP addresses. Hooks can also be used to mask sensitive data in the clone.

7. The cloned MongoDB cluster is then available for use.

When cloning, you can create a clone from the source storage array or from a backup storage array

depending on where you want the clone to be created.

4.5 Patch and Upgrade an Application Cluster with One Click

Because of the potential risk that the patch is not compatible with the application cluster’s configuration or

data, patching or upgrading an application cluster can be challenging. To create failure-proof patches and

upgrades, it is best to test them on an exact copy (clone) of the application cluster first. If everything looks

good after applying the patch or the upgrade to the clone, then it can safely be applied to the source

database.

Robin Systems significantly simplifies this process through one-click automation of the validation

procedure for a patch or an upgrade (Figure 14). Also, to catch any incompatibilities with the cluster

topology, data, or configuration, you should test the patch or the upgrade before you apply it.



Figure 14) Update and patch a MongoDB cluster in one click.

The process works as follows:

1. An updated MongoDB bundle is uploaded to the Robin application catalog. This new bundle describes which versions of the previous bundles that it can upgrade.

2. After an upgrade is available, all running instances of MongoDB can see it, much like how an update is created for apps that run on a smartphone. See Figure 14.

3. After you click the Update(s) Available notification, the details of the patch or upgrade are presented.

4. Click Rolling Upgrade to make sure that the patch or the update is applied in the sequence that is indicated in the manifest file for the bundle.

5. Click Test Upgrade to create a clone of the entire MongoDB cluster and to apply the patch or to update to the clone. If the update fails in the clone, then it is incompatible with the running cluster. A failure might be due to incompatibility with the cluster configuration, topology, or current on-disk data. The clone can be purged after informing the user that the patch or upgrade could not be applied to the MongoDB cluster. If the update succeeds in the clone, then the patch can be rolled into the running MongoDB cluster.



4.6 Scale Cluster Resources with One Click

Handling transient load spikes requires the dynamic adjustment of the CPU, memory, and I/O resources

that are assigned to each container. Robin simplifies this adjustment by making it a one-click operation

Figure 15). Robin’s REST API can be used to automate dynamic adjustment of resources, for example, to

increase when a load spike occurs and to decrease after the spike ends.

Figure 15) MongoDB cluster resources can be dynamically adjusted with one click.

In the background, Robin takes the following actions:

• CPU and memory:

The container’s cgroup settings are modified to reflect the new core and memory allocations.

Several /proc and /sys entries are virtualized to reflect the new core and memory allocations.

Docker Engine does not perform this step; rather, Robin takes care of this process because it is very important for applications that query /proc and /sys entries to determine resources.

The sysinfo(2) system call is virtualized to reflect the new core and memory allocations.

Again, this step is not performed by Docker Engine. Instead, Robin takes care of it.

• I/O:

Robin programs the ONTAP API server to make changes to the IOPS settings for the LUNs. This step is carried out by creating a QoS policy group for the FlexVol volumes that correspond to the MongoDB data volumes.

4.7 Scale Out a Cluster with One Click

Handling permanent cluster growth—for example, to increase the number of QueryRouters of a

MongoDB cluster—can also be accomplished through a one-click operation or a single REST API call.

Figure 16 shows an example.



Robin automatically executes the following workflow:

1. New infrastructure resources are allocated to accommodate the additional components, which includes CPU cores, memory, storage capacity, and so on.

2. A new placement plan is generated to determine the best servers to spawn the containers on. This plan accounts for any antiaffinity or affinity, isolation, and multitenancy policies that were assigned to the MongoDB cluster when it was first provisioned.

3. ONTAP APIs are invoked to assign the required storage volumes.

4. New IP addresses are allocated from the pool of available IP addresses.

5. The containers are brought online with the IP addresses and storage volumes.

6. Cgroup, /proc, /sys, and sysinfo(2) are virtualized for each container.

7. Application-specific hook scripts are executed to make the MongoDB cluster aware of the newly added containers.

Figure 16) MongoDB services can be scaled out in one click.

5 Solution Validation

To validate the Robin Systems and NetApp ONTAP joint solution, we spun up a sharded MongoDB

cluster. The setup is presented in Figure 17.



Figure 17) Test setup for validating the Robin and NetApp solution for MongoDB.

5.1 MongoDB Cluster Configuration

A MongoDB cluster with two query routers, three config servers, and three shards was spun up by using

Robin as shown in Figure 17. MongoDB version 3.4 was used in this test. The configuration for each

service is described in Table 1.

Table 1) MongoDB cluster configuration details.

Service Name Number Cores Memory Storage

Config server 3 8 cores/cfg server 16GB/cfg server 1 volume of 128GB/cfg server

Query routers 2 16 cores/router 64GB/router N/A

Shards 3 24 cores/shard 128GB/shard 8 volumes of 128GB/shard

5.2 Performance Benchmarks

We validated MongoDB performance with the Robin Systems and NetApp storage by using the Yahoo!

Cloud Serving Benchmark (YCSB) test framework. YCSB is a framework and a common set of workloads

for evaluating the performance of different key-value and cloud-serving stores. It has two components:

• The YCSB client. An extensive workload generator.

• The core workloads. A set of scenarios to be executed by the generator.

5.3 Setup for YCSB

Three physical Linux servers were used to host the Robin cluster. Two servers (Host1 and Host2 in

Figure 17) were configured with 72 cores and 512GB of RAM, and the third server (Host3) was

configured with 32 cores and 128GB of RAM. The disparity among the hosts is due to a lack of availability

of similar configurations at the time of running the test. Each host was running 64-bit Red Hat Enterprise

Linux version 7.4. The hosts were connected by using 10Gb Ethernet to a network switch. Robin software

version 3.1 was used for this test.



NetApp FAS8080 arrays were used in a two-node HA configuration with a capacity of 10.8TB of SSDs

(12 SSDs of 900GB each) and 145.2TB of HDDs (40 HDDs of 3.63TB each). The arrays ran the 9.2

version of ONTAP.

5.4 MongoDB Cluster Configuration for YCSB

A MongoDB cluster with a three-member replica set was created to run the YCSB benchmark as shown

in Figure 18. Each member in the replica set had the following configuration:

• 24 cores

• 128GB memory

• 8 volumes of 128GB each

Figure 18) Test setup for running YCSB.

5.5 YCSB Results

We benchmarked the solution with the following YCSB workloads:

• Workload A. An update-heavy workload that has a mix of 50% reads and 50% writes. An application example is a session store that records recent actions.

• Workload B. A read-mostly workload that has a mix of 95% reads and 5% writes. An application example is photo tagging, in which 5% of the I/O is the addition of tags and 95% of the I/O is tag reading.

• Workload C. A read-only workload with 100% reads. An application example is a user profile cache, for which profiles are constructed elsewhere (for example, Hadoop deployments).

Figure 19 through Figure 21 show the respective results for each workload.



Figure 19) YCSB workload A: 50% reads and 50% writes.

The following command was used to run workload A (threads were 8, 16, 32, and 64):

/root/YCSB/bin/ycsb run mongodb -s -P /root/YCSB/workloads/workloada -p

mongodb.url=”mongodb://mongodb-s1:27017/ycsb1” -p operationcount=120000000 -p

maxexecutiontime=1800 -threads 16

Figure 20) YCSB workload B: 95% reads and 5% writes.

The following command was used to run workload B (threads were 8, 16, 32, and 64).

/root/YCSB/bin/ycsb run mongodb -s -P /root/YCSB/workloads/workloadb -p



47,972

83,393 85,857 88,579

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

8 16 32 64

0

100

200

300

400

500

600

700

800

Thro

ughput

(ops/s

ec)

Number of threads

Read L

ate

ncy in u

sec (

99th

perc

entile

)

Workload A

Throughput (ops/sec) Read latency (99th percentile)

52,972

104,707

185,735

265,959

0

50000

100000

150000

200000

250000

300000

8 16 32 64

0

100

200

300

400

500

600

700

Thro

ughput

(ops/s

ec)

Number of threads

Read L

ate

ncy in u

sec (

99th

perc

entile

)

Workload B




Figure 21) YCSB workload C: 100% reads.

The following command was used to run workload C (threads were 8, 16, 32, and 64).

/root/YCSB/bin/ycsb run mongodb -s -P /root/YCSB/workloads/workloadc -p



These results demonstrate that this solution has equivalent performance numbers compared with creating

MongoDB volumes manually (MongoDB on the NetApp Data Fabric). Therefore, this solution simplifies

and enables multiple one-click application workflows without compromising performance.

6 Summary

This technical report discusses the benefits of a joint Robin Systems application-aware platform as a

service and the NetApp ONTAP Data Fabric solution. This joint offering can dramatically simplify the

deployment and lifecycle management of any big data, NoSQL, or SQL database application.

Where to Find Additional Information

To learn more about the information described in this document, refer to the following documents and

websites:

• Robin Systems https://robinsystems.com/resources/

• NetApp OnCommand API Services https://mysupport.netapp.com/documentation/productlibrary/index.html?productID=62040

• NetApp AFF https://www.netapp.com/us/products/storage-systems/all-flash-array/aff-a-series.aspx

Version History

59,613

119,793

212,133

320,733

0

50000

100000

150000

200000

250000

300000

350000

8 16 32 64

0

50

100

150

200

250

300

350

400

Thro

ughput

(ops/s

ec)

Number of threads

Read L

ate

ncy in u

sec (

99th

perc

entile

)

Workload C


https://www.netapp.com/us/media/tr-4492.pdf

https://robinsystems.com/resources/

https://mysupport.netapp.com/documentation/productlibrary/index.html?productID=62040

https://www.netapp.com/us/products/storage-systems/all-flash-array/aff-a-series.aspx



Version Date Document Version History

Version 1.0 March 2018 Initial release.



Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exact product and feature versions described in this document are supported for your specific environment. The NetApp IMT defines the product components and versions that can be used to construct configurations that are supported by NetApp. Specific results depend on each customer’s installation in accordance with published specifications.

Copyright Information

Copyright © 1994–2018 NetApp, Inc. All Rights Reserved. Printed in the U.S. No part of this document covered by copyright may be reproduced in any form or by any means—graphic, electronic, or mechanical, including photocopying, recording, taping, or storage in an electronic retrieval system—without prior written permission of the copyright owner.

Software derived from copyrighted NetApp material is subject to the following license and disclaimer:

THIS SOFTWARE IS PROVIDED BY NETAPP “AS IS” AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL NETAPP BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

NetApp reserves the right to change any products described herein at any time, and without notice. NetApp assumes no responsibility or liability arising from the use of products described herein, except as expressly agreed to in writing by NetApp. The use or purchase of this product does not convey a license under any patent rights, trademark rights, or any other intellectual property rights of NetApp.

The product described in this manual may be protected by one or more U.S. patents, foreign patents, or pending applications.

RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.277-7103 (October 1988) and FAR 52-227-19 (June 1987).

Trademark Information

NETAPP, the NETAPP logo, and the marks listed at http://www.netapp.com/TM are trademarks of NetApp, Inc. Other company and product names may be trademarks of their respective owners.

http://mysupport.netapp.com/matrix

http://www.netapp.com/TM

tr-4680 enterprise data apps aas with robin systems and ... · 1 introduction data drives today’s...

Documents