Persistent Storage for Linux Containers Louis Gordon Senior Storage Solutions Architect, Red Hat

Linux Containers in brief

A Software packaging concept that typically includes an application and all of its runtime dependencies

• Higher Quality Software

Releases

• Shorter Test Cycles

• Easier Application

Management

HOST OS

SERVER

CONTAINER

LIBS

APP

CONTAINER

LIBS

APP

Benefits

Greater Portability, Automation, and Integration

Code Registry

Push image

Code & Build Test Deploy

Pull image

The Red Hat Stack – From PaaS to Storage

DevOps Tools and User Experience

Language Runtimes and Middleware

Databases and Other Services

Container Orchestration and Management

Container API

Storage

Container Host

Typical workloads for Containers

Base: 194 IT operations and development decision-makers at enterprises in APAC, EMEA, and North America Source: A commissioned study conducted by Forrester Consulting on behalf of Red Hat, January 2015

“For which workloads or application use cases have you used/do you anticipate to use containers?”

Scalable, Cost Effective, Distributed Storage for Containers

Why Containers require persistent Storage?

• Web & presentation layers should be stateless ... but ...

• That’s probably < 5% of all application instances in a DC

• Most other infrastructure applications / services require persistence for

storing application and configuration data

• Building a different infrastructure for stateful vs. stateless apps ?

• Modern IT requires Standardization (e.g. Cloud Computing)

• Software-defined DC is a hybrid cloud enabler

The requirements for Container Storage?

• Scalable – Scale out capacity & performance

• Resilient – Likely there will be important data

• Flexible – Allow different application access profiles

• Software-defined – To allow flexible deployment models

• Open – For customer choice and lowest TCO

RED HATSTORAGE➔

Current Options for Persistent Storage

Shared Filesystems: • NFS • GlusterFS

Block Storage: • Ceph RBD • iSCSI or FC • GCE Persistent Disk • AWS Elastic Block Store

Storage Options ComparisonNFS-Filer GlusterFS Ceph

RBDiSCSI / FC GCE PD AWS EBS

Scalability - + ++ - ++ ++

Availability + ++ ++ + + +

Cost - ++ + ++ - -

Deployment Flexibility

- ++ ++ - - -

Data Privacy

++ ++ ++ ++ -- --

Sharing Data

++ ++ +/- - - -

Use Cases Generic Data Sharing

Generic Data Sharing

Very Large Deployments, Object Store

High Performance

Block

Cheap Block Storage

Cheap Block Storage

STORAGE APPLIANCE

COMPUTE NODE + + + +

+

STORAGE NODE

COMPUTE NODE + + + +

+ + + +

+

+

+

+

+

+

+

+

Scale-Out Storage

• Increased Performance & Capacity

• Avoid Fork-Lift Upgrades

• Built-In Redundancy

Why Bother?

PROPRIETARY HARDWARE

HARDWARE-BASED INTELLIGENCE

SCALE-UP ARCHITECTURE

CLOSED DEVELOPMENT PROCESS

Common, off-the-shelf hardware

Lower cost, standardized supply chain

Scale-out architecture

Increased operational flexibility

Software-based intelligence

More programmability, agility, and control

Open development process

More flexible, well-integrated technology

Persistent Storage requires Redundancy

• Red Hat Storage includes important data redundancy features

• 2-way and 3-way replication

• Erasure Coding

• Geo-Replication

• Snapshots

Red Hat Storage and Containers

Node 1NGINX

Container

Node 2NGINX

Container

Persistent Storage for OSE/RHEL Atomic

Node 1NGINX

Container

Node 3MySQL

Container

File

sha

re

File

sha

re B

lock

Dev

ice

/

Obj

ect

Stor

e

Atomic Host Atomic HostOPENSHIFTOPENSHIFT

Gluster

Ceph

Gluster

Host OS

Container Container

Host OS

Container Container

/share (Gluster/NFS)

Container Volume mapping options

Gluster Volume

Brick 1 Brick 2 Brick 3 Brick n

/sha

re

(Glu

ster

/NFS

)

/share/sub1 ➔ /data

/share/sub2 ➔ /data

Host OS

Container Container

Host OS

Container Container

/shared-volume (Gluster-Fuse or NFS)

Sharing amongst hosts

/shared-volume /shared-volume /shared-volume /shared-volume

Gluster Volume

Brick 1 Brick 2 Brick 3 Brick n

Host OS

Container Container

Host OS

Container Container

Mapping Ceph RBDs to containers

/shared ➔ /data

Ceph

OSD OSD OSD OSD

Ceph

RBD

+

FS

Ceph

RBD

+ F

S /shared Ceph RBD + FS

/host-share ➔ /data /local-mnt

/shared ➔ /data/h

ost-

shar

e

App Container App Container App Container App Container

Containerized Gluster Storage

/shared-volume /shared-volume

Gluster Container (privileged)

/bricks/brick2

Gluster Container (privileged)

/bricks/brick1

/shared-volume (Gluster-Fuse or NFS)

Host OS Host OS

Gluster Volume

/shared-volume /shared-volume

Hyperconverged Containers and Storage

• Lower TCO

• Unified

Orchestration

• Ease of Use

• Greater control

ContainerJBoss NGINX

NGINX

ContainerMASTER

Kubernetes Node 2

ContainerElastic Spark

MongoDB

Container

Kubernetes Node 3

Redis PostgreSQL

Container

Kubernetes Node 1

Kubernetes Node 4

Container Container Container Container

Containerized Ceph Storage

/local-mnt1

Container

RHCS

Container

RHCS

Ceph RBDs (and Object Storage)

Host OS Host OS

Ceph Pool

/local-mnt2 /local-mnt3 /local-mnt4

Flexible Deployment Options

x86 x86 x86

Physical

VM VM VM

Virtualized

VM VM VM

Public / Private Cloud

OpenStack, AWS, Azure, GCE

VMware, RHEV, KVM

Container

JBoss

NGINX

NGINX Container

MASTER

Kubernetes Node 2

Container

Elastic

Spark

MongoDB Container

Kubernetes Node 3

Redis

PostgreSQL Container

Kubernetes Node 1

Kubernetes

Node 4

Container

JBoss

NGINX

NGINX Container

MASTER

Kubernetes Node 2

Container

Elastic

Spark

MongoDB Container

Kubernetes Node 3

Redis

PostgreSQL Container

Kubernetes Node 1

Kubernetes

Node 4

Container

JBoss

NGINX

NGINX Container

MASTER

Kubernetes Node 2

Container

Elastic

Spark

MongoDB Container

Kubernetes Node 3

Redis

PostgreSQL Container

Kubernetes Node 1

Kubernetes

Node 4

Customer Case Studies

Customer Case Study: CapitalOne

Business Challenge: • A leading diversified bank with 65 million customers • Fast growing business and customer base • Need to be disruptive and different • Analytics plays a big role in growth strategy

Solution Description: • Predefined docker images with a wide variety of analytics

tools • Self-service Portal for developers to pick and instantiate • Integrated monitoring and metrics • Automated lifecycle management of containers • High availability through MESOS • Shared and consolidated Storage Platform with Gluster

Solution Benefits: • More agile application development

• Larger choice of technologies

• Optimal resource usage and performance

Customer Case Study: CapitalOne

Customer Case Study: Verizon

Docker containers running

Business Challenge:

• Verizon is the largest provider of cellular services in the United States with

more than 100 million subscribers. Pretty much every subscriber takes

‘selfies’ and snaps of the kittens and these need to be backed up. Whilst many

users use Apple’s iCloud, or Yahoo’s Flickr, Verizon also has their own branded

Cloud offering – Verizon Cloud.

• Verizon wished to use a file-based format rather than converting all these

images to Objects and to have an extremely efficient architecture in terms of

the use of their server infrastructure.

Solution Description: • Simplified Deployment via containers • Seamless upgrade and rollback • Dedicated and Containerized Gluster for FSaaS • Dynamic scaling (up) of capacity as needed • Performance and health metrics collection via container monitoring agents

and sending alerts.

Solution Benefits: • Quicker time to market for new services • Increased subscriber ‘stickiness’ and improving customer

satisfaction. • Reduced infrastructure Costs • Infrastructure can grow as demand grows

ETH/IBapp app app

app app app

Converged Computing ArchitectureMixed App and Storage Workloads

App-only servers

Converged servers

app

app +

Storage-only servers

app app app app

app app app app

app app app app

app app

app

Storage stack imposes only3% - 10% load on compute processing

• Applications and storage stacks can co-exist on same compute substrate.

• This achieves higher server

utilization and lower operational costs across the cloud

Customer Case Study: Verizon

Summary

• Most containerized applications will require Persistent Storage

• Software-defined Storage allows hyper-convergence for applications and

storage

• Red Hat Storage and OpenShift Enterprise provide a complete PaaS solution

with full deployment flexibility from on-premise to hybrid clouds

Thank You