Containerized Storage for Containers

OpenEBS Online Hangout #3September 14, 2017@OpenEBS

Agenda

Why OpenEBS?

OpenEBS Community Updates

Demo - OpenEBS on AWS

Demo - Benchmarking DB on Containers

Q/A

● Back in the day of IPX/SPX

○ We had many servers with all a

little storage

● Performance was not at the level

of current 15K SAS drives

● Back-ups jobs were hard to

manage

● Networking was really fast back

then

The idea was to ‘consolidate’ all storage

islands into a large one such that all

things become easier and faster

Why do we have centralised storage?

● Network of storage systems

○ FC loops as ethernet was not

fast enough

● Special purpose file systems to

implement advanced storage

features

○ Deduplication

○ Snapshots

○ Clone

○ Replication

○ Etc

● Scale as far as the controller

would let you

Storage (attached) Networks

● Container change the way we

typically consume storage from

an application perspective

○ Number of mounts points

high

● This alone creates scalability

problems

● Solution for OpenEBS would

be to explore a distributed

solution

● But…

○ Applications are already

distributed!

Apps changed; storage is still the same

● Distributed filesystems are inherently difficult to develop and more importantly hard and

nasty to maintain in production

● Require more often than not, specialised client drivers to really leverage distributed

nature. NFS and SMB are not distributed client protocols

● We could have lipsticked something already there

○ Linux comes with distributed filesystems within kernel-tree

Not Yet Another Distributed Filesystem

● Application have native scalability features, load balancers, database sharding, Paxos and RAFT

(applications go down stack)

● Does not mean that we don't need storage HA by definition but it does show

○ Data availability is not exclusively controlled at the storage layer

○ App performance is now loosely coupled from storage

○ Does the developer know best what the micro services need?

● How we manage and deploy application has also changed

● Applications have changed and somebody has forgotten to tell storage

What app changes are we talking about?

● Bringing enterprise class storage features to individual storage containers○ No compromise on features

● Cloud native design build using the same containers technology it stores for○ Don't monkey patch existing storage with plugins

● Implemented in user space to fully leverage high speed fabrics and devices○ 24 cores, 40gbe networking

● Declarative provisioning and protection policies○ You decide what the storage should look like

● Multi-cloud○ Data mobility included

● Not a cluster storage instance but a cluster of storage instances

Why OpenEBS ?

Community

Updates

Growing Community!

Growing Community and Spaces!

Hacktoberfest

Find out the following tags in Github

first-timers-only

hacktoberfest

for-new-contributors

Upcoming Events

OpenEBS on AWS

Demo

Pre-requisites

● AWS account.● User with Identity and Access Management permission.

● uday with IAMFullAccess permission.● uday user’s SecretAccessKey and AccessKeyID as Environment Variables.

export AWS_ACCESS_KEY_ID=<access key>export AWS_SECRET_ACCESS_KEY=<secret key>

State : Current -> Desired

Kubernetes - Cluster

Deployment - Is It Easy?

OpenEBS GitHub Repo:● Clone OpenEBS GitHub repo.

https://github.com/openebs/openebs● Deployment Instructions and Automation Script.

○ https://github.com/openebs/openebs/tree/master/e2e/terraform○ OpenEBS cloud deployment script oebs-cloud.sh.

Decode Deployment

OpenEBS Deployment in Cloud - Tools Used

Benchmarking

percona on

OpenEBS

Demo

Benchmarking Setup on AWS

Steps to run the percona TPC-C benchmark

● Deploy the percona pod on kubernetes (percona.yaml)

● Setup the percona monitoring by deploying the PMM-server & registering the percona host into it (pmm-server.yaml)

● Create the TPC-C benchmark specifications via configmap (tpcc.conf)

● Prepare database, load data per spec & start benchmark (tpcc-bench.yaml)

❖ YAMLs : https://github.com/openebs/performance-benchmark

❖ Custom images : https://github.com/openebs/test-storage

Running Percona on OpenEBS

● Percona server is an instrumented & enhanced open-source drop in replacement for MySQL DB. Better performing InnoDB, called XtraDB

● Percona official image on dockerhub has 5M+ pulls

Monitoring Percona using PMM

● Percona Monitoring and Management (PMM) is an open-source platform from Percona for managing and monitoring MySQL performance

● PMM-Client sits on the DB host & collects server & general system metrics

● PMM-Server aggregates collected data & presents it on a dashboard.

Benchmarking Percona : TPC-C

● 1 - Company● N - Warehouses ● 100K Items / WH● 10 - Districts / WH● 3000 Customers / District● Stocks, History, Orders● User think times !

Next steps ?

● Resulting metric tpmC (New orders per minute) accounts for ~45% of the Transactions only

● Analyze results, repeat with new set of tpcc specs OR mysql/innodb startup variables !

● Doesn’t reveal everything about the underlying storage’s performance, but aids Apple to Apple comparison under standard operating environment

● The steps we have used are generic & can be applied with any storage volume !

● Let us collaborate !!

What do you think?

https://github.com/openebs

Join the community

#slackSlack.openebs.io

blog.openebs.io

@openebs

We have reached the End of the hangout !

Thank you for joining us !

See you at the next hangout !