Why software defined storage matters?Sergey Goncharov

Solution Architect, Red Hatsgonchar@redhat.com

AGENDA● Storage and Datacenter evolution

● Red Hat Storage portfolio● Red Hat Gluster Storage

● Red Hat Ceph Storage● Red Hat Storage roadmap and future

● Q/A

3

WHAT IS SOFTWARE-DEFINED STORAGE?

SERVER-BASED CENTRALIZED CONTROL OPEN ECOSYSTEM

4

THE DATACENTER IS EVOLVING

DEVELOPMENTMODEL

APPLICATIONARCHITECTURE

DEPLOYMENTAND PACKAGING

APPLICATIONINFRASTRUCTURE

STORAGE

Waterfall

Agile

DEVOPS

Monolithic

N-Tier

MICROSERVICES

Virtual Services

CONTAINERS

Bare Metal

Hosted

HYBRID CLOUD

Data Center

Scale Out

SOFTWARE-DEFINED STORAGE

Scale Up

5

SERVER-BASED STORAGE

Server-based storage is the use of software and standard hardware to provide services traditionally provided by single-purpose storage appliances (similar to server virtualization, which uses software to emulate servers), providing increased agility and efficiency.

Appliance

USER

Appliance

USER

Appliance

USER

Distributed Cluster of Services

USER USER USER

6

STORAGE ORCHESTRATION

Storage orchestration is the ability to provision, grow, shrink, and decommission storage resources on-demand and programmatically, providing increased control and integration of storage into a software-defined data center.

Web Console API Command Line

A browser interface designed for

managing distributed storage

A full API for automation andintegration with outside

systems

A robust, scriptable command-lineinterface for expert operators

Full life cycle management for distributed, software-defined data services

PROVISION INSTALL CONFIGURE TUNE MONITOR

7

GROWING INNOVATION COMMUNITIES

• Contributions from Intel, SanDisk, CERN, and Yahoo

• Presenting Ceph Days in cities around the world and quarterly virtual Ceph Developer Summit events

• Over 11M downloads in the last 12 months

• Increased development velocity, authorship, and discussion has resulted in rapid feature expansion

97 AUTHORS/MO

2,453 COMMITS/MO

260 POSTERS/MO

33 AUTHORS/MO

97 COMMITS/MO

138 POSTERS/MO

8

THE FUTURE OF STORAGEOPEN, SOFTWARE-DEFINED STORAGE

Standardized, unified, open platforms

Standard Computersand Disks

Ceph Gluster +++

Control Plane (API, GUI)

Custom GUI

Proprietary Hardware

Proprietary Software

Custom GUI

Proprietary Hardware

Proprietary Software

USER USER USER

ADMIN ADMIN ADMIN

Custom GUI

Proprietary Hardware

Proprietary Software

USER ADMIN

Sta

nd

ard

Ha

rdw

are

Op

en

So

urce

So

ftwa

re

TRADITIONAL STORAGEComplex proprietary silos

9

THE RED HAT STORAGE PORTFOLIO

Gluster management

Ceph data service Gluster data service

Ceph management

OP

EN

SO

UR

CE

SO

FT

WA

RE

STA

ND

AR

DH

AR

DW

AR

E

Share-nothing, scale-out architecture provides durability and adapts to changing demands

Self-managing and self-healing features reduce operational overhead

Standards-based interfaces and full APIs ease integration with applications and systems

Supported by the experts at Red Hat

10

HOW STORAGE FITS

RED HAT

STORAGE

PHYSICAL VIRTUALIZATION PRIVATE CLOUD CONTAINERS PUBLIC CLOUD

RED HAT GLUSTER STORAGEOpen source, software-defined storage for unstructured file data at

petabyte scale

Media, video Machine, Log Data

GeoSpatial Persistent Storage

Documents

RED HAT GLUSTER STORAGE ADVANTAGES

OPEN

SCALABLE

ACCESSIBLE

MODULAR

ALWAYS-ON

Open, software-defined distributed file and object store

No Metadata Server

Multi-Protocol the Same Data

No Kernel Dependencies

High-Availability across data, systems and applications

• Based on GlusterFS open source community project• Uses proven local file system (XFS)• Data is stored in native format

• Uses an elastic hashing algorithm for data placement• Uses local filesystem’s xattrs to store metadata • Nothing shared scale-out architecture

• Global name space• NFS, SMB, object, HDFS, Gluster native protocol• Posix compliant

• GlusterFS is based on filesystem in userspace (FUSE)• Modular stackable arch allows easy addition of features ...without being tied to any kernel versioniant

• Synchronous replication with self-healing for server failure • Asynchronous geo-replication for site failure

GLUSTER ARCHITECTUREDistributed scale out storage using industry standard hardware

NFS CIFS FUSE

SERVERS WITH LOCAL DISKS

Aggregates systems to one cohesive unit and presents using common protocols

WHY RED HAT CEPH STORAGE FOR OBJECTS?

• Proven at Web-Scale for Object Storage

• Flexible Storage for Your Applications

• Data Protection, Reliability & Availability Enterprises Demand

• Open, Community Based Software Defined Storage for Object Storage

• Cost Effective Object Storage to Help You Maximize Your Storage Budget

15

CEPH ARCHITECTURE

APPLICATION HOST / VM CLIENT

RGW

A web services gateway for object storage, compatible

with S3 & Swift

RBD

A reliable, fully-distributed block device with cloud

platform integration

CEPHFS*

A distributed file system with POSIX semantics &

scale-out metadata

LIBRADOS

A library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby, PHP)

RADOS

A software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors

* TECH PREVIEW

NEW INTEGRATED MANAGEMENT WITH RED HAT STORAGE CONSOLE 2

• Easy to use graphical interface to manage storage cluster life cycle

• Ansible-based deployment tools for installation, importation, and granular configuration from CLI or GUI

• Monitoring and troubleshooting with statistical and graphical information about cluster components

RADOS. RGW, RBD, CALAMARI

RED HAT STORAGE CONSOLE 2

CLUSTER OF STANDARD SERVERS AND MEDIA

CLIENTSADMINS

ROAD MAP: RED HAT GLUSTER STORAGE

Gluster 3.7, RHEL 6, 7 Gluster 3.8, RHEL 6, 7 Gluster 4, RHEL 7

TODAY (v3.1) V3.2 (H1-2016) FUTURE (v4.0 and beyond)C

OR

EFI

LES

EC

MG

MT

CO

RE

FILE

SE

CM

GM

T

CO

RE

FILE

SE

CM

GM

T

ERASURE CODING

BIT ROT DETECTION

TIERING

SNAP SCHEDULE

ACTIVE/ACTIVE NFSV4

SMB 3 (BASIC SUBSET)

SELINUX

SSL ENCRYPTION( IN-FLIGHT)

DEVICE MANAGEMENT

GEO-REPLICATION,SNAPSHOTS

DASHBOARD

INODE QUOTAS

FASTER SELF-HEAL

CONTROLLED REBALANCE

SMB 3(ADVANCED FEATURES)

MULTI-CHANNEL

AT-REST ENCRYPTION

DYNAMIC PROVISIONING OF RESOURCES

COMPRESSION

DEDUPLICATION

HIGHLY SCALABLE CTRL PLANE

NEXT-GEN REPLICATION DISTRIBUTION

pNFS

QOS

CLIENT SIDE CACHING

NEW UI

GLUSTER REST API

CORE • Bluestore (tech preview)• Performance optimizations

CORE • Bluestore general availability• Data checksums• End-to-end SSL• Improved multi-threading

RED HAT CEPH STORAGE 2 (today)

MANAGEMENT• Red Hat Storage Console 2• Lifecycle management• Ansible support

OBJECT• Support for global clusters• AD, LDAP, Keystone v3• Expanded API compatibility• NFS gateway (tech preview)

BLOCK• Mirroring

RED HAT CEPH STORAGE 2.1 (Q4 2016)

OBJECT• AWS Secure Token Service (STS)

BLOCK• iSCSI

UPSTREAM ACTIVITY (in the future)

FILE• Active/active metadata servers• Snapshots• Hypervisor integration

OBJECT• Metadata search• Tiering to public cloud• Object-level encryption• Compression

BLOCK

• Distributed client-side cache• Improved iSCSI support