WHY SOFTWARE DEFINED STORAGE MATTERS

Red Hat Storage

September 2016

Web, mobile, social media, cloud IoT, ”digitilization”, apps für alle, big data analytics, simulations, constructions…

Media and entertainment A staggering amount of content is created during today’s optimized production processes.

Medicin and research More granularity = higher resolution=more storage. Compliance regulations means saving info as long as the patient lives.

Video on-demand services Rapid growth of video on-demand has resulted in 50% of households using VOD services.

THE DATA EXPLOSION

Solution nr 1: PROPRIETARY APPLIANCES

Easy to start with but complex to manage +PB size Vendor lock-in leads to roadmap glued to the vendors development team Price premium of component costs drives increase customer TCO (vendor recovery is in the upgrades)

THE TRADITIONAL APPROACH TO STORAGE ADMINS

MANAGEMENT INTERFACE

DATA

FC/NFS/SMB/ISCSI ENDPOINTS

“Propriatory” HW and Servers

SERVERS AND MEDIA

PROPRIETARY SOFTWARE

Solution nr 2: PUBLIC CLOUD STORAGE

Complexity hidden from end users, pay-as-you-go pricing, easy to consume No lock in but moving data in and out of data usually costs money Hidden costs (bw, SLA´s, performance) gives uncertain TCO. Ex: The official cost for object 3 cent/GB/month can be 5-8 cents in the end.

CONVENIENT STORAGE AS A SERVICE ADMINS

MANAGEMENT INTERFACE

DATA

CLOUD STORAGE APIS

STANDARD SERVERS AND MEDIA

LINUX + PROPRIETARY SOFTWARE

SOFTWARE DEFINED STORAGE BY RED HAT

Complexity hidden from end users, “higher level management”, No vendor lock in. At all. In any way Built with Open Source solutions. No vendor lock in. (Linux below, No “secret sauce” above). Standards driven with lowest TCO. Object: >1cent Block >2c/m/Gb

STORAGE AS A SERVICE tailored for you ADMINS

MANAGEMENT INTERFACE

DATA

NFS/SMB/RBD/iSCSI + APIs

STANDARD SERVERS AND MEDIA

RHEL + Open Source Storage Software

Why Red Hat?

RED HAT PRODUCT PROCESS We participate in and create community- powered upstream projects.

We integrate upstream projects, fostering open community platforms.

We enable software and hardware partners, customers, and academia to participate at every stage of development.

We commercialize these platforms together with a rich ecosystem of services and certifications.

PARTICIPATE

INTEGRATE

STABILIZE

1M+ PROJECTS (upstream projects)

(community platforms)

(supported products platforms, and solutions)

®

Close cooperation with extended ecosystem by massive testing, Building of ref architectures and agreements with vendors to be able to provide full support

PROPRIETARY HARDWARE

SCALE-UP ARCHITECTURE

HARDWARE-BASED INTELLIGENCE

CLOSED DEVELOPMENT PROCESS

Common, off-the-shelf hardware

Scale-out architecture

Software-based intelligence

Open development process

Lower cost, standardized supply chain

Increased operational flexibility

More programmability, agility, and control

More flexible, well-integrated technology

FOUR IMPORTANT DIFFERENCES

NO Vendor lock in – in any way: HW – choose best bang for the buck Technology – free to choose any tech supported by Linux Software – Even Red Hat is possible to leave – annual subs!

Cruical part of Red Hat Strategy Not something we may or may not invest in!

PHYSICAL VIRTUAL PRIVATE CLOUD CONTAINERS PUBLIC CLOUD

Gluster – your next Scale Out NAS

GLUSTER ARCHITECTURE Distributed scale out share nothing storage using industry standard hardware

NFS CIFS FUSE

SERVERS WITH LOCAL DISKS

Aggregates systems to one cohesive unit and presents using common protocols

WHERE IS GLUSTER DEPLOYED?

Red Hat Gluster Storage

PHYSICAL VIRTUAL CONTAINERS CLOUD

Gluster: Open Shift and containers

Offers persistent storage to applications running in containers

Applications and storage can co-exist on the same hardware

Allows for higher server utilization and lowers operational costs

Storage generates only 3%–10% overhead on converged servers

If you evaluate OSE: make sure you evaluate Gluster at the same time!

Container JBoss NGINX

NGINX Container OpenShift

Master

OpenShift Node 2

Container Elastic Spark

MongoDB Container

OpenShift Node 3

Gluster shared volumes

Redis

PostgreSQL

Container

OpenShift Node 1

OpenShift Node 4

Distributed-Replicated Volume

FAULT-TOLERANT DATA PLACEMENT

MOUNT POINT

Creates an FT distributed volume by mirroring same file across 2 bricks

FILE 1

server1

DISTRIBUTED-REPLICATED VOLUME

Replicated Vol 0 Replicated Vol 1

BRICK (exp 1)

FILE 1

server2

BRICK (exp 2)

FILE 2

server3

BRICK (exp 3)

FILE 2

server4

BRICK (exp 4)

GEO REPLICATION V.S. REPLICATED VOLUMES

Mirrors data across geographically distributed trusted storage pools. Provides high-availability.

Mirrors data across bricks within one trusted storage pool.

Backups of data for disaster recovery. Provides high-availability.

Asynchronous replication: checks for changes in files periodically. Syncs them on detecting differences.

Synchronous replication: each and every file operation is applied to all the bricks

Potential of data loss: minutes/hours Potential of data loss: none

Geo-Replication Replicated Volumes

ERASURE CODING Storing more data with less hardware

o  RECONSTRUCT corrupted or lost data

o  ELIMINATES the need for RAID

o  CONSUMES FAR LESS SPACE than replication

o  APPROPRIATE for capacity-optimized use cases.

How efficent? you ask. 10 servers with 10 drives x 2TB – 1 drive breaks

=3 minutes rebuild of protection.

4TB=6 minutes. 8TB=? Whole node?

2 3 4 x 1 y

FILE

ERASURE CODED VOLUME

STORAGE CLUSTER

TIERING Cost-effective flash acceleration

AUTOMATED promotion and demotion of data between “hot” and “cold” sub volumes

BASED on frequency of access.

FILE

STORAGE CLUSTER

COLD SUBVOLUME (SLOWER SPINNING DISK)

HOT SUBVOLUME (FASTER FLASH)

DEPLOYMENT IN AWS/AZURE/GCP

Build Gluster volumes across AWS Availability Zones

Red Hat Gluster Storage Amazon Machine Images (AMIs)

High availability + Multiple EBS devices pooled

No application rewrites required

Scale-out performance and capacity availability as needed

Azure & GCP is also supported along with AWS

GLUSTER FS Duplicate-Replicate Volume (Primary)

GLUSTER FS Duplicate-Replicate Volume (Primary)

Primary Datacenter Secondary Datacenter

Geo-Replication

RHSS RHGS

RHSS RHGS

EBS VOLUMES

Replicated bricks

EBS VOLUMES

EBS VOLUMES

Replicated bricks

EBS VOLUMES

BIT ROT DETECTING Detection of silent data corruption

RED HAT GLUSTER STORAGE 3.1 provides a mechanism to scan data periodically and detect bit-rot.

CHECKSUMS are computed when files are accessed and compared against previously stored values.

IF THEY DO NOT MATCH an error is logged for the storage admin.

0 0 0 0 0 X 0 0

ADMIN

!!

Ceph – your next Object storage beast

CEPH ARCHITECTURE Distributed scale out shared nothing storage using industry standard hardware

RBD S3 FUSE

SERVERS WITH LOCAL DISKS

Aggregates systems to one cohesive unit and presents using common protocols

NFS iSCSI

TECHNICAL BENEFITS OF CEPH ●  Massive scalability, to

support petabytes of data

●  Maximum uptime, with no single point of failure

●  Reduced

maintenance, via self-management and self-healing

●  Operational efficiency,

by distributing data dynamically among servers and disks

COMPLETE STORAGE FOR OPENSTACK

●  Deeply integrated with modular architecture and components for ephemeral & persistent storage o  Nova, Cinder, Manila, Glance, Keystone, Swift

●  Optimized for Database as a Service with MySQL workloads

●  Bundled in Red Hat OpenStack Platform – 64TB included in first installation

(a.k.a. First fix is free)

ADVANTAGES FOR OPENSTACK USERS ●  Instantaneous booting of 1 or 100s of VMs ●  Instant backups via seamless data migration between Glance, Cinder, Nova ●  Tiered I/O performance within single cluster ●  Multi-site replication for disaster recovery or archiving

We´re doing Constant improvements to RGW, Ceph’s object storage interface: •  Support for authentication using Active

Directory, LDAP & OpenStack Keystone v3 •  Greater compatibility with the Amazon S3

and OpenStack Swift object storage APIs •  AWS v4 signatures, object versioning, bulk deletes

•  New NFS gateway for bulk import and export of object data (tech preview)

OBJECT STORAGE IMPROVEMENTS

Global object storage clusters with single namespace •  Enables deployment of clusters across

multiple geographic locations in an asynch but still active/active mode

•  Clusters synchronize, allowing users to read from or write to the closest one

Multi-site replication for block devices •  Replicates virtual block devices across

regions for disaster recovery and archival

NEW MULTISITE CAPABILITIES

STORAGE CLUSTER US-EAST

STORAGE CLUSTER US-WEST

PERFORMANCE BOOST WITH BLUESTORE

•  New storage backend optimized for modern media - Tech preview in RHCS 2

•  Replaces FileStore, optimized for HDDs – file system for object data management impacted performance

•  Benefits of BlueStore: •  2-3X performance •  Mitigates need for underlying file system or dedicated journal device •  Flexible media topologies (SSDs, K/V drives, persistent memory

devices) to help optimize data or metadata I/O •  Native checksumming to detect data corruption on media

OPENSTACK FILE SUPPORT WITH CephFS •  POSIX-compliant file system to store data in accordance

with OpenStack Manila Project •  Tech Preview RHCS 2 – Manila Tech Preview OSP 7, 8 •  FUSE or kernel clients (RHEL 7.2) •  CephFS Manila driver (tech preview) with OSP 9 •  Roadmap: robustness, hardening, then scalability

November 2015: Manila Tech

Preview in OSP 7

July 2016: CephFS tech preview in RHCS 2 and clients

August 2016: CephFS Manila

driver tech preview in OSP 9

November 2016: Manila production support in OSP 10

2017: CephFS production

support RHCS 2.x

INTEGRATED MANAGEMENT WITH RED HAT STORAGE CONSOLE 2

•  Easy to use graphical interface to manage storage cluster life cycle

•  Ansible-based deployment tools for installation, importation, and granular configuration from CLI or GUI

•  Monitoring and troubleshooting with statistical and graphical information about cluster components

RADOS. RGW, RBD, CALAMARI

RED HAT STORAGE CONSOLE 2

CLUSTER OF STANDARD SERVERS AND MEDIA

CLIENTS ADMINS

Is the future GUI for Gluster + common API connection point to storage

iSCSI and Ceph

•  Planned for Tech Preview in Ceph 2.1 (planned for November) •  HA implementation via iSCSI GW (same method as for object) •  Initial support for MS plus Linux •  Future support VMware, incl API´s like VASA and VAAI during

2017 •  In community – test it, you like it!

Red Hat gives Support Stable code Lifecycles Upgrades Documentation Training

Why Red Hat and not Co****ity? Community gives Support? Google it Alfa and Beta level code Lifecycles? What lifecycles? Upgrades? What upgrades? Documentation? Training?

= Production environments = Test environments

Conclusion

With Red Hat Storage Gluster and Ceph you get No Vendor lock in whatsoever Modern scale out architecture with super resilience Storage that integrates where you want it “Future proven” technology that will enable you to handle the storage explosion

What’s not to like?

You need Ceph.... or Gluster

You wanna test? May we suggest a RED HAT STORAGE “Jumpstart ”?

o  Startup meeting – goals and success criterias, hw req/design

o  Three days consulting for pre-production cluster implementation delivered remotely or at customer site (travel and expenses n.i)

o  Six months pre-production cluster support plus possibility to talk to SSA + Consultant who helped you.

o  Product knowledge transfer to necessary staff

o  Helps ensure Ceph or Gluster configured and deployed for desired workloads

Will be your best POC/Pilot ever! After 6 months (or shorter) you are ready to go! All this starting at the equivalent of 1000 pizzas (200 in Norway) =10 000€

bit.ly/cephtestdrive bit.ly/glustertestdrive

WANNA TEST NOW? TEST DRIVES!