walter graf ceph appliance: eternus cd10000 s2 · pdf fileceph appliance: eternus cd10000 s2...
TRANSCRIPT
0 © 2015 Fujitsu INTERNAL USE ONLY
Ceph Appliance: ETERNUS CD10000 S2
Walter Graf
1 © 2015 Fujitsu INTERNAL USE ONLY
Summary Upfront
New Storage Application Areas
How to address with Ceph
Fully integrated and Easy To Use ETERNUS CD10000 S2
2 © 2015 Fujitsu INTERNAL USE ONLY
New Storage Application Areas
Cloud Infrastructures with OpenStack
Unstructured Data Repositories using Amazon S3 / OpenStack
Swift
3 © 2015 Fujitsu INTERNAL USE ONLY
The Infrastructure Challenge
Agile Development Lengthy Deployment
4 © 2015 Fujitsu INTERNAL USE ONLY
The “DevOps” Way
Deployment Automation
Infrastructure as Code
Speeds up Deployment
Makes Q/A possible
5 © 2015 Fujitsu INTERNAL USE ONLY
ETERNUS CD10000 OpenStack use case
„DBCE“ Cloud Marketplace
„DARZ“ OpenStack Cloud
ETERNUS CD10000 Cloud Storage
Time To Market Quicker Booking & Billing
Agility Automation Flexibility
Cost Reduction
Cinder
Nova
Neutron
Competitive Advantage
6 © 2015 Fujitsu INTERNAL USE ONLY
Customer / Storage Profile
Service Provider
Large Private Cloud Operators
Own Software Development
Web Frontend
Open Source Minded
Following Cloud Programming Paradigms
Self Managing & Self Healing Storage
Immortal System
7 © 2015 Fujitsu INTERNAL USE ONLY
The scale of unstructured data is mind-boggling
Data growth 18% p.a. structured data
41% p.a. unstructured data
About 85% is unstructured data
8 © 2015 Fujitsu INTERNAL USE ONLY
The “Unstructured Data” Opportunity
Big Data repositories Internet of Things
Car Sensor Data
…
Sync & Share „Enterprise Dropbox“
ownCloud, Seafile, ctera, Data Space
…
Create competitive advantages
9 © 2015 Fujitsu INTERNAL USE ONLY
Object Storage Architecture
Ceph Object Store (ETERNUS CD10000)
RESTful API [S3,Swift]
Ceph Object API
iRODS, Seafile, …
Rados Gateway
Ceph Object API
Owncloud, iRODS, …
10 © 2015 Fujitsu INTERNAL USE ONLY
University of Mainz
Johannes Gutenberg University of Mainz
~ 36,500 students; > 500 professors
> 150+ departments; > 5,500 employees; multiple institutes and clinics
Central IT department - ZDV
IT services and HPC center
11 © 2015 Fujitsu INTERNAL USE ONLY
Customer / Storage Profile
Large repositories > 100 TB
Seamless scalability up to multi PB range
Cost efficient storage for less frequently accessed data
Easy accessible (no tape)
Immortal system
No vendor lockin
Viewing object storage as superior over file system storage
12 © 2015 Fujitsu INTERNAL USE ONLY
Ceph
Scalable
Disaster Resilient
Self-managing
Self-Healing
13 © 2015 Fujitsu INTERNAL USE ONLY
OpenStack.org User Survey May 2015
14 © 2015 Fujitsu INTERNAL USE ONLY
Key Components
Calculate instead of lookup
Use objects instead of blocks or files
Automatically place, replicate, balance, and migrate data
15 © 2015 Fujitsu INTERNAL USE ONLY
True Scalability, Disaster Resilience, Seamless Lifecycle
Access network
Controller 1 Controller 2
Limited Scalability
DR only with multiple systems and
additional software
Major migration efforts at EOSL
Unlimited Scalability
Built in DR
Seamless migration to new nodes, no EOSL
Node 1 Node 2 Node 3 Node 4
…
RAID Ceph
16 © 2015 Fujitsu INTERNAL USE ONLY
Ceph Software Architecture
Cluster Monitors <10
cluster membership
authentication
cluster state
cluster map
Topology
Authentication
Meta Data Server* (MDS) 10s
for POSIX only
Namespace mgmt.
Metadata ops (open, stat, rename, …)
POSIX
meta data only
Object Storage Daemons (OSD) 10,000s
stores all data / metadata
organises all data in flexibly sized containers
Clients
bulk data traffic
block file object
*) still experimental
17 © 2015 Fujitsu INTERNAL USE ONLY
From Object to OSD
Object
Placement Group #b
OSD#i
(primary copy)
CRUSH
ruleset
OSD#k
(2nd copy)
OSD#l
(3rd copy)
func(pool, obj_name)
Placement Group #a Placement Group #c
18 © 2015 Fujitsu INTERNAL USE ONLY
Crush Ruleset Example
1. In fire section 1
i. Using CRUSH select host
ii. Using CRUSH select & remember 1 osd on this host
2. In fire section 2
i. Using CRUSH select 1 host
ii. Using CRUSH select & remember 1 osd on this host
3. In fire section 3
i. Using CRUSH select 1 host
ii. Using CRUSH select & remember 1 osd on this host
3 replicas in 3 fire sections
19 © 2015 Fujitsu INTERNAL USE ONLY
Recover
21 © 2015 Fujitsu INTERNAL USE ONLY
Why ETERNUS CD10000?
Enterprise Class Architecture – ETERNUS CD10000 S2
End to End Software plus Hardware Integration & Service
Lifecycle Management
Overcome Downsides of SDS
22 © 2015 Fujitsu INTERNAL USE ONLY
ETERNUS CD10000 > 50 PB
> 200 nodes
23 © 2015 Fujitsu INTERNAL USE ONLY
ETERNUS CD10000 Architecture
CD10k Backend Network
Mgmt Node
Customer Network
Data
Mgmt
Replica, Recovery, Balancing, Migration
Gateway Node
Gateway Node
…
CD10k Mgmt Network
raw capacity > 50 PB
Storage Node
… Storage
Node Storage
Node Storage
Node
Monitor Node
optional component
24 © 2015 Fujitsu INTERNAL USE ONLY
Storage Node
44 TB
ETERNUS CD10000 – Flex Nodes
PRIMERGY
2540-M1
(44 TB)
PCIe
SSD
optional
Storage Node
92 TB
PRIMERGY
2540-M1
(44 TB)
ETERNUS JX40 S2
(48 TB)
PCIe
SSD
Storage Node
140 TB
PRIMERGY
2540-M1
(44 TB)
ETERNUS JX40 S2
(48 TB)
ETERNUS JX40 S2
(48 TB)
PCIe
SSD
Storage Node
188 TB
PRIMERGY
2540-M1
(44 TB)
ETERNUS JX40 S2
(48 TB)
ETERNUS JX40 S2
(48 TB)
ETERNUS JX40 S2
(48 TB)
PCIe
SSD
25 © 2015 Fujitsu INTERNAL USE ONLY
Density Node
Advanced 1U integrated storage node
Based on Intel® Atom™ Processor, which reduces total power
consumption by 50% with no reduction in performance
Equipped with 12 drives, one node provides 60TB capacity
Local SSD Cache option
Innovative drive mounting design eliminates rotational vibration
Ideally suited for use in Erasure Coded pools
26 © 2015 Fujitsu INTERNAL USE ONLY
ETERNUS CD10000 S2 – Node optimizations
PRIMERGY 2540-M1 (44 TB)
ETERNUS JX40 S2 (48 TB)
ETERNUS JX40 S2 (48 TB)
ETERNUS JX40 S2 (48 TB)
PCIe SSD
2.5“ 900 GB 3.5“ 4 TB disks
Modular Capacity Extensions
PCIe SSD becoming optional
PRIMERGY 2530-M1
PRIMERGY 2530-M1
PRIMERGY 2530-M1
1U Mgmt Node
Optional Monitor Node
Optional RADOS GW
27 © 2015 Fujitsu INTERNAL USE ONLY
Efficiency gains with ETERNUS CD10000 S2
Up to factor 6 less cost per GB with Flex Nodes
Up to 75% reduction in capacity investment for data redundancy with Erasure Coding
Up to factor 10 less costs per GB with Density Nodes
Up to factor 10 higher storage density with Density Nodes
Compared to ETERNUS CD10000 S1
Up to 50% less power consumption with Density Nodes
29 © 2015 Fujitsu INTERNAL USE ONLY
How To Connect Storage Nodes
10 GbE Switch
10 GbE Switch
10 GbE Switch
8 x 10 GbE
Storage Node
Storage Node
1 x 10 GbE
1 x 10 GbE
2 x 10 GbE Storage Node connection
in a non-redundant configuration
Storage Node connection in a redundant configuration
30 © 2015 Fujitsu INTERNAL USE ONLY
Switch Cascading
8 x 10 GbE 8 x 10 GbE
2 x 40 GbE
2 x 40 GbE
2 x 40 GbE
2 x 40 GbE
2 x 40 GbE
2 x 40 GbE
10 GbE Switch
10 GbE Switch
10 GbE Switch
10 GbE Switch
10 GbE Switch 10 GbE Switch …
…
…
Uplink connection in a non-redundant configuration
Uplink connection in a redundant configuration
31 © 2015 Fujitsu INTERNAL USE ONLY
How To Connect Zones (1)
8 x 10 GbE (for redundancy) 8 x 10 GbE
10 GbE Switch
10 GbE Switch
10 GbE Switch
10 GbE Switch
QSFP+
QSFP+
QSFP+
QSFP+
QSFP+
QSFP+
QSFP+
QSFP+
long distance optical cable*
*) depending on cable length MMF or SMF cables / SFPs / QSFPs are needed required cables, SFPs, QSFPs not part of ETERNUS CD10000 portfolio
32 © 2015 Fujitsu INTERNAL USE ONLY
How To Connect Zones (2)
8 x 10 GbE (for redundancy) 8 x 10 GbE
10 GbE Switch
10 GbE Switch
10 GbE Switch
10 GbE Switch
QSFP+
QSFP+
QSFP+
QSFP+
QSFP+
QSFP+
QSFP+
QSFP+
optical cable**
**) required cables, SFPs, QSFPs not part of ETERNUS CD10000 portfolio
optical cable** Customer Network*
Customer Network* optical cable** optical cable**
*) required components are project specific and are not part of ETERNUS CD10000 portfolio
34 © 2015 Fujitsu INTERNAL USE ONLY
Ceph is Open Source …
Stay compatible
Contribute instead of fork
Based on Ceph Enterprise Releases
Offer solution maintenance instead of separate contracts for
hardware and software
All backed by Red Hat
35 © 2015 Fujitsu INTERNAL USE ONLY
Immortal System Concept
Normal Storage
ETERNUS CD10000
storage nodes
other components
initial configuration
initial purchase time normal EOSL
storage upgrade
storage upgrade
new storage nodes
new storage nodes
replacement of old storage nodes and other components
prolonged EOSL prolonged EOSL prolonged EOSL
36 © 2015 Fujitsu INTERNAL USE ONLY
Ceph Challenge: Installation
Manual Ceph Installation (Plain Ceph)
Setting-up a Ceph cluster with 240+ OSDs
• 1.5 – 2 admin days
Adding an additional node
• from 3 admin hours up to half a day
Automated installation (CD10000)
Setting-up a Ceph cluster with 240+ OSDs
• 1 hour
Adding an additional node
• 0.5 hour
37 © 2015 Fujitsu INTERNAL USE ONLY
Ceph Admin Challenge: Replacing an HDD
Plain Ceph
taking the failed disk offline in Ceph
taking the failed disk offline on OS /
Controller Level
identify (right) hard drive in server
exchanging hard drive
partitioning hard drive on OS level
Make and mount file system
bring the disk up in Ceph again
On ETERNUS CD10000
vsm_cli <cluster> replace-disk-out
<node> <dev>
exchange hard drive
vsm_cli <cluster> replace-disk-in
<node> <dev>
38 © 2015 Fujitsu INTERNAL USE ONLY
Ceph Admin Challenge: Adding a Node
Plain Ceph
Install hardware
Install OS
Configure OS
Configure network
Configure S/W repositories
Configure ntp
Configure sudo
Configure ssh
Partition disks (OSDs, Journals)
Make filesystems
Deploy monitor (optional)
Deploy OSDs
On ETERNUS CD10000
Install hardware
• hardware will automatically PXE boot
and install the current cluster
environment including current
configuration
Make node available to GUI
Add node to cluster with mouse click
on GUI
39 © 2015 Fujitsu INTERNAL USE ONLY
ETERNUS CD10000 = Ceph++
+ Hardware / Software Integration
E2E Solution Contract
Automated Deployment
Cloud Storage
+
+
+ Immortal System
Automated Management +
40 © 2015 Fujitsu INTERNAL USE ONLY
ETERNUS CD10000 …
Addresses new storage application areas
Avoids Vendor Lockin
Addresses Key Storage Requirements
scalability, DR, automation, immortal system, seamless migration
Offers optimised price capacity ratio
Stay tuned for further information: http://partners.ts.fujitsu.com
Corporate Partner Portal > Portfolio catalog > Products > Storage solutions > Hyperscale Storage
WP „RAID: End of Era?“, Presentations, Product Insights, Sizing Tool
On Youtube: Search for „ETERNUS CD10000“
41 © 2015 Fujitsu INTERNAL USE ONLY
42 © 2015 Fujitsu INTERNAL USE ONLY
ETERNUS CD10000 Backup
Rapid Infrastructure Provisioning with OpenStack
43 © 2015 Fujitsu INTERNAL USE ONLY
The backup principle - still relevant?
Does the 3-2-1 backup principle still rule?
Have at least 3 copies of your data
Keep these backups on 2 different media
Store 1 backup offsite
But, what about, amount of data, bandwith, backup window, cost?
44 © 2015 Fujitsu INTERNAL USE ONLY
Data protection strategies for petabyte storage
The traditional way
Dominated by explicit requirements
Legacy apps processes
Insists on 3-2-1 principle
New concepts are demanded!
Leverage Ceph functions
Backup-in-the-box
New cloud way
Backup only what you need
Move data as less as possible
Ceph provides
• HW redundancy across sites
• Logical level protection based on Snapshots
OpenStack / Sync & Share make use of the Ceph functions
What we see: Two camps of customers/ mindsets
45 © 2015 Fujitsu INTERNAL USE ONLY
New: backup-in-the-box
Backup-in-the-box
Pool based backup
Full, incremental, synthetic full
Pool or object-based recovery
To the original pool or to another
CD10000 storage nodes
Ceph Cluster
Pool A Backup
Pool Pool B