leveraging swift storage policies using scality ring
TRANSCRIPT
Leveraging Swift Storage Policies
Nicolas TrangezTechnical Architect [email protected]@eikke / @scality
Copyright Scality 2015
About Scality Ring
● Petabyte-scale software-defined object & file storage, fully scale-out
● Hardware-agnostic
● Core: Scality “Ring”
– Replication
– Erasure Coding (ARC)
– Interaction with “Ring” through connectors:
● Plain HTTP (sproxyd)
● S3, CDMI
● FUSE, NFS, CIFS/SMB
● Application-specific (e-mail servers,...)
Copyright Scality 2015
Copyright Scality 2015
Scality & OpenStack● Dedicated engineering team
● Integrating with OpenStack Storage projects
– Cinder (data volumes), in-tree
– Glance (VM images), currently out-of-tree
– Swift (object storage), out-of-tree
– 2015 H2: Manila (shared filesystem)
● Single system to manage, unify OpenStack storage requirements
– Eliminates multiple storage silos
– Consolidates 80% of OpenStack data storage
– Scalable to petabytes and beyond
● Team encouraged to work on any OpenStack project(s) as part of job
Copyright Scality 2014
Copyright Scality 2015
Scality Ring Scality Sproxyd
Swift AccountsService + Storage
Swift ContainersService + Storage
CustomerApplication
Scality Sproxyd
Swift Object Service
swift-scality-backend
Swift Object Service
swift-scality-backend
Swift Object Service
swift-scality-backend
Swift Proxy ServiceSwift Proxy ServiceSwift Proxy Service
scality-sproxyd-client
Copyright Scality 2015
Swift Storage Policies
● Allow admin to define policies/classes of storage
– With different performance, pricing models, SLA's,...
● Allow tenant to assign a policy to a container (only at creation time)
● Objects in containers stored according to container policy
– Swift upstream: replication count, SSD/spindle, EC (beta in Kilo)
– Scality: map to sets of Sproxyd connectors in different configurations
Copyright Scality 2015
Use Case #1: Performance vs. Cost
● Replication vs. Erasure Coding
– Storage consumption: *N vs *((n + k)/ n)
– Low time-to-first-byte vs. higher latency
– Efficient random access vs. mostly-streaming
● Spindles vs. SSD
● Depending on application, transfer data between containers/policies over time
– Keep costs under control
Copyright Scality 2015
Use Case #2: Geo-Distributed Storage
● Geo-distributed datacenters add network latency impact
● Keep operations as local as possible
– Configuration of SPs takes location of endpoints into account
– Sort on distance
● Regulations impose locality constraints
– E.g. privacy laws
– Storage Policies used to ensure data placement complies
Copyright Scality 2015
Use Case #2A: Multi-datacenter, Single Cluster
● Active/Active, managed as a single Ring
● Location-aware allocation: both Replication and ARC/EC can spread data across disparate domains for site failure tolerance
● Let Swift object servers talk to 'closeby' connectors
– Reduce number of WAN latency hits
Copyright Scality 2015
Use Case #2B: Multi-datacenter, Multiple Clusters
● Active/Passive, source Ring asynchronously synced to target
– Site-level Disaster Recovery (within time window) without latency hit
– Ensure data is replicated to 'hot' access points
– Works really well for immutable objects
– E.g. movie production teams around the world, working locally, data synced to LA HQ continuously (or at night, or ...)
● Ensure Swift talks to 'source' Rings for CUD operations, local Rings for R operations (taking stale reads into account, fallback to source Ring on HTTP 404)
Copyright Scality 2015
Implementation
● Re-use existing Scality Swift back-end functionality
– HTTP connection pooling
– Detect sproxyd outage & flapping using φ-accrual failure detector
● Keep config in 1 file, shared across all servers
– Ease of deployment
● Optional per-server location preference hints
[ring:paris-rep3]location = parissproxyd_endpoints = http://paris1.int/rep3, http://paris2.int/rep3
[ring:paris-arc6+3]location = parissproxyd_endpoints = http://paris1.int/arc6+3, http://paris2.int/arc6+3
[ring:sfo-arc6+3]location = sfosproxyd_endpoints = http://sfo1.int/arc6+3
[ring:nyc-arc6+3]location = nycsproxyd_endpoints = http://nyc1.int/arc6+3
[storage-policy:1]read = sfo-arc6+3, paris-arc6+3write = nyc-arc6+3
[storage-policy:2]read = nyc-arc6+3write = paris-rep3
[storage-policy:3]write = paris-rep3
Copyright Scality 201413
Scality, a disruptive software company
Founded 2009
70+ production deployments including Comcast, Dailymotion, RTL
San Francisco (HQ), Washington DC, Boston (R&D), Paris (R&D), Tokyo, Singapore
135 employees, increasing at high rate
300% sales growth in 2014
Join us, we’re hiring !! (www.scality.com/careers)
Questions & Answers
Visit Scality booth #T13
www.scality.com