emc world 2016 - code.06 delivering ecs object services to cloud foundry applications

1© Copyright 2016 EMC Corporation. All rights reserved. 1© Copyright 2016 EMC Corporation. All rights reserved.

DELIVERING OBJECT SERVICES TO CLOUD FOUNDRY APPLICATIONSTRAVIS RHODEN – DEVELOPER ADVOCATE, EMC {code}

AARON SPIEGEL – ADVISORY SYSTEMS ENGINEER


❤️

Elastic Cloud Storage for cloud

4© Copyright 2016 EMC Corporation. All rights reserved.

EMC ELASTIC CLOUD STORAGE

CLOUD FOUNDRY SERVICE BROKER (PART I)

SPRING CLOUD CONNECTOR INTEGRATION


• Connect Java applications to cloud services

• Support Cloud Foundry, Heroku & more

• Protect Developers from technology lock-in

• Two libraries are available supporting AWS S3/IAM SDKs directly, and jclouds blobstore SDK

SPRING CLOUD CONNECTORS


❤️

Elastic Cloud Storage for


EMC ELASTIC CLOUD STORAGE

CLOUD FOUNDRY SERVICE BROKER (PART II)

DEPLOYMENT & REGISTRATION


• Create, Modify and Delete Object Storage Buckets

• Bind one or more CF apps to buckets, with unique credentials for each app

• Customize CF Marketplace bucket plans

• Customize bucket ACLs for applications

• Upgrade or downgrade bucket plans

• Specify an ECS namespace and replication group

• Customize services & plans via YAML configuration

THE ECS CLOUD FOUNDRY SERVICE BROKER

https://github.com/emccode/ecs-cf-service-broker

https://github.com/emccode/ecs-cf-service-broker


A COMPLETE PRIVATE OR HOSTED CLOUD SOLUTION

Cloud Foundry Architecture Stack

CF App Blobstore

User Application

BOSH Blobstore

Object/HadoopData


◈ Declarative Formats

◈ Clean Contract

◈ Suitable for Cloud Platforms

◈Minimize divergence

◈ Scalable

http://12factor.net/


http://12factor.net/backing-services

IV. Backing servicesTreat backing services as attached resources


IV. Backing servicesTreat backing services as attached resources

◈ Object doesn’t require host operating system configuration Contrary to NAS mounts, which introduce snow-flakes HTTP stateless sharable access with unique credentials

◈ Simplified object storage semantics No need to manipulate directory structures Single boundless namespace Less development overhead for capacity, permission and lock checks

◈ HTTP accessible assets Direct content access to clients: binary & static content Removes application server from data flow


CLOUD FOUNDRY SERVICE BROKER API

• Natural user-experience for

backing services

• Catalog & marketplace for

both private & public clouds

• Supports all kind of services:

— Free/Paid— Database— Message queues— Caches— Binary stores

http://docs.cloudfoundry.org/services/api.html


ECS - CLOUD SCALE OBJECT STORAGE PLATFORMOBJECT STORAGE SERVICES ON COMMODITY HARDWARE

ECS Software HDFS

STORAGE

OBJECTSTORAGE

CommodityPlatforms

FILESTORAGE

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwi6mcyM2cLJAhWCGz4KHYB4AaQQjRwIBw&url=http://www.amazon.co.uk/Ability-ASI-BeAnywhere-Remote-Access/dp/B001GQ2BWC&psig=AFQjCNF3aWCbxik3d86Ybag8g2fwALB7ig&ust=1449335306113233

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwi6mcyM2cLJAhWCGz4KHYB4AaQQjRwIBw&url=http://www.amazon.co.uk/Ability-ASI-BeAnywhere-Remote-Access/dp/B001GQ2BWC&psig=AFQjCNF3aWCbxik3d86Ybag8g2fwALB7ig&ust=1449335306113233


COMPREHENSIVE MULTI-PROTOCOL DATA ACCESS

• Simultaneous access to underlying data through multiple interfaces (Protocols)

– Object, HDFS, File

• HDFS compatible with Cloudera, Hortonworks, Pivotal etc.

• Support for S3, Swift, Atmos and Centera CAS APIs object

• NFS V3 FILE Support

• Extensions to APIs– Byte-Range updates, Atomic appends, Rich ACLs,

MetaData Search, etc.

ATMOS


ECS ELASTIC SCALABILITY

Limitless Scale: Each layer is independently scalable, highly available, and has no single point of failure.

Scale-Out Architecture: Scale by adding more nodes, no special nodes or roles

Global Namespace: Any node has full system view of data and meta-data

Persistence Layer

Storage Engine

JBODs

OBJECT

HDFSOBJECT

HDFSOBJECT

HDFS

Global Namespace


GEO-REPLICATED DATA PROTECTION

• Handles local hardware and full data center failures– Disk, Node, Rack, Data Center are

failure domains

– Component failures are recovered using fragments from local site without WAN traffic

• Local hardware failure recovery requires no WAN traffic

• New Hybrid encoding approach enables low storage overhead

Site 1

Site 3

Site 2


• Bucket level metering – Reporting on capacity, object count,

bandwidth in/out, objects created and deleted

• Object capacity quota management– Set quota limits on bucket or

namespace

• Deep dive monitoring and auditing – Displayed in GUI and REST API

EASY TO MANAGE MULTI-TENANT ARCHITECTURE


Data Lake Foundation

• Active archive analytics with HDFS support

• Unstructured data repository

• Multi-protocol access

Modern Application Platform

• Multiple APIs(S3, Swift, Atmos, HDFS)

• New application models

• Mobile app platforms

• Active-active anywhere access to geo-replicated content

• Simple App Dev with Strong Consistency

• Faster time to value with data-in-place analytics

• Global analytics on geo-distributed data and archives

L.A.

Memphis

UK

SUPPORT MODERN APPLICATIONS & ANALYTICS SIMPLY & EASILY


SUMMARY


@EMCcode@spiegela@codenrhoden

emccode.comcommunity.emccode.com

Come visit us at Booth #1044 or in the vLab

Questions?

emc world 2016 - code.06 delivering ecs object services to cloud foundry applications

Technology