©2015,  Amazon  Web  Services,  Inc.  or  its  affiliates.  All  rights  reserved

Intro to AWS: Storage Services Matt McClean, AWS Solutions Architect

AWS storage options

•  Scalable object storage •  Inexpensive archive storage •  Persistent block storage •  Shared file system •  Turnkey gateway solution

We are constantly producing more data

From all types of industries

Object Storage

Amazon S3 Simple Storage Service

2012 2013 2014 2015

95% year-over-year increase in data transfer to and from S3 (Q4 2014 vs Q4 2013, not including Amazon use)

S3 usage

99.999999999% durability

Reduced redundancy option 99.99% saves ~20%

S3 resources: buckets and objects

S3 website: static content

1 PB raw storage

800 TB usable storage

600 TB allocated storage

400 TB application data

S3 capacity pricing—pay only for what you use!

Amazon S3

S3 continuous cost reduction •  Available through 11 regions globally •  Priced at per GB-month rates •  8 price reductions since launch •  51% average S3 capacity fee

reduction on 4/1/2014

•  TCO: comparing on-premises to S3 –  Can be challenging for some

customers –  We can help!

S3

Amazon EMR

Amazon S3—New Features

S3 event notifications Delivers notifications to Amazon SNS, Amazon SQS, or AWS Lambda when events occur in S3

S3

Events

SNS topic

SQS queue

Lambda function

Notifications

Notifications

Notifications

Foo() { … }

•  Preserve, retrieve, and restore every version of every object stored in your bucket

•  S3 automatically adds new versions and preserves deleted objects with delete markers unless an explicit versioned DELETE operation is made

•  Easily control the number of versions kept by using lifecycle expiration policies

•  Easy to turn on in the AWS Management Console

S3 versioning

S3 cross-region replication Automated, fast, and reliable asynchronous replication of data across AWS regions

Source (Ireland)

•  Only replicates new PUTs. Once S3 is configured, all new uploads into a source bucket will be replicated

•  Entire bucket or prefix based

•  1:1 replication between any 2 regions

•  Versioning required

Use cases: •  Compliance—store data hundreds of miles apart •  Lower latency—distribute data to regional customers •  Security—create remote replicas managed by separate AWS accounts

Destination (Frankfurt)

S3 use cases

•  Web-scale storage capacity and performance for web applications

•  Single-origin store with delivery through Amazon CloudFront

•  Staging and persistent store for Big Data applications

•  Storage target for backup and active archive

Amazon Glacier Low-cost archiving service

$0.01 per GB-month

$120 per TB/year

99.999999999% durability

3–5 hours data retrieval

Storage costs vs.

retrieval costs

Amazon Glacier benefits

•  Reduce cost for long-term archiving •  Leverage unlimited storage capacity •  Replace tape museums •  Improve durability

Amazon S3–Amazon Glacier integration Policy-based archiving service

S3 Lifecycle Policies

→•  Key prefix “logs/”

•  Transitions objects to Glacier 30 days after creation date

•  Delete 365 days after creation date

<LifecycleConfiguration> <Rule>

<ID>archive-in-30-days</ID <Prefix>logs/</Prefix>

<Status></Status> <Transition>

<Days>30</Days>

<StorageClass>GLACIER</StorageClass> </Transition>

<Expiration>

<Days>365</Days> </Expiration>

</Rule> </LifecycleConfiguration>

SoundCloud—leveraging Amazon Glacier for audio transcoding •  World’s leading social sound platform •  Audio files must be transcoded and stored

in multiple formats

S3 Amazon Glacier

Use cases for S3 lifecycle policies

•  Data tiering in the cloud

•  Manage object versioning for logical data protection

•  Deletion of data from Amazon Glacier by policy

Block Storage

Instance storage

Amazon EBS Elastic Block Storage

1 GB 16 TB

Ú

EBS

General Purpose (SSD)

Up to 16 TB

10,000 IOPS

Up to 160 Mbps

Provisioned IOPS (SSD)

Up to 16 TB

20,000 IOPS

Up to 320 Mbps

Price Performance EBS

Magnetic General Purpose Provisioned IOPS

Use cases Infrequent data access

Boot volumes Small to med DBs

Dev and Test

I/O intensive Relational DBs

NoSQL DBs

Storage media Magnetic disk-backed SSD-backed SSD-backed

Max IOPS 40–200 IOPS 10,000 IOPS 20,000 IOPS Latency (random read) 20–40 ms 1–2 ms 1–2 ms

Availability Designed for 99.999% Designed for 99.999% Designed for 99.999%

Price $.05/GB-month $.05/million I/O $.10/GB-month $.125/GB-month

$.065/provisioned IOPS

Amazon EBS snapshots

Amazon EBS use cases

•  Persistent block storage for Amazon EC2 •  Transactional workloads •  File system for an instance—NTFS, ExtFS, and so on

Introducing Amazon Elastic File System Cloud-based file system

What is Amazon EFS?

•  Fully managed file system for EC2 instances •  Provides standard file system semantics •  Works with standard operating system APIs •  Sharable across thousands of instances •  Elastically grows to petabyte scale •  Delivers performance for a wide variety of workloads •  Highly available and durable •  NFS v4–based

Amazon EFS is designed for a broad range of use cases, such as…

•  Content repositories •  Development environments •  Home directories •  Web server farms •  Big Data

Amazon EFS Benefits...

… to app owners and developers?

… to your business?

•  Easy to move existing code, applications, and tools used today with existing NFS servers to the AWS cloud

•  Simple shared file storage solution for new cloud-native applications

•  Predictable pricing with no up-front investment •  Increased agility •  Spend less time managing file storage and more time

focusing on your business

… to IT administrators?

•  Eliminates need to manage and maintain file system storage at scale

AWS Storage Gateway

AWS Storage Gateway Your on-ramp to AWS cloud storage: •  Back up into S3 •  DR on EC2 •  Archive into Amazon Glacier •  iSCSI or VTL interface

Internet Web Services API

HTTP(S)

S3 S3

&/or Amazon Glacier

Partner Solutions

Summary: the Amazon storage portfolio

S3 •  Object storage: data presented as buckets of objects

EFS •  File storage (analogous to NAS): data presented as a file system

Amazon Glacier •  Archival storage: data presented as vaults/archives of objects

Storage Gateway

•  Back up and archive data into S3 and Amazon Glacier

EBS •  Block storage (analogous to SAN): data presented as disk volumes

LONDON