© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

Dougal Ballantyne, HPC Solutions Architect

October 7th, 2015

Amazon EBS Deep DiveSTG402

What to Expect from the Session

Amazon EBS

• Volumes

• Snapshots

Encryption

Q&A

Amazon EBS overview

For most builders AWS is get in and go!

Source: http://www.trucksplanet.com/catalog/model.php?id=1020

A “normal” hard drive

EBS =

What is Amazon EBS?

• Network block storage as a service

• Designed for five nines of availability

• EBS volumes attach to any EC2 instance in

the same Availability Zone

• Provides point-in-time snapshots to Amazon

S3

More about Amazon EBS

• It’s a service!

• It’s independent of Amazon EC2

• It has regional and AZ availability goals

• All EBS volumes are designed for 99.999% availability

• Over 2 million volumes are created per day

A few definitions…

• IOPS: Input/output operations per second (#)

• Throughput: Read/write rate to storage (MB/s)

• Latency: Delay between request and completion (ms)

• Capacity: Volume of data that can be stored (GB)

• Block size: Size of each I/O (KB)

Volume management

• Volume management, use tags

• Delete on termination flags, can be changed anytime

• It is persistent storage, it will stay around!

• Every customer should have an approach, example tag

scratch volumes

Volume Initialization

• Newly created volumes

• Just attach, mount and go!

• Pre-warming not recommended

• Volumes restored from snapshots

• EBS implements lazy loading from Amazon S3 so that you

can begin using the volume right away.

• For performance sensitive applications we recommend

reading the entire volume to accelerate the loading of data

from S3 and avoid potential latency increase.

EBS Volume Types

Magnetic General Purpose (SSD) Provisioned IOPS

(SSD)

EBS Volume Types

IOPS Baseline: 100-10,000 (3 / GiB)

IOPS Burst: 30 minutes @ 3,000

Throughput: Up to 160 MB/s

Latency: Single-digit ms

Performance Consistency: 99%

Most workloadsGeneral Purpose (SSD)

General Purpose (SSD) – Burst & baseline

16 KiB I/O size

(2) Max I/O credit per bucket is 5.4M

(1) Always accumulating 3

IOPS per GiB per second

(3) You can spend up to

3000 IOPS per second

Understanding General Purpose (SSD) bursting

Baseline performance = 3 IOPS per GiB or 100 IOPS

Minutes to empty a full I/O credit

bucket for various volume sizes

The larger the volume, the longer it takes to empty the I/O credit bucket

1 TB or larger volume will never exhaust its I/O credit bucket

Minutes to empty a full I/O credit

bucket for various volume sizes

The larger the volume, the longer it takes to empty the I/O credit bucket

1 TB or larger volume will never exhaust its I/O credit bucket

General Purpose (SSD) volumes example

Microsoft Windows 30 GiB boot volume:

Gets initial I/O credit of 5.4M

Could burst for up to 30 mins @ 3000 IOPS

Always accumulating 90 I/O credits per

second

Improved instance boot time

m3.medium

Volume type Access time OS

GP2 4:33 Windows Server 2012

Magnetic 7:16 Windows Server 2012

GP2 0:45 CentOS6

Magnetic 1:16 CentOS6

40% Reduction in access times by using General Purpose SSD

Database volume

1 TB PIOPS volume with 4K IOPS = $526.40 per month per volume

GP2 1 TB volume with 3000 IOPS = $102.40

GP2 2 x 500 GB volume at 3K, Burst to 6K = $102.40

80% cost savings, 50% more peak I/O with General

Purpose SSD

Guidelines for sizing

General Purpose (SSD) volumes

Generic boot, developer, test/dev, and web apps:

Provision GB required for your applications

Database apps:

1. Calculate the IOPS required in steady state

2. Perform this calculation: (steady state IOPS) ÷ 3 = GB to provision

Note: I/O bursts will support:

• Database load or table scan operations

• Spike in I/O workload

EBS Volume Types

IOPS: 100-20,000 (Customer

Provisioned)

Throughput: Up to 320 MB/s

Latency: Single-digit ms

Performance Consistency: 99.9%

Mission Critical workloadsProvisioned IOPS (SSD)

Provisioned IOPS (SSD)

16 KiB I/O size

EBS Volume Types

IOPS: typically 100, best effort

Throughput: 40-90 MB/s

Latency: Read 10-40ms, Write 2-10ms

Best for infrequently accessed data

Magnetic

EBS volume types - summary

General Purpose (SSD) Provisioned IOPS (SSD) Magnetic

Recommend use cases

Boot volumes

Small to med DBs

Dev and test

I/O-intensive workloads

Large DBsCold storage

Storage media SSD-backed SSD-backed Magnetic-backed

Volume size 1 GB - 16 TB 4 GB - 16 TB 1 GB - 1 TB

Max IOPS per volume 10,000 IOPS 20,000 IOPS ~100 IOPS

Burst < 1 TB to 3000 IOPS baseline baseline

Read and write peak throughput 160 MB/s 320 MB/s ~50-90 MBps

Max IOPS per node (16k) 48,000 48,000 48,000

Peak throughput node 800 MB/s 800 MB/s 800 MB/s

Latency (random read) 1-2 ms 1-2 ms 20-40 ms

API Name gp2 io1 standard

Price* $.10/GB-month$.125/GB-month

$.065/provisioned IOPS

$.05/GB-month

$.05/ 1M I/O

Why is General Purpose SSD the default?

High baseline level of performance

Burst to higher level of IOPS

Single, capacity-based pricing dimension

• Makes forecasting very easy

• Eliminates sizing complexity

Attractive price/GB/price/IOPS density

Always use General Purpose

(SSD) for boot volumes

Migrating to General Purpose (SSD) volumes

Change volume type during launch

Use EBS snapshots

You may be able to resize the file system

Use General Purpose (SSD) sizing guide

Benefits of using EBS snapshots

More durable than an EBS volume

• Stored in Amazon S3

Incremental (space-efficient)

• First snapshot is a clone

• Pay only for what you use

Availability Zone-independent

• Clone into any AZ

Can be copied efficiently across regions

What happens?

• EBS volume is made up of blocks

• Only blocks written to are marked as updated

• At the point of create snapshot, we make a list of blocks

to copy

• Snapshots copy only updated blocks to S3

• No need to wait for snapshot to complete

• Future snapshots copy only what has changed since last

snapshot

Tagging snapshots

Use tags to add

metadata to snapshots:

• Type (daily, weekly)

• Version

• Instance Id

• Volume Id

• Application stack

Tools to manage snapshots

• Customers have told us these work:

• Skeddly – www.skeddly.com

• ec2-consistent-snapshot – github.com/alestic/ec2-consistent-

snapshot/

EBS optimized instances

• Most instance families support the EBS-optimized flag

• EBS-optimized instances now support up to 4 GB/s

• Drive 32,000 16K IOPS or 500 MB/s

• Available by default on newer instance types

• EC2 *.8xlarge instances support 10 Gb/s network

• Max IOPS per node supported is ~48,000 IOPS @ 16K I/O

Encryption

Why encrypt data volumes?

Security:

Protects against someone who might gain unauthorized physical access to the volume

Can help with internal or external compliance efforts:

• Chief Information Security Officer wants encryption to protect sensitive corporate

data

• Third-party auditors want to see evidence that sensitive customer data is

encrypted

Ease of use and operating cost reduction:

Unlike open-source or third-party solutions, such as Trend Micro SecureCloud, SafeNet

ProtectV, etc., EBS encryption offers:

• “Checkbox” encryption at no extra cost

• Automated, secure key management

AWS Key Management Service

Managed service simplifies creation, control, rotation, and use of encryption keys in your applications

Integrated with AWS Server-side encryption• Amazon S3, EBS, Amazon RDS, Amazon Redshift, Amazon WorkMail, and

Amazon Elastic Transcoder

Integrated with Client-side encryption• AWS SDKs, S3 Encryption Client, DynamoDB Encryption Client

Integrated with AWS CloudTrail to provide auditable logs for regulatory and compliance activities

Available in all commercial regions except China

AWS Key Management ServiceIntegrated with AWS IAM Console

Server-side encryption in AWSAmazon EBS

Your Application or

AWS Service

+

Data Key Encrypted Data Key

Encrypted

Data

Master Key(s) in

Customer’s Account

AWS

Key Management Service

1. Application requests encryption key to use to encrypt data, passes reference to master key in account.

2. Client request authenticated based on master key permissions.

3. New data encryption key created - copy encrypted under master key.

4. Plaintext and encrypted data key returned to the client.

5. Plaintext data key used to encrypt data and then deleted.

6. Encrypted data key stored for later use and sent back to AWS KMS for when decryption occurs.

AWS Key Management ServiceHow Keys are Used to Protect Your Data

Summary

Use encryption if

you need itTake snapshotsSelect the right

instance for your

workload

Select the right

volume for your

workload

Q&A

Remember to complete

your evaluations!

Thank you!