aws webcast - introduction to ebs
DESCRIPTION
Elastic Block Store is one of the fundamental components of a best-practices cloud architecture. Join Senior Solutions Architect Miles Ward for a detailed review of the service, a clear breakdown on cost and approaches for price estimates, easy methods for extracting maximum performance from both Standard and Provisioned-IOPS EBS volumes, configuration nuances for both Windows and Linux users, RAID guidance, RDBMS and NoSQL storage configuration tuning, as well as several examples of EBS delivering amazing value at scale.TRANSCRIPT
![Page 1: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/1.jpg)
Presenter Name
Presenter Title
Month Day, Year
Understanding Amazon EBS Availability and Performance
Miles Ward Senior Manager, Solutions Architecture
![Page 2: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/2.jpg)
Agenda
• Overview of Elastic Block Store
• Some key concepts: block size, IOPS, throughput
• Performance
• Availability
![Page 3: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/3.jpg)
Storage Options on AWS
Block Storage (Elastic Block Store)
Object Storage (S3, Glacier)
Use for: • Access to raw
unformatted block level storage
• Persistent Storage
Use for: • Pictures, videos,
highly durable media storage
• Cold storage for long-term archive
![Page 4: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/4.jpg)
Amazon Elastic Block Store (EBS) Elastic Block Storage: Persistent Storage for EC2
Feature Details
High performance file system
Mount EBS as drives and format as required
Flexible size Volumes from 1GB to 1TB in size
Secure Private to your instances
Available Replicated within an Availability Zone
Backups Volumes can be snapshotted for point in time restore
Monitoring Detailed metrics captured via Cloud Watch
![Page 5: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/5.jpg)
What are some of our customers doing with EBS?
Enterprises
Enterprise workloads are built on block
storage
Oracle, SAP, Microsoft
Applications
Convenient, cost-effective,
reliable file server
Gaming/Social/ Mobile/Education
Very high performance
and consistent IO
for NoSQL and relational
DBs
Marketing / Analytics
Fast sequential IO
access
![Page 6: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/6.jpg)
Key EBS concepts
• Standard EBS Volumes
• Provisioned IOPS EBS
Volumes
• Block Size
• Queue Depth
• Snapshots
![Page 7: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/7.jpg)
Standard and Provisioned IOPS Volume Types
Standard Volumes Provisioned IOPS Volumes
Optimized for Workloads with low or moderate IOPS needs and occasional bursts.
Transactional workloads requiring consistent IOPS.
Volume Attributes
Up to 1 TB, average 100 IOPS per volume. Best effort performance. Can be striped together for larger size and higher IOPS.
Up to 1TB, 2,000 IOPS per volume. Consistent IOPS. Can be striped together for larger size and higher IOPS.
Workloads File server, Log processing, Websites, Analytics, Boot, etc.
Business applications, MongoDB, SQL server, MySQL, Postgres, Oracle, etc.
![Page 8: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/8.jpg)
Introducing Provisioned IOPS Volumes
❶ Select a new type of Provisioned IOPS volumes
❸ Specify the number of IOs per second your application needs, up to 4000 PIOPS per volume. The volume will deliver the specified IO per second.
❷ Specify the volume capacity
$ ec2-create-volume --size 500 --availability-zone us-east-1b --type
io1 –iops 2000
![Page 9: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/9.jpg)
Consistent IO performance
Provisioned IOPS volumes deliver within 10% of the IOPS performance 99.9% of the time over a given year.
![Page 10: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/10.jpg)
Stab
le
Testing random 4K reads
EBS
PIOPS
+ SSD
![Page 11: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/11.jpg)
Before PIOPS:
After PIOPS:
End-to-end latency
![Page 12: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/12.jpg)
When should I use Provisioned IOPS?
• Provisioned IOPS volumes are designed for running transactional applications that require high and consistent IO: o Relational Databases
o NoSQL Databases, e.g. MongoDB
o Productivity applications, e.g. Microsoft Exchange
o Enterprise Applications
• Standard volumes are designed for boot volumes, running applications with moderate and bursty IO, and sequential IO access
![Page 13: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/13.jpg)
IOPS vs. Throughput
One input operation or “block”
The network throughput available on the connection between EC2 and EBS
The IOPS provisioned
![Page 14: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/14.jpg)
IOPS vs. Throughput
Bigger than 16k…
Greater than the mbps limit…
Greater than the IOPS provisioned…
![Page 15: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/15.jpg)
A little math
• Networks are in mbit/s, disks tend to be in MB/s.
• 1000mbit/s = 125 MB/s
• 500mbit/s = 67.5 MB/s
• Blocks/ IOs are in KB
• ~7,500 16KB blocks fit in 1000mbit/s
• ~30,000 4KB blocks fit in 1000mbit/s
![Page 16: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/16.jpg)
Block Size
![Page 17: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/17.jpg)
Queue Depth
Maintain a number of pending I/O requests to get the most out of your Provisioned IOPS volume. The volumes must maintain an average queue length of 1 (rounded up to the nearest whole number) for every 200 provisioned IOPS in a minute
![Page 18: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/18.jpg)
Performance
• Architecting for Performance
– Avoid IO saturation
– EBS Optimized
– Avoid throughput saturation
– RAID
• Achieving Consistent Performance
– Pre-warm PIOPS volumes
– Plan for snapshot
• Snapshot Performance
![Page 19: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/19.jpg)
Architecting for Performance: Use EBS Optimized Instances
![Page 20: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/20.jpg)
USING Provisioned IOPS Volumes
❶ Select a new type of Provisioned IOPS volumes
❸ SPECIFY ENOUGH IOPS!!
❷ Specify the volume capacity
$ ec2-create-volume –size 400 --availability-zone us-east-1b --type
io1 –iops 4000
![Page 21: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/21.jpg)
Architecting for Performance: Avoid Throughput Saturation
• Example:
– Cluster Compute instance types have 2Gb/s bandwidth to EBS, more than 8 PIOPS volumes at 2000 IOPS each will saturate 2 Gb/s network
– EBS Optimized M3.2Xlarge instance has 1 Gb/s bandwidth dedicated to EBS, more than 16 PIOPS volumes at 500 IOPS each will saturate the 1 GB/s network
![Page 22: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/22.jpg)
RAID
• RAID 10: provides increased redundancy
– Replace EBS volume without application downtime
– Increases read throughput
– However, 50% reduction of provisioned aggregate write performance
– E.g., MongoDB optimized around the benefits of RAID 10
• RAID 0:
– All EBS volumes are replicated in the same AZ
– Increased throughput
![Page 23: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/23.jpg)
Achieving Consistent Performance: Pre-warm PIOPS volumes
• There is a 5 to 50 percent performance reduction in IOPS when you first access the data on a PIOPS volume.
• Write to all blocks on volumes before first use
– $ dd of=/dev/md0 if=/dev/null
![Page 24: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/24.jpg)
Snapshots
• Create snapshots (backups) of any Amazon EBS volume.
• The volume need not be attached to a running instance in order to take a
snapshot.
• These snapshots can be used to create multiple new Amazon EBS volumes,
expand the size of a volume, or move volumes across Availability Zones.
• The snapshots can be shared with specific AWS accounts or made public.
• You can use this functionality to increase the size of an existing volume, rapidly
replicate development and testing environments, or use Snapshot Copy to copy
snapshots to another region for disaster recovery or regional expansion.
![Page 25: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/25.jpg)
Snapshot Performance
• To improve snapshot performance
– Increase the frequency of snapshots
![Page 26: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/26.jpg)
Achieving Consistent Performance: Plan for Snapshot
To minimize the impact of snapshots on performance of a master node:
– create snapshots from a read replica of your data
– Plan snapshots during off-peak usage
– Use cross-region snapshot copy to keep distant regional copies for HA / low latency local access
![Page 27: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/27.jpg)
Performance – Minimum production DB scale
• Always use Elastic Block Store (EBS) o Significant write cache o Great random IO performance o Enhanced durability compared to instance stores
![Page 28: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/28.jpg)
Performance – production scale
• Use Provisioned IOPS for all workloads targeting above 100 IOPS.
• Move up to higher bandwidth instance types (m1.xlarge, c1.xlarge, m2.4xlarge)
• Increase EBS volume size to >= 400 GB
• Increase number of volumes in RAID set
![Page 29: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/29.jpg)
Performance – Extra-large scale
• Leverage Cluster Compute instance types o More bandwidth to EBS o Ex. CC2 will make excellent primary nodes,
particularly when paired with a large number of EBS volumes (= 8)
• Improve RAID configuration with: o effective_io_concurrency = # of stripes in RAID
set
![Page 30: AWS Webcast - Introduction to EBS](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6cc164a7959b5318b45a9/html5/thumbnails/30.jpg)
• Amazon Provisioned IOPS o http://copperegg.com/amazon-provisioned-iops-ebs/
• Benchmarking EBS performance: o http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerformance.html
• Stratalux: Putting Amazon’s Provisioned IOPS to the test o http://www.stratalux.com/2012/08/09/putting-amazon’s-provisioned-iops-to-the-test/
Get started on EBS
today! aws.amazon.com/ebs
Questions: e-mail: [email protected]
@milesward