advanced ebs snapshot management (stg402) | aws re:invent 2013
DESCRIPTION
Amazon EBS Snapshot is an easy to use feature that backs up your data on an Amazon EBS volume. This session covers filesystem selection (XFS, ext*, etc.), quiescing of the filesystem, tagging of snapshots, and life-cycle management of snapshots. In this session, we introduce two new OSS tools. One tool manages arrays of snapshots using tagging, making it easier to snapshot and recover a RAID array of Amazon EBS volumes; another tool manages snapshots of root volumes across an AWS account, and automatically snapshotting and applying lifecycle management to root volume snapshots.TRANSCRIPT
![Page 1: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/1.jpg)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
Advanced EBS Snapshot Management
Craig Carl, AWS
Session STG402
![Page 2: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/2.jpg)
EBS
• Network-attached block devices for Amazon EC2 instances
• Protected, durable within an AZ – AFR between 0.1% – 0.5%
• Standard and PIOPS – PIOPS is for databases!
![Page 3: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/3.jpg)
Snapshots
• More durable than an Amazon EBS volume – Stored in Amazon S3
• Differential (space-efficient) – First snapshot is a clone
– Pay for only what you use
• AZ-independent – Clone into any AZ
• Can be copied efficiently across regions
![Page 4: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/4.jpg)
Why Take Snapshots
• More durable than Amazon EBS
• Versioning
• Backups
• Cost
![Page 5: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/5.jpg)
Individual snapshots are easy
create-snapshot
delete-snapshot
copy-snapshot
![Page 6: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/6.jpg)
Managing Thousands of Snapshots Isn’t Easy
• A snapshot should have a lifecycle – Grandfather-father-son
– x versions
– x dailies, y weeklies, z monthlies
– A lifecycle might span regions
• Snapshots of arrays
![Page 7: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/7.jpg)
Introducing arche
• arche – https://github.com/AWSLabs/arche
• OSS, built and maintained by AWS
– Accepting pull requests from the community
![Page 8: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/8.jpg)
arche
• Two primary pieces – arche takes snapshots and manages lifecycles
• Takes snapshots based on a SWF schedule
– Tags snapshots
• Deletes snapshots based on a policy
– Limited to number of days now
– pierus is a CLI for restoring snapshots.
![Page 9: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/9.jpg)
arche Uses Tags
KEY VALUE
arche 1
retention <int> (days)
array <str>
dest-region <str>
• on volumes
![Page 10: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/10.jpg)
arche Uses a Lot of Tags
KEY VALUE
arche 1
src_instance <str>
src_device <str>
expire_date date
array <str>
src-region <str>
• on snapshots
![Page 11: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/11.jpg)
Accepting pull requests!
• File system quiesce support
• More advanced lifecycles
• Multi region copy
• AMI create support
![Page 13: Advanced EBS Snapshot Management (STG402) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022051513/547e929b5806b5b85e8b4682/html5/thumbnails/13.jpg)
We are sincerely eager to hear
your feedback on this
presentation and on re:Invent.
Please fill out an evaluation form
when you have a chance.