dependable cloud architecture - cloud develop edition
DESCRIPTION
This is a talk I gave at Cloud Develop 2013. It was an adapted from a workshop session that Brent Stineman and I did in Jan of 2013 for CodeMash.TRANSCRIPT
![Page 1: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/1.jpg)
Image: xkcd.com
Dependable Cloud Architecture
@mikewo
Mike Wood
http://mvwood.com
![Page 2: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/2.jpg)
“Failure is alwaysan option.”
Image: Discovery Channel, Fair Use
![Page 3: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/3.jpg)
Protection From:
What are we looking for?
Check out: http://bit.ly/wazbizcontImages: Office ClipArt & Godzilla Releasing Corp (Fair Use)
Hardware Failure Data Corruption Network Failure Loss of Facilities
![Page 4: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/4.jpg)
Image: FOX, Fair Use
Human Error
![Page 5: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/5.jpg)
What we’re trying to achieve
1. Monitoring2. Resilient Solutions
![Page 6: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/6.jpg)
Image: Office ClipArt
Cost vs Risk
99.999% $1, … ,000.00
To get more 9’s here add more 0’s here.
![Page 7: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/7.jpg)
Image: NASA
Monitoring
![Page 8: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/8.jpg)
Functional Transparency
Image: Office ClipArt
Logging Messages
Hardware Health
Dependent Services Health
![Page 9: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/9.jpg)
Telemetry
![Page 10: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/10.jpg)
Image: NASA
Analyze your Data
![Page 11: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/11.jpg)
Resilience
![Page 12: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/12.jpg)
Remember: Failure is always an option.
Common Points of Failure• Machine\application crashes• Throttling (exceeding capacity)• Connectivity\Network• External service dependencies
Focus less on the uptime of hardware and more about how the solution handles it WHEN
something fails!
![Page 13: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/13.jpg)
Try/catch != Resilient
![Page 14: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/14.jpg)
Image: Michael Wood
Decompose your system…
![Page 15: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/15.jpg)
Request bufferingRetry Policies
• Wait and try again• Queue until available
Queuing Enables• Asynchronous workloads• Temporal Decoupling• Load Levelling
Check out: http://bit.ly/wazrequestbuffer
![Page 16: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/16.jpg)
Capacity BufferingContent Delivery Networks (CDN’s)
Distributed Application Cache
Local Content Cache
Enables recovery during outages or
spikes in load
![Page 17: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/17.jpg)
Dynamic Addressing & Configuration
![Page 18: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/18.jpg)
Dept. of Redundancy Dept.
Have a backup, somewhere elseMore than one? Cost to benefit Ratio?
Ready StateHot = full capacityWarm = scaled down, but ready to growCold = mothballed, starts from zero
Image: Mr. White
![Page 19: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/19.jpg)
Redundancy - Its about probability
95% uptime 95% uptime 95% uptime 95% uptime
1 box : 5% downtime or 438hrs per year
2 boxes : 5/100 * 5/100 = 25/10,000 = 0.25% downtime or 22hrs per year
4 boxes : 5/100 * 5/100 * 5/100 * 5/100 = 625/100,000,0000.000625% downtime or 3.285 MINUTES per year
(that’s 18 ½ days!)
![Page 20: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/20.jpg)
Always carry a spare75% Capacity, half of our load 75% Capacity, half of our load
50% more capacity then needed• Can absorb of temporary spikes• Time to react if need to add capacity
100% of load, 150% Capacity0% Capacity, redirect all load
Over allocated, but still functioning• Degrade, but don’t fail
SYSTEM FAILURE!!!
![Page 21: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/21.jpg)
Accessible vs. Available
Image: Twitter, Fair Use
![Page 22: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/22.jpg)
Availability via Degradation
Image: Michael Wood
![Page 23: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/23.jpg)
Total Outage duration =
Time to Detect+ Time to Diagnose+ Time to Decide+ Time to ActImage: Office ClipArt
![Page 24: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/24.jpg)
Images: Gizmodo
Virtualization and Automation
![Page 25: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/25.jpg)
Images: Orion Pictures owns Terminator Franchise
![Page 26: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/26.jpg)
The “HI” Point
Check out: http://bit.ly/wazinternals
![Page 27: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/27.jpg)
Image: NASA
![Page 28: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/28.jpg)
“Don't be too proud of this technological terror you've constructed…”
ADMIT:• Your Solution WILL fail at some point• You can learn from others just as
well as yourself
DO:• Root cause analysis• Read other root cause analysis
DON’T:• Get cocky• Stick your head in the sand
![Page 29: Dependable Cloud Architecture - Cloud Develop Edition](https://reader038.vdocuments.site/reader038/viewer/2022103109/546e9ea3af79596c298b5797/html5/thumbnails/29.jpg)
Questions
@mikewo
Mike Wood
http://mvwood.com
http://bit.ly/CloudFailSafe