webscale computing mike culver

45
Webscale Computing Mike Culver Amazon Web Services

Upload: marrim

Post on 17-Jan-2016

48 views

Category:

Documents


0 download

DESCRIPTION

Webscale Computing Mike Culver. Amazon Web Services. Amazon Has Three Parts. 1. 3. 2. Ideas Meet Pagers. What if Gravity Sets In?. How Do You Survive This?. Don’t Do the Muck. What is Cloud Computing?. Remote computing capacity On-demand Infinite scale XML Web services. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Webscale Computing Mike Culver

Webscale ComputingMike Culver

Amazon Web Services

Page 2: Webscale Computing Mike Culver

Amazon Has Three Parts

1

2

3

Page 3: Webscale Computing Mike Culver
Page 4: Webscale Computing Mike Culver

Ideas Meet Pagers

Page 5: Webscale Computing Mike Culver

What if Gravity Sets In?

Page 6: Webscale Computing Mike Culver

How Do You Survive This?

Page 7: Webscale Computing Mike Culver

Don’t Do the Muck

Page 8: Webscale Computing Mike Culver

What is Cloud Computing?

o Remote computing capacityo On-demando Infinite scaleo XML Web services

Page 9: Webscale Computing Mike Culver
Page 10: Webscale Computing Mike Culver

Fallacies of Distributed Computing

1. The network is reliable.

2. Latency is zero.

3. Bandwidth is infinite.

4. The network is secure.

5. Topology doesn't change.

6. There is one administrator.

7. Transport cost is zero.

8. The network is homogeneous.

Wikipedia.com

Page 11: Webscale Computing Mike Culver

2001 2002 2003 2004 2005 2006 2007

Bandwidth Consumed byAmazon Web Services

Bandwidth Consumed byAmazon’s Global Websites

2008

Page 12: Webscale Computing Mike Culver

Amazon Web Services Are...

Building block services that allow developers to innovate and make money

Infrastructure As a ServiceAmazon Simple Storage Service

Amazon Elastic Compute Cloud

Amazon Simple Queue Service

Amazon SimpleDB

Commerce As a ServiceAmazon Flexible Payments Service

Fulfillment Web Service

People As a ServiceAmazon Mechanical Turk

Alexa Web ServicesAlexa Web Information Service

Alexa Top Sites

Alexa Site Thumbnail

Alexa Web Search Platform

Page 13: Webscale Computing Mike Culver

Amazon Simple Storage Service (Amazon S3)

Highly scalable data storage in-the-cloud

Programmatic access via web services API

Simple to get going, simple to use

Highly available and durable

Pay-as-you-go:

Storage: $0.15 / GB / month

Data Transfer: starts at $0.17 / GB

Requests: nominal charges

Page 14: Webscale Computing Mike Culver

Billions of Objects Stored

.8 Billion

5 Billion

10 Billion

Q3 06 Q1 07 Q2 07

14 Billion

Q4 07

18 Billion

Q1 08

22 Billion

Q2 08

Page 15: Webscale Computing Mike Culver

Amazon S3 Namespace

Amazon S3

bucket bucket

object object objectobject

bucket

object object

Page 16: Webscale Computing Mike Culver

Amazon S3 Namespace

Amazon S3

mculver-images media.mydomain.com

Beach.jpg img1.jpg img2.jpg2005/party/hat.jpg

public.blueorigin.com

index.html img/pic1.jpg

Page 17: Webscale Computing Mike Culver

Internet Backup on a StickOr Stick Backed Up on the Net?

Page 18: Webscale Computing Mike Culver

Open Source Backup

Page 19: Webscale Computing Mike Culver

1 TB: Basecamp and Campfire Tools

Page 20: Webscale Computing Mike Culver

Amazon Elastic Compute Cloud(Amazon EC2)

Resizable compute capacity in the cloud

Obtain and boot new server instances in minutes

Quickly scale capacity, up or down, as your computing requirements change

Full root access to a blank Linux machine

Simple Web service management interface

Changes the economics of computing

Page 21: Webscale Computing Mike Culver

You just lost

customers

You just lost

customers

Predictions Cost Money

InfrastructureCost $

time

LargeCapital

Expenditure

LargeCapital

Expenditure

PredictedDemand

TraditionalHardware

ActualDemand

AutomatedVirtualization

Page 22: Webscale Computing Mike Culver

Resilient Infrastructure Requires Physical Isolation

Availability Zone A Availability Zone B

Availability Zone C

Page 23: Webscale Computing Mike Culver

Three Flavors of Amazon Machine Images

Public AMIs: Use pre-configured, template AMIs to get up and running immediately. Choose from Fedora, Movable Type, Ubuntu configurations, and more

Private AMIs: Create an Amazon Machine Image (AMI) containing your applications, libraries, data and associated configuration settings

Paid AMIs: Set a price for your AMI and let others purchase and use it (Single payment and/or per hour)

Page 24: Webscale Computing Mike Culver

Virtual Machine Choices

Standard High CPU

Small Large Extra Large

Medium Extra Large

Bits 32 64 64 32 64

RAM 1.7 GB 7.5 GB 15 GB 1.7 GB 7 GB

Disk 160 GB 850 GB 1690 GB 350 GB 1690 GB

EC2 Compute Units

1 4 8 5 20

I/O Performance

Medium High High High High

Firewall Yes Yes Yes Yes Yes

Per Hour $0.10 $0.40 $0.80 $0.20 $0.80

Page 25: Webscale Computing Mike Culver

$5 or $30,000 For a Prototype?

$30,000 worth of in-house servers $30,000 worth of in-house servers

DISA paid a total of $5DISA paid a total of $5

Page 26: Webscale Computing Mike Culver

Build Your Own Virtual ISP

Page 27: Webscale Computing Mike Culver

Operating Systems as a Service

Page 28: Webscale Computing Mike Culver
Page 29: Webscale Computing Mike Culver

4 TB Data 100 Nodes 11 Million PDFs

100 instances x 24 hours x $0.10 / Hr = $240

Page 30: Webscale Computing Mike Culver
Page 31: Webscale Computing Mike Culver

From 50 to 3400 Instances in 3 Days

Page 32: Webscale Computing Mike Culver

Amazon Simple Queue Service(Amazon SQS)

A distributed queue in the cloudUsed for storing messages traveling between computersReliable

Runs within Amazon's high-availability data centersMessages are stored redundantly across multiple servers and locations

Scalable to millions of messages a daySimple: Only 6 methodsPlatform agnosticProvides access control and message locking

Page 33: Webscale Computing Mike Culver
Page 34: Webscale Computing Mike Culver
Page 35: Webscale Computing Mike Culver
Page 36: Webscale Computing Mike Culver
Page 37: Webscale Computing Mike Culver
Page 38: Webscale Computing Mike Culver
Page 39: Webscale Computing Mike Culver
Page 40: Webscale Computing Mike Culver
Page 41: Webscale Computing Mike Culver

Amazon SimpleDBLimited Beta

Page 42: Webscale Computing Mike Culver

Databases Should Just WorkBut They’d Better Scale…

Easy: no administration

Web Service API

Flexible (no schemas)

Scalable: create new domains as

Your data grows

Request throughput increases

Durable

Multiple nodes

Replicated data centers

Page 43: Webscale Computing Mike Culver

Amazon SimpleDB

item description color material

123 Sweater Blue, Red

456 Dress shirt White, Blue

789 Shoes Black Leather

PUT (item, 123), (description, Sweater), (color, Blue), (color, Red)

PUT (item, 456), (description, Dress shirt), (color, White), (color, Blue)

PUT (item, 789), (description, Shoes), (color, Black), (material, Leather)

QueryDomain = MyStore

[‘description’ = ‘Sweater’]

Page 44: Webscale Computing Mike Culver

Learn More About AWS

Explore http://aws.amazon.com

Read our blog at http://aws.typepad.com

Email me at [email protected]

Page 45: Webscale Computing Mike Culver

Demo