aws summit berlin 2013 - your first week with ec2

Ryan Shuttleworth, Technical Evangelist

Your first week with EC2(and other AWS things…)

things you should know/think about up front

some best practices for getting started

essential technologies to dive into and get familiar with

architectural principles you should immerse yourself in

What are we going to cover?Your first 5 days with EC2…

hear a ‘looking back at our first year’ customer storycompressed into 5 days

What are we going to cover?Your first 5 days with EC2…

organise your house1DAY

Users & Roles

Start as you mean to go on

Secure your console with IAM roles

A little time spent now will save headaches later

Users & Roles

Start as you mean to go on

Secure your console with IAM roles

A little time spent now will save headaches later

Accounts & Billing

Create a structure that makes sense

Dev & Test accounts vs production

Consolidated billing

Resource tagging

Master Accountaws.invoices@mycompany.com

Billing

Division Badmin@divisionB.com

User2Dev2Admin2

IAM

Master Accountaws.invoices@mycompany.com

consolidated billing information

Billing

Division Badmin@divisionB.com

User2Dev2Admin2

IAM

Tags:Own=DivProj=P

Tags:Own=DivProj=Q

Tags:Own=DivProj=R

Master Accountaws.invoices@mycompany.com

consolidated billing information

Billing

Tags: (key-value)

e.g Own=DivProj=R

Operating Co. Aadmin@opcoa.com

User1Dev1Admin1

IAM

Tags:Own=OpCoProj=A

Tags:Own=OpCoProj=B

Tags:Own=OpCoProj=C

Division Badmin@divisionB.com

User2Dev2Admin2

IAM

Tags:Own=DivProj=P

Tags:Own=DivProj=Q

Tags:Own=DivProj=R

Business Unit Cadmin@busUnitC.com

User3Dev3Admin3

IAM

Tags:Own=BusCProj=X

Tags:Own=BusCProj=Y

Tags:Own=BusCProj=Z

Master Accountaws.invoices@mycompany.com

consolidated billing information

Billing

Operating Co. Aadmin@opcoa.com

User1Dev1Admin1

IAM

Tags:Own=OpCoProj=A

Tags:Own=OpCoProj=B

Tags:Own=OpCoProj=C

Division Badmin@divisionB.com

User2Dev2Admin2

IAM

Tags:Own=DivProj=P

Tags:Own=DivProj=Q

Tags:Own=DivProj=R

Business Unit Cadmin@busUnitC.com

User3Dev3Admin3

IAM

Tags:Own=BusCProj=X

Tags:Own=BusCProj=Y

Tags:Own=BusCProj=Z

Master Accountaws.invoices@mycompany.com

Alert:

Reached $1,250consolidated billing information

Billing

Alert:

Reached $3,000Alert:

Reached $500

Master Accountaws.invoices@mycompany.com

consolidated billing information

Programmatic billing access

S3 CSV

Billing

Operating Co. Aadmin@opcoa.com

User1Dev1Admin1

IAM

Tags:Own=OpCoProj=A

Tags:Own=OpCoProj=B

Tags:Own=OpCoProj=C

Division Badmin@divisionB.com

User2Dev2Admin2

IAM

Tags:Own=DivProj=P

Tags:Own=DivProj=Q

Tags:Own=DivProj=R

Business Unit Cadmin@busUnitC.com

User3Dev3Admin3

IAM

Tags:Own=BusCProj=X

Tags:Own=BusCProj=Y

Tags:Own=BusCProj=Z

Operating Co. Aadmin@opcoa.com

User1Dev1Admin1

IAM

Tags:Own=OpCoProj=A

Tags:Own=OpCoProj=B

Tags:Own=OpCoProj=C

Division Badmin@divisionB.com

User2Dev2Admin2

IAM

Tags:Own=DivProj=P

Tags:Own=DivProj=Q

Tags:Own=DivProj=R

Business Unit Cadmin@busUnitC.com

User3Dev3Admin3

IAM

Tags:Own=BusCProj=X

Tags:Own=BusCProj=Y

Tags:Own=BusCProj=Z

Master Accountaws.invoices@mycompany.com

consolidated billing information

Programmatic billing access

S3 CSV

Billing

Secrets & Keys

Secrets & Keys

Your front door keys

Secrets & Keys

Your front door keys

Control access to your instances

Key management strategy

Secrets & Keys

Control access to your APIs

Your front door keys

Use IAM Roles to distribute to instances

Control access to your instances

Key management strategy

learn the basics2DAY

What is EC2?

Elastic capacity Flexible

Complete control

Reliable

Inexpensive

Secure

Disposable compute

Instance

Instance Unit of scale

Unit of resilience

Unit of control

Instance Unit of scale

Unit of resilience

Unit of control

Your stack

Instance

Instance

Instance

Instance

Unit of scale

Unit of resilience

Unit of control

Scal

e ou

t

Instance

Instance

Instance

Instance

Unit of scale

Unit of resilience

Unit of control

Instance

Instance

Instance

Unit of scale

Unit of resilience

Unit of control

Instance

Instance

Instance

Unit of scale

Unit of resilience

Unit of control

Instance

Think differentlyCompute is transient

Programmatic resources

Treat your datacentre resources like code

Distributed systems

Programmatic resources

Design for decoupled systems up front

Treat your datacentre resources like code

Late binding

Distributed systems

Programmatic resources

Decide what it will run on at point of deployment

Design for decoupled systems up front

Treat your datacentre resources like code

go wild with tools3DAY

Compute

Storage

Security Scaling

Database

NetworkingMonitoring

Messaging

Workflow

DNSLoad Balancing

BackupCDN

APIAccess everything

via CLI, API or Console

Achieve the highest levels of automation

sophistication with ease

Everything is programmable

CLI Tools

ec2-run-instances ami-54cf5c3d--instance-count 2--group webservers--key mykey--instance-type m1.small

$>

>>> import boto.ec2>>> conn = boto.ec2.connect_to_region("us-east-1")>>> conn.run_instances( 'ami-54cf5c3d', key_name='mykey', instance_type='m1.small', security_groups=['webservers'])

Python boto

Resources created programmatically

Resources created programmaticallyConfigure automatically

Bake an AMI

Start an instance

Configure the instance

Create an AMI from your instance

Start new ones from the AMI

Bootstrapping

Bake an AMI

Start an instance

Configure the instance

Create an AMI from your instance

Start new ones from the AMI

Bootstrapping

ec2-run-instances<your ami-

id>

$>

Bake an AMI Configure dynamically

Start an instance

Configure the instance

Create an AMI from your instance

Start new ones from the AMI

Bootstrapping

Launch an instance

Use metadata service and cloud-init to

perform actions on instance when it

launches

vs

Bootstrapping

Bake an AMI

Build your base images and setup

custom initialisation scripts

Maintain your ‘golden’ base

Bootstrapping

Use bootstrapping to pass custom

information in and perform post launch

tasks like pulling code from SVN

+

Bootstrapping

Configure dynamically

Bake an AMI

Bootstrapping

Time consuming configuration (e.g startup time)

Static configurations(e.g less change management)

Bootstrapping

Configure dynamically

Bake an AMI

Bootstrapping

Continuous deployment(e.g latest code)

Environment specific(e.g dev-test-prod)

Bootstrapping

Configure dynamically

Goal is bring an instance up in a useful state

The balance will vary depending upon your application

Instance request

User data

User-data

Instance request

User data

Meta-data service

User-data

Instance request

User data

Instance

Meta-data service

User-data

#!/bin/shyum -y install httpd php mysql php-mysqlchkconfig httpd on/etc/init.d/httpd start

Shell script in user-data will be executed on launch:

Tip

Tip

Amazon Windows EC2Config Service executes user-data on launch:<script>dir > c:\test.log</script>

<powershell>any command that you can run</powershell>

<powershell>Read-S3Object -BucketName myS3Bucket -Key myFolder/myFile.zip -File c:\destinationFile.zip

</powershell>

AWS Powershell Tools

Why do this?

AutomationLess fingers, less mistakes

AvailabilityDrive higher

availability with self-healing

SecurityInstances

locked down by default

FlexibleShell,

Powershell, CloudFormation,Chef, Puppet,

OpsWorks

ScaleManage large scale

deployments and drive autoscaling

EfficiencyAudit and manage your estate with less time & effort

try something new4DAY

UnconstrainedEC2 resources

UnconstrainedComplimentary services

My little instance(created programmatically)

A bit of S3 code(pulled down automatically from S3)

>>> from boto.s3.key import Key>>> k = Key(bucket)>>> k.key = 'foobar'>>> k.set_contents_from_string('This is a test of S3')

My little instance(created programmatically)

A bit of S3 code(installed automatically)

>>> from boto.s3.key import Key>>> k = Key(bucket)>>> k.key = 'foobar'>>> k.set_contents_from_string('This is a test of S3')

My little instance(created programmatically)

UNLIMITED storage from my little instance

Services instead of softwareRemoves undifferentiated heavy lifting

Services instead of softwareRemoves undifferentiated heavy lifting

S3 for object storage

SQS for queues

RDS for databases

CloudWatch for monitoring

put something together5DAY

1. Use multiple availability zones

2. Use RDS with replicas and slaves

3. Use auto-scaling groups

4. Use Elastic Load Balancing

5. Use Route53 to host DNS zones

Find out more about model architectures in:

Building Web Scale Applications session

Get set up right from the start

Understand programmable compute

Use the tools and have a play

Explore the services beyond EC2

Build something! (you can always throw it away)

SummaryLessons learned…

aws.typepad.com

Thank you