aws 101, london - september 2014

AWS 101 – Welcome! Ian Massingham - Technical Evangelist

30 September 2014

[email protected]

@IanMmmm

Thanks to our sponsors

Before we start…

…we’d love your feedback!

background

Consumer Business

Tens of millions of active

customer accounts

13 countries: US, UK, Germany, Japan, France, Canada, China,

Italy, Brazil, Mexico, India, Spain, Australia

Seller Business

Sell on Amazon websites

Use Amazon technology for your own retail website

Leverage Amazon’s

massive fulfillment centre network

IT Infrastructure Business

Cloud computing

infrastructure for hosting web-scale solutions

Hundreds of thousands of

registered customers in over 190 countries

How did Amazon…

… get into Cloud Computing?

AWS Mission

Enable businesses and developers to use web

services* to build scalable, sophisticated applications.

*What people now call “the cloud”

Not excess capacity!

Not excess capacity!

Startups on AWS

Find out more at : aws.amazon.com/solutions/case-studies

Powering the Most Popular Internet Businesses


Enterprises on AWS


Each day AWS adds the equivalent server capacity to power Amazon when it was a

global, $7B enterprise

utility computing

On demand Pay as you go

Uniform Available

Utility computing

Utility computing


Available Uniform

Utility computing

Utility computing

Compute

Storage

Security Scaling

Database

Networking Monitoring

Messaging

Workflow

DNS Load Balancing

Backup CDN


Uniform Available

On a global footprint

Region

US-WEST (N. California) EU-WEST (Ireland)

ASIA PAC (Tokyo)

ASIA PAC (Singapore)

US-WEST (Oregon)

SOUTH AMERICA (Sao Paulo)

US-EAST (Virginia)

GOV CLOUD

ASIA PAC (Sydney)

CHINA (Beijing)

At the end of a web service aws ec2 run-instances --image-id ami-a813fadf --count 3 --placement AvailabilityZone=eu-west-1a --instance-type m1.small

aws ec2 run-instances --image-id ami-a813fadf --count 5 --placement AvailabilityZone=eu-west-1c --instance-type m1.medium

and a rich Management Console

elasticity

Traditional IT capacity

Elastic capacity

Capacity

Time Your IT needs

On and Off Fast Growth

Variable peaks Predictable peaks

Elastic capacity

Elastic capacity

On and Off Fast Growth

Predictable peaks Variable peaks

WASTE

CUSTOMER DISSATISFACTION

Elastic capacity

Fast Growth On and Off

Predictable peaks Variable peaks

From one instance…

…to thousands

and back…

exploiting elasticity

Sunday Monday Tuesday Wednesday Thursday Friday Saturday

Typical weekly traffic to Amazon.com

November traffic to Amazon.com

November

November traffic to Amazon.com Provisioned capacity

November

November traffic to Amazon.com 76%

24%

Provisioned capacity

November

November 10th 2010 Turned off last physical web server of

Amazon.com

November 10th 2010 Turned off last physical web server of

Amazon.com

October 31st 2011 Turned off last web servers supporting

European business

November traffic to Amazon.com

November

Building a Top500 HPC Cluster on AWS

the toolbox

Compute Storage

AWS Global Infrastructure

Database

App Services

Deployment & Administration

Networking

Reference Model security

Compute Storage


Database

App Services


Networking

Global infrastructure

Regions An independent collection of AWS resources in a defined geography A solid foundation for meeting location-dependent privacy and compliance requirements


Availability Zones Designed as independent failure zones Physically separated within a typical metropolitan region

Compute Storage


Database

App Services


Networking


Edge Locations To deliver content to end users with lower latency A global network of edge locations Supports global DNS infrastructure (Route53) and Cloud Front CDN

Dallas(2)

St.Louis

Miami

Jacksonville Los Angeles (2)

Palo Alto

Seattle

Ashburn(3)

Newark New York (3)

Dublin

London(2)

Amsterdam (2)

Stockholm

Frankfurt(2) Paris(2)

Singapore(2)

Hong Kong (2)

Tokyo (2)

Sao Paulo

South Bend

San Jose

Osaka Milan

Sydney

Madrid

Seoul

Mumbai

Chennai

Compute Storage


Database

App Services


Networking

Compute Storage


Database

App Services


Networking

Networking

Direct Connect Dedicated connection to AWS

VPN Connection Secure internet connection to AWS

Virtual Private Cloud Private, isolated section of the AWS Cloud

Route 53 Highly available and scalable Domain Name Service

Compute Storage


Database

App Services


Networking

Compute

Vertical Scaling From $0.02/hr

Elastic Compute Cloud (EC2) Basic unit of compute capacity Range of CPU, memory & local disk options 13 Instance types available, from micro to cluster compute

Feature Details

Flexible Run windows or linux distributions

Scalable Wide range of instance types from micro to cluster compute

Machine Images Configurations can be saved as machine images (AMIs) from which new instances can be created

Full control Full root or administrator rights

Secure Full firewall control via Security Groups

Monitoring Publishes metrics to Cloud Watch

Inexpensive On-demand, Reserved and Spot instance types

VM Import/Export Import and export VM images to transfer configurations in and out of EC2

Compute

Auto-scaling Automatic provisioning of compute resources based upon demand, configuration or schedule

Trigger auto-scaling policy

Feature Details

Control Define minimum and maximum instance pool sizes and when scaling and cool down occurs

Integrated to CloudWatch Use metrics gathered by CloudWatch to drive scaling

Instance types Run auto scaling for on-demand instances and spot. Compatible with VPC

aws autoscaling create-auto-scaling-group --auto-scaling-group-name MyGroup --launch-configuration-name MyConfig --availability-zones eu-west-1a --min-size 4 --max-size 200

Compute Storage


Database

App Services


Networking

Compute

Elastic Load Balancing Create highly scalable applications Distribute load across EC2 instances in multiple availability zones

Feature Details

Auto-scaling Automatically scales to handle request volume

Available Load balance across instances in multiple availability zones

Health checks Automatically checks health of instances and takes them in or out of service

Session stickiness Route requests to the same instance

Secure sockets layer Supports SSL offload from web and application servers with flexible cipher support

Monitoring Publishes metrics to Cloud Watch

Compute Storage


Database

App Services


Networking

Compute Storage


Database

App Services


Networking

Storage

S3 - Durable storage, any object 99.999999999% durability of objects Unlimited storage of objects of any type Up to 5TB size per object

Feature Details

Flexible object store Buckets act like drives, folder structures within

Access control Granular control over object permissions

Server-side encryption 256bit AES encryption of objects

Multi-part uploads Improved throughput & control

Object versioning Archive old objects and version new ones

Object expiry Automatically remove old objects

Access logging Full audit log of bucket/object actions

Web content hosting Serve content as web site with built in page handling

Notifications Receive notifications on key events

Import/Export Physical device import/export service

Compute Storage


Database

App Services


Networking

Storage

Elastic Block Store High performance block storage device 1GB to 1TB in size Mount as drives to instances

Feature Details High performance file

system Mount EBS as drives and format as required

Flexible size Volumes from 1GB to 1TB in size

Flexible performance Magnetic, EBS General Purpose (SSD) and EBS Provisioned IOPS (SSD) volume types

Secure Private to your instances

Available Replicated within an Availability Zone

Backups Volumes can be snapshotted for point in time restore

Monitoring Detailed metrics captured via Cloud Watch

Compute Storage


Database

App Services


Networking

Database Relational Database Service Database-as-a-Service No need to install or manage database instances Scalable and fault tolerant configurations

Feature Details

Platform support Create MySQL, PostgreSQL, Microsoft SQL Server and Oracle RDBMS

Preconfigured Get started instantly with sensible default settings

Automated patching Keep your database platform up to date automatically

Backups Automatic backups and point in time recovery and full DB backups

Backups Volumes can be snapshotted for point in time restore

Failover Automated failover to slave hosts in event of a failure

Replication Easily create read-replicas of your data and seamlessly replicate data across availability zones

Compute Storage


Database

App Services


Networking

Database

Amazon Relational Database Service (Amazon RDS) databases stores forum threads, site content, and project configuration data. High availability Multi-AZ database deployment to handle live game metadata and user-generated content. Enterprise-grade fault tolerance for protecting customer data. By managing time-consuming database administration tasks, Amazon RDS allows SEGA to focus on business critical applications.

Compute Storage


Database

App Services


Networking

Database

DynamoDB Provisioned throughput NoSQL database Fast, predictable performance Fully distributed, fault tolerant architecture

Feature Details

Provisioned throughput Dial up or down provisioned read/write capacity

Predictable performance Average single digit millisecond latencies from SSD backed infrastructure

Strong consistency Be sure you are reading the most up to date values

Fault tolerant Data replicated across availability zones

Monitoring Integrated to Cloud Watch

Secure Integrates with AWS Identity and Access Management (IAM)

Elastic MapReduce Integrates with Elastic MapReduce for complex analytics on large datasets

Compute Storage


Database

App Services


Networking

Database

Redshift Managed Massively Parallel Petabyte Scale Data Warehouse Streaming Backup/Restore to S3 Extensive Security 2 TB -> 1.6 PB

RDS Dynamo DB

Redshift

Compute Storage


Database

App Services


Networking

Application Services

CloudFront World-wide content distribution network Easily distribute content to end users with low latency, high data transfer speeds, and no commitments.

Feature Details

Fast Multiple world-wide edge locations to serve content as close to your users as possible

Integrated with other services Works seamlessly with S3 and EC2 origin servers

Dynamic content Supports static and dynamic content from origin servers

Streaming Supports rtmp from S3 and includes support for live streaming from Adobe FMS and Microsoft Media Server

London

Paris

NY

Served from S3 /images/*

3

Served from EC2 *.php

2

Single CNAME www.mysite.com

1

Application Services

Amazon SQS

Processing task/processing trigger

Processing results Amazon SQS Reliable, highly scalable, queue service for storing messages as they travel between instances

Feature Details

Reliable Messages stored redundantly across multiple availability zones

Simple Simple APIs to send and receive messages

Scalable Unlimited number of messages

Secure Authentication of queues to ensure controlled access

Compute Storage


Database

App Services


Networking

Compute Storage


Database

App Services


Networking

Deployment & Admin

Elastic Beanstalk One-click deployment from Eclipse, Visual Studio and Git Rapid deployment of applications All AWS resources automatically created

Feature Details

Platform support Containers for Java, .net and PHP

Resource creation Creates load balancer, instances, autoscaling and monitoring automatically

Monitoring & Logs Integrated with Cloud Watch and consolidates server logs

Versioning Manage versions of applications and easily rollback deployments

Notifications Receive alerts on key events

Full resource access Access all underlying AWS resources as necessary

Deployment & Admin

OpsWorks DevOps focused managed application stacks Underlying Chef recipes allow for complete customisation

Feature Details

Platform support Chef recipes allows for community expansion for platform components such as Solr, NgniX etc

Resource creation Customizable deployments, rollback, partial deployments, patch management, automatic instance scaling, and auto healing

Layered Manage logical application layers and combine into stacks.

Compute Storage


Database

App Services


Networking

Cloud Formation Automate creation of ‘stacks’ in a repeatable way Scripting framework for AWS resource creation

Feature Details

Platform support Support for AWS resources from EC2 to IAM

Resource creation Creates AWS resources behind the scenes and reports on progress

Declarative Specify stacks in JSON format and source control your environments

Customizable Drive stack creation with parameters

Deployment & Admin

Compute Storage


Database

App Services


Networking

Deployment & Admin

Identity & Access Management Granular control of user rights with AWS Automated granting of EC2 service rights

Software Developer Kits Comprehensive support of programming models for using AWS services

Compute Storage


Database

App Services


Networking

+ others WorkSpaces Cloud Search

Simple Email Service Simple Workflow Service

Simple Notification Service ElastiCache (Memcache & Redis)

Elastic MapReduce CloudWatch

…and more to come!

security & compliance

Foundation Services

Compute Storage Database Networking

AWS Global Infrastructure Regions

Availability Zones Edge Locations A

maz

on

Shared responsibility

Foundation Services

Compute Storage Database Networking

Client-side Data Encryption & Data Integrity Authentication

Server-side Encryption (File System and/or Data)

Network Traffic Protection (Encryption/Integrity/Identity)

Platform, Applications, Identity & Access Management

Operating System, Network & Firewall Configuration

Customer Data A

maz

on

Shared responsibility

You

AWS Global Infrastructure Regions

Availability Zones Edge Locations

Certifications

SOC 1 Type 2 (formerly SAS-70)

ISO 27001

PCI DSS for EC2, S3, EBS, VPC, RDS, ELB, IAM

FISMA Moderate Compliant Controls

HIPAA & ITAR Compliant Architecture

Physical Security

Datacenters in nondescript facilities

Physical access strictly controlled

Must pass two-factor authentication at least twice for

floor access

Physical access logged and audited

HW, SW, Network

Systematic change management

Phased updates deployment

Safe storage decommission

Automated monitoring and self-audit

Advanced network protection

Security standards

http://aws.amazon.com/security

So what are we going to build

today?

A dynamic web application featuring:

A highly available, fault tolerant architecture Elasticity to deal with spikes in load

A fully managed database Static content served through a CDN

Shaun Pearce AWS Solutions Architect

A dynamic web application featuring:

A highly available, fault tolerant architecture Elasticity to deal with spikes in load

A fully managed database Static content served through a CDN

Region

Instance

Availability Zone Availability Zone

Region

Instance

S3


Region

Instance

Cloud Front

S3


Region

Instance

Cloud Front

S3

RDS Availability Zone Availability Zone

Region

Instance

Elastic Load Balancer

Cloud Front

S3 Auto scaling Group


Instance

But before we start….

Region

Cloud Front


Region

Instance


bootstrapping passing data to an instance

Instance request

User data

Instance request

User data

Instance

Meta-data service

Script executed on launch: <script> ipconfig /all > c:\ipconfig.txt netstat > c:\netstat.txt

</script>

Script executed on launch: #!/bin/sh yum -y install httpd php php-mysql chkconfig httpd on /etc/init.d/httpd start

security groups instance firewalling

Security Group

instance

Port 80 (HTTP)

Port 22 (SSH)

Name Description Protocol Port range IP Address, range, or another security group

Region

Instance


index.php Reads instance meta-data

Some php code that gets the data <?php

// get the instance id $url = "http://169.254.169.254/latest/meta-data/instance-id"; $instance_id = file_get_contents($url); // get the AZ where the instance is running $url = "http://169.254.169.254/latest/meta-data/placement/availability-zone";

$zone = file_get_contents($url);

// get the security group it is in $url = "http://169.254.169.254/latest/meta-data/security-groups";

$group = file_get_contents($url); // get the public DNS name $url = "http://169.254.169.254/latest/meta-data/public-hostname";

$hostname = file_get_contents($url); ?>

And displays it

Instance ID: <?php echo $instance_id; ?> Availability Zone: <?php echo $zone; ?> Security Group: <?php echo $group; ?>

Region

Instance


S3 For Static Website Hosting

Region

Instance

S3


CloudFront Global CDN

Region

Instance

Cloud Front

S3


Amazon RDS Managed Relational DB

Region

Instance

Cloud Front

S3


auto-scaling with elastic load balancing

Region

Instance


Cloud Front



Instance


Region

Availability Zone

Instance Instance Instance

Elastic Load Balancer Auto scaling Group


Region

Availability Zone

Instance Instance Instance Instance Instance Instance



Region

Availability Zone

Instance Instance Instance



Region

Availability Zone


Instance Instance


Region

Availability Zone

Instance Instance


Instance

Describes what Auto Scaling will create when adding

Instances

- AMI - - Instance Type - - Security Group -

- Instance Key Pair -

Only one active launch configuration at a time

Auto Scaling will terminate instances with old launch

configuration first rolling update

Auto Scaling managed grouping of EC2 instances

Automatic health check to maintain pool size

Automatically scale the number

of instances by policy – Min, Max, Desired

Automatic Integration with ELB

Automatic distribution & balancing across AZs

Parameters for performing an Auto Scaling action

Scale Up/Down and by how much

ChangeInCapacity (+/- #) ExactCapacity (#)

ChangeInPercent (+/- %)

Cool Down (seconds)

Policy can be triggered by CloudWatch events

Launch Configuration Auto-Scaling Group Auto-Scaling Policy

Region

Instance


Cloud Front



Instance

add some load from another EC2 instance

questions?

Next Steps

We typically see customers start by trying our services

Get started now at : aws.amazon.com/getting-started

You can get started for free…

Get started now at : aws.amazon.com/free

Design your application for the AWS Cloud

More details on the AWS Architecture Center at : aws.amazon.com/architecture

AWS Training & Certification Certification

aws.amazon.com/certification

Demonstrate your skills, knowledge, and expertise

with the AWS platform

Self-Paced Labs

aws.amazon.com/training/ self-paced-labs

Try products, gain new skills, and get hands-on

practice working with AWS technologies

aws.amazon.com/training

Training

Skill up and gain confidence to design, develop, deploy and

manage your applications on AWS

We’d love your feedback!

Thank You – AWS 101 Ian Massingham - Technical Evangelist

30 September 2014

[email protected]

@IanMmmm

aws 101, london - september 2014

Technology