best practices for getting started session

Best Practices for Getting Started with AWS

[email protected]@IanMmmm

Ian Massingham — Technical Evangelist

mailto:[email protected]

Getting Started with AWS: Agenda

Eight best practices you should focus on when getting started

Resources you can use to learn more

Getting Started with AWS

http://aws.amazon.com/getting-started/

Getting Started with AWS


Choose Your First Use Case Well

1

Chose Your First Use Case Well

Make your first project a S.M.A.R.T one


Dev & Test

Spin environments up and down on demand

Decouple development and test environments

from operations constraints

Explore elasticity in a sandboxed environment



Dev & Test





Backup & DR Take part of your data or

business applications step- by-step into non-

production DR use

Understand cloud dynamics and test during

controlled failover



Dev & Test







production DR use


controlled failover

Greenfield Project

Embody best practice of cloud computing in

unconstrained greenfield projects

Self contained web projects, document

archiving etc



Dev & Test







production DR use


controlled failover

Greenfield Project

Embody best practice of cloud computing in

unconstrained greenfield projects

Self contained web projects, document

archiving etc

Pain point

Move specific service aspects causing undue cost or management

burden

Workflows, search indexing, media

streaming, document archiving, constrained

databases


Plan Evolution and Set Goals

Understand services

Test performance

Architect for scale

Develop team capabilities

Implement monitoring

Change control and management

Security management

Scalability

Automate corrective actions

Auto-scaling

Zero downtime deployments

System backup and recovery

Proof of Concept Production Automation

Sam

ple

Act

iviti

es

Lay Out Your Foundations

2

Accounts

Create an account structure that makes sense

Use accounts like environments where you need separation and

control

e.g. Dev Sandboxes Test Environments

Business Units Products & Services


BillingAccounts



control



Control access to billing information

Use IAM users to keep billing information in the master account

Consolidate billing into a single account

Let one account pick up the bill for multiple ‘sub accounts’

Setup billing alerts and automated bill reporting

Get CloudWatch notifications when billing reaches a point and output

csv reports to S3 for analysis


Enable delivery of billing reports with resources & tags

Billing preferences

Billing Settings

BillingMaster Account

[email protected]

Billing

Consolidated Billing Relationship

Master [email protected]

Division [email protected]

User2Dev2Admin2

IAM

Billing

Consolidated Billing Relationship



User2Dev2Admin2

IAM

Tags: Own=DivProj=P

Tags: Own=DivProj=Q

Tags: Own=DivProj=R

Tags: (key-value) e.g Own=Div

Proj=R

Billing

Consolidated Billing Relationships


Business Unit [email protected]

User3Dev3Admin3

IAM

Tags: Own=BusCProj=X

Tags: Own=BusCProj=Y

Tags: Own=BusCProj=Z


User2Dev2Admin2

IAM

Tags: Own=DivProj=P

Tags: Own=DivProj=Q

Tags: Own=DivProj=R

Operating Co. [email protected]

User1Dev1Admin1

IAM

Tags: Own=OpCoProj=A

Tags: Own=OpCoProj=B

Tags: Own=OpCoProj=C

Billing




User3Dev3Admin3

IAM





User2Dev2Admin2

IAM

Tags: Own=DivProj=P

Tags: Own=DivProj=Q

Tags: Own=DivProj=R


User1Dev1Admin1

IAM




Alert:

Reached $500 Alert:

Reached $3500 Alert:

Reached $1250

S3CSV

Billing

ANALYSIS

Programmatic Billing Access




User3Dev3Admin3

IAM





User2Dev2Admin2

IAM

Tags: Own=DivProj=P

Tags: Own=DivProj=Q

Tags: Own=DivProj=R


User1Dev1Admin1

IAM




3rd Party Cost Management Tools

Access KeysBillingAccounts



control










Decide upon a key management strategy

Control access to EC2 instances via SSH and embedded public key:

e.g. EC2 Key Pair per group of instances, EC2 Key Pair per

account

Consider SSH key rotation & automation

Limit exposure to private key compromise by rotating keys and replacing authorized_keys listings

on running instances Consider bootstrap automation to

grant developer access with developer unique keypairs


Groups & RolesAccess KeysBillingAccounts



control










Decide upon a key management strategy

Control access to EC2 instances via SSH and embedded public key:

e.g. EC2 Key Pair per group of instances, EC2 Key Pair per

account

Consider SSH key rotation & automation

Limit exposure to private key compromise by rotating keys and replacing authorized_keys listings

on running instances Consider bootstrap automation to

grant developer access with developer unique keypairs

Use IAM Groups to manage console users and API

access Provide developers with IAM user

login and unique API access credentials

Control & restrict what IAM users can do by placing them in groups

with associated policies

Assign EC2 Instances IAM roles

Let AWS manage API access credentials on running instances by assigning a system entitlement to

an instance e.g. instance can only read S3

bucket


Identity & Access Management - IAMAccount

ApplicationsAdministrators Developers

Jim

Gavin

Steve

Nigel

Stephen

Ingest

Console

Reporting



Jim

Gavin

Steve

Nigel

Stephen

Ingest

Console

Reporting

Groups

Multi-factor Authentication



Jim

Gavin

Steve

Nigel

Stephen

Ingest

Console

Reporting

Groups Roles

Multi-factor Authentication

AWS API Credentials

IAM Policies{"Statement":[{"Effect":"Allow","Action":["elasticbeanstalk:*","ec2:*","elasticloadbalancing:*","autoscaling:*","cloudwatch:*","s3:*","sns:*"],"Resource":"*"}]}

Create a policy to assign permissions to a user, group, role or resource.

Policies are created using JSON. A policy consists of one or more statements, each of which describes one set of permissions.

Policies control access to AWS APIs

Identity and Access Management - IAM

For more details on IAM, visit:

aws.amazon.com/iam

http://aws.amazon.com/iam

Think Security3

Foundation Services

Compute Storage Database Networking

AWS Global Infrastructure Regions

Availability ZonesEdge Locations

Client-side Data Encryption & Data Integrity Authentication

Server-side Encryption (File System and/or Data)

Network Traffic Protection(Encryption/Integrity/Identity)

Platform, Applications, Identity & Access Management

Operating System, Network & Firewall Configuration

Customer Data

Amaz

onYo

u

Shared Security Responsibility

Understand your customer & determine your security stance

Leverage AWS Security

External Audience

Regulatory Audience

Internal Audience

Architecture

Administration

IAM

Certifications

White Papers

QSA Process

Your Processes

Your Certifications Penetration Test Results


Engage with security assessors early in your adoption cycle


Don’t fear assessment – AWS meets high standards (PCI DSS, ISO27001)

Security assessments take time, so allow for this in your planning

Undertake architecture reviews early in your design/deployment process



Use comprehensive materials and certifications provided by AWS


For more details on AWS Security, visit: aws.amazon.com/security

Risk and compliance white paper AWS security processes white paper CSA consensus assessments initiative questionnaire

(requires NDA)



Use comprehensive materials and certifications provided by AWS

Build upon the security features of AWS to implement ‘security by design’


Direct Connect & VPNVirtual Private CloudControl & AuditTiered Access

IAM Control users and allow use IAM Roles to provide API credentials for instances to enable access to

AWS resources via APIs

APIs vs Instance Provide developers with API credentials with separately

controlled access to SSH keys/administrative logins

Temporary Credentials Provide temporary API credentials

for access to AWS resources

Instance firewalls Firewall control on instances via

Security Groups

AWS CloudTrail The AWS API call history recorded

by CloudTrail enables security analysis, resource change

tracking, and compliance auditing

AWS Config A fully managed service that provides you with an AWS

resource inventory, configuration history, and configuration change

notifications to enable security and governance

Subnet control Create low level networking

constraints for resource access, such as public and private

subnets, internet gateways and NATs

Bastion hosts Only allow access for

management of production resources from a bastion host. Turn off when not needed and

restrict startup via MFA

VPC Peering Connect privately to other VPCs-

Peer VPCs together to share resources across multiple virtual networks owned by your or other

AWS accounts.

Private connections to VPC Secured access to resources in AWS over software or hardware VPN and dedicated network links

Because your VPC can be hosted behind your corporate firewall, you

can seamlessly move your IT resources into the cloud without changing how your users access

these applications.

Build on AWS Security Features

Build on the Strengths of the AWS Cloud

4

e.g. Application performance improvement by migration of static content to Amazon S3 & CloudFront

Review application architectures early – assess their fit for the cloud

Can cloud benefits be delivered with minimum effort & outlay?

e.g. variable capacity requirements, ‘standard’ technology stacks, reference architectures*

e.g. Faster development cycles for dev/test, reduced cap-ex for application environmentsWill cloud yield top-line growth, cost savings or agility improvements?

e.g. fully scripted deployments, IAM & EC2 instance roles, rolling deploymentsCan automation lead to a more robust, agile & secure services?


1234

Disposable compute

Design systems that can tolerate instance failures

Scala

bility

Avail

abilit

yCo

st O

ptim

isatio

n


✖ ✖

Dispose of compute when it is not required

✖ ✖

Disposable compute

Flexible capacityDesign systems that can dynamically scale from zero to hundreds of instances

Scala

bility

Avail

abilit

yCo

st O

ptim

isatio

n


✖ ✖ ✖

Use Auto-scaling (events, schedules etc) to drive capacity availability

✖ ✖ ✖

Disposable compute

Flexible capacity

Cost effective storageUse Amazon S3 for durable & cost effective storage

Scala

bility

Avail

abilit

yCo

st O

ptim

isatio

n


✖ ✖ ✖

Deploy & scale relational databases with RDS & use DynamoDB for high throughput NoSQL tables

✖ ✖ ✖

Disposable compute

Flexible capacity

Cost effective storage

Automation and control

Automate everything from deployment, to scaling, to instance recovery from failure

Scala

bility

Avail

abilit

yCo

st O

ptim

isatio

n


✖ ✖ ✖

Create instance for your OS choice

Configure environment

Install software

Create AMI from instance

Launch fully configured instances from AMI

AMICustom machine

image

Instances

Auto-scaling Manual deployments

Programmatic deployments

Bootstrapping - Custom AMIs

12345

ami-idami-launch-indexami-manifest-pathblock-device-mappinghostnameinstance-actioninstance-idInstance-typekernel-id

local-hostnamelocal-ipv4macnetworkplacementprofilepublic-hostnamepublic-ipv4public-keysreservation-id

http://169.254.169.254/latest/meta-data

The metadata service contains & provides information about an instance

Metadata Service

Receive custom data to drive

bootstrapping

Custom or standard machine image

Bootstrapping - Metadata Service

AMI

Instances



Metadata Service

Receive custom data to drive

bootstrapping

Custom or standard machine image


AMI

Instances

+ user dataScripts in user-data field of metadata will be executed on launch For example

#!/bin/shyum-yinstallhttpdchkconfighttpdon/etc/init.d/httpdstart

<powershell> …</powershell>

or




+ user data

Install software e.g. web server, app server, proxy

Pull data and application packages from S3

Publish metadata for instance to other systems e.g. monitoring systems

Setup security profile of instance based upon intended use e.g. pull latest config

1. Use multiple availability zones

2. Use RDS with replicas and slaves

3. Use auto-scaling groups

4. Use Elastic Load Balancing

5. Use Route53 to host DNS zones

Auto-ScalingRDSRoute 53Elastic Load Balancing

Use at regional level Combined with autoscaling will balance requests and resource

capacity across availability zones

Within VPC Use to load balance between

application tiers within an availability zone

Instance migrations Easily move instances from dev

environments to test environments by moving between ELBs

Leverage SLA Improve application reliability with

Route 53’s SLA on requests served

Weighted routing Perform A/B analysis, and staged application roll-outs by moving a

portion of traffic to new infrastructure

Control TTLs and updates Take absolute control of DNS

updates for more decisive system updates

Scale databases without admin overhead

Choose instance size for databases and scale up over time

Add high availability from management console

Create master-slave configurations and read-replicas.

AWS takes care of the failover and recreation of a new slave in event

of master DB loss

Dynamically scale resources & control costs Only provision the resources that

are required with scale up and cool down policies that match

demand


For more details, visit the AWS architecture center: aws.amazon.com/architecture

http://aws.amazon.com/architecture

Services not Software5

AWS CloudInfrastructure & Services

YourBusiness

More Time to Focus onYour Business

Configuring Cloud Services

70%

30%70%

Self Managed Software & Infrastructure

30%

Managing All of the “Undifferentiated Heavy Lifting”

Services Not Software

Relational Database ServiceEasy to set up, operate, and scale Handles time-consuming database management tasks, such as backups, patch management, and replication Supports MySQL, Oracle, Microsoft SQL Server, and PostgreSQL, with Amazon Aurora in preview

NoSQL Database ServiceFast, predictable performance

Supports document & key-value data models Fully distributed, fault tolerant architecture

Amazon RDS

Amazon DynamoDB


Amazon SQS

Processing task/processing trigger

Processing results

Simple Queue ServiceFast, reliable, scalable, fully managed message queuing service Transmit any volume of data, at any level of throughput

Amazon SQS

Amazon EMR

Elastic MapReduceUses Hadoop, an open source framework, to distribute your data and processing across EC2 instances Integrates with other AWS services, such S3 & DynamoDB Supports the broad Hadoop tools ecosystem


Optimise Your Costs6

Use the Right Instance Types

Use Auto Scaling

Turn Off Unused Instances

Use Reserved Instances

1234

Use Spot Instances 5

Use Storage Classes6Offload Your Architecture7Use Services, Not Software8Use Consolidated Billing9Use Cost Management Tools10

G2

GPU enabled

M3

General purpose

Memory optimized

R3

CR1M2

Storage and IO optimized

C4

Compute optimized

C1 CC2

I2

HI1

HS1

CG1M1 C3

Use the Right Instance Types

Linux from $0.013/hour Windows from $0.018/hour

Pay as you go for computing capacity

Low cost and flexibility

Pay only for what you use, no up-front commitments or long-term contracts

Ideal for applications being developed or tested on EC2 for the fist time

Use Cases:

Applications with short term, spiky, or unpredictable workloads;

Application development or testing

On-demand Instances

1 or 3 year terms

Three payment options: All Upfront, Partial Upfront & No Upfront

Cost reduced in comparison to the on-demand purchasing option

Predictable pricing, plus reserved capacity helps to ensure that compute capacity is

available when needed

Use Cases:

Applications with steady state or predictable usage

Applications that require reserved capacity, including disaster recovery

Reserved Instances

Bid on unused EC2 capacity

Name your own price for EC2 computing capacity. Instances will run whenever your

bid exceeds to the current Spot Price

Spot Price varies in real-time based on supply/demand, determined automatically

Cost / Large Scale, dynamic workload handling

Use Cases:

Applications with flexible start and end times, or which can be accelerated with

additional computing capacity

Applications only feasible at very low compute prices

Spot Instances

Instance Purchasing Options

For more details, visit EC2 purchasing options: aws.amazon.com/ec2/purchasing-options/


Use Tools & Frameworks

7

Access everything via CLI, API or Console

Use one of 9 (soon to be 10) fully supported SDKs to create or make use of existing AWS resources within your own code

Leverage a broad ecosystem of open source, free and commercially licensed tools to work with AWS Services

Achieve the highest levels of automation to support continuous deployment, define your infrastructure-as-code or automate your development, operations or DevOps processes

Find out more at: aws.amazon.com/developers/getting-started/

Everything is Programmable

https://aws.amazon.com/developers/getting-started/

https://aws.amazon.com/developers/getting-started/

AWS Deployment & Management Tools

AWS Elastic Beanstalk

AWS OpsWorks

AWS CloudFormation

AWS CodeDeploy

Get Supported8

Get Supported: AWS Support Options

Four Support Tiers are Available.

Chose from:

Basic Developer Business Enterprise

For more details on AWS Support, visit: aws.amazon.com/premiumsupport


Get Supported: Trusted Advisor

Operating systems on EC2 instances:

Ubuntu Server Red Hat Enterprise Linux and Fedora SUSE Linux (SLES and openSUSE) CentOS Linux Microsoft Windows Server 2003 R2 Microsoft Windows Server 2008 Microsoft Windows Server 2008 R2 Microsoft Windows Server 2012

Infrastructure components:

Sendmail and Postfix MTAs OpenVPN and RRAS SSH, SFTP, and FTP LVM and Software RAID

Web servers:

Apache IIS Nginx

Databases:

MySQL Microsoft SQL Server

Get Supported: 3rd Party Software

For more details on AWS Support, visit: aws.amazon.com/premiumsupport


Resources You Can Use to Learn More

aws.amazon.com/getting-started/

aws.amazon.com/premiumsupport

aws.amazon.com/architecture

aws.amazon.com/security

aws.amazon.com/campaigns/emea-getting-started


http://aws.amazon.com/premiumsupport


http://aws.amazon.com/security

http://aws.amazon.com/campaigns/emea-getting-started

Certification

aws.amazon.com/certification

Self-Paced Labs

aws.amazon.com/training/self-paced-labs

Try products, gain new skills, and get hands-on

practice working with AWS technologies

aws.amazon.com/training

Training

Validate your proven skills and expertise with the

AWS platform

Build technical expertise to design and operate

scalable, efficient applications on AWS

AWS Training & Certification

http://aws.amazon.com/certification

http://aws.amazon.com/training/self-paced-labs/

http://aws.amazon.com/training/self-paced-labs/

http://aws.amazon.com/training/

Follow us fo

r more

events

& webina

rs

@AWScloud for Global AWS News & Announcements

@AWS_UKI for local AWS events & news

@IanMmmmIan Massingham — Technical Evangelist

best practices for getting started session

Technology