Leveraging AWS for Lean Development Lifecycle

Ryan Shuttleworth – Technical Evangelist @ryanAWS

What is the “lean development cycle”?

Fast

Flexible Pivot

Iterate Agility

Code not documentation

MVP Sprint Working software

Continuous integration

Idea Product

Idea Product

Least friction

Highest quality

Idea Product

A Startup Journey (your experiences may vary, but the

following is based on real experiences…)

“Democracy is the art of running the circus from

the monkey cage.”

H. L. Mencken (1880 - 1956)

“Democracy is the art of running the circus from

the monkey cage.”

H. L. Mencken (1880 - 1956)

Outside it’s nice and neat: Have a picnic on the lawn at the zoo

Inside it’s chaos: Screaming, swinging, banana-skins

“Democracy is the art of running the circus from

the monkey cage.”

H. L. Mencken (1880 - 1956)

Outside it’s nice and neat: Have a picnic on the lawn at the zoo

Inside it’s chaos: Screaming, swinging, banana-skins

Sound like a startup? Let everyone feel safe and comfortable looking at you from the outside

“Democracy is the art of running the circus from

the monkey cage.”

H. L. Mencken (1880 - 1956)

Outside it’s nice and neat: Have a picnic on the lawn at the zoo

Inside it’s chaos: Screaming, swinging, banana-skins

Sound like a startup? Let everyone feel safe and comfortable looking at you from the outside Manage the chaos on the inside

“Democracy is the art of running the circus from

the startup monkey cage.”

H. L. Mencken (1880 - 1956)

“Democracy is the art of running the circus from

the startup monkey cage.” (and why the AWS cloud is the perfect place for one)

The idea… 1 You've got a cracking idea, but don't know if there is point or a market

The idea… 1 You've got a cracking idea, but don't know if there is point or a market

1x free tier instance Zero cost

What

AWS

Take-away

CloudFormation a Wordpress site Go buy a $30 template Make it look real

Wait and see what traffic/interest you get

The idea… 1 You've got a cracking idea, but don't know if there is point or a market

http://aws.amazon.com/cloudformation/aws-cloudformation-templates/

2 Analytics prove you have a point, web hits increase traffic. A lot.

Proving it…

2 Analytics prove you have a point, web hits increase traffic. A lot.

CDN your marketing presence Easy, 60 minute update You look global

What

AWS

Take-away You now have interest Keep it ticking over Define the Minimum Viable Product (you can get away with)

Proving it…

They came, but then they go because it's too slow to respond They came, signed up for a beta, proving you might actually need to build something…

3 You’ve got some people to give something to, so give them something

Customers…

3 You’ve got some people to give something to, so give them something

Beanstalk – sow the seed, don’t worry about complex operations yet Multi-AZ – small instances, but highly available

What

AWS

Take-away

First phase application – make sure you launch it on a positive SLA Features may be lacking, but it needs to be up!

Deliver something simple, deliver it well Keep those early adopters happy

Customers…

3 You’ve got some people to give something to, so give them something

Beanstalk – sow the seed, don’t worry about complex operations yet Multi-AZ – small instances, but highly available

What

AWS

Take-away

First phase application – make sure you launch it on a positive SLA Features may be lacking, but it needs to be up!

Deliver something simple, deliver it well Keep those early adopters happy

Customers…

Application package is deployed into Beanstalk

Elastic Beanstalk

CloudFormation

Tools

User Application

Application Service

HTTP Service

Language Interpreter

Operating System

Host

Which creates the container in EC2

Elastic Beanstalk

CloudFormation

Tools

Beanstalk takes care of the environment…

Elastic Beanstalk

CloudFormation

Tools

…adding Elastic Load Balancer…

Elastic Beanstalk

CloudFormation

Tools

…Auto Scaling groups

Elastic Beanstalk

CloudFormation

Tools

…and launching instances

Elastic Beanstalk

CloudFormation

Tools

…all wired into the Beanstalk framework

Elastic Beanstalk

CloudFormation

Tools

…and published under a CNAME

Elastic Beanstalk

CloudFormation

Tools

…with logs and app versions held in S3

Elastic Beanstalk

CloudFormation

Tools

4 Inevitably, 3am on a Saturday night, things start to break….

Stuff breaks….

4 Inevitably, 3am on a Saturday night, things start to break….

You started on small instances – up them. You’ve plenty to play with

What

AWS

Take-away

Your database grew – you haven’t optimised it Your application is naïve – it doesn’t cache anything

Engineering doesn’t happen overnight With AWS you bought some time – 4-6wks on the clock to make life easy again

Stuff breaks….

5 Got customers, demand is growing and now have functionality to deliver

Survival…

5 Got customers, demand is growing and now have functionality to deliver

Reserve instances – reduce some cost now Build dev-test disposable environments - Need to test things before we deliver (Beanstalk, CloudFormation)

What

AWS

Take-away

Optimise the data layer Size instance types properly

We bought time to pay down technical debt Reduced costs, started to act like a real outfit

Survival…

6 Cold sweats at 2.30am – service outage – we could have lost everything!

Resilience…

6 Cold sweats at 2.30am – service outage – we could have lost everything!

RDS resiliency S3 snapshots and EBS/AMI management

What

AWS

Take-away

How resilient is my RDS setup? What are my snapshot strategies? What is my point in time recovery period for RDS?

Few clicks (and a little more cost) and it’s taken care of 35 days of backup history on RDS Slave failover - automatic

Resilience…

7 You have customers from the real world, and they are EVERYWHERE

Scale…

7 You have customers from the real world, and they are EVERYWHERE

CDN static content served from application CMSs CDN dynamic content where you can, play with very low TTLs Review Azs, ASGs for origin servers

What

AWS

Take-away

Now your application is your bottleneck App hits are very high, database is fine Do the bare minimum of heavy lifting in your application

Push load away from your core application Again, buy time (spend more time on features) Keeping reviewing so you don’t accrue too much technical debt

Scale…

9 You grew your service, now focus on ‘supportability ratio’

Automate…

9 You grew your service, now focus on ‘supportability ratio’

CLI, APIs CloudFormation, CloudInit

What

AWS

Take-away

More customers to less staff is better – the ratio Focus on tools and automation to keep your team lean Build a DevOps team (you have one by now anyway!)

No more 2.30am problems DevOps from iPhone (on a Friday night at the pub)

Automate…