why startups are still on aws

WHY STARTUPS ARE (STILL)

ON AMAZON WEB SERVICES

Danilo Poccia | Solutions Architect

Startups?

http://www.flickr.com/photos/klearchos/5857451393/

« A startup is a human institution

designed to create a new

product or service under

conditions of extreme

uncertainty. »

Eric Ries, The Lean Startup

Startups fail?

Assume customer is known

Assume features are known

Assume solution is known

Waterfall approach You know the problem and the solution

TEST LAUNCH BUILD SPEC

Known set of

requirements

Known ways to

satisfy them

Agile methodologies You know the problem, not the solution

TEST LAUNCH BUILD SPEC

Known set of

requirements

Unknown ways

to satisfy them

VALID?

You don't know precisely

what problem you solve

Lean startups:

LEARN & ADAPT

1. Focus on a simple implementation

of your idea

1. Focus on a simple implementation of

your idea

2. Start with a minimal core set of

features


your idea


features

3. Release and listen to your users


your idea


features


4. Question your initial assumptions

based on feedback


your idea


features


4. Question your initial assumptions

based on feedback

5. Rinse and repeat

MVP Minimum Viable Product

MVP Smallest thing I can do to test my idea?

« If you're not embarrassed

when you ship your first version

you waited too long »

Matt Mullenweg

CEO & Founder of WordPress

Staying lean is creating the

smallest viable product and

then iterate around it You don't know precisely your user's needs

RELEASE EVAL BUILD IDEA

Unknown set of

requirements

Unknown ways

to satisfy them

ITERATE OR

PIVOT

RELEASE EVAL BUILD IDEA ITERATE

OR

PIVOT

YOUR problem you have the idea

you don't have the resources

It should be cheap and validate ideas

SPEED AND AGILITY

Experiment

Often

Fail quickly

at a low cost

More

Innovation

Experiment

Infrequently

Failure is

expensive

Less

Innovation

“ON-PREMISE”


OR

PIVOT

you created a fantastic webapp

http://www.flickr.com/photos/scobleizer/3985020876/

people love your app

http://www.flickr.com/photos/grantrobertson/448648883

1

and everyone wants to use it!

http://www.flickr.com/photos/livingos/4480894461/

the traditional way…

invest on infrastructure

capacity planning…

http://www.flickr.com/photos/mutsmuts/4695658106

/

…capacity guessing

once it's deployed…

maintenance?

monitoring?

log analysis?

test environments?

http://www.old-computers.com/news

Unable to

serve

customers

Infrastructure

Cost $

time

Large

Capital

Expenditure

Opportunity

Cost

Predicted

Demand

Traditional

Hardware

Actual

Demand

Automated

Virtualization

with AWS…

“When we started musiXmatch in 2010, we wanted

to focus on building a great user experience for our

users and our customers", says Francesco Delfino,

cofounder, "We choose Amazon Web Services

because it allowed us to freely define and fine tune

our server architecture, while shielding us from

common hardware issues.

During these years, AWS sustained our growth

providing the resources we needed as soon as our

traffic increased: there is no bigger cost for a startup

than missing the opportunity to scale exactly when

you need it.”

“Amazon Web Services gives us the

power to scale our infrastructure

without worrying about capacity limits.

Our infrastructure is distributed and

currently runs on 3 regions.”

www.spreaker.com


OR

PIVOT

Data based

decision making

1. Collect as much data as you can


2. Do A/B testing


2. Do A/B testing

3. Drive your development by

user's feedback

Store valuable data sources Server logs, click streams, application events, …

"Hadoop is a reliable storage and data analysis system"

HDFS MapReduce

Deploying a Hadoop cluster is hard

http://eddie.niese.net/20090313/dont-pity-incompetence/

Amazon Elastic

MapReduce Hadoop + The AWS Cloud

Elastic Data Warehouse

Expand to

25 instances

Data Warehouse

(Steady State)

Data Warehouse

(Batch Processing)

Shrink to

9 instances

Data Warehouse

(Steady State)

3.5 billion records, 71 million unique cookies, 1.7 million targeted ads required per day

Targeted Ad

User recently

purchased a

sports movie and

is searching for

video games (1.7 Million per day)

Leveraged AWS and Elastic MapReduce 100 node cluster on demand

Processing time dropped from 2+ days to 8 hours

Increased ROAS by 500%

We use EC2, Auto

Scaling and Elastic Load

Balancing to deliver ads

along side our main

infrastructure.

We automatically absorb

peaks through scripts

monitoring our local farm,

and change entries in the

Route 53 (DNS) when

needed, allowing more

traffic towards AWS load

balancers.

www.dotandmedia.com


OR

PIVOT

Iterate Enhance your product, get more feedback

to prepare next releases

The platform grows with you Add or remove components as needed

Scale up – scale down

Pay as you go

Pivot If no traction, pivot to address a different

vertical, a different problem

POSSIBLE

OFFERING

POSSIBLE

OFFERING

POSSIBLE

OFFERING

POSSIBLE

OFFERING

PROBLEM

SPACE

POSSIBLE

OFFERING

POSSIBLE

OFFERING

POSSIBLE

OFFERING

PROBLEM

SPACE

POSSIBLE

OFFERING

Getting the right features matters less

Knowing what's not working,

quickly, matters more

Experiment and

Innovate

Lean startup goal is to accelerate the cycle

CONTINUOUS

INTEGRATION

DEVELOPER

SOURCE CODE

REPOSITORY

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

PICK

TASKS

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

SUBMIT

CODE

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

SCHEDULE

BUILD

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

RECURENT

BUILDS

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

CODE

FETCH

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

CODE QUALITY

TESTS

TEST

RESULTS

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

BUILD OUTPUT

SOURCE CODE

REPOSITORY

PROJECT MANAGEMENT

SERVER

CONTINUOUS

INTEGRATION SERVER

DOCS

BINARIES /

PACKAGES

SOURCE CODE

REPOSITORY

DNS

CONTINUOUS

INTEGRATION SERVER

PROJECT

MANAGEMENT SERVER

BUILDS

PAIN POINTS: • UNIT TESTS INCOMPLETE

• MOCKS MAINTENANCE

• TEST ENV EXPENSIVE

• TEST ENV ≠ PROD

KEY = ITERATION

ITERATIVELY MODIFY THE

SYSTEM TO BETTER MEET

THE EXPECTATIONS OF

YOUR USERS

ON-DEMAND

PAY AS YOU GO

ELASTIC

AWS

CLOUDFORMATION STACK-BASED DEPLOYMENT

SERVICE

CLOUDFORMATION

TEMPLATE

{

"Description" : "Create RDS with username and

password",

"Resources" : {

"MyDB" : {

"Type" : "AWS::RDS::DBInstance",

"Properties" : {

"AllocatedStorage" : "500",

"DBInstanceClass" : "db.m1.small",

"Engine" : "MySQL",

"EngineVersion" : "5.5",

"MasterUsername" : "MyName",

"MasterUserPassword" : "MyPassword"

}

}

}

}

"AWS::CloudFormation::Init" : {

"config" : {

"packages" : {

"yum" : {

"mysql" : [],

"mysql-server" : [],

"httpd" : [],

"php" : [],

"php-mysql" : []

}

},

"sources" : {

"/var/www/html" :

"https://s3.amazonaws.com/my-builds/build-

v4.zip"

}

}

}

{

"Parameters" : {

"KeyName" : {

"Description" : "Name of an existing EC2

KeyPair to enable SSH

access to the instance",

"Type" : "String"

}

},

}

CLOUDFORMATION

TEMPLATE

PROCEDURAL

DEFINITION Create it programmatically

KNOWN

CONFIGURATION Store stack configuration in

source control

PARAMETER

DRIVEN Dynamic and user-driven

templates

COLLABORATION Share templates with ease

as just files

Template ELBs to front

secondary cache

~100 Nginx

secondary cache

servers

2-3 Nginx mid-tier

cache servers

Stack

CLOUDFORMATION TEMPLATE

VIDEO CACHING

INFRASTRUCTURE

APPLICATION

VERSIONS

& INFRASTRUCTURE

VERSIONS

CLOUDFORMATION

TEMPLATE

TEST ENVIRONMENTS

30,000 REQUESTS / SECOND

1 TB TRAFFIC / DAY

“…AWS seemed to be the best solution available to allow a small, independent company to rapidly develop and test a completely new infrastructure,

and host it.

We also loved the flexibility that AWS allowed us, when spinning up smaller test environments, for

beta trials, QA, localization, and during development. The low initial cost was also

crucial.”

Alex Evans, CTO

CONTINUOUS

INTEGRATION

CONTINUOUS

DEPLOYMENT

CONTINUOUS

DEPLOYMENT SMALL, FREQUENT CHANGES

CONSTANTLY INTEGRATING INTO

PRODUCTION.

SOFTWARE DEPLOY

≠ PRODUCT LAUNCH

1.5 BILLION PAGE VIEWS

OCTOBER 2012

$83 MILLION IN TRANSACTIONS

4.2 MILLION ITEMS SOLD

30 DEPLOYS PER DAY = 1 DEPLOY EVERY 20 MINUTES

« Complexity arises when the

dependencies among the

elements become important. »

Scott E. Page, John H. Miller,

Complex Adaptive Systems: An Introduction to

Computational Models of Social Life

“Production is truly the only place

you can validate your code.”

HTTPS://GITHUB.COM/ETSY/DEPLOYINATOR

HTTP://CODEASCRAFT.ETSY.COM/

HTTP://SORCERY.SMUGMUG.COM/

AWS OPSWORKS MANAGING THE COMPLETE

APPLICATION LIFECYCLE

A stack represents

your application. One

stack might be used

for staging and

another for production.

A layer defines how to

setup and configure a

set of instances and

related resources

such as volumes

and software.

Tell OpsWorks where

it can find your code

and define any

additional deployment

tasks. OpsWorks will

take care of deploying

your app.

Scale your stack

based on time or

load. Clone your

production stack to a

different region.

Automate workflows

for common tasks.

STACK | LAYER | APP | INSTANCE

GETTING STARTED WITH OPSWORKS

YOUR STACKS IN THE DASHBOARD

STACK OVERVIEW

LAYERS IN A STACK

INSTANCES IN A STACK

APPS IN A STACK

DEPLOYMENTS & COMMANDS

YOU CAN BRING YOUR

OWN CHEF RECIPES OR

LEVERAGE HUNDREDS

OF COMMUNITY-BUILT

CONFIGURATIONS

14 BILLION REQUESTS/MONTH

50 000 DATABASE UPDATES / SEC

NO CACHE

“AWS OpsWorks gives us the tools we

need to automate operations.

We can scale Monster World, one of the

largest Facebook games, to millions of

users without ever needing more than two

backend developers”

Jesper Richter-Reichhelm

Head of engineering – Wooga

A / B TESTING

LOAD TESTING

USING AMAZON EC2 TO SIMULATE

2.4 MILLION PLAYERS

DATA-DRIVEN

ARCHITECTURES

METRICS @ETSY

COST-ORIENTED

ARCHITECTURES

PHP+APACHE+VARNISH

NGINX+NODEJS

11.6s

Mean time

between

deployments

(weekday)

1,079

Max number of

deployments in a

single hour

10,000

Mean number of

hosts

simultaneously

receiving a

deployment

30,000

Max number of

hosts

simultaneously

receiving a

deployment

DEPLOYMENTS AT

AMAZON.COM

CONTINUOUS

DEPLOYMENT

=

CONTINUOUS

EXPERIMENTATION

CONTINUOUS

DEPLOYMENT

=

CONTINUOUS

IMPROVEMENT

FASTEST TIME TO

MARKET

www.chili-tv.it

http://www.chili-tv.it/



“The AWS Cloud brings business agility as Shell is able to deploy services much more quickly”

Johan Krebbers

Vice President of Architecture

“AWS has enabled Soundtracker to

perform nimble development” says

Daniele Calabrese, Founder and CEO,

“allowing our service to scale quickly,

effectively, and seamlessly.”

focus on your business

Stefano Pochet, founder and CEO of Nealab, explains,

“Amazon Web Services fits the need for flexibility and

reliability required by high-traffic Web applications. In

addition, it makes it easier and cheaper to maintain our

entire infrastructure.”

…

“AWS has allowed us to forget about hardware and

focus on software. For us, AWS has really made Web

development easier.”

why startups are still on aws

Technology