zero to cloud (oscon 2014)

Zero to Cloud with @NetflixOSS

Justin Ryan (@quidryan)

Who We Are• Justin Ryan • Michael Tripoli • Various other Netflix Employees in the Audience

#netflixoss

#netflixoss

Netflix is the world's leading Internet television network with over 48 million members in more than 40 countries enjoying more than one billion hours of TV shows and movies per month, including original series. For a low monthly price, Netflix members can watch as much as they want, anytime, anywhere, on nearly any Internet-connected screen. Members can play, pause and resume watching, all without commercials or commitments. (Credit: ir.netflix.com)

Tutorial Format• 30 Minutes of Lecture • 30 Minutes on Console • 30 Minutes on Jumphost • 30 Minutes deploying Asgard • 30 Minutes Edda and Eureka • 30 Minutes Karyon

#netflixoss

Tutorial Purpose• Help adoption of @NetflixOSS components • Answer questions along the way • Less trail blazing for us

#netflixoss

Assumptions• Stay Local (using us-west-2) • Non-destructive operations • Follow instructions closely • Security through obscurity • Not a “developing for the cloud” tutorial • This will cost money

#netflixoss

#netflixoss

http://nflx.it/zerotocloud

#netflixoss

Step 1 - Sign Up for AWS

Network

#netflixoss

Jumphost

Laptop

Asgard InstanceInstanceInstances

InstanceInstanceInstancesElastic Load

Balancer

Elastic Load

Balancer

SSH

HTTP

#netflixoss

Step 2 - AWS Console

#netflixoss

Step 3 - Key Pair

#netflixoss

Step 4 - Create Jumphost

#netflixoss

Step 5 - Create Role

#netflixoss

Step 6 - Create User

Cloud Deployment Model

#netflixoss

Auto Scaling Group

Elastic Load

Balancer

Launch Configuration

Security Group

Amazon Machine Image

Configuration

Runtime

InstanceInstanceInstances

#netflixoss

Step 7 - Security Group

Immutable Infrastructure• Simplifies Operations • Baking with Aminator

#netflixoss

Foundation

Base AMI

Application

#netflixoss

Step 8 - Foundation AMI

#netflixoss

Step 9 - Setup Jumphost

#netflixoss

Step 10 - Credentials

Building• Bake a system package (RPM, DEB) • Use ospackage plugin to generate

#netflixoss

Foundation

Base AMI

Application

WAR DEBJava

Python

#netflixoss

Step 11 - Base AMI

netflix.github.io

#netflixoss

@netflixoss

#netflixoss

Who• Platform Infrastructure • Cloud Database Engineering • Cloud Solutions Engineering Tools • Cloud and Platform Engineering • (Anyone with code that’s not a competitive advantage)

#netflixoss

What• Infrastructure • Caching • Database • Cloud • Building • Deployment

!

#netflixoss

• Testing • Monitoring • Networking • Robustness • Security • Configuration

Not• Streaming • Encoding • Merchandising • Movie metadata • Recommendations • 10’ UI

#netflixoss

Open source infrastructure• Github • Cloudbees for Builds • http://techblog.netflix.com/ • http://twitter.com/netflixoss • Apache 2.0 license (when possible) • Meetups - Season 2 going on right now!

#netflixoss

#netflixoss

Step 12 - Bake Asgard

Islands of Applications

#netflixoss

Search Auto-complete Ratings

Signup API Streaming Starts

Islands of Applications

#netflixoss

Search Auto-complete Ratings

Signup API Streaming Starts

Application Application Application

Application Application Application

Inventing the Application• Problem

- Application is not an Amazon concept • Solution

- Create an Application domain in SimpleDB - Enforce naming conventions on Amazon objects

#netflixoss

Red/Black Deploy

#netflixoss

Elastic Load

Balancer

InstanceInstanceInstances

api-usprod-v007

InstanceInstanceInstances

api-usprod-v008

Fast Rollback

#netflixoss

Elastic Load

Balancer

InstanceInstanceInstances

api-usprod-v007

InstanceInstanceInstances

api-usprod-v008

Inventing the Cluster• Problem

- Two ASGs on purpose - Different names • Solution

- Maintain same prefix - Append version - Derive Cluster from ASG names

#netflixoss

Asgard vs Console• Asgard

- Provide Application and Cluster - Simplifies object construction - Introduce workflow - Tightly couple with internal systems • Console

- Access to all objects

#netflixoss

#netflixoss

Step 13 - Asgard/Asgard

#netflixoss

Step 14 - Deploy Edda

Mid-tier Calling

#netflixoss

EurekaData Plane

Control Plane

InstanceInstanceApp A InstanceInstanceApp B

#netflixoss

Step 14 - Deploy Eureka

#netflixoss

Extra - Karyon

Gift Cards

#netflixoss

Thanks, Amazon!

Thanks!Justin Ryan (@quidryan)

slideshare.net/quidryan