Zero To Cloud (OSCon 2014)

Download Zero To Cloud (OSCon 2014)

Post on 13-Jul-2015

144 views

Category:

Engineering

0 download

TRANSCRIPT

  • Zero to Cloud with @NetflixOSS

    Justin Ryan (@quidryan)

  • Who We Are Justin Ryan Michael Tripoli Various other Netflix Employees in the Audience

    #netflixoss

  • #netflixoss

    Netflix is the world's leading Internet television network with over 48 million members in more than 40 countries enjoying more than one billion hours of TV shows and movies per month, including original series. For a low monthly price, Netflix members can watch as much as they want, anytime, anywhere, on nearly any Internet-connected screen. Members can play, pause and resume watching, all without commercials or commitments. (Credit: ir.netflix.com)

  • Tutorial Format 30 Minutes of Lecture 30 Minutes on Console 30 Minutes on Jumphost 30 Minutes deploying Asgard 30 Minutes Edda and Eureka 30 Minutes Karyon

    #netflixoss

  • Tutorial Purpose Help adoption of @NetflixOSS components Answer questions along the way Less trail blazing for us

    #netflixoss

  • Assumptions Stay Local (using us-west-2) Non-destructive operations Follow instructions closely Security through obscurity Not a developing for the cloud tutorial This will cost money

    #netflixoss

  • #netflixoss

    http://nflx.it/zerotocloud

  • #netflixoss

    Step 1 - Sign Up for AWS

  • Network

    #netflixoss

    Jumphost

    Laptop

    Asgard InstanceInstanceInstances

    InstanceInstanceInstancesElastic Load

    Balancer

    Elastic Load

    Balancer

    SSH

    HTTP

  • #netflixoss

    Step 2 - AWS Console

  • #netflixoss

    Step 3 - Key Pair

  • #netflixoss

    Step 4 - Create Jumphost

  • #netflixoss

    Step 5 - Create Role

  • #netflixoss

    Step 6 - Create User

  • Cloud Deployment Model

    #netflixoss

    Auto Scaling Group

    Elastic Load

    Balancer

    Launch Configuration

    Security Group

    Amazon Machine Image

    Configuration

    Runtime

    InstanceInstanceInstances

  • #netflixoss

    Step 7 - Security Group

  • Immutable Infrastructure Simplifies Operations Baking with Aminator

    #netflixoss

    Foundation

    Base AMI

    Application

  • #netflixoss

    Step 8 - Foundation AMI

  • #netflixoss

    Step 9 - Setup Jumphost

  • #netflixoss

    Step 10 - Credentials

  • Building Bake a system package (RPM, DEB) Use ospackage plugin to generate

    #netflixoss

    Foundation

    Base AMI

    Application

    WAR DEBJava

    Python

  • #netflixoss

    Step 11 - Base AMI

  • netflix.github.io

    #netflixoss

  • @netflixoss

    #netflixoss

  • Who Platform Infrastructure Cloud Database Engineering Cloud Solutions Engineering Tools Cloud and Platform Engineering (Anyone with code thats not a competitive advantage)

    #netflixoss

  • What Infrastructure Caching Database Cloud Building Deployment

    !

    #netflixoss

    Testing Monitoring Networking Robustness Security Configuration

  • Not Streaming Encoding Merchandising Movie metadata Recommendations 10 UI

    #netflixoss

  • Open source infrastructure Github Cloudbees for Builds http://techblog.netflix.com/ http://twitter.com/netflixoss Apache 2.0 license (when possible) Meetups - Season 2 going on right now!

    #netflixoss

  • #netflixoss

    Step 12 - Bake Asgard

  • Islands of Applications

    #netflixoss

    Search Auto-complete Ratings

    Signup API Streaming Starts

  • Islands of Applications

    #netflixoss

    Search Auto-complete Ratings

    Signup API Streaming Starts

    Application Application Application

    Application Application Application

  • Inventing the Application Problem

    - Application is not an Amazon concept Solution

    - Create an Application domain in SimpleDB - Enforce naming conventions on Amazon objects

    #netflixoss

  • Red/Black Deploy

    #netflixoss

    Elastic Load

    Balancer

    InstanceInstanceInstances

    api-usprod-v007

    InstanceInstanceInstances

    api-usprod-v008

  • Fast Rollback

    #netflixoss

    Elastic Load

    Balancer

    InstanceInstanceInstances

    api-usprod-v007

    InstanceInstanceInstances

    api-usprod-v008

  • Inventing the Cluster Problem

    - Two ASGs on purpose - Different names Solution

    - Maintain same prefix - Append version - Derive Cluster from ASG names

    #netflixoss

  • Asgard vs Console Asgard

    - Provide Application and Cluster - Simplifies object construction - Introduce workflow - Tightly couple with internal systems Console

    - Access to all objects

    #netflixoss

  • #netflixoss

    Step 13 - Asgard/Asgard

  • #netflixoss

    Step 14 - Deploy Edda

  • Mid-tier Calling

    #netflixoss

    EurekaData Plane

    Control Plane

    InstanceInstanceApp A InstanceInstanceApp B

  • #netflixoss

    Step 14 - Deploy Eureka

  • #netflixoss

    Extra - Karyon

  • Gift Cards

    #netflixoss

    Thanks,

    Amazon!

  • Thanks!Justin Ryan (@quidryan)

    slideshare.net/quidryan