choosing the right framework for running docker containers in prod

146
Choosing the Right Framework for Running Docker Containers in Prod Presented by: Josh Padnick Phoenix DevOps

Upload: josh-padnick

Post on 11-Jan-2017

8.211 views

Category:

Software


0 download

TRANSCRIPT

Page 1: Choosing the Right Framework for Running Docker Containers in Prod

Choosing the Right Framework for Running

Docker Containers in ProdPresented by: Josh Padnick

Phoenix DevOps

Page 2: Choosing the Right Framework for Running Docker Containers in Prod

Docker is a game-changer.

Page 3: Choosing the Right Framework for Running Docker Containers in Prod

Why Devs • Lightweight

Containers are just isolated processes. We can start a new container in seconds.

• Portable My Mac, the linux EC2 instance, and your Windows PC all run the exact same container.

• EcosystemI can easily share images, manage private images, and use “official” images for virtually all open source software.

Page 4: Choosing the Right Framework for Running Docker Containers in Prod

Why Devs • Squeeze More Resources out of a Single Server

Did you know this dirty secret of the Infrastructure-as-a-Service world?

85%

15%

In Use Free

SOURCE: http://radar.oreilly.com/2014/12/why-the-data-center-needs-an-operating-system.html

Typical Data Center Resource Utilization

Page 5: Choosing the Right Framework for Running Docker Containers in Prod

So can I run multiple containers in a single VM?

Page 6: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Something like this?

Page 7: Choosing the Right Framework for Running Docker Containers in Prod

Yes! But, well…

Page 8: Choosing the Right Framework for Running Docker Containers in Prod

The Gartner Tech HypecycleAny guesses where the “multi-container VM” paradigm is?

SOURCE: https://setandbma.wordpress.com/2012/05/28/technology-adoption-shift/

Page 9: Choosing the Right Framework for Running Docker Containers in Prod

My Take on ThisIn reality, the exact spot varies by team,

so this is a bit of a generalization.

Page 10: Choosing the Right Framework for Running Docker Containers in Prod

Today’s talk is about our options for that red dot.

For each option, we’ll cover:

• Pro’s • Con’s • When to use

Page 11: Choosing the Right Framework for Running Docker Containers in Prod

• Full-stack web-app engineer for 12+ years.

• Since I’ve worked with many different teams, I generally help accelerate the DevOps/AWS learning curve for teams.

• PhxDevOps Clients include: Intel, Infusionsoft, American Bible Society, CÜR Music, plus multiple startups and web design companies.

Josh Padnick

These slides are posted on http://joshpadnick.com

I help software teams scale their app using DevOps and AWS.http://PhoenixDevOps.com

@OhMyGoshJosh

My LinkedIn

My GitHub

Want to know more about building scalable apps on AWS? Check out a 12,000+ word article I wrote on how at http://bit.ly/1EtYRbL.

Page 12: Choosing the Right Framework for Running Docker Containers in Prod

Disclaimers• I have a bias toward AWS and may leave

out solutions from other IaaS providers such as Azure.

• The solutions we cover today are deep and diverse. This talk reflects my own experiences but your mileage may vary!

Page 13: Choosing the Right Framework for Running Docker Containers in Prod

Agenda• CoreOS in 60 seconds

• Theory of Multi-Container VM’s

• The Three Paradigms of Multi-Container VM’s

• Cover all the Major Solutions

Page 14: Choosing the Right Framework for Running Docker Containers in Prod

We only have 60 minutes. So we’ll move fast.

Page 15: Choosing the Right Framework for Running Docker Containers in Prod

CoreOS in 60 Seconds

Page 16: Choosing the Right Framework for Running Docker Containers in Prod

What is ? • It’s super stripped-down linux. You don’t even

get a package manager.

• The idea is you run everything as a container.

• CoreOS is based on ChromiumOS, which itself is based on Gentoo Linux.

• Uses systemd for init.

Page 17: Choosing the Right Framework for Running Docker Containers in Prod

CoreOS and This Presentation• Because CoreOS is “built for Docker”, many

solutions use it as their default OS.

• In reality, you can usually use any OS that runs Docker natively, but CoreOS is often the “recommended” linux distro for Docker.

Page 18: Choosing the Right Framework for Running Docker Containers in Prod

Theory ofMulti-Container VM’s

Page 19: Choosing the Right Framework for Running Docker Containers in Prod

What does it take to run this?

VM-1 VM-2

Service A Service AService B

Service C

Service B

Page 20: Choosing the Right Framework for Running Docker Containers in Prod

Docker BuilderWe need

somewhere to build our image.

Page 21: Choosing the Right Framework for Running Docker Containers in Prod

• If we build from a fresh environment each time, every Docker image/layer is downloaded from scratch.

• Ideally, our “Docker Builder” has pre-downloaded (“seeded”) all our most popular Docker images.

• In practice, this is managed by your build tool, like Jenkins, CircleCI, Shippable, etc.

Docker BuilderWe need

somewhere to build our image.

Page 22: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Docker BuilderWe need

somewhere to build our image.

Page 23: Choosing the Right Framework for Running Docker Containers in Prod

• Main options here are:

• Cloud

• Docker Hub

• Quay.io

• On-Premise

• Docker Trusted Registry (Paid)

• Docker Distribution (Free)

• Quay.io (Paid)

Docker RegistryWe need

somewhere to store our built

images.

Docker BuilderWe need

somewhere to build our image.

Page 24: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Docker BuilderWe need

somewhere to build our image.

Page 25: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 26: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Page 27: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

“Increase Service C container count from 1 to 2.”

Page 28: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Service C

Page 29: Choosing the Right Framework for Running Docker Containers in Prod

Docker BuilderWe need

somewhere to build our image.

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Container SchedulingWe need something to decide which host will

launch our new container?

Page 30: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Container SchedulingWe need something to decide which host will

launch our new container?• One of the most important

considerations when choosing a host is “who’s got the memory and CPU I need?”

• But we also need to know who’s in a different Availability Zone / Data Center so we can achieve high fault tolerance.

Docker BuilderWe need

somewhere to build our image.

Page 31: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Routing / Load BalancingWe need a way to route a

request to any of our containers.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 32: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Page 33: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

GET ServiceB HTTP/1.1

Page 34: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Routing / Load Balancing Solution

Page 35: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Routing / Load BalancingWe need a way to route a

request to any of our containers.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 36: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Service DiscoveryWhen we launch new

containers, we need to tell our router they exist.

Routing / Load BalancingWe need a way to route a

request to any of our containers.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 37: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Routing / Load Balancing Solution

Page 38: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Routing / Load Balancing Solution

“Increase Service C container count from 1 to 2.”

Page 39: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Routing / Load Balancing Solution

Service C

Page 40: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Routing / Load Balancing Solution

Service C

Page 41: Choosing the Right Framework for Running Docker Containers in Prod

VM-1 VM-2

Service A Service AService B

Service C

Service B

Routing / Load Balancing Solution

Service C

Page 42: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Service DiscoveryWhen we launch new

containers, we need to tell our router they exist.

Routing / Load BalancingWe need a way to route a

request to any of our containers.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 43: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Auto-Restart Failed Containers

Something needs to know that a container failed and

auto-restart it.

Service DiscoveryWhen we launch new

containers, we need to tell our router they exist.

Routing / Load BalancingWe need a way to route a

request to any of our containers.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 44: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Auto-Restart Failed Containers

Something needs to know that a container failed and

auto-restart it.

Extract Container LogsWe need a way to read logs

from all containers.Service Discovery

When we launch new containers, we need to tell

our router they exist.

Routing / Load BalancingWe need a way to route a

request to any of our containers.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 45: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Extract Container LogsWe need a way to read logs

from all containers.

Monitor EverythingWe need to monitor cluster resources and individual

containers.

Auto-Restart Failed Containers

Something needs to know that a container failed and

auto-restart it.

Service DiscoveryWhen we launch new

containers, we need to tell our router they exist.

Routing / Load BalancingWe need a way to route a

request to any of our containers.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 46: Choosing the Right Framework for Running Docker Containers in Prod

Docker RegistryWe need

somewhere to store our built

images.

Automated DeploymentWe need a way to deploy our Docker image into the cluster.

Extract Container LogsWe need a way to read logs

from all containers.

Monitor EverythingWe need to monitor cluster resources and individual

containers.

Auto-Restart Failed Containers

Something needs to know that a container failed and

auto-restart it.

Service DiscoveryWhen we launch new

containers, we need to tell our router they exist.

Routing / Load BalancingWe need a way to route a

request to any of our containers.

Container SchedulingWe need something to decide which host will

launch our new container?

Docker BuilderWe need

somewhere to build our image.

Page 47: Choosing the Right Framework for Running Docker Containers in Prod

Does our cluster have “state”?• Yes!

• Router needs to know which containers are from which services.

• We need to know which hosts are actually in our cluster.

Page 48: Choosing the Right Framework for Running Docker Containers in Prod

Storing Cluster State• This topic alone warrants full books.

• One option for storing state is to simply use a database like PostgreSQL.

• But the more popular option is for each host in the cluster to store state in an eventually consistent way using a “consensus algorithm.” I call this a cluster datastore.

• The most popular such solutions are: etcd, consul, and zookeeper.

Page 49: Choosing the Right Framework for Running Docker Containers in Prod

Unit of Container Deployment• We need something that describes what kind of

container to deploy.

• This is typically a declarative file in either YAML or JSON that declares all aspects of our docker run command, whether 2+ containers are run together, etc.

Page 50: Choosing the Right Framework for Running Docker Containers in Prod

The Theory in Summary• Docker builder

• Docker registry

• Automated deployment

• Container scheduling

• Routing / Load Balancing

• Service discovery

• Auto-restart failed containers

• Logging

• Monitoring

• Cluster datastore

• Unit of Container Deployment

Page 51: Choosing the Right Framework for Running Docker Containers in Prod

Paradigms of Multi-Container VM’s

Page 52: Choosing the Right Framework for Running Docker Containers in Prod

Three Paradigms• Cluster Frameworks

• Platform-as-a-Service (PaaS)

• Data Center Operating Systems

Page 53: Choosing the Right Framework for Running Docker Containers in Prod

Paradigm #1:Cluster Frameworks

Page 54: Choosing the Right Framework for Running Docker Containers in Prod

The Big Idea• You control the infrastructure (e.g. AWS, Azure)

• You’re given an unopinionated set of primitives on top of which you can build your own solution.

• Primitives include launching containers, but not full deployment.

Page 55: Choosing the Right Framework for Running Docker Containers in Prod

Major Cluster Frameworks

+ Fleet

Docker Swarm

(We’ll cover each of these today)

Page 56: Choosing the Right Framework for Running Docker Containers in Prod

Paradigm #2:Platform-as-a-Service (PaaS)

Page 57: Choosing the Right Framework for Running Docker Containers in Prod

The Big Idea• You control the infrastructure (e.g. AWS, Azure)

• Install the PaaS tool on top of your own infrastructure.

• PaaS tool typically sits on top of a Cluster Framework.

• You’re not 100% sure how it works, but it solves your needs today and you can always deep dive later, or (hopefully) get commercial support.

Page 58: Choosing the Right Framework for Running Docker Containers in Prod

Major PaaS Solutions

(We’ll cover Deis in depth shortly)

https://github.com/remind101/empire

Page 59: Choosing the Right Framework for Running Docker Containers in Prod

Paradigm #3:Data Center Operating Systems

Page 60: Choosing the Right Framework for Running Docker Containers in Prod

The Big Idea• You control the infrastructure (e.g. AWS, Azure)

• You’re given an opinionated framework which has everything you need to deploy.

• You operate at the abstraction level of “cluster” and really don’t care when individual hosts die.

• These tend to be the most powerful, and the most complex.

Page 61: Choosing the Right Framework for Running Docker Containers in Prod

Major Data Center Operating System Frameworks

(We’ll cover each of these today)

( + ? )

Page 62: Choosing the Right Framework for Running Docker Containers in Prod

Hybrid Paradigms:Where A runs on B

Page 63: Choosing the Right Framework for Running Docker Containers in Prod

Hybrid Combo #1:Run Kubernetes on Mesos

Page 64: Choosing the Right Framework for Running Docker Containers in Prod

Hybrid Combo #2:Run Kubernetes as a Hosted Service

Page 65: Choosing the Right Framework for Running Docker Containers in Prod

Hybrid Combo #3:Run the Open Source PaaS “Empire” on

top of EC2 Container Service

https://github.com/remind101/empire

Page 66: Choosing the Right Framework for Running Docker Containers in Prod

Hybrid Combo #4:Use Docker Swarm as the UX to Mesos.

Page 67: Choosing the Right Framework for Running Docker Containers in Prod

Mentally Managing the Hybrids

• Don’t get too caught up on these exotic combinations.

• Focus first on one of the “non-hybrid” technologies.

• Then evaluate what can be run on top of your choice technology, and whether it will make life easier for you.

Page 68: Choosing the Right Framework for Running Docker Containers in Prod

The Major Solutions

Page 69: Choosing the Right Framework for Running Docker Containers in Prod

Disclaimers• We just don’t have enough time to cover

each solution in depth.

• We only get about 3 minutes per solution, so let’s get started!

Page 70: Choosing the Right Framework for Running Docker Containers in Prod

Cluster Framework Solution:

+ Fleet

Page 71: Choosing the Right Framework for Running Docker Containers in Prod

How It Works• Launch a CoreOS cluster on the IaaS platform of your

choice (e.g. AWS, Azure, VMWare, etc.)

• CoreOS comes with a CLI tool fleet that enables launching containers, but does not constitute a full deployment system.

• Best thought of as a set of primitives you can work with, not a full-fledged framework.

• Define systemd unit files to describe the Docker container you want to launch.

Page 72: Choosing the Right Framework for Running Docker Containers in Prod

Docker Builder Roll Your Own

Docker Registry Roll Your Own

Deployment: Scheduling Built into fleet, but no resource-aware scheduling

Deployment: Routing Roll Your Own

Deployment: Service Discovery Roll Your Own

Auto-Restart Failed Containers Built into Fleet

Monitoring Roll Your Own

Logging Roll Your Own

Cluster Data Store etcd

Unit of Deployment Systemd unit file

Page 73: Choosing the Right Framework for Running Docker Containers in Prod

Pro’s• Relatively mature/stable among Cluster Frameworks.

• Once you’ve setup etcd, everything else “just works”.

• RESTful API into fleet allows for easily building out your own custom solution.

• Fleet will auto-restart failed containers.

• Tagging cluster nodes allows for clever distribution of containers (e.g. across Availability Zones).

• CoreOS gives us a well-defined method for updating individual cluster nodes to the latest CoreOS.

• Commercial support available.

Page 74: Choosing the Right Framework for Running Docker Containers in Prod

Con’s• Setting up etcd can be painful.

• Fleet does not allow resource-aware scheduling, so containers may run out of resources.

• Fleet does not expose a primitive for “transferring” a container from one cluster node to another.

• No built-in way to monitor cluster-wide resource consumption.

• Not usable for a production cluster without significant setup overhead (e.g. setting up service discovery).

• Learning fleet ultimately requires learning systemd and discovering what fleet commands actually do.

Page 75: Choosing the Right Framework for Running Docker Containers in Prod

When To Use It• You want to learn the foundations of CoreOS.

• You want high customizability over your setup and can tolerate non-resource-aware scheduling.

• You’re willing to manually handle many operations such as launching additional containers.

Page 76: Choosing the Right Framework for Running Docker Containers in Prod

Cluster Framework Solution:

Page 77: Choosing the Right Framework for Running Docker Containers in Prod

How It Works• Launch at least 3 EC2 instances in AWS.

• Install the ECS agent on each node (or launch an AMI with the agent pre-installed).

• Cluster setup “just works”

• Define a “Task Definition” to describe how one or more Docker containers should be launched.

• Define a “Service” that launches one or more instances of the Task Definition, and ECS auto-deploys your Tasks (containers).

Page 78: Choosing the Right Framework for Running Docker Containers in Prod
Page 79: Choosing the Right Framework for Running Docker Containers in Prod
Page 80: Choosing the Right Framework for Running Docker Containers in Prod
Page 81: Choosing the Right Framework for Running Docker Containers in Prod
Page 82: Choosing the Right Framework for Running Docker Containers in Prod
Page 83: Choosing the Right Framework for Running Docker Containers in Prod
Page 84: Choosing the Right Framework for Running Docker Containers in Prod

Docker Builder Roll Your Own

Docker Registry Roll Your Own

Deployment: Scheduling Resource-aware, pluggable scheduler. Can be swapped w/ custom one.

Deployment: Routing Leverages AWS Elastic Load Balancers

Deployment: Service Discovery Built in to services

Auto-Restart Failed Containers Built in to services

Monitoring Basic monitoring included at cluster level.

Logging Roll Your Own

Cluster Data Store Zookeeper (but this is hidden to us)

Unit of Deployment Task Definition

Page 85: Choosing the Right Framework for Running Docker Containers in Prod

Pro’s• Very easy to set up.

• Simple UX.

• Low learning curve.

• Covers most of what you need out of the box, including built-in routing and service discovery.

• Presumably AWS will keep improving it.

• Supported via AWS.

Page 86: Choosing the Right Framework for Running Docker Containers in Prod

Con’s• Doesn’t support dynamic port mapping from container to host.

• Each service requires its own Elastic Load Balancer, which is $18/month. (Unless you’re willing to expose a service on a port other than 80/443)

• Supports rolling deployments provided you have a spare node to launch a new service instance on. Blue/Green deployments are claimed as a feature, but require out-of-band customization.

• Not recommended to leverage the existing Zookeeper cluster datastore in use, so you may have to run two cluster datastores (e.g Zookeeper + Consul).

• Use of “private subnets” requires two separate clusters, one for public and one for private.

Page 87: Choosing the Right Framework for Running Docker Containers in Prod

When To Use It• You use AWS and…

• You want to get up and running quickly with your Docker-based microservices framework.

• You want to run your monolith using containers today, knowing you can migrate to other cluster tech’s in the future.

• You want an official solution with official support.

• You want to minimize the number of vendors/tech’s you work with.

Page 88: Choosing the Right Framework for Running Docker Containers in Prod

Cluster Framework Solution:

Docker Swarm

Page 89: Choosing the Right Framework for Running Docker Containers in Prod

How It Works• You use “docker-machine” to launch multiple EC2

instances (or other VMs). Each EC2 instance is configured with the Docker daemon and the docker-swarm agent (which is just a container).

• You launch one or more “Swarm Masters”, one of which is the “master leader.” You use this to control your cluster.

• You can now run Docker CLI commands to launch containers on your cluster.

Page 90: Choosing the Right Framework for Running Docker Containers in Prod
Page 91: Choosing the Right Framework for Running Docker Containers in Prod
Page 92: Choosing the Right Framework for Running Docker Containers in Prod

Docker Builder Roll Your Own

Docker Registry Roll Your Own

Deployment: Scheduling Resource-aware, pluggable scheduler. Can be swapped w/ custom one.

Deployment: Routing Roll Your Own

Deployment: Service Discovery Roll Your Own

Auto-Restart Failed Containers Open GitHub Issue: https://github.com/docker/swarm/issues/599

Monitoring Roll Your Own

Logging Roll Your Own

Cluster Data Store Pluggable!See https://docs.docker.com/

swarm/discovery/ Unit of Deployment Docker container, or Docker compose manifest

Page 93: Choosing the Right Framework for Running Docker Containers in Prod

Pro’s• Use the Docker CLI you’ve come to know and love.

• You can run other Docker tools that call the old Docker CLI directly on top of Swarm and they will “just work”. Does this matter?

• Potentially simpler to program against compared to CoreOS fleet.

• Resource-aware scheduling.

• Open source event bus available allows for interesting possibilities in response to cluster events (esp. around service discovery).

• Official Docker solution.

Page 94: Choosing the Right Framework for Running Docker Containers in Prod

Con’s• Not yet recommended for production.

• My own experience with docker-machine and docker-swarm have been underwhelming in terms of stability.

• Other than “official Docker solution” and “use Docker CLI”, I don’t see any superior features to alternatives.

Page 95: Choosing the Right Framework for Running Docker Containers in Prod

When To Use It• For experiments and curiosity.

• Docker swarm is intriguing, but under heavy development and doesn’t yet present a clear value proposition compared to alternatives.

• But check back in 6 months, and it may be a solid contender. See Project Orca for an exciting opinionated take on the UX. https://youtu.be/8vSPpPSd00w?t=1h25m16s

Page 96: Choosing the Right Framework for Running Docker Containers in Prod

PaaS Solution:

Page 97: Choosing the Right Framework for Running Docker Containers in Prod

How Deis Works

Control Plane Component

Cluster

Control Plane Component

Control Plane Component

Data Plane Component

Data Plane Component

Routerper Host

Routerper Host

Page 98: Choosing the Right Framework for Running Docker Containers in Prod

How Deis Works

Control Plane Component

Cluster

Control Plane Component

Control Plane Component

Data Plane Component

Data Plane Component

Routerper Host

Routerper Host

Service A Service B Service B Service C

Page 99: Choosing the Right Framework for Running Docker Containers in Prod

Deis Workflow for Dev’s (basically, Heroku on your own infrastructure)

SOURCE: http://docs.deis.io/en/latest/understanding_deis/concepts/

Page 100: Choosing the Right Framework for Running Docker Containers in Prod

How Deis Works for Operators

SOURCE: http://docs.deis.io/en/latest/understanding_deis/architecture/

Page 101: Choosing the Right Framework for Running Docker Containers in Prod

Docker Builder Included!

Docker Registry Included!

Deployment: Scheduling Defaults to resource-unaware fleet. Pluggable schedulers are in tech-preview.

Deployment: Routing Included!

Deployment: Service Discovery Included!

Auto-Restart Failed Containers Included!

Monitoring Roll Your Own

Logging Built-in logspout means you can send your logs anywhere.

Cluster Data Store PostgreSQL + Ceph

Unit of Deployment Heroku buildpack, Dockerfile, or Docker image.

Page 102: Choosing the Right Framework for Running Docker Containers in Prod

Pro’s• Everything in one package, ready to go. Get up and

running pretty quickly.

• Nice workflow for dev’s

• Open source

• Great community

• Good paradigm of what you would eventually need to build.

• Commercial support available through Engine Yard.

Page 103: Choosing the Right Framework for Running Docker Containers in Prod

Con’s• Learning curve for operators can feel steep.

• When the PaaS fails, it’s time to start climbing the learning curve. For example, I once terminated a node, broke the ceph cluster and had to dig into the guts to figure out how to fix it.

• Deis’s architectural opinions may differ from your own.

Page 104: Choosing the Right Framework for Running Docker Containers in Prod

Data Center Operating Solution #1:

Page 105: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Cluster

Page 106: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Page 107: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Our job is to store cluster state!

Page 108: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

Page 109: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

We make “resource offers” to “frameworks”.

Page 110: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

Page 111: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

We run “tasks”, which are Docker containers. We take our orders

from the Master.

Page 112: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

A Resource Offer:“Master, I have 2 CPU cores, 8 GB RAM,

and 25GB of disk space available!”

Page 113: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

A Resource Offer:“Master, why go for 2 CPUs when

I’ve got 3 CPUs and 10 GBs of RAM!”

Page 114: Choosing the Right Framework for Running Docker Containers in Prod

You’ve just seen Tier 1 of the Mesos resource scheduling algorithm.

Page 115: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Master Forwards Resource Offers“I have received your offers and they will

be forwarded to whomever I please.”

Page 116: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Page 117: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Master Forwards Resource Offers“Marathon, I choose you first! I can offer

you 3 CPUs and 10 GB of RAM.”

Page 118: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Frameworks Accept/Reject Offers:“Yawn. Pass.”

Page 119: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Master Forwards Resource Offers“Chronos, surely you have need of resources. I offer you the same!”

Page 120: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Frameworks Accept/Reject Offers:“Let’s do this.

I need 1 GB of RAM and 1 CPU core.”

Page 121: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Master Schedules Task on a Slave“It shall be so!

Agent #22, you shall run this task.”

Page 122: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Slave Receives the Task:“Strength and honor, sire!”

Page 123: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Slave starts running the task as a docker container.

Page 124: Choosing the Right Framework for Running Docker Containers in Prod

How Mesos Works

Zookeeper

Cluster

Zookeeper Zookeeper

Mesos Master

Mesos Master

Mesos Master

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

MesosSlave

(aka Agent)

Framework:Chronos

Framework:Marathon

Framework:ApacheSpark

Framework:Cassandra

Page 125: Choosing the Right Framework for Running Docker Containers in Prod

Docker Builder Roll Your Own

Docker Registry Roll Your Own

Deployment: Scheduling Resource-aware scheduler. Can run other “frameworks” side by side.

Deployment: Routing Roll Your Own

Deployment: Service Discovery Roll Your Own

Auto-Restart Failed Containers Included!

Monitoring Roll Your Own

Logging Roll Your Own

Cluster Data Store Zookeeper

Unit of Deployment Mesos Task (which will usually be a Docker container, usually submitted through Marathon.

Page 126: Choosing the Right Framework for Running Docker Containers in Prod

But wait, there’s more!• Setting up Mesos involves coordinating many

different moving pieces.

• Also, there’s no immediate way to gain a cluster-wide view of total memory/CPU/disk space use.

• Also, the learning curve can be steep.

Page 127: Choosing the Right Framework for Running Docker Containers in Prod

Mesosphere DCOS is meant to solve these problems.

• Offers “turn-key” setup (though the setup itself is not really production-grade).

• Offers a fancy UI for viewing cluster resource usage.

• Offers a special CLI for installing frameworks with 1 command.

• It’s very much in active development and would work best with a Mesosphere support plan.

Page 128: Choosing the Right Framework for Running Docker Containers in Prod

Pro’s• I find the Mesos abstraction the most intuitive when it

comes to managing cluster resources.

• Scalability is off the charts. Verizon, Siri, Yelp, Twitter and OpenTable all use Mesos.

• Growing community.

• Multiple “frameworks” already supported such as Apache Spark and Cassandra.

• Solomon Hykes called it the “gold standard” for running Docker containers in a cluster.

Page 129: Choosing the Right Framework for Running Docker Containers in Prod

Con’s• It can take weeks to setup if you need to do it right.

• The learning curve for dev’s is manageable, but for operators there are many moving pieces.

• There are certain edge cases that are rare but that would affect cluster performance over time.

• If you want to run Mesos on CoreOS, either you need to violate the CoreOS way, or run Mesos Master / Slave (Agent) in docker containers which is officially not recommended.

Page 130: Choosing the Right Framework for Running Docker Containers in Prod

When To Use It• You’re running multiple microservices, and you anticipate

significant scale.

• You want to squeeze as much possible utilization out of your large cluster as possible.

• You’re ready to adopt the cluster as the primary abstraction and expect to co-mingle prod and dev, multiple services, and multiple frameworks.

• Note: Smallest company I met at MesosCon was ~60 employees. That is probably the lower limit of company size before Mesos makes sense (IMO).

Page 131: Choosing the Right Framework for Running Docker Containers in Prod

Mesos + Docker Swarm• At MesosCon (August 2015), Docker showed

Docker Swarm as the CLI-based way to control Mesos deployments.

Page 132: Choosing the Right Framework for Running Docker Containers in Prod

Data Center Operating Solution #2:

Page 133: Choosing the Right Framework for Running Docker Containers in Prod

Disclaimers• Mr. Padnick may or may not have any

actual real-world experience with Kubernetes but felt it necessary to include it here for the sake of completeness.

Page 134: Choosing the Right Framework for Running Docker Containers in Prod

Kubernetes Pods• A pod is a group of docker containers that

should be run together.

Pod

Web Server

Content Management

Server

SOURCE: Illustrations reproduced from https://www.youtube.com/watch?v=Fcb4aoSAZ98

Page 135: Choosing the Right Framework for Running Docker Containers in Prod

Kubernetes Labels• A label is a set of key-value pairs that attach to allow Kubernetes

to identify groups of pods.

• Concept of labels is baked into most APIs.

Pod

SOURCE: Illustrations reproduced from https://www.youtube.com/watch?v=Fcb4aoSAZ98

FE

Pod

BI, FE

Pod

v2

Page 136: Choosing the Right Framework for Running Docker Containers in Prod

Kubernetes Replication Controllers

• A replication controller is a definition: “I want to run this pod 5 times.”

• If one of the pods fails, Kubernetes will auto-restart a new one.

SOURCE: Illustrations reproduced from https://www.youtube.com/watch?v=Fcb4aoSAZ98

Pod

v1

Pod

v1

ReplicationController

#Pods = 2 Label selector: v1

Page 137: Choosing the Right Framework for Running Docker Containers in Prod

Kubernetes Cluster

Node

Kubernetes Masteretcd

API Server Controller Manager Server

Scheduler Server

Kubernetes Masteretcd

API Server Controller Manager Server

Scheduler Server

kubeletagentproxy

Pod

FE

Pod

v2

Pod

v1

Node

Node

Node

Node

Node

Node

Page 138: Choosing the Right Framework for Running Docker Containers in Prod

Docker Builder Roll Your Own

Docker Registry Roll Your Own

Deployment: Scheduling Resource-aware scheduler.

Deployment: Routing Included!

Deployment: Service Discovery Included!

Auto-Restart Failed Containers Included!

Monitoring Optimal support with Google Cloud Engine. Limited support for others.

Logging Optimal support with Google Cloud Engine. Limited support for others.

Cluster Data Store etcd

Unit of Deployment Pod

Page 139: Choosing the Right Framework for Running Docker Containers in Prod

Pro’s• Produced by google.

• Very well-documented.

• Open source.

• The “successor” to CoreOS + Fleet. Commercially supported by CoreOS as tectonic.

• If run in Google Cloud Engine, can potentially be quite powerful.

Page 140: Choosing the Right Framework for Running Docker Containers in Prod

Con’s• Preferential support for Google Cloud Engine.

• Produced by Google but not necessarily the exact system Google uses to run its own cluster (though based on it).

• I may or may not be aware of additional issues.

Page 141: Choosing the Right Framework for Running Docker Containers in Prod

When To Use It• You’re running Google Cloud Engine

• You have prior experience from working at Google

• I may or may not be aware of add’l use cases.

Page 142: Choosing the Right Framework for Running Docker Containers in Prod

Mesos + Kubernetes• You can run kubernetes on top of Mesos as an

alternative to Marathon.

Page 143: Choosing the Right Framework for Running Docker Containers in Prod

Final Thoughts

Page 144: Choosing the Right Framework for Running Docker Containers in Prod

Closing Thoughts• To get started quickly, choose EC2 Container

Service.

• To get a feel for the core technologies, choose a PaaS like Deis and slowly learn CoreOS.

• To run multi-container VM’s at (potentially huge) scale, choose Mesos.

• There are many more “satellite” projects I didn’t cover solving unique problems!

Page 145: Choosing the Right Framework for Running Docker Containers in Prod

Now have fun and docker on!

Page 146: Choosing the Right Framework for Running Docker Containers in Prod

Q&A