KE N COC H RANE ENGINEER ING | DOCKER | @KenCochrane

The Future of Applications

• The Matrix from Hell and the Need for Containers • Docker & Containers: how they work • Docker Overview and Docker Ecosystem • Current Usage and Adoption • Creating an Open Environment • Docker Roadmap & Business Model

Agenda

Apps Have Fundamentally Changed

~2000 2014

Long*lived Development*is*iterative*and*constant

Monolithic*and*built*on*a*single*stack

Built*from*loosely*coupled*components

Deployed*to*a*single*serverDeployed*to*a*multitude*of*servers

User DBStatic website

Web frontend

QueueAnalytics DB

Background workers API endpoint

nginx 1.5 + modsecurity + openssl + bootstrap 2

postgresql + pgv8 + v8 hadoop + hive + thrift + OpenJDK

Ruby + Rails + sass + Unicorn

Redis + redis-sentinel

Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs

Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client

Development VM

QA server

Public Cloud

Disaster recoveryContributor’s laptop

Production Servers

Production Cluster

Customer Data Center

Mul@plicity*of*Stacks

Mul@plicity*of*hardw

are*

environm

entsDo*services*and*apps*

interact*appropriately?Can*I*m

igrate*smoothly

*and*quickly?

The Problem in 2014

Static website ? ? ? ? ? ? ?Web frontend ? ? ? ? ? ? ?Background workers ? ? ? ? ? ? ?User DB ? ? ? ? ? ? ?Analytics DB ? ? ? ? ? ? ?Queue ? ? ? ? ? ? ?

Development VM

QA Server

Single Prod Server Onsite Cluster Public Cloud Contributor’s

laptopCustomer Servers

The Matrix From Hell

An Inspiration…and Some Really Ancient History: Cargo Transport Pre-1960

Mul@plicity*of*Goo

dsMul@pilicity*of*m

etho

ds

for*transpor@ng

/storing

Do*I*w

orry*about*how*

goods*interact*(e.g.*coff

ee*beans*next*to*spices)

Can*I*transport*quickly*and*sm

oothly*(e.g.*from

*boat*to*train*to*truck)

A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.M

ul@plicity*of*Goo

dsMul@pilicity*of*m

etho

ds

for*transpor@ng

/storing

Do*I*w

orry*about*how*

goods*interact*(e.g.*coff

ee*beans*next*to*spices)

Can*I*transport*quickly*and*sm

oothly**(e.g.*from

*boat*to*train*to*truck)

…in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another

Solution: Intermodal Shipping Container

• 90% of all cargo now shipped in a standard container • Order of magnitude reduction in cost and time to load and unload ships • 5000 ships deliver 200M containers per year

This Spawned an Intermodal Shipping Container Ecosystem

An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…

Let’s Create a Shipping Container System for Applications

Static website User DB Analytics DBQueueWeb frontend

Mul@plicity*of*Stacks

Mul@plicity*of*hardw

are*

environm

ents

Do*services*and*apps

*interact*appropriately?

Development VM

QA server

Public Cloud

Contributor’s !laptop

Production Cluster

Customer Data Center

…that can be manipulated using standard operations and run consistently on virtually any hardware platform

Can*I*migrate*sm

oothly*and*quickly

And Eliminate the Matrix from Hell

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development VM

QA Server

Single Prod Server

Onsite Cluster

Public Cloud

Contributor’s Laptop

Customer Servers

Hypervisor (Type 2)Host OSServer

VM

Containers are isolated, but share OS kernel and, where appropriate, bins/libraries…result is significantly faster deployment, much less overhead, easier migration, faster restart

App*A

Bins/*Libs

Guest*OS

App*A’

Bins/*Libs

Guest*OS

App*B

Bins/*Libs

Guest*OS

App*A’

Host OS

Server

App*A

App*B

App*B’

App*B’

App*B’Container

Bins/Libs Bins/Libs

Step One: Create a Lightweight Container (vs. VMs)

A set of slides from IBM: Comparing Docker (Blue) to VM (Red)

•Container technology has been around for a while (c.f. LXC, Solaris Zones, BSD Jails) • But, their use was largely limited to

specialized organizations, with special tools & training. Containers were not portable

Type 1 Hypervisor Linux Containers docker

Step 2: Make the Containers Easy to Use, Standardized, Interoperable, Automatable

Step 2: cont.

With%Docker,%Containers%get%the%following:%•Ease*of**use,*tooling*

•ReTusable*components*

•Ability*to*run*on*any*Linux*server*today:*physical,*virtual,*VM,*cloud,*OpenStack,*+++*T (Stay*tuned*for*other*O/S’s)*

•Ability*to*move*between*any*of*the*above*in*a*maYer*of*secondsTno*modifica@on*or*delay*

•Ability*to*share*containerized*components*

•Interoperability*with*all*exis@ng*devops*tools*

•Self*contained*environment—no*dependency*hell*

•Tools*for*how*containers*work*together:*linking,*nes@ng,*discovery,*orchestra@on,*++

Step 3: Make Containers Super Lightweight

Bins/*Libs

App*A

App*Δ

*B/L

Every app, every copy of an app, and every slight modification of the app requires a new virtual server

App*A

VMs Containers

App*A

!!!

App*A

App*A’

Guest*OS

VMs

(No OS to take up space, resources,*or require restart)

Original AppNo OS. Can*Share bins/libs

Copy of AppCopy on write allows us to only save the diffs Between container A and container A’

Modified App

Guest*OS

Guest*OS

Bins/Libs

Bins/Libs

Bins/Libs

App*A

App*A

App*A’

An Open Platform to Build, Ship, and Run Distributed Applications

Step 4: Build an Open Platform

An Open Platform…

API

Engine Hub

open source software at the heart of the Docker platform 

cloud-based platform services for distributed applications

API

An Open Platform…

API

Engine Hub

cloud-based platform services for distributed applications

API

open source software at the heart of the Docker platform 

Any App

• + 17K apps • + 9K projects

• Physical • Virtual cloud

Any Infrastructure

…to Build, Ship, and Run

Build Ship

Run

Dev

QA

Source

Staging

DockerFile

Source Code Repository

TESTTEST

TESTTEST

TEST

Mac/Win Dev Machine

Doc

ker

Analytics DB

Users Collab

Provenance Policy

Docker Hub

Registries

Public Curated Private

Docker Hub API

Third Party Tools

QA Machine

Doc

ker

Doc

ker

Boot2Docker

Linux OS

Physical

Virtual

Cloud

Infrastructure Management

Infrastructure Management

GCE RAX IBM ++

Prod Machine Prod MachineProd Machine

Doc

ker

Doc

ker

Doc

ker

Doc

ker

Doc

ker

Doc

ker

VM VM VM

Doc

ker

Doc

ker

Doc

ker

Doc

ker

Doc

ker

Doc

ker

Linux OS Linux OS Linux OS

…Distributed Applications

Data

User DB

Queue

Analytics DB

API Endpoint

Web Frontend

Data

Background Workers

Development VM

QA Server

Disaster Recovery

Contributor’s LaptopProduction Servers

Production Cluster

Customer Data Center

Public Cloud

Step 5: Build an Ecosystem

Official Repos & 14K+ Dockerized Apps

Community*• 500+ Contributors • 250+ Meetups on Docker • 6M Downloads • 10K Projects on GitHub

Support*• Enterprise Support • Robust Documentation • Implementation, Integration, Training

• Network of Partners

The Docker Platform Docker Engine Docker Hub

Partners

Content

Users

Build, Ship, and Run

Partner Ecosystem

Who is Using Docker?

And hundreds of others… these are just a sample of those who have published use cases

Step 6: Build Systems and Infrastructure to Take Advantage of This New Paradigm

BARE METAL SERVERS

API-drivenInstantly Available Highly SpecializedNo

Hypervisor

OnMetal

Conclusion

•Multiple forces are driving a reconsideration of how applications should be created, built, deployed, scaled, and managed

•We believe that the right approach is to decouple application management from infrastructure management

•Container based approach (vs. VM approach) provides right level of abstraction

• Enables infrastructure to be managed consistently and stably

• Enables applications to be built flexibly and deployed flexibly • Provides greater degree of visibility, control, and management

of what runs where and what components are allowed • Massive cost, speed, efficiency savings • Docker is becoming the standard for containerization

Rackspace is a true leader in enabling this revolution!

Conclusion (cont.)

Learn More

Docker project:www.docker.com

Follow Docker on Twitter: @docker

Take the Docker interactive tutorial: www.docker.com/tryit/

Go to the Docker repository on GitHub: github.com/docker/docker

Go to a meetup: www.docker.com/community/