building web scale apps with docker and mesos by alex rukletsov (mesosphere)
DESCRIPTION
Operating apps at web scale has become the new normal, but has been out of reach for most companies. Join us as we show you how to deploy and manage your Docker containers at scale. See how easy it is to build highly-available, fault-tolerant web scale apps using Docker with the Mesos cluster scheduler. Docker plus Mesos is a new way to scale applications. Together they give you capabilities similar to Google’s Borg, the Googleplex’s secret weapon of scalability and fault tolerance.TRANSCRIPT
Building Web Scale Apps with Docker and Mesos
Alexander RukletsovSoftware Engineer @ Mesosphere
Why should we care?
Applications in the Cloud Era
Client-Server Era:Small apps, big servers
Cloud Era:Big apps, small servers
Serv Serv Serv Serv
VirtualizationAggregation
App
Server
App App App App
Major Components
● Hardware
● Orchestration, deployment and isolation
● Cluster and resource management
● Scale-aware applications, service discovery, etc.
Major Components
● Hardware
● Orchestration, deployment and isolation
● Cluster and resource management
● Scale-aware applications, service discovery
Applications in the Cloud Era
Client-Server Era:Small apps, big servers
Cloud Era:Big apps, small servers
Serv Serv Serv Serv
VirtualizationAggregation
App
Server
App App App App
“Divide and rule [and package]”
—Philip II of Macedon, paraphrased
“Divide et impera [et sarcina]”
Containers
● Lightweight Linux execution environment
● Static application composition
● Reliable deployment
● Unit of resource isolation
● Execution isolation
● Multi-tenancy without heavyweight VMs
Docker
● Open source
● Configurable layers
● Reproducible
● Version-controlled
● Plenty of other people’s containers
Docker
● Open source● Configurable layers● Reproducible● Version-controlled● Plenty of other people’s containers
● First-class citizen in Mesos and the Mesosphere stack
● Kubernetes employs and promotes Docker
© Gerard Julien/AFP
Run everything in containers!
What about container management?
© Unknown
© Unknown
Apache Mesos facts
● Created in 2009 at UC Berkeley, hardened in Twitter● Top-level Apache project● Mesosphere, Twitter, and Airbnb are major users /
contributors● Scales to 10 000s of nodes, production grade● Packages and support through Mesosphere● Google officially endorsed Mesos for Kubernetes● Built-in containerization, including Docker
Mesos as a Distributed OS kernel
● Two level resource scheduling● Launch tasks across the cluster● Communication between tasks (like IPC)● APIs for building “native” applications (aka frameworks):
program against the datacenter● APIs in C++, Python, JVM-languages, Go and counting● Pluggable CPU, memory, IO isolation● Multi-tenant workloads● Failure detection● Easy failover and HA
How Mesos works (HA mode)
Zookeeper
MesosMasterMesos
MasterMesosMaster
MesosMaster
MesosSlave
Executor Task
Task
Framework TaskScheduler
Application
Executor
Task
TaskExecutor
MesosSlave
© ect.nl
Manage containers with Mesos!
Mesos + Docker =
Native Docker support in Mesosmessage DockerInfo {
required string image = 1;
// Network options.
enum Network {
HOST = 1;
BRIDGE = 2;
NONE = 3;
}
message PortMapping {
required uint32 host_port = 1;
required uint32 container_port = 2;
optional string protocol = 3; // Protocol to expose (ie: tcp, udp).
}
optional Network network = 2 [default = HOST];
repeated PortMapping port_mappings = 3;
optional bool privileged = 4 [default = false];
// Allowing arbitrary parameters to be passed to docker CLI.
repeated Parameter parameters = 5;
}
Native Docker support in MesosCommandInfo command;
command.set_value("dd if=/dev/zero of=/dev/null");
ContainerInfo::DockerInfo dockerInfo;
dockerInfo.set_image("mesosphere/inky");
ContainerInfo containerInfo;
containerInfo.set_type(ContainerInfo::DOCKER);
containerInfo.mutable_docker()->CopyFrom(dockerInfo);
TaskInfo task;
task.set_name("");
task.mutable_task_id()->set_value("1");
task.mutable_slave_id()->CopyFrom(offer.slave_id());
task.mutable_resources()->CopyFrom(offer.resource());
task.mutable_command()->CopyFrom(command);
task.mutable_container()->CopyFrom(containerInfo);
vector<TaskInfo> tasks;
tasks.push_back(task);
driver.launchTasks(offer.id(), tasks);
Native Docker support in Marathon// nginx-task.json
{
"container": {
"type": "DOCKER",
"docker": {
"image": "nginx",
"network": "BRIDGE",
"portMappings": [
{ "containerPort": 80,
"hostPort": 0,
"servicePort": 80,
"protocol": "tcp" }
]
}
},
"id": "nginx",
"instances": "1",
"cpus": "0.25",
"mem": "256",
"uris": []
}
$ cat nginx-task.json | http http://dev1.mesosphere.com:8080/v2/apps
Future Docker Swarm API support
Mesos
Mesos Framework
Serv
DockerApp
Docker Swarm API
DockerApp
DockerApp
DockerApp
DockerApp
DockerApp
DockerApp
Serv Serv Serv Serv
What Mesos contributes
● Multi-framework: weighted fair sharing, roles, etc.● Run Docker containers alongside other popular frameworks
(e.g. Spark, Rails, Hadoop, …)● Run services and batch apps in the same cluster● Advanced scheduling: resources, constraints, global view● High resource availability, cluster self-healing● Proven at scale, battle-tested in production● GUI / CLI cluster management console
Ways of running Dockers
● Marathoneasy to setup, reliable orchestration
● Multiple Marathonsbenefit from two level scheduling, reservations, framework
roles
● Custom frameworkfine-grained management
“L’homme est libre au moment qu’il veut l’être”
“Man is free at the instant he wants to be”
—Voltaire
Cluster configuration example #1
AWS, DigitalOcean, GCE
Mesos
Hardware
Kernel
Apps
API
DockerApp
DockerApp
Services
Services REST API“Marathon”
DockerApp
AppDocker
AppApp
Cluster configuration example #2
AWS, DigitalOcean, GCE
Mesos
Hardware
Kernel
Apps
API
Services
Services REST API“Marathon”
Kubernetes
DockerApp
DockerApp
DockerApp
AppDocker
AppApp
Docker
Cluster configuration example #3
Serv Serv Serv Serv
Mesos
Spark, MPI, Hadoop, Storm
ServServServServ
Mesos SDKJava, Python, C++, Go
Services REST API“Marathon” (init)
Batch REST API“Chronos” (cron)
Serv
App Recurring Jobs(ETL, backups)
Hardware
Native Long running Batch
Apps
API
Kernel
AppDocker
App
AWS, DO, GCE
Docker
Cluster configuration example #4
Mesos
Spark, MPI, Hadoop, Storm
ServServ
Mesos SDKJava, Python, C++, Go
Services REST API“Marathon” (init)
Batch REST API“Chronos” (cron)
Serv
App Recurring Jobs(ETL, backups)
Hardware
Native Long running Batch
Apps
API
Kernel
AppDocker
App
Serv
© Reuters
Mesos is not only for “big” players!
Tiny clusters for everybody
● Mesosphere on GCE
● Mesosphere on DigitalOcean
● Mesosphere on AWS
● Portable Mesosphere on a USB stick
google.mesosphere.com
digitalocean.mesosphere.com
elastic.mesosphere.io
Demo: Mesos-on-Mesosphere
TaskTaskTask
Mesos-on-Mesosphere
Zookeeper
MesosSlaveMarathon
MesosMaster
MesosMasterMesos
MasterMesosMaster
MesosSlave
...
Task
Docker
DockerDockerDocker
MesosSlave
Mesos-on-Mesosphere
Zookeeper
MesosSlaveMarathon
MesosMaster
MesosMasterMesos
MasterMesosMaster
MesosSlave
...
MesosMaster
Docker
MesosSlave
MesosSlave
Docker
MesosSlave
MesosMaster
Summary
● Complete stack for large (and small) distributed apps
● Multi-tenancy
● Resource optimizations
● Easy to deploy
● No vendor lock-in
