Docker PaaS ( Mesos+AWS )

Oskars Gavriševs@2015

Agenda :

● Overview of Docker Paas solutions in market.

● Mesos in details.

● Torture of Mesos high availability.

● Lessons learned from AWS deployment.

Docker PaaS

Docker PaaS• Kubernetes ( By Google )

• Mesos ( By Apache Foundation )

----------------------------------------------------

• EC2 Container Service ( By Amazone )

• CoreOS ( By 3x devs )

• Stratos ( By Apache )

• Rancher ( By Rancher Labs )

• SmartOS

Kubernetes concepts. • Pods

• Masters

• Minions

• Replication controllers

• Service

• Labels

• Kubelets

Minions !

Kubernetes architecture.

Kubernetes minion.

Kubernetes on open source pills .

Amazon EC2 container service (ECS) , concepts.

• Cluster

• Container instance

• Task definition

• Task

• Service

ECS architecture.

ECS disadvantages.

• Scheduling algorithm is not transparent.

• Limited docker functionality (No “Host” networking).

• Only one container per one host with ELB.

• No Labels.

• No Health checks.

ECS, know how.

• How one container can start other container on same host ?

ECS, know how (2).

CoreOS

Rancher

Mesos in details

Mesos Architecture

Mesos Master(s)

• High availability

• Masters form quorum.

• Leader election.

• Register frameworks, nodes.

• Min. 3x masters (1x off), 5x recommended ( 2x off )

Mesos Slaves

● Provides resources.

● Can consist of multiple executors.

● Executors executes tasks.

Mesos Frameworks

Mesos Frameworks (2)

Marathon ( Long running services )

Hadoop ( Big Data Processing )

Chronos / Jenkins ( Batch scheduling )

ElasticSearch ( Data storage )

* http://mesos.apache.org/documentation/latest/frameworks/

What so special with this mesos ?

High availability

• If master fails tasks can continue to run.

• New leader election.

• Failed master can recover jobs.

Health checks

Component disconnection handling

• Slave disconnects from Zookeeper (don’t know which master is leader).

Component disconnection handling (1)

• Master disconnects from Zookeeper (enters leaderless state).

Warning suicidal master !

Component disconnection handling(2)

• Slave doesn’t respond to master.

Component disconnection handling(3)

• Master disconnects from nodes.

Component disconnection handling(4)

• Leading master dies.

Better resource utilization

Better resource utilization (2)

Mesos API

• http://master:5050/master/state.json

Task labels.

Constrains

DEMO

CPU / Meme / Disk quotas

CPU / Meme / Disk quotas (2)

DEMO

Marathon

Marathon docker scaling

• Scale up / down with one click.

Marathon port mapping and LB

DEMO

Chronos

Job dependencies

DEMO

Lessons learned

Lessons

• Unable to deploy on AWS ECS due to ‘host != networking’.

Lessons (2)

• It's almost impossible to pass dynamic variables to container.

https://github.com/ogavrisevs/Mesos-Docker-Cluster