Workshop mesos docker devoxx fr 2016

Download Workshop mesos docker devoxx fr 2016

Post on 15-Apr-2017

419 views

Category:

Technology

1 download

Embed Size (px)

TRANSCRIPT

  • #DevoxxFR

    Dploiement d'une app distribue sur Mesos avec Docker, Marathon et

    MesosDNS

    Cyril Collen @sxarvipa Bertrand Dickel @bdickele Julia Mateo @juliamateodc

    1

  • #DevoxxFR 2

    Dveloppeurs Java

    Qui sommes nous ?

  • #DevoxxFR

    1re partie du workshop

    3

    Prsenter cm-voting, une webapp distribue

    Dployer cm-voting avec docker links

    Dployer cm-voting avec docker compose

  • #DevoxxFR

    2me partie du workshop

    4

    Simuler un cluster Mesos en local

    Dployer cm-voting

  • #DevoxxFR

    Vous aurez besoin de

    5

    - Un laptop

    - VirtualBox et la VM fournie dans les cls USB

  • #DevoxxFR

    The webapp

    6

  • #DevoxxFR

    The webapp

    7

  • #DevoxxFR

    Docker

    Open platform for developers and sysadmins to build, ship, and run distributed applications.

    8

  • #DevoxxFR

    Docker

    9

    Server

    Host OS

    Hypervisor

    GuestOS Guest OS Guest OS Guest OS

    Libs/Bins Libs/Bins Libs/Bins

    App 1 App 2 App 3 App 4

    Libs/Bins

    Server

    Host OS

    Docker Engine

    Libs/Bins Libs/Bins

    App 1 App 2 App 3 App 4

    c1 c2 c3 c4

    VM1 VM2 VM3 VM4

  • #DevoxxFR

    Container and images

    10

    Docker Images

    libcontainer, Union Filesystem

    Centos Ubuntu

    Jetty

    add App.war

    Container 1

    OracleDB

    Container 2

    Read only

    Writable

    Read only

    Docker Images

    Writable

  • #DevoxxFR

    Avant de commencer

    11

    - Dmarrez la VM :

    login : root

    password : osboxes.org

    - Activer le rseau

    Lets begin !

  • #DevoxxFR

    Dploiement cm-voting avec Docker links

    12

    - Code disponible dans la VM (/root/cm-voting) et sur github :

    https://github.com/karesti/cm-voting

    - Dmarrer docker service :

    >> service docker start

    - Dmarrer un container mongo :

    >> docker run -i -t -d name mongo_cmvoting -p 27017:27017 mongo

    - Dmarrer cm-voting :

    >> cd /root/cm-voting

    >> docker build -t cmvoting .

    >> docker run -i -t -d p 9000:9000 --link mongo_cmvoting:mongo cmvoting

  • #DevoxxFR

    Deploy cm-voting with docker-compose

    13

    >> cd /root/cm-voting

    >> docker-compose up

  • #DevoxxFR

    Mesos

    14

  • #DevoxxFR

    Quest-ce Mesos?

    15

    Systme de dploiement

    Plateforme dexcution

  • #DevoxxFR

    Systme de dploiement

    Systme de dploiement intelligent

    Ansible, Chef, Puppet -> configuration statique

    Mesos assigne dynamiquement les resources en fonction de la charge du cluster (ou dautres facteurs externes)

    16

    Quest-ce Mesos?

  • #DevoxxFR

    Plateforme dexcution

    Hosting dapplications plus flexible : Si un slave tombe, lapplication est dploye sur une nouvelle instance (framework Marathon)

    HAProxy pour faire du load balancing

    Alternative aux solutions Paas payantes (Heroku, Azure, Openshift...)

    17

    Quest-ce Mesos?

  • #DevoxxFR 18

    Quest-ce Mesos?

    ZKMaster Master

    Executor

    Executor Slaves

    Executor Executor

    Executor Executor

    Executor Executor Executor

  • #DevoxxFR 19

    Quest-ce Mesos?

    mesos.apache.org/documentation/latest/mesos-architecture/

  • #DevoxxFR

    Mesos frameworks : Marathon

    20

    Marathon est un framework Mesos crit en Scala Fournit un moyen facile pour dployer des conteneurs

    Docker

    Gestion dapplications haute disponibilit Rest API oriente dveloppeurs

  • #DevoxxFR

    Mesos frameworks : Marathon

    21

    ZKMaster Master

    Executor

    Executor Slaves

    Executor Executor

    Executor Executor

    Executor Executor Executor

  • #DevoxxFR

    Mesos frameworks : Marathon

    22

    ZK Master Master

    Executor

    Executor Slaves

    Executor Executor

    Executor Executor

    Executor Executor Executor

    Marathon

  • #DevoxxFR

    Service discovery avec MesosDNS

    23

    cm-voting

    Mesos slave 1

    Mesos slave 2

    REPLICATION, LOAD BALANCING...

  • #DevoxxFR

    Service discovery avec MesosDNS

    24

    cm-voting

    Mesos slave 1

    Mesos slave 2

    AND FAILOVER

  • #DevoxxFR

    Service discovery avec MesosDNS

    25

    https://github.com/mesosphere/mesos-dns

    https://github.com/mesosphere/mesos-dns

  • #DevoxxFR

    2me partie du workshop : deploy cmvoting sur Mesos

    26

  • #DevoxxFR

    Create mesos cluster

    Docker binding

    Chaque port simule un noeud du cluster

    27

  • #DevoxxFR

    Docker binding

    28

    Host

    /root/mongo/data

    Host

    /var/run/docker.sock

    Docker container

    /data/db

    Docker container

    /var/run/docker.sock

  • #DevoxxFR

    Create cluster (I)

    29

    Dployer zookeeper :

    >> docker run -d -e SERVER_ID=1 -p 2181:2181 garland/zookeeper

    Dployer mesos master

    >> docker run -d -p 5050:5050 -e "MESOS_HOSTNAME=${HOST_IP}" \\

    -e "MESOS_IP=${HOST_IP}" -e "MESOS_QUORUM=1" \\

    -e "MESOS_ZK=zk://${HOST_IP}:2181/mesos" --name mesos-master \\

    -e "MESOS_LOG_DIR=/var/log/mesos" --net host --restart always \\ mesoscloud/mesos-master:0.23.0-centos-7

  • #DevoxxFR

    Create cluster (II)

    30

    Dployer marathon :

    >> docker run -d -e "MARATHON_HOSTNAME=${HOST_IP}" \\-e "MARATHON_HTTPS_ADDRESS=$ {HOST_IP}\\ -e "MARATHON_HTTP_ADDRESS=${HOST_IP}" \\-e "MARATHON_MASTER=zk://${HOST_IP}:2181/mesos" \\-e "MARATHON_ZK=zk://${HOST_IP}:2181/marathon" \\--name marathon --net host --restart always \\mesoscloud/marathon:v1.0.0-RC1

  • #DevoxxFR

    Create cluster (III)

    31

    >> docker run -p 5051:5051 -d -e "MESOS_HOSTNAME=${HOST_IP}" \\-e MESOS_PORT=5051 -e "MESOS_IP=$ {HOST_IP}" \\-e "MESOS_MASTER=zk://${HOST_IP}:2181/mesos" \\-v /sys/fs/cgroup:/sys/fs/cgroup \\-v /var/run/ docker.sock:/var/run/docker.sock \\--name slave1 --net host --privileged --restart \\ always mesoscloud/mesos-slave:0.23.0-centos-7

    >> docker run -p 5052:5052 -d -e "MESOS_HOSTNAME=${HOST_IP}" \\-e MESOS_PORT=5052 -e "MESOS_IP=$ {HOST_IP}" \\-e "MESOS_MASTER=zk://${HOST_IP}:2181/mesos" \\-v /sys/fs/cgroup:/sys/fs/cgroup \\-v /var/run/ docker.sock:/var/run/docker.sock \\--name slave2 --net host --privileged --restart \\ always mesoscloud/mesos-slave:0.23.0-centos-7

    >> docker run -p 5053:5053 -d -e "MESOS_HOSTNAME=${HOST_IP}" \\-e MESOS_PORT=5053 -e "MESOS_IP=$ {HOST_IP}" \\-e "MESOS_MASTER=zk://${HOST_IP}:2181/mesos" \\-v /sys/fs/cgroup:/sys/fs/cgroup \\-v /var/run/ docker.sock:/var/run/docker.sock \\--name slave3 --net host --privileged --restart \\ always mesoscloud/mesos-slave:0.23.0-centos-7

    Slave 3

    Slave 2

    Slave 1

  • #DevoxxFR

    Mesos GUI

    32

  • #DevoxxFR

    Marathon GUI

    33

  • #DevoxxFR

    Dployer cm-voting Deploy MesosDNS :

    curl -H "Content-Type: application/json" -X POST -d '{"id":"mesos-dns", "cmd":"/usr/bin/mesos-dns -v=2 -config=/config.json", "cpus":0.5, "mem":500, "container":{"type":"DOCKER", "docker":{"image":"mesosdns", "network":"HOST"}}}' http://10.0.2.15:8080/v2/apps?force=true

    Deploy Mongo

    curl -H "Content-Type: application/json" -X POST -d '{"id":"mongo", "cmd":"cat /etc/resolv.conf > /tmp/temp && echo \"nameserver 10.0.2.15\" > /etc/resolv.conf && cat /tmp/temp >> /etc/resolv.conf && mongod", "cpus":0.5, "mem":500, "container":{"type":"DOCKER", "docker":{"image":"mongo", "privileged":true, "network":"BRIDGE", "portMappings":[{"containerPort": 27017, "hostPort": 0}]}}}' http://10.0.2.15:8080/v2/apps?force=true

    34

    http://10.0.2.15:8080/v2/apps?force=true

  • #DevoxxFR

    Is MesosDNS working?

    Add this line to the beginning of /etc/resolv.conf :

    nameserver 10.0.2.15

    Ask MesosDNS for Mongo service

    >> dig mongo.marathon.mesos

    >> dig _mongo._tcp.marathon.mesos SRV

    35

  • #DevoxxFR

    Dployer cm-voting

    Deploy cmvoting

    curl -H "Content-Type: application/json" -X POST -d '{"id":"cmvoting", "cmd":"cat /etc/resolv.conf > /tmp/temp && echo \"nameserver 10.0.2.15\" > /etc/resolv.conf && cat /tmp/temp >> /etc/resolv.conf && revel run github.com/karesti/cm-voting prod 9000", "cpus":0.5, "instances":1, "mem":250, "container":{"type":"DOCKER", "docker":{"image":"cmvoting", "network":"HOST"}}, "env" : {"SERVICE_NAME":"mongo"}}' http://10.0.2.15:8080/v2/apps?force=true

    36

    http://10.0.2.15:8080/v2/apps?force=true

  • #DevoxxFR

    Vrifier que cmvoting tourne

    37

  • #DevoxxFR

    Load balancing : lb-marathon

    38

    Outil qui gre HAProxy

    Il se souscrit levent bus de Marathon

    Il met jour la configuration du HAProxy en temps rel

    Health checking

    SSL support

  • #DevoxxFR

    Demo

    39

  • #DevoxxFR 40

    GESTION DU FAILOVER LENT

    NO HEALTH CHECKING

    LES REQUETES PEUVENT ETRE

    MISES EN CACHE

    SERVICE PORTS NON IDENTIFIABLES

    MesosDNS drawbacks

  • #DevoxxFR

    Service discovery avec lb-marathon

    41

    cm-voting

    cm-voting

    lb-marathon

    l