Docker deploy automation with Fabric

1MOSCOW, October 2016

$ fab

RINAT KHABIBIEV

2

https://github.com/renskiy https://www.facebook.com/rinat.khabibiev

Since 2008

http://www.redmadrobot.com

5

https://www.docker.com/company

SOME DOCKER STATISTICS

6

DOCKER, CAVEATS

Open source

Network overhead

https://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf

7

DOCKER, BENEFITS

Simple requirements

CI/CD integration

Doesn’t require Internet connection to update App

The 12-factor App (12factor.net)

8

REDMADROBOT: TOTAL DOCKERIZATION

Python (Django)

Cron

PostgreSQL

RabbitMQ

Nginx

Elasticsearch

Redis

9

AVAILABLE DEPLOY TOOLS

Vagrant

Fabric

Ansible

Capistrano

Docker compose

Kubernetes

?

10

DEPLOY TOOL REQUIREMENTS

Arbitrary environment support

Easy adaptation (copy/paste)

Easy customization

No special education or experience requirements

Docker support

DB migrations apply and rollback

Support of private registry and complex networks

FABRICIO

11

https://github.com/renskiy/fabricio

DOCKER DEPLOY AUTOMATION TOOL

12

FABRICIO DEPLOY CONFIG EXAMPLE

# fabfile.py

from fabricio import docker, tasks

class NginxContainer(docker.Container):

image = docker.Image('nginx:1.9')

ports = '80:80'

nginx = tasks.DockerTasks(

container=NginxContainer(name='web'),

hosts=['user@example.com'],

)

13

FABRICIO DEPLOY PROCESS

$ fab --list

Available commands:

nginx backup -> pull -> migrate -> update

nginx.deploy backup -> pull -> migrate -> update

nginx.pull pull Docker image from registry

nginx.rollback rollback Docker container to previous version

nginx.update start new Docker container if necessary

$ fab nginx

[user@example.com] Executing task ‘pull’

[user@example.com] run: docker pull nginx:stable

[user@example.com] out: 1.9: Pulling from library/nginx

...

$ docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

ec44b023adf0 nginx:1.9 "nginx -g" 2 minutes ago Up 2 minutes 0.0.0.0:80->80/tcp web

14

FABRICIO FEATURES

Build Docker images

Create containers from images with provided tags

Unlimited infrastructures

Parallel execution (Fabric feature)

Rollback containers to previous version

Work with public and private Docker registries

DB migrations and rollback, backup and restore

15

FABRICIO COMMAND PARAMS

# show detailed information about command

$ fab --display nginx

Displaying detailed information for task 'nginx':

backup -> pull -> migrate -> update

Arguments: self, tag=None, force=False, migrate=True, backup=False

# deploy container using image with provided tag

$ fab nginx:1.10

# force new container

$ fab nginx:force=yes

# combo!

$ fab nginx:1.10,force=yes

# rollback container to previous version

$ fab nginx.rollback

16

FABRICIO DOCKER CONTAINER DEFINITION

class Container(object):

image = None # type: Image

cmd = None

stop_timeout = 10

user = None

ports = None

env = None

volumes = None

links = None

hosts = None

network = None

restart_policy = None

stop_signal = None

def __init__(self, name, **options):

self.name = name

...

18

FABRICIO AND PRIVATE DOCKER REGISTRY

# start local Docker registry

$ docker run -d -p 5000:5000 --name registry -v /data/registry:/var/lib/registry registry:2

nginx = tasks.PullDockerTasks(

container=NginxContainer('web'),

hosts=['user@example.com'],

)

$ fab --list

Available commands:

nginx prepare -> push -> backup -> pull -> migrate -> update

nginx.deploy prepare -> push -> backup -> pull -> migrate -> update

nginx.prepare prepare Docker image

nginx.pull pull Docker image from registry

nginx.push push Docker image to registry

nginx.rollback rollback Docker container to previous version

nginx.update start new Docker container if necessary

19

BUILD DOCKER IMAGES WITH FABRICIO

class MyContainer(docker.Container):

image = docker.Image('my_image')

app = tasks.BuildDockerTasks(

container=MyContainer('my_service'),

hosts=['user@example.com'],

build_path='.',

)

$ fab --list

Available commands:

app prepare -> push -> backup -> pull -> migrate -> update

app.deploy prepare -> push -> backup -> pull -> migrate -> update

app.prepare prepare Docker image

app.pull pull Docker image from registry

app.push push Docker image to registry

app.rollback rollback Docker container to previous version

app.update start new Docker container if necessary

20

FABRICIO ROLES AND INFRASTRUCTURES

from fabric import api as fab

@tasks.infrastructure

def production():

fab.env.update(roledefs={'front': ['user@example.com']})

nginx = tasks.DockerTasks(

container=NginxContainer(name='web'),

roles=['front'],

)

$ fab --list

Available commands:

production select production infrastructure to run task(s) on

production.confirm automatically confirm production infrastructure selection

...

$ fab production nginx

Are you sure you want to select production infrastructure to run task(s) on? [y/N]

21

FABRICIO: DEPLOYING DJANGO PROJECTS

from fabric import api as fab

from fabricio import docker, tasks

from fabricio.apps.python.django import DjangoContainer

class BaseDjangoContainer(DjangoContainer):

image = docker.Image('my_django')

@property

def env(self):

return 'DJANGO_SETTINGS_MODULE=settings.{}'.format(

fab.env.infrastructure,

)

django = tasks.BuildDockerTasks(

container=BaseDjangoContainer('api'),

hosts=['user@example.com'],

migrate_commands=True,

)

22

FABRICIO: DEPLOYING DJANGO PROJECTS

$ fab --list

Available commands:

django prepare -> push -> backup -> pull -> migrate -> update

django.deploy prepare -> push -> backup -> pull -> migrate -> update

django.migrate apply migrations

django.migrate_back remove previously applied migrations if any

django.prepare prepare Docker image

django.pull pull Docker image from registry

django.push push Docker image to registry

django.rollback rollback Docker container to previous version

django.update start new Docker container if necessary

23

FABRICIO: DATA BACKUP AND RESTORE

from fabricio.apps.db.postgres import PostgresqlBackupMixin

class BackupDjangoContainer(BaseDjangoContainer,

PostgresqlBackupMixin):

volumes = '/data/backup/postgres:/backup'

db_backup_dir = '/backup'

django = tasks.BuildDockerTasks(

container=BackupDjangoContainer('api'),

hosts=['user@example.com'],

backup_commands=True,

)

$ fab --list

Available commands:

django.backup backup data

django.restore restore data

...

24

FABRICIO REQUIREMENTS AND INSTALL

Python 2.6 or 2.7

Docker CLI (Linux/Mac/Windows)

Docker 1.9 or greater recommended (remote side)

# virtualenv install

$ pip install --upgrade fabricio

# macOS system-wide install

$ sudo pip install --upgrade fabricio six==1.4.1

25

FABRICIO ROADMAP

Master-Slave configurations for PostgreSQL

Docker Swarm support

docker-py integration

MySQL?

Non-Django frameworks?

26

27

USEFUL LINKS & QUESTIONS

Fabricio: https://github.com/renskiy/fabricio

Author of Fabricio: https://www.facebook.com/rinat.khabibiev

Хабра-блог Redmadrobot: https://habrahabr.ru/company/redmadrobot

Redmadrobot on Facebook: https://www.facebook.com/redmadrobot

The 12-factor App (SaaS dev patterns): https://12factor.net

Docker image with cron: https://hub.docker.com/r/renskiy/cron

Questions