continuous integration && docker

CONTINUOUSINTEGRATION &&

DOCKERGOING PARALLEL

by Michael Bodnarchuk @davert

WHO AM I?Web developer with Ruby,JavaScript and PHP experienceLead developer of

Space Traveler

Codeception PHPTesting Framework

WHAT IS ALL ABOUT?Development vs Testing EnvironmentUsing Docker for TestingOverview of CI ServersRunning Paralell Jobs

DEVELOPMENTENVIRONMENT

use same OSuse same PHP/Java/Ruby/Python versionuse same services (Nginx, MySQL, Mongo)use same configurationssetup fake smtp server

TESTING ENVIRONMENTsame as Development Environmentprepared to run on CI serveruses database with data fixturesshould be cleaned after tests executionmay contain additional services like Selenium

TESTING+DEVELOPMENTSETUP

Use provisioning: Chef, Ansible, Puppet, ...Create/delete databases for each test executionSolve isolation issues for parallel testing

OR USE

WHAT IS DOCKERDocker is an open platform distributed appsRuns and manages isolated Linux containersProvides GitHub-like infrastructure

Host

Container 1

Container 2

Container 3

Container ...

Docker Clientdocker pulldocker rundocker ...

Docker Index

Docker Daemon

DOCKER IMAGESIsolated FilesystemBuilt by DockerUses Incremental FS DriversCan be stored in Docker Registry

IMAGES BUILT FROM DOCKERFILEFROM fedora:20

RUN yum install -y mongodb-server && yum clean all

RUN mkdir -p /var/lib/mongodb && \ touch /var/lib/mongodb/.keep && \ chown -R mongodb:mongodb /var/lib/mongodb

ADD mongodb.conf /etc/mongodb.conf

VOLUME [ "/data/db" ]

EXPOSE 27017

USER mongodbWORKDIR /var/lib/mongodbCMD mongod

DOCKER CONTAINERRunning container != booting OSNo daemons will be executed on startOnly one process can be run at once

RUN CONTAINER$ docker run -it -p 27017:27017 -v /var/www/project/data:/data mongo1

CONTINUOUS INTEGRATION (CI) IS A DEVELOPMENTPRACTICE THAT REQUIRES DEVELOPERS TO INTEGRATE CODEINTO A SHARED REPOSITORY SEVERAL TIMES A DAY. EACH

CHECK-IN IS THEN VERIFIED BY AN AUTOMATED BUILD,ALLOWING TEAMS TO DETECT PROBLEMS EARLY.

thoughtworks.com/continuous-integration

TRAVIS CI

TRAVIS CI FEATURESSaaSFree for open source projectsSimple configurationVirtualization via OpenVPN/DockerConcurrent buildsPlans start from 129$/month

JENKINS CI

JENKINS FEATURESFree HostedDozens of PluginsThe Most Popular and TrustedOld-School Interface

RUNNING PARALLELBUILDS

TravisCI: out-of-box. Runs same tests suites for differentlanguage versions. I.e. PHP 5.3, 5.4, 5.5JenkinsCI: using multi-configuration project (MatrixReloadedplugin). Builds can be executed on slave servers.

BUILD MATRIX1. Configure axis2. Combinate values3. Use current values as environment vars

CONFIGURATION MATRIX EXAMPLE

RUN CONCURENT JOBdocker run -i -t -v $WORKSPACE:/project test_container ./runtests.sh $SUITE

WHAT'S INSIDE TEST CONTAINER?Test container contain prepared database./runtests.sh starts all required services (nginx, mysql,selenium, redis, etc)Container stops when tests are finishedNo supervisors: we can execute only one process per run

RUNTESTS.SH#!/bin/shecho "Starting Services...."service elasticsearch start > /dev/null 2>&1service nginx start > /dev/null 2>&1service php5-fpm start > /dev/null 2>&1service mysql start > /dev/null 2>&1service memcached start > /dev/null 2>&1phantomjs --webdriver=4444 > /dev/null 2>&1 &mailcatcher -f > /dev/null 2>&1 &

echo "Running tests"cd /project/$1 # switch to applicationcodecept run $2 # run tests from specific suite

LESSONS LEARNEDstopped containers should be removedcontainers should be auto-rebuilt on test data updates

CONCLUSIONSTest execution time can be improved by running tests inparallelDocker makes easy to run isolated concurrent builds

QUESTIONS?Talk by GitHub: Projects: , ,

Michael Bodnarchuk @davertDavertMik

Codeception Robo Task Runner JSter