Docker for (java-)devsHow Vimond handles the Docker hype

Why

It makes the life of devs and ops easier

if you do things right…

And to do that you must understand how it works.

Image vs container illustration from http://docs.docker.com/terms/container/

Image vs container

• an image is the blueprint for the container

• a container is an instance of the image

• the container is writeable, while an image is not (after creation)

• the container state can be committed to form an image

Base images

Choosing a base image

• They come in all sizes, for specialised or general purposes.

• Majority based on some version of Ubuntu, but many are also small (of size in MB) with a limited toolset.

Choosing a base imageThis is one of the questions dividing the community into two camps:

Should my container run one process, or many?

my process as PID 1

vs

using an init process?

Images inheritancephusion/baseimage:0.9.16

vimond/vimond-base

vimond/micros-base

vimond/gatekeeper vimond/eventservice vimond/< micros >vimond/< micros >vimond/< micros >

Tags / Versions

• Use some time to think through Docker tagging

• Tag is normally a version, but don’t need to be

• Special tag “latest” appended if no tag specified

• Metadata is a new feature - might change the scenarios where tags was used earlier. Have not looked into it yet.

The Dockerfile

Dissecting a DockerfileFROM phusion/baseimage:0.9.16 # Set correct environment variables.ENV HOME /root# Regenerate SSH host keys. # You may also comment out this instruction; the# init system will auto-generate one during boot.RUN /etc/my_init.d/00_regen_ssh_host_keys.sh# Use baseimage-docker's init system.CMD ["/sbin/my_init"] # ...put your own build instructions here...#RUN apt-get update && apt-get -y upgrade# Clean up APT when done.#RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

Dissecting a DockerfileFROM vimond.artifactoryonline.com/vimond-base# automatically accept oracle licenseRUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true \ | /usr/bin/debconf-set-selectionsRUN add-apt-repository ppa:webupd8team/java \ && apt-get update \ && apt-get -y upgrade \ && apt-get -y install \ oracle-java8-set-default \ oracle-java8-unlimited-jce-policy \ oracle-java8-installer \ libsnappy-java \ python-software-properties # Clean up APT when done.RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*ENV JAVA_HOME /usr/lib/jvm/java-8-oracle

Dissecting a Dockerfile# vim:set ft=dockerfile:FROM vimond.artifactoryonline.com/vimond-base-java-8MAINTAINER Olve Sæther Hansen <olve@vimond.com># Set correct environment variables.ENV HOME /root# Regenerate SSH host keys. baseimage-docker does not contain any, so you# have to do that yourself. You may also comment out this instruction; the# init system will auto-generate one during boot.RUN /etc/my_init.d/00_regen_ssh_host_keys.sh# Use baseimage-docker's init system.CMD ["/sbin/my_init"] # automatically accept oracle license#RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true \ # | /usr/bin/debconf-set-selectionsRUN apt-get update \ && apt-get -y upgrade \ && apt-get -y install \ python-pip \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*RUN pip install cqlsh#Swaps (ubuntu) dash with bash for easier sourceingRUN rm /bin/sh && ln -s /bin/bash /bin/shCOPY docker-service.sh /tmp/docker-service.shCOPY docker-service-startup-command.sh /etc/my_init.d/docker-service-startup-command.shRUN chmod a+x /etc/my_init.d/docker-service-startup-command.shONBUILD COPY docker/docker-config.yml docker/docker.properties build/libs/*.jar target/*.jar /tmp/ONBUILD RUN rm -fv /tmp/*tests*.jar#Notes on variables below. $UPPER_CASE means variables to be evaluated at runtime, all file names in#/etc/container_environment will be a variable name with the file content as value.#Variables with $lower_case means the var should used only in the Dockerfile image build phase.#This is for keeping the confusion at bay.ONBUILD RUN source /tmp/docker.properties \ && useradd -ms /bin/bash -d /opt/$service_name -G docker_env $service_name \ && mkdir /var/log/${service_name} \ && mkdir /etc/service/${service_name} \ && mv /tmp/docker-service.sh /etc/service/$service_name/run \ && echo $service_name >> /etc/container_environment/SERVICE_NAME \ [snip]

Dissecting a DockerfileFROM vimond.artifactoryonline.com/micros-baseimageMAINTAINER Olve Sæther Hansen <olve@vimond.com>EXPOSE 18080EXPOSE 18081ENV SERVICE_18080_NAME eventservice_backendsENV SERVICE_18081_NAME eventservice_adminENV SERVICE_9010_NAME eventservice_jmxENV SERVICE_18080_CHECK_HTTP /versionENV SERVICE_18080_CHECK_INTERVAL 15s ENV SERVICE_18081_CHECK_HTTP /healthcheckENV SERVICE_18081_CHECK_INTERVAL 15s ENV SERVICE_18080_TAGS "haproxy_lb_http,public,service,haproxy_backend"ENV SERVICE_18081_TAGS "admin_http_no_lb,private,haproxy_backend"ENV SERVICE_9010_TAGS "tcp,private"

Ways of building images

• Image from direct container changes

• Dynamic Dockerfile

• Static Dockerfile

Things to think about• size of image (.dockerignore)

• compile inside or outside of image?

• what to do about secrets?

• what do do about dependencies

• what to do about access to dependencies (repos)

• (sure I forgot something - anyone?)

Lets build some images

When your image is built, where should it go?

hub.docker.com

• the GitHub for Docker

• IMO best for public images, as rights management and building can be cumbersome for private repositories

hub.<yoursite>.com

• The registry is open source, so it is possible to run it locally

• Have not tried this - might be a sound solution

<yoursite>.artifactoryonline.com

• We went for this solution, as we already store our jar-files, npm builds and gems in Artifactory.

• Some other nice features for image promotion

• Probable other solutions, both onsite or as a service in this space

Volumes

Mounted Volumes• Mount files or folders from host to the container in RO or

RW mode.

• Useful for

• static data,

• complex configurations

• getting logs out of the container

• any dynamic data created in container that you want to keep

Data Volume Container

• Data volumes can be created from scratch or from a docker container which has a volume.

• use “—volume-from” command when starting/creating other container

• useful for dynamic data that can be versioned, shared and used in different environments (e.g. databases)

What is going on inside a container

• Can be a bit cumbersome, but possible to introspect in many ways.

• top (processes are exposed to host)

• ssh (if ssh server enabled- re multiple processes)

• docker exec -t -i <container id> bash -l

• docker inspect <container id>

Enough already, can't we just fire them up?

Docker Compose

• previously known as fig

• Tool for setting up several containers linked together.

• Handled by a single configuration

• We use it to set up infrastructure and perform integration tests

Putting everything together

Configuration

Configuration

• use production optimised defaults

• use a sane convention and override when needed

• make configuration expressible via system variables when possible

(meaning no lists, maps etc)

Discovery

Discovery

• Keep service discovery out of the service

• Keep service discovery out of the image containing the service

• Use Docker events and have a dedicated discovery agent running and reporting

Routing

Routing• We have now many services/containers running, each on

a random port on different machines.

• How do we route traffic to and from?

• Docker Compose can help (also in prod systems)

• We decided to write HAProxy configs on the fly based on Docker events (via a service discovery system - Consul and Registrator)

• Consul-template reacts to events in consul, and rewrites configuration files

HAProxy service routing• Typically two HAProxy servers with identical configuration

• One front-end on port 80 that selects backend based on DNS

• One backend for each service

• Many load balanced servers (potentially) for each backend

• Service outage will rewrite configuration file and restart HAProxy

• Service additions will rewrite configuration file and restart HAProxy

Other interesting Docker tools and notes

• Docker Machine

• Docker Swarm

• ECS from Amazon

• https://github.com/gliderlabs/logspout

• https://github.com/GoogleCloudPlatform/kubernetes

• Many others, but difficult to separate hype from promising projects.

?