Docker tips and tricks

Docker Beijing Meetup Group

Jérôme Petazzoni (@jpetazzo)

Grumpy French DevOps- Go away or I will replace you with a very small shell script

Wrote dotCloud PAAS deployment tools- EC2, LXC, Puppet, Python, Shell, ØMQ...

Docker contributor- Security, networking...

Runs all kinds of crazy things in Docker- Docker-in-Docker, VPN-in-Docker,

KVM-in-Docker, Xorg-in-Docker...

Outline

Some new features that you should know aboutThe Docker orchestration flowchartMeasuring and optimizing container performanceYou should use volumes

latest features

Docker 0.11

SELinux integration(works better with CentOS)

DNS integration for links(access linked containers by hostname)

docker run --net- use host networking for high speed

- share network of another container

Docker 0.12

docker pause/unpausemore importantly: 1.0 release candidate :-)

Docker 1.0

It's “production-ready!”you can buy support contracts, training...(in addition to the traditional t-shirts and stickers )☺

Docker 1.1

.dockerignore(don't upload your .git anymore!)

docker logs --tail- further logging improvements on the way

(truncate)

Docker 1.2

New cool options for docker run

--restart=always/no/on-failure

--cap-add=NETADMIN

--cap-drop=CHOWN

--device=/dev/kvm:/dev/kvm

Docker 1.3 (almost there)

docker exec(replaces nsenter)

docker create(lifecycle management)

Signature(for official images)

--security-opts(customize SELinux/AppArmor)

Docker X.X: Windows Server Containers

Windows Server Containers

orchestration

Orchestration

There's more than one way to do it- describe your stack in files

(Fig, Maestro-NG, Ansible and other CMs)

- submit requests through an API(Mesos, Kubernetes, Helios...)

- implement something that looks like a PAAS(Flynn, Deis, OpenShift...)

- OpenStack (because OpenStack can do everything!)

Introducing the Docker orchestration

flowchart

Do you (want to) use OpenStack?

Yes- if you are building a PAAS, keep an eye on Solum

(and consider contributing)

- if you are moving VM workloads to containers, use Nova(that's probably what you already have; just enable the Docker driver)

- otherwise, use Heat(and use Docker resources in your Heat templates)

No- go to next slide

Are you looking for a PAAS?

Good question: to PAAS or not to PAAS?

PAAS does not solve problems- PAAS puts all* your problems in one place

- now you have N identical problems instead of N different problems

All your applications must be standardized- so that they all have the same problem (instead of different ones)

It's much harder to operate a PAAS than a single app- in other words: PAAS is great if you have many apps

*Well, not all your problems, but things like database failover, high availability, scaling...

Are you looking for a PAAS?

Are you looking for a PAAS?

Yes- CloudFoundry (Ruby, but increasing % Go)

- Deis (Python, Docker-ish, runs on top of CoreOS)

- Dokku (A few 100s of line of Bash!)

- Flynn (Go, bleeding edge)

- Tsuru (Go, more mature)

- OpenShift geard (Go again!)

Choose wisely (or go to the next slide)- http://blog.lusis.org/blog/2014/06/14/paas-for-realists/

“I don’t think ANY of the current private PaaS solutions are a fit right now.”

If you have only one host

Fig (www.fig.sh)

fig.yml:web: build: . command: python app.py links: - db ports: - "8000:8000"db: image: postgres

If you have a few hosts (10s)

Maestro-NG(https://github.com/signalfuse/maestro-ng)- fig-like YAML file

- can talk to multiple hosts

- manual placement

Your favorite Configuration Management system- Ansible, Chef, Puppet, Salt: have Docker modules

- use CM to deploy hosts and start containers

- use Dockerfiles to deploy code & dependencies, libraries, packages

If you have many hosts (100s)

Helios- Java

- needs ZK, a master server, and one agent per host

<empty spot><empty spot><empty spot>

Hmmm... There might be a start-up opportunity there

If you have many many hosts (1000s)

Mesos- C++

- needs ZK, a master server, and one agent per host

- and probably a few other standby servers for HA

- and frameworks; e.g.:https://github.com/VoltFramework/volthttps://github.com/mesosphere/marathon

Kubernetes- work in progress

performance

Gathering metrics

cgroups give us per-container...- CPU usage

- memory usage (fine-grained: cache and resident set size)

- I/O usage (per device, reads vs writes, in bytes and in ops)

cgroups don't give us...- network metrics (have to do tricks with network namespaces)

https://github.com/google/cadvisor

http://jpetazzo.github.io/2013/10/08/docker-containers-metrics/

CPU performance

Nothing to doCPU performance is native in all benchmarks

I/O performance

Working set should be on a volumeVolume performance is native in all benchmarks

Memory performance

Memory control group has an overheadOverhead happens when memory is given by the kernel to the container, or reclaimed back

Overhead is not related to memory allocationsDisabling the memory control group = native speedBut it is a global operation (affects all containers)… And requires a reboot

Network performance

Linux bridge = overheadIPTables = overheaddocker run --net host = native speed- but loss of isolation

SR/IOV and macvlan = almost native speed- better performance than VMs

- maintain isolation

volumes

What is a volume?

Special directory in a containerMapped to normal directory on the hostCan be shared by multiple containers

When should we use volumes?

Bypass copy-on-write system- fast I/O path with zero overhead

- keep data across container upgrades

Use specific storage device in container- e.g. SAN, or fast SSD RAID for database...)

Share data between containers- this is cool, and let's see why!

Logging with volumes

Write log files to a volumedocker run --name logs -v /var/log busybox true

docker run --volumes-from logs myapp

Inspect logsdocker run --rm --volumes-from logs ubuntu bash

Ship logs to something else (logstash, syslog...)docker run --volumes-from logs pipestash

Backups with volumes

Data files should be in a volumedocker run --name mysqldata -v /var/lib/mysql busybox true

docker run --volumes-from mysqldata mysql

Run backup job in a separate containerdocker run --rm --volumes-from mysqldata mysqlbackup \ tar -cJf- /var/lib/mysql | stream-it-to-the-cloud.py

Of course, you can use anything fancier than tar(e.g. rsync, tarsnap...)

Moving containers and volumes around

If the container is stateless (web app...):- get the image to the new machine

- start the new container

- reconfigure load balancers

If the container is stateful (DB...):- Flocker

- Flocker

- Flocker

- or move volumes around and do the network plumbing yourself

More information about volumes

Docker Docs:https://docs.docker.com/userguide/dockervolumes/

Additional insights:http://blog.docker.com/2014/06/why-you-dont-need-to-run-sshd-in-docker/

Dockeradvanced concepts

Containers, containers everywhere!

Not an actual book (yet)

Thank you!Questions?

www.docker.com@docker@jpetazzo