docker cluster with swarm, consul, registrator and consul-template

Docker clusterWith swarm, consul, registrator and consul-

template

jmaitrehenry

2

Overview

• Create a swarm cluster• Manage a swarm cluster• Service discovery• Service auto-configuration

2015-06-15

3

1 linux server…

2015-06-15

An host running Linux

4

With docker…

2015-06-15

An host running Linux

Docker daemon

5

2 linux servers with docker…

2015-06-15

An host running Linux

Docker daemon

An host running Linux

Docker daemon

6

How can I manage them?

• Individually? Why not if you like that• On a cluster with a manager? Yeah!

2015-06-15

7

Docker swarm

2015-06-15

Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual host.

https://github.com/docker/swarm

8

How do I add a node to swarm ?

• Swarm need to register node IP and docker tcp port somewhere• Consul have a Key/Value store and swarm can register node

on it

2015-06-15

9

What is Consul?

Consul is a distributed, highly available system and provides:• Service discovery• Failure detection• Key/Value store• Bonus: supports multiple datacenter

2015-06-15

10

Create a swarm cluster

2015-06-15An host running Linux

Docker daemon

An host running Linux

Docker daemon

Consul host

Consuldaemon

Swarm cluster

11

How do I register an node to consul ?

• With swarm !

docker run -d swarm:0.3.0-rc2 join --addr=10.1.1.231:2375 consul://10.1.8.32:8500/dc1

2015-06-15

12

How do I register an node to consul ?

• swarm:0.3.0-rc2 ? Support docker login• --addr=<public host IP>:<port of docker http API>• consul://<IP of consul server>:<port>/<datacenter>docker run -d swarm:0.3.0-rc2 join --addr=10.1.1.231:2375 consul://10.1.8.32:8500/dc1

2015-06-15

13

Create a swarm cluster

2015-06-15An host running Linux

Docker daemon

An host running Linux

Docker daemon

Consul host

Consuldaemon Register node to consul K/V store

Swarm cluster

14

How can I manage my swarm cluster?

• With swarm !docker run -d -p 2375:2375 swarm manage consul://10.1.8.32:8500/dc1

2015-06-15

15

How can I manage my swarm cluster?

# export DOCKER_HOST= tcp://<manager IP>:<port>

# docker <docker cmd>

Without TLS enabled (dev only please ) : # unset DOCKER_TLS_VERIFY

2015-06-15

16

How can I manage my swarm cluster?

2015-06-15

17

Manage a swarm cluster

2015-06-15

An host

Docker daemo

n

Consul host

Consuldaemon

Register node

My laptopDocker daemo

n

Docker daemo

n

An host

manager

Swarm cluster

18

Manage a swarm cluster

2015-06-15

An host

Docker daemo

n

Consul host

Consuldaemon

Register node

My laptopDocker daemo

n

Docker daemo

n

An host

managerGet node list

Swarm cluster

19

Manage a swarm cluster

2015-06-15

An host

Docker daemo

n

Consul host

Consuldaemon

Register node

My laptopDocker daemo

n

Docker daemo

n

An host

manager

Manage hosts

Get node list

Swarm cluster

20

Ok, fine, I have 3 docker containers but how they know each other?

• First, we need to register service somewhere• Second, each service can ask somewhere where are other

service

2015-06-15

21

Service discovery

Service discovery is a key component of most distributed systems and service oriented architectures. The problem seems simple at first: How do clients determine the IP and port for a service that exist on multiple hosts?

2015-06-15

22

Oh, consul is a service discovery!

• Yes it is!Service list: http://10.1.8.32:8500/v1/catalog/services

2015-06-15

23

Consul• Service details:

http://10.1.8.32:8500/v1/catalog/service/statsd-8125

2015-06-15

Consul server

24

Consul• Service details:

http://10.1.8.32:8500/v1/catalog/service/statsd-8125

2015-06-15

Consul server

Docker node

25

Consul• Service details:

http://10.1.8.32:8500/v1/catalog/service/statsd-8125

2015-06-15

Consul server

Docker node

Service address

26

How can docker register services in consul?

• With a service registry bridge for Docker like Registrator

2015-06-15

27

Registrator

• Service registry bridge for Docker• « Registrator automatically register/deregisters services for

Docker containers based on published ports and metadata from the container environment.  »• https://github.com/gliderlabs/registrator

2015-06-15

28

Registrator

2015-06-15

On each swarm node

29

Registrator

2015-06-15

On each swarm node

Local docker socket

30

Registrator

2015-06-15

Public node IP

On each swarm node

Local docker socket

31

Demo

2015-06-15

322015-06-15

332015-06-15

34

Ok, I have some nginx running, how can I have a single entry point?

• With a load balancer (haproxy) with an auto update configuration

2015-06-15

35

Consul template

« The daemon consul-template queries a Consul instance and updates any number of specified templates on the filesystem. As an added bonus, consul-template can optionally run arbitrary commands when the update process completes »

https://github.com/hashicorp/consul-template

2015-06-15

36

Consul template - command

2015-06-15

37

Consul template – where to run it?

• Where you want, the only condition is to have access to consul server.• You could have it in a container• You could have it somewhere else

2015-06-15

38

Consul template – template file

2015-06-15

Service name

39

Consul template – template file

2015-06-15

Service nameService IP

40

Consul template – template file

2015-06-15

Service nameService IP

Service port

41

Consul template - testing

2015-06-15

42

Consul template - testing

2015-06-15

Dump generated template and exit

43

Thanks!

2015-06-15