docker service registration and discovery
TRANSCRIPT
Service Registration
and Discovery with
Michael Richardson @m_richo
$whoami
• Michael Richardson
• Engineer @ odecee
• @m_richo
• http://www.mricho.com
• I like DevOps*
• Docker has grown on me in the last 6 months
* My definition of DevOps is likely different from yours
Agenda
• What is Service Registration & Discovery?
• Consul
– tool to store information about services
• Registrator
– tool to register Docker services
• Consul-template
– tool to query registered services and apply configuration.
• Demo
Service Registration and Discovery
What is it?
Service Registration and Discovery
What is it?
“Service Discovery Protocols (SDP) are
network protocols which allow automatic
detection of devices and services offered by
these devices on a computer network.”
Source: http://en.wikipedia.org/wiki/Service_discovery
Service Registration and Discovery
What is it?
Basically… how do clients determine the
network details of desired services.
Service Registration and Discovery
Existing techniques?
• DNS (perhaps Dynamic DNS)
• Puppet exported resources
• Chef Search
• many others
Service Registration and Discovery
These approaches can struggle in a docker world
Image source: http://www.ideachampions.com/weblogs/archives/2011/09/the_top_100_lam.shtml
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS
– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP
based DNS isn’t sufficient. *
* SRV records can be an option
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP based DNS isn’t sufficient. *
• Config Management agents
* SRV records can be an option
Service Registration and Discovery
These approaches can struggle in a docker world
• DNS– TTL and caching can often be an issue
– Just try doing interesting DNS things in an enterprise.
– Containers don’t have a unique IP address. Simple IP based DNS isn’t sufficient. *
• Config Management agents– I’m resistant to running agents within a container.
– CM agent run intervals are probably not frequent enough.
* SRV records can be an option
Service Registration and Discovery
Are there any new tools for Services Discovery?
Service Registration and Discovery
• Zookeeper
• Doozer
• Etcd
• SmartStack
• Eureka
• SkyDNS
• consul
Yep. Quite a few…
Service Registration and Discovery
• Zookeeper
• Doozer
• Etcd
• SmartStack
• Eureka
• SkyDNS
• consul
Yep. Quite a few…
Consul
“Service Discovery and
configuration made easy.
Distributed, highly available
and datacenter aware.”
Consul
Key features…
• Service Discovery
• Health Checking
• Key/Value Storage
• Multi Datacenter
Consul
Key features…
• Service Discovery
• Health Checking
• Key/Value Storage
• Multi Datacenter
Consul - example
Consul - example
Consul service definition for nginx web server
Consul + Docker
How should I use consul with docker?
Consul + Docker
How should I use consul with docker?
Option 1: run 1 Consul client agent in every
container to register its service
Or
Option 2: run 1 consul client agent per Docker
host that is responsible for all the containers on it
Consul + Docker
How should I use consul with docker?
Option 1: run 1 Consul client agent in every
container to register its service
Or
Option 2: run 1 consul client agent per Docker
host that is responsible for all the containers on it
Option 2 makes more sense to me
Consul + Docker
Docker host running some containers
Consul + Docker
Add a Consul Agent as a container
Consul + Docker
Which talks to the consul cluster
Consul + Docker
Next Question?
How do we register the
container services with the
consul agent?
Consul + Docker
Question?
How do we register the
container services with the
consul agent?
Use
Registrator!
Registrator
• https://github.com/gliderlabs/registrator
• Automatically registers/deregisters services for docker
containers
• Supported Registry services include consul, etcd and SkyDNS
2
• Reads events from Docker Socket (/var/run/docker.sock)
• Run on each Docker host*
* Ideally run it as a container
Registrator
Installation instructions
Registrator
Installation instructions
Registrator
Starting Registrator
Registrator
Starting Registrator
Registrator
Starting Registrator
pass docker socket to container
Specify service registry endpoint (eg consul)
How does it work
How does it work
• Start containers for consul
and registrator
How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
• Start new container
How does it work
• Start containers for consul
and registrator
• Registrator watches for
container start and die
events
• Start new container
• Registrator informs consul
of service A’s metadata
(IP/port/name/tag)
Service information is now in
Consul. How can we access it?
• Consul’s HTTP API
• Consul’s DNS API
• Consul-template (via HTTP
API)
Consul Access
• Consul’s HTTP API
• Consul’s DNS API
• Consul-template (via HTTP
API)
Consul Access
Consul-template
What does it do?
Consul-template
What does it do?
• Populate template files with data from
Consul.
• Optionally run commands when consul
data changes.
Consul-template – example 1
Consul-template – example 1
Consul-template – example 2
Consul-template – example 2
Time for a Demo
links
• https://consul.io/
• https://github.com/hashicorp/consul-template
• https://github.com/gliderlabs/registrator
• https://github.com/CiscoCloud/microservices-
infrastructure
• http://jasonwilder.com/blog/2014/02/04/service-
discovery-in-the-cloud/
Questions?
Thanks for listening
@m_richo