service discovery in a microservice architecture using consul

Service Discovery in a Architecture

using Microservice

Consul

Who Am ICurrently doing Devops, Scala stu�

At Equeris, lean startup within EquensDocker, Consul, Scala, Cloud and other buzzwords.

Email me at: I write at: Twitter:

jos.dirksen@gmail.comhttp://www.smartjava.org

@josdirksen

Follow alongSources, presentation etc:

Demo heavy sessionhttps://github.com/josdirksen/next-build-consul

Microservices?

What are microservices?Small, �ne-grained easy to replace components.Organized around capabilities.Di�erent languages and backends (whatever �ts best).Fault tolerant, resiliant, automated deployements.

"Small Autonomous services that work together",Sam Newman

From three tier to Microservices

Running Microservices is hardWhere is my other service or database?Am I healthy, is the other one healthy?Where do I store con�guration?How do I handle redundancy and failover?...

, says everyone"Distributed systems are hard"

ServiceDiscovery

Basic approachHardcoded IP Address or LookupDNS

Lookup is nice!Requires managing names (con�g �les), DNS ServerHow to handle failover?

DNS

Now with failoverPoint to a loadbalancerDNS

Works nicely with !How to check health and register services?Programmatic access to LB?

DNS

What would be nice

Does : Lightweight ( , REST) support failoverHas �exible and manages

Lookups DNSHealth checking Con�guration

Consul"Consul [..] provides an opinionated framework for

service discovery and eliminates the guess-workand development e�ort. Clients simply register

services and then perform discovery using a DNSor HTTP interface. Other systems require a home-

rolled solution." - consul.io

Main FeaturesService discovery through REST and DNSSimple registration using REST APIDistributed KV store for con�gurationProvides extensive health checking

All in one package

Good to knowMulti DC-readyAPI for distributed locksEasy HA SetupEvent system

Consul = Zookeeper + Nagios + DNSMasq +Scriptings + ...

Consul Architecture

Service Registration Flow1. Service calls Consul Agent with registration message:

.2. Agent communicates registration with Consul Server3. Agent checks health of Service.4. If check succeeds mark as Healthy, if not mark as Unhealthy,

communicate results with Server5. When a lookup for Service occurs, only return Healthy services

http://agent_host/v1/agent/service/register

Sample: registration message{ "Name": "service1", "address": "10.0.0.12", "port": 8080, "Check": { "http": "http://10.0.0.12:8080/health", "interval": "5s" } }

Send when a new service starts upCheck types: script, http, tcp, TTL, Docker

Sample: DNS Lookup$dig @nb-consul.local backend-service.service.consul

; <<>> DiG 9.8.3-P1 <<>> @nb-consul.local backend-service.service.consul ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27716 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION: ;backend-service.service.consul. IN A

;; ANSWER SECTION: backend-service.service.consul. 0 IN A 10.0.9.3 backend-service.service.consul. 0 IN A 10.0.9.2 backend-service.service.consul. 0 IN A 10.0.9.4

Consul provides a DNS ServerWorks great with Docker ( : will show in Demo)teaser

Sample: REST Lookup$ curl -s http://192.168.99.106:8500/v1/catalog/service/backend-service [{ "Node": "cf2f293e423c", "Address": "192.168.99.111", "ServiceID": "backend-service", "ServiceName": "backend-service", "ServiceAddress": "10.0.9.2", "ServicePort": 8080 },{ "Node": "072b4ea1abc1", "Address": "192.168.99.112", "ServiceID": "backend-service", "ServiceName": "backend-service", "ServiceAddress": "10.0.9.3", "ServicePort": 8080 }]

DEMO

Closer look: DNS LookupIn code:

resp, err := http.Get("http://backend-service:8081/") if err != nil { fmt.Println(err) } else { defer resp.Body.Close() body, _ := ioutil.ReadAll(resp.Body) w.Header().Set("Content-Type",resp.Header.Get("Content-Type")) w.Write(body) }

For docker:

dns: 192.168.99.106 dns_search: service.consul

ConsulEcosystem

Consul templateRender template based on Consul stateSetup reverse proxy: Nginx, Apache, haproxy

global daemon maxconn {{key "service/haproxy/maxconn"}}

defaults mode {{key "service/haproxy/mode"}}{{range ls "service/haproxy/timeouts"}} timeout {{.Key}} {{.Value}}{{end}}

listen http-in bind *:8000{{range service "release.web"}} server {{.Node}} {{.Address}}:{{.Port}}{{end}}

Demo

Envconsul"III. Con�g Store con�g in the environment",

http://12factor.net/

More settings > More complexity

$ envconsul \ -consul demo.consul.io \ -prefix redis/config \ redis-server [opts...]

Vault: help in managing secrets

DEMO ENVCONSUL

DEMO ENVCONSUL

Prometheus & Grafana

More informationlinks:

This presentation, sources and docker stu�:

https://www.consul.io/https://github.com/hashicorp/consul-templatehttps://github.com/hashicorp/envconsul

https://github.com/josdirksen/next-build-consul

Thank You!