soa with consul

33
Service Oriented Architecture with Consul Rajesh Sharma @broncha VantageBit

Upload: rajesh-sharma

Post on 13-Apr-2017

68 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Soa with consul

Service Oriented Architecturewith Consul

Rajesh Sharma

@broncha

VantageBit

Page 2: Soa with consul
Page 3: Soa with consul

Problems when you scale aSOA/microservices

Host resolution (service discovery)

Con�guration updates

Page 4: Soa with consul

SolutionsApache zookeeper

etcd

doozerd

consul

Page 5: Soa with consul

ConsulIt is a tool for discovering and con�guring services inyour infrastructure, with features like

Service Discovery

Health Checking

Key/Value Store

Page 6: Soa with consul

InstallationJust download a binary �le and you are good to go.

Page 7: Soa with consul

Con�guration

Page 8: Soa with consul

Server nodescon�g.json

{ "server": true, "data_dir": "/var/consul", "encrypt": "2346rxJm2aFuMNxWCKWnvPBA==", "log_level": "INFO", "enable_syslog": true, "bind_addr": "172.31.0.195", "start_join": ["172.31.12.223", "172.31.28.140"], "ui": true}

Page 9: Soa with consul

Client nodes

Page 10: Soa with consul

con�g.json

{ "server": false, "data_dir": "/var/consul", "encrypt": "2346rxJm2aFuMNxWCKWnvPBA==", "log_level": "INFO", "enable_syslog": true, "bind_addr": "172.31.21.186", "start_join": ["172.31.28.140","172.31.12.223","172.31.0.195"}

Page 11: Soa with consul

services.json

{ "services": [ { "id": "application@ip-172-31-21-186", "name": "application", "address": "172.31.21.186", "port": 8080, "checks": [ { "name": "Application/Apache on port 8080", "tcp": "localhost:8080", "interval": "10s", "timeout": "1s" } ] } ]}

Page 12: Soa with consul

The HTTP Interface

Page 13: Soa with consul

Catalog API

Page 14: Soa with consul

/v1/catalog/nodes

[ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "baz", "Address": "10.1.10.11", "TaggedAddresses": { "lan": "10.1.10.11", "wan": "10.1.10.11" }, "Meta": { "instance_type": "t2.medium" } } ...]

Page 15: Soa with consul

/v1/catalog/services

{ "consul": [], "redis": [], "postgresql": [ "primary", "secondary" ]}

Page 16: Soa with consul

/v1/catalog/service/<service>

[ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "192.168.10.10", "Meta": { "instance_type": "t2.medium" }, "CreateIndex": 51, "ModifyIndex": 51, "ServiceAddress": "172.17.0.3", "ServiceEnableTagOverride": false, "ServiceID": "32a2a47f7992:nodea:5000", "ServiceName": "foobar", "ServicePort": 5000, "ServiceTags": [ "tacos" ] }

Page 17: Soa with consul

/v1/catalog/node/<node>

{ "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Services": { "consul": { "ID": "consul", "Service": "consul", "Port": 8300 }, "redis": { "ID": "redis", "Service": "redis", "Tags": [ "v1" ], "Port": 6379

Page 18: Soa with consul

Health Check API

Page 19: Soa with consul

/v1/health/node/<node>

[ { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "CheckID": "serfHealth", "Name": "Serf Health Status", "Status": "passing", "Notes": "", "Output": "", "ServiceID": "", "ServiceName": "" }, { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check", "Status": "passing", "Notes": "",

Page 20: Soa with consul

/v1/health/checks/<service>

[ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check", "Status": "passing", "Notes": "", "Output": "", "ServiceID": "redis", "ServiceName": "redis" }]

Page 21: Soa with consul

/v1/health/service/<service>

[ { "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Service": { "ID": "redis", "Service": "redis", "Tags": null, "Address": "10.1.10.12", "Port": 6379 }, "Checks": [ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check",

Page 22: Soa with consul

/v1/health/service/<service>?passing

[ { "Node": { "ID": "40e4a748-2192-161a-0510-9bf59fe950b5", "Node": "foobar", "Address": "10.1.10.12" }, "Service": { "ID": "redis", "Service": "redis", "Tags": null, "Address": "10.1.10.12", "Port": 6379 }, "Checks": [ { "Node": "foobar", "CheckID": "service:redis", "Name": "Service 'redis' check",

Page 23: Soa with consul

The Key-Value Store API

Page 24: Soa with consul

GET /v1/kv/<key>

[ { "CreateIndex": 100, "ModifyIndex": 200, "LockIndex": 200, "Key": "zip", "Flags": 0, "Value": "dGVzdA==", "Session": "adf4238a-882b-9ddc-4a9d-5b6758e4159e" }]

PUT /v1/kv/<key>

DELETE /v1/kv/<key>

Page 25: Soa with consul

The DNS InterfaceInstead of making HTTP API requests to Consul, ahost can use the DNS server directly via namelookupsredis.service.us-east-1.consul

master.rabbitmq.service.us-east-1.consul

slave.redis.service.consul

Consul will listen on 127.0.0.1:8600 for DNSqueries in the consul. domain

We can use dnsmasq to route DNS queries for consul. domains to 127.0.0.1:8600

Unhealthy nodes are automatically �ltered

Page 26: Soa with consul

Node Lookups

<node>.node[.datacenter].<domain>

$ dig @127.0.0.1 -p 8600 foo.node.consul ANY

; <<>> DiG 9.8.3-P1 <<>> @127.0.0.1 -p 8600 foo.node.consul ANY; (1 server found);; truncating comment sections for brevity

;; QUESTION SECTION:;foo.node.consul. IN ANY

;; ANSWER SECTION:foo.node.consul. 0 IN A 10.1.10.12

;; AUTHORITY SECTION:consul. 0 IN SOA ns.consul. postmaster.consul. 1392836399 3600 600 86400 0

Page 27: Soa with consul

Service Lookups

[tag.]<service>.service[.datacenter].<domain>

$ dig master.redis.service.consul

; <<>> DiG 9.10.3-P4-Ubuntu <<>> master.redis.service.consul;; truncating comment sections for brevity

;; QUESTION SECTION:;master.redis.service.consul. IN A

;; ANSWER SECTION:master.redis.service.consul. 0 IN A 172.31.28.140

;; Query time: 2 msec;; SERVER: 127.0.0.1#53(127.0.0.1);; WHEN: Wed Feb 01 10:36:28 UTC 2017;; MSG SIZE rcvd: 88

Page 28: Soa with consul

Distributed lock and Leaderelection

Page 29: Soa with consul

Pick a key

service/<servicename>/leader`

Create a session

$ curl -X PUT -d '{"Name": "rabbitmq-service"}' \ http://localhost:8500/v1/session/create # response { "ID": "4ca8e74b-6350-7587-addf-a18084928f3c" }

Page 30: Soa with consul

Acquire a session for given key

$ curl -X PUT -d <body> \ http://localhost:8500/v1/kv/<key>?acquire=<session>

If return value is true key was acquired and thisnode is the leader

Else some other node already acquired that keyand is the leader

Page 31: Soa with consul

Other nodes can discover the leader node usingthe session info API

$ curl http://localhost:8500/v1/kv/<key>[ { "Session": "4ca8e74b-6350-7587-addf-a18084928f3c", "Value": "Ym9keQ==", "Flags": 0, "Key": "<key>", "LockIndex": 1, "ModifyIndex": 29, "CreateIndex": 29 }]

If the key has no associated Session, then there isno leader

Page 32: Soa with consul

The nodes would be watching the key to check if itwas released and try to become the leader

$ curl \http://localhost:8500/v1/session/info/4ca8e74b-6350-7587-addf-a18084928f3c[ { "LockDelay": 1.5e+10, "Checks": [ "serfHealth" ], "Node": "consul-primary-bjsiobmvdij6-node-lhe5ihreel7y" "Name": "dbservice", "ID": "4ca8e74b-6350-7587-addf-a18084928f3c", "CreateIndex": 28 }]

Page 33: Soa with consul

Questions?