dynamic inventory: no more host lists!

Dynamic inventory:No more host list!

#ansibleja Ansible勉強会#1

2014/06/10@tagomoris

14年6月10日火曜日

TAGOMORI Satoshi (@tagomoris)LINE Corp.

14年6月10日火曜日

Ansible

Batch server operations

add routing table entry

deploy ssh keys

update openssl

Install/update middleware

Hadoop, Presto, Fluentd (and its config/code ...)

14年6月10日火曜日

Ansible: good points

Minimal managed node requirements

Simple tool chain and files

YAML

and many built-in modules

Less stress of shell scripts

Dynamic inventory

14年6月10日火曜日

Ansible: bad points

Less japanese case-studies

Less japanese books and blog entries

Less japanese users? -> No!

14年6月10日火曜日

Inventory

Hosts and groups

Host variables / Group variables

Group of groups

‘group_vars/’ and ‘host_vars/’

14年6月10日火曜日

Inventory: example

mail01.local globalip=192.168.0.11

[blog]www[01:30].blog.local virtualip=203.0.113.63app[01:50].blog.localmaster01.db.blog.local server_id=....slave0[1:4].blog.local server_id=..????

[linenantoka]www[01:10].nantoka.line.local

14年6月10日火曜日

ROLEs and TYPEs

Roles (web, app, db, hadoop, ...)

Groups (service-names, ...)

OSs, Server models, Disk types (HDD, SSD, ioDrive, ...)

IP addrs (local, global, virtual)

... and many more

14年6月10日火曜日

YabitzOur host management tool/server:

WebUI and WebAPI (json)

hostname[s], status, service, CPU, memory, disk, rack position, size, local/global/virtual IPs, virtual/parent server, ....

tagschanges(history, diff)anything

links to other internal web tools

14年6月10日火曜日

Dynamic inventory

Dynamic inventory plugins

Cobbler external inventory script

Example for linux install server

AWS EC2 external inventory script

Digital Ocean, Google Compute Engine, Linode, Zabbix, ...

Mix inventories (static and dynamic)

14年6月10日火曜日

Custom dynamic inventory

Original scripts AS dynamic inventory

“Developing Dynamic Inventory Sources”http://docs.ansible.com/developing_inventory.html

executable program

implement `--list`

implement `--host HOSTNAME`

14年6月10日火曜日

JSON: for `--list`

Dynamic inventory: output{ "databases": { "hosts": [ "host1.local", "host2.local" ], "vars": { "a": true } }, "webservers": [ "host2.local", "host3.local" ], "tokyo": { "hosts": [ "host1.local", "host4.local", "host5.local" ], "vars": { "b": false }, "children": [ "shibuya", "ebisu" ] }, "shibuya": [ "host6.local" ], "ebisu": [ "host7.local" ]}

14年6月10日火曜日

JSON: for `--host HOSTNAME`

Dynamic inventory: output

{ “localip”: “192.0.2.53”, “virtualip”: “203.0.113.53”, “global_fqdn”: “tagomoris.example.com”,}

14年6月10日火曜日

Dynamic inventory reference

1. inventory --list

2. inventory --host host1.local

3. inventory --host host2.local

4. inventory --host host4.local

5. ...

14年6月10日火曜日

Playbook execution

$ ansible-playbook PLAYBOOK_PATH.yml

14年6月10日火曜日

Yabitz inventory

ansible-playbook

bin/yabitz_inventory

Yabitz

playbooks/PLAYBOOK.yaml

Servers

14年6月10日火曜日

Role based playbooks

Flat playbook directory

playbooks/*.yamlplaybooks/SERVICE_NAME/*.yaml

“hosts” are always roleshosts: centoshosts: centos6hosts: hadoop-slavehosts: hadoop-all-nodes:fluentd-all-nodes

14年6月10日火曜日

Yabitz tags and ROLEsTags

Server-type-tag (by studio3104 crawler)

centoscentos5, centos6

Role-based-tag

hadoop-all-nodeshadoop-master, service-hadoop-master2hadoop-slave

14年6月10日火曜日

Implementation examples

ansible.cfg[defaults]hostfile=/path/to/bin/yabitz_inventory

commands$ ansible-playbook playbooks/update-openssl.yaml

$ TARGET_HOSTS=host1.local,host2.local \

ansible-playbook playbooks/SERVICE/foo.yaml

14年6月10日火曜日

No more host lists!

Ansible inventory is very simple and understandable

But we cannot use it for real world operations

especially for hundreds or thousands of hosts...

Use dynamic inventory!

14年6月10日火曜日

Role based playbook testing

Serverspec

testing without breaking operations

providing double check (beside of ansible)

14年6月10日火曜日

Serverspec directoriesfrom http://serverspec.org/tutorial.html

host based spec directory treespec/SERVERNAME/function_spec.rb

and exec (`rake spec`)

“How to share serverspec tests among hosts”http://serverspec.org/advanced_tips.htmlstatic file based test sharing

14年6月10日火曜日

Specs and roles

Role based provisioning

roles-to-hosts resolution MUST be done dynamically

Helper command to get roles from playbooks

and role based spec directorybin/kick-spec

spec/ROLE/*_spec.rb

14年6月10日火曜日

Implementation example(2)

$ bin/kick-spec playbooks/BOOK.yaml

$ bin/kick-spec playbooks/BOOK.yaml \

--host host1.local --host host2.local

$ bin/kick-spec --role latest_openssl \

--host host5.local

14年6月10日火曜日

Conclusion

DO NOT write same data in 2 or more files

USE master data from anywhere

Dynamic inventory makes it possible!

14年6月10日火曜日

NO MORE HOST LISTS!!!!!!!!

Thanks!

14年6月10日火曜日

Appendix: “_meta”(1)Special attribute of JSON for `--list`

To include host variables in single JSON result

No more exec: `--host HOST`

{ “shibuya”: [ “host6.local” ], “ebisu”: [ “host7.local” ], “_meta”: { “hostvars”: { “host6.local”: { “var1”: “value1”, “var2”: “value2” }, “host7.local”: { .... } } }}

14年6月10日火曜日

Without `_meta`:

`--list` x1 and `--host` x1000 !

With `_meta`:

`--list` x1 only!

Appendix: “_meta”(2)

14年6月10日火曜日

Benchmark: time before task startsw/ yabitz_inventoryfor thousands of hosts

Appendix: “_meta”(3)

w/o _meta w/ _meta

Elapsed time 18min! 10sec!!!!

14年6月10日火曜日