Deploying OpenStack MarconiCreating Parallel Universes with SaltStack

Oz Akan, Cloud Engineering Manager, Rackspace

Outline

•Marconi•Why SaltStack?•Universe | Environment•Salt Concepts•Framework•Summary

Marconi

Marconi

Marconi

Message Queue

Marconi

6data centers

Marconi

360servers

Marconi

5 Billion

transactions per day

Marconi

…butsomething more

remarkable

Marconi

load balancersweb servers

catalog databasesqueues databases

zenoss masterzenoss collectors

graylog serverselastic search servers

bastionsusage tracking workers

usage tracking databases

Marconi

from nothing

Marconi

Marconi

45 minutesfrom nothing to web

scale

Why Salt?

Challenges

human mitsakes

Challenges

scaleof web

Challenges

sc li gdynamism

a n

Challenges

environmentsmultip

le

environments

Universe | Environment

Universe | Environment

laws | rulesdefined by

Salt Concepts

Salt Concepts

grainon minions

Salt Concepts

pillaron master

Salt Concepts

environmentmaps to a folder

Salt Concepts

directory overlayfor states and

pillar

Directory Overlay Example

file_roots: prod: - /srv/salt/prod - /srv/salt/base

Salt Concepts

minequery minions

Salt Concepts

mapfor salt-cloud

Salt Concepts

overstatemore

orchestration

Salt Concepts / overstate exampleset-mongodb_server:

match: 'G@environment_id:marconi-prod-ord and G@roles:mongodb_server'

sls:

- mongodb_server

require:

- set-firewall

set-mongodb_replica:

match: 'G@environment_id:marconi-prod-ord and G@roles:mongodb_server and G@mongodb_role:primary'

sls:

- mongodb_server.replica

require:

- set-mongodb_server

Framework

Framework

roleis many things

Framework / role

grainrole: web_server

Framework / role

formulasif..else in db_servermongodb

Framework / role / formulas example

# queues_server/init.sls

{% if 'roles' in grains and 'queues_server' in grains['roles'] %}

include:

- marconi

- memcached

- queues_server.kernel

- queues_server.install

{% endif %}

Framework / role

minionspillar

Framework / role / minions example

# pillar/minions.sls

minions:

cdb1a-cqp-ord:

roles:

- mongodb_server

attributes:

mongodb_replica_set: catalog-rs1

mongodb_role: primary

db_type: catalog

…

web4a-cqp-ord:

roles:

- queues_server

attributes:

mongodb_replica_set: catalog-rs1

queues_api: queue

Framework / role

devicespillar

Framework / role / devices example

# pillar/devices.sls

devices:

load_balancers:

text: 'cloud load balancers'

addresses:

- 10.183.250.0/23

marconi-endpoint:

text: 'marconi ORD endpoint'

fqdn: ord.queues.api.rackspacecloud.com

protocol: https

address: 192.237.142.76

…

graylog_lb:

text: 'graylog load balancer'

fqdn: log.marconi-graylog.com

Framework / role

networkspillar

Framework / role / networks example

# pillar/networks.slsnetworks: vpn-all: text: ’vpn networks' addresses: - '10.1.2.3/22' - '10.2.3.4/24’

… salt-master: text: 'salt master servers' addresses: - '10.178.129.47/32' - '162.200.150.120/32'

Framework / role

roles pillarsections per formula

Framework / role / pillar example

# pillar/roles.sls

roles:

role:

text:

attributes:

flags:

clients:

minions:

networks:

devices

Framework / role / pillar example

# pillar/roles.sls

roles:

mongodb_server:

text: 'marconi mongodb database server’

attributes:

- mongodb_replica_set

- mongodb_role

- db_type

flags:

- mongodb_replica_set_configured

Framework / role / pillar example

# pillar/roles.sls

roles:

mongodb_server:

clients:

minions:

-

roles: ['bastion_server']

protocols:

-

name: tcp

ports: ['22']

states: ['NEW','ESTABLISHED']

text: 'ssh access'

Framework / role / pillar example

# pillar/roles.sls

roles:

mongodb_server:

clients:

minions:

-

roles: ['queues_server','mongodb_server','memcached_server','bastion_server']

protocols:

-

name: icmp

types: ['0','8']

text: 'ping access'

Framework / role / pillar example

# pillar/roles.sls

roles:

mongodb_server:

clients:

networks:

-

name: vpn-all

protocols:

-

name: icmp

types: ['0','8']

text: 'ping access from zenoss server'

Framework / role / pillar example

# pillar/roles.sls

roles:

web_server:

clients:

devices

-

name: load_balancers

protocols:

-

name: tcp

ports: ['443']

text: 'http access from lb to server'

states: ['NEW','ESTABLISHED','RELATED']

Framework

environmentis many things

Framework / environment

• project• purpose• location

set of grains

Framework / environment

environment_idproject-purpose-

location

Framework / environment

/etc/salt/masterfile_roots,

pillar_roots

Framework / environment example

file_roots:

base:

- /srv/salt/marconi/base

marconi-prod-lon:

- /srv/salt/marconi/prod-lon

- /srv/salt/marconi/base

marconi-test-lon:

- /srv/salt/marconi/test-lon

- /srv/salt/marconi/base

pillar_roots:

base:

- /srv/salt/marconi/base/pillar

marconi-prod-lon:

- /srv/salt/marconi/prod-lon/pillar

- /srv/salt/marconi/base/pillar

marconi-test-lon:

- /srv/salt/marconi/test-lon/pillar

- /srv/salt/marconi/base/pillar

Framework / environment example

# folder layoutroot@salt1a:/srv/salt/marconi# ls -1

base

prev-ord

prod-dfw

prod-hkg

prod-iad

prod-lon

prod-ord

prod-syd

test-ord

…

Framework

mine in formulas

Framework / mine

firewalljinja template

Framework / mine / firewall {%- if 'scope' in minion %}

{%- if minion.scope == 'project' %}

{%- for key, value in salt['mine.get']('project:' + grains['project'], 'grains.items', expr_form='grain').items() %}

{%- if role in value['roles'] %}

-A INPUT -s {{ key }} -j {{ role|upper }}

…

{%- elif minion.scope == 'environment_id' %}

{%- for key, value in salt['mine.get']('environment_id:' + grains['environment_id'], 'grains.items', expr_form='grain').items() %}

{%- if role in value['roles'] %}

-A INPUT -s {{ key }} -j {{ role|upper }}

…

Framework / mine

hostsjinja template

Framework / mine / hosts{%- for key, value in salt['mine.get']('environment_id:' + grains['environment_id'], 'grains.items', expr_form='grain').items() %}

{{ value['id'] }}:

host:

- present

- ip: {{ salt['mine.get'](value['id'], 'network.ip_addrs').values()[0][0] }}

{%- endfor %}

Framework / mine / zenoss hosts{%- if 'roles' in grains and 'zenoss_server' in grains['roles'] %}

{%- for key, value in salt['mine.get']('roles:zenoss_server', 'grains.items', expr_form='grain').items() %}

{%- if value['project'] == pillar['project'] %}

host_{{ value['id'] }}:

host:

- present

- name: {{ value['id'] }}

- ip: {{ salt['mine.get'](value['id'], 'network.ip_addrs').values()[0][1] }}

{%- endif %}

{%- endfor %}

{%- endif %}

Summary

•grains• mark minions (project, purpose, location, role)

•pillar• to define global rules per role

•salt-mine • to be able to query minions in the environment

•environments and directory overlay

Multiple Environments

Q&A