SalterUsing Go to Provision Saltstack Clusters on AWS

Dave SmithVP, Engineering

@dizzyd

Salter - https://github.com/orchestrate-io/salter

Motivations

• Orchestrate manages clusters on AWS

• Clusters should be reproducible

• Love saltstack, but saltcloud didn’t exist yet

Salter - https://github.com/orchestrate-io/salter

Background

• Gen0: Ruby app (agh!)

• Gen1: Python app (better…)

• Gen2: Golang app (best!)

Salter - https://github.com/orchestrate-io/salter

What’s It Do?

• Constructs a reproducible cluster from a specification file

• Bootstraps salt master + minions into the cluster

• Provides commands to manage the cluster

Salter - https://github.com/orchestrate-io/salter

Notable Features

• Multithreaded instance initiation

• Sandboxing of credentials based on AWS key

• Clean logging of highstate for multiple nodes

Salter - https://github.com/orchestrate-io/salter

Cluster Spec

• nodes - count, roles, sgroup

• sgroups - text-based rules

• aws - flavor, ami, region, zone, keyname

• salt - data dir, userdata file

• salt.grains

Salter - https://github.com/orchestrate-io/salter

Cluster Spec Example[nodes.prod-api]roles = ["api"]count = 3sgroup = “api”

[sgroups.api]rules = [ # SSH access "tcp:22:22:0.0.0.0/0", # Monitoring "icmp:8:-1:monitor", # Ping “tcp:5666:5666:monitor",# NRPE "tcp:123:123:monitor", # NTP "udp:123:123:monitor", # NTP (udp)]

[nodes.prod-monitor]roles = ["monitor"]sgroup = “monitor"

[aws]ssh_username = "ubuntu"flavor = "m1.xlarge"ami = "ami-271a484e"region = "us-east-1"

Salter - https://github.com/orchestrate-io/salter

Lifecycle Of A Cluster

launch upload

highstate

ssh teardown

sgroups

Salter - https://github.com/orchestrate-io/salter

Launching A Cluster

$ salter -c staging.cfg -a launchUsing data dir: .salter/data/68ae66ab14900staging (i-4ca8171f): startedstaging (54.211.183.243): running 16:37:08 up 0 min, 0 users, load average: 0.75, 0.19, 0.06

Salter - https://github.com/orchestrate-io/salter

Uploading Salt Config$ salter -c staging.cfg uploadUsing data dir: .salter/data/680b14900Uploading deploy/salt to 54.80.213.168:/srv/salt...building file list ... donecreated directory /srv/salt./pillar/pillar/global.slspillar/top.slspillar/users.sls…Running saltutil.sync_all...Running mine.update...Running saltutil.refresh_pillar...

Salter - https://github.com/orchestrate-io/salter

Highstating

$ salter -c staging.cfg highstateUsing data dir: .salter/data/680cb14900 staging: 0 errors 917 changes 352 states. master: 1 errors 917 changes 352 states.

Salter - https://github.com/orchestrate-io/salter

Golang Joy

• Static compilation / easy distribution

• Parallelism constructs

• Fun, new lang with good library support for AWS and SSH

Salter - https://github.com/orchestrate-io/salter

Golang Pain

• Source/library management

• Forking a library requires code changes (?!)

• Typing system can be awkward

• for-each construct requires reflection

Salter - https://github.com/orchestrate-io/salter

Nifty Utility Features

• ssh - never lookup an IP again

• hosts - get an easy hosts file for whole cluster

• info - see internal/external IPs

• csshx - access all nodes of a certain type

Questions?