saltconf14 - ryan lane, wikimedia - immediate consistency with trebuchet deploy and saltstack
DESCRIPTION
Deployment is often omitted from configuration management and orchestration, which typically focuses on the infrastructure and not the core application that the infrastructure supports. This presentation will give an overview of a deployment system I wrote using SaltStack to provide immediately consistent deployment for software. The deployment system has the ability for manual deploys or for continuous deployments using a Heroku-like workflow. It has support for multiple deployment schemes, with a default of Git deployment. The deployment system uses a combination of Salt peer runner calls, runners, modules, returners, pillars and grains as well as a deployment host running Apache, Sartoris (a git-deploy frontend), and Redis (for reporting).TRANSCRIPT
Immediate ConsistencyWith Trebuchet Deploy
Deployment is anignored problem
Sync
Common
All
Php
- Poor immediate consistency
- Poor reporting during deployment
- No reporting after deployment
- Difficult to modify and maintain
- Non-extensible
- No service orchestration
- Difficult to use
sync-commonsync-common-allsync-common-filesync-dblistsync-docrootsync-dirsync-filesync-wikiversionsscapapache-gracefulapache-graceful-allsync-apache
+ Has eventual consistency
+ Has fast transmission mechanism via tiered rsync
TrebuchetDeployment
System
+ Immediate consistency via deployment module
+ Eventual consistency via states
+ Reporting via returners
+ Easy modification and extension via modules and runners
+ Service orchestration via reactors, runners and modules
+ Targeting via grains or pillars
+ Configuration via pillars
deployment_config: parent_dir: /srv/deployment redis: db: 0 host: reporting.example.org port: 6379repo_config: test/testrepo: grain: testrepo upstream: https://gerrit.wikimedia.org/r/p/test fetch_module_calls: saltutil.refresh_pillars checkout_module_calls: testrepo.gen_css service_name: test-service
- Git is slow for large binaries
- No canary support yet
git deploy startgit deploy syncgit deploy abortgit deploy revertgit deploy service <action>git deploy reportgit deploy <extension>
TrebuchetTrigger
Mission
Enable the world to treat Wikimedia's infrastructure like any of
its projects: anyone can edit.
TrebuchetPaaS
Join us!
Ryan Lane
Lyft
trebuchet-deploy on GitHub
IRC (Freenode):
#trebuchet-deploy
Come to the puppet + salt
integration talk later today!