delivery pipelines at symphony talent - present and future
TRANSCRIPT
Delivery PipelinesPresent and Future
Nathan JonesOctober 2016
Nathan Jones
DevOps Lead // Symphony Talent
@_ncjones
Context
4
40
124
Dev Teams of 4-6
Deployable Components
Git Repositories
320 AWS EC2 Instances
1 Prod Environment (Multi-Tenant SAAS)
Logical Pipeline
App Source CI Build Test Staging Prod
Deploy Deploy Deploy
Test TestTestBuildPublish
Merge PR
Build
${app}-${version}.tar.gz
App Source
RAKE
Publish
Builds S3 Bucket
${app}-${version}.tar.gz
Deploy
BuildsS3 Bucket
Build Agent
App Server
3 Download & Configure App
/${env}/${app}/version.txt
1 Promote App version to Env
App Servers
2 Trigger Puppet Agents
Test
App Source CI Build Test Staging Prod
Pre-Publish Post-Deploy
Post-Deploy Testing
Deployed App
Build Agent
CucumberTest Suite
Build Agent
Pre-Publish Testing
Containerized App
CucumberTest Suite
The Good...
Same artifact for all envs
Automated Deploy
Configuration as Code (Puppet)
Automated Tests
✓
✓
✓
✓
Puppet Changes Not Isolated
App Code CI Build Test Staging Prod
PuppetCode
Ad-hoc Infrastructure Changes
App Code CI Build Test Staging Prod
OS Patching
AWS Resources
Manual Pipeline Transitions
Too Many Test Environments
Staging ProdApp Code CI Build
Test
Team 5Team 4
Team 3Team 2
Team 1
The Good...
Puppet not isolated per env
The Bad...
Ad-hoc infrastructure changes
Ad-hoc server patching
Too many test environments
Manual pipeline state transitions
✗
✗
✗
✗
✗
Same artifact for all envs
Automated Deploy
Configuration as Code (Puppet)
Automated Tests
✓
✓
✓
✓
Masterless Puppet (in progress) Puppet not isolated per env
Terraform: infrastructure as code
Packer: patch AMIs not VMs
Pipeline tool (future)
Solves itself (future)
Ad-hoc infrastructure changes
Ad-hoc server patching
Too many test environments
Manual pipeline state transitions
✓
✓
✓
✓
✓
Current Work
Pipeline Tools
Jenkins PipelinesAWS CodePipeline
Unified Pipelines
Application
Staging Prod
Puppet Configuration
Terraform Resources
Packer Machine Images
Deploy
Test
Deploy
Test
Build Pipelines Deploy Pipeline