building a scalable ci platform using docker, drone and rancher
TRANSCRIPT
© 2015 Rancher Labs, Inc.© 2015 Rancher Labs, Inc .
Building a Scalable CI Deployment with Docker, Drone and RancherAugust 13, 2015
#ranchermeetup
© 2015 Rancher Labs, Inc.2
Shannon WilliamsCo-Founder/VP Marketing @smw355
Darren ShepherdCo-Founder/Chief Architect @ibuildthecloudFreenode: darren0
#ranchermeetup
© 2015 Rancher Labs, Inc.3
Bill MaxwellDirector of DevOps @CloudNautique
Brad RydzewskiFounder of Drone.io @bradrydzewski @droneiowww.drone.io
#ranchermeetup
© 2015 Rancher Labs, Inc.4
First things first…
This is a not a !#ranchermeetup
#ranchermeetup
© 2015 Rancher Labs, Inc.5
There are rules for a meetup!• Questions are always welcome• Demo, then demo some more• Keep the slides to a minimum• Open to proposals• Give away stuff with your logo
on it
#ranchermeetup
© 2015 Rancher Labs, Inc.6
Tweets = Shirts
First 10 Rancher Screenshots posteds on Twitter about today’s
#RancherMeetup will get a Rancher shirt
Follow @rancher_labs
#ranchermeetup
© 2015 Rancher Labs, Inc.7
Agenda• Why is Docker becoming so popular in DevOps• How we run our CI process at Rancher• Demo: Drone, Docker, Rancher• Drone.io Overview and RoadMap• Latest Rancher News: LibCompose, Rolling Upgrades, Virtual Host
LB, Sidekicks, etc.
#ranchermeetup
© 2015 Rancher Labs, Inc.8
Containers are growing in popularity because they improve the entire DevOps pipeline
• Quick onboarding of developers• Consistent software builds• Accurate test environments• More frequent code releases• Immutable production environments• Orchestrated upgrades
© 2015 Rancher Labs, Inc.9
Many organizations are looking at how to incorporate Docker into their pipelines
Source Code Repo
CI/CD
Build and Test tools
Docker Tools
Develop Build Package Test Deploy/Upgrade Operate
© 2015 Rancher Labs, Inc.10
Open source software for building a Private Container Service
© 2015 Rancher Labs, Inc.11
What is a container service?Central service where users and teams can create container environments for running services and applications
Includes:1. Container Orchestration2. Container Infrastructure3. Container Management
© 2015 Rancher Labs, Inc.12
Environment
What is a container service?
Private Container Service
Resources
Resources (Clouds, VMs, Servers)
Environment
Resources
DevOpsPipeline
User Directory Container Registry
Ops Tools
© 2015 Rancher Labs, Inc.
Build Environment
DevOps builds software in the container service
- DevOps teams create Build Environments- Adds resources and Jenkins slaves- Developer Pushes Code to Repo- Jenkins creates build containers- Build and Unit Tests run in build container- Outputs Artifacts- Jenkins creates Packaging container- Push Docker Image to registry- Build environments scale elastically
BuildDocker File
PackagingDocker File
TestCode Build Docker
File
Develop Build Package Test Deploy/Upgrade Operate
© 2015 Rancher Labs, Inc.
Test Environment
DevOps tests software in the container service
- Create Test Environments- Upload Compose & Test files- Build triggers Jenkins- Jenkins triggers Rancher to deploy compose- Rancher creates containers, networks,
storage, etc.- Test container conducts integration tests
Develop Build Package Test Deploy/Upgrade Operate
Test docker-compose.yml lb: image: haproxy links: - webtomcat: image: app:buildn links: - db - cachecache: image: memcachedb: image: mysql:testn
Hosts
db- mysql:testn
cache- memcache
Lb-haproxy
tomcat-app:buildn
Test
Test
Compose Images
© 2015 Rancher Labs, Inc.
Production Environment
Hosts
DevOps runs production in the container service
- Production Compose- Rancher deploys
environment- Update Scaling- Health Checks & HA- Upgrade- Backup & Recovery
Develop Build Package Test Deploy/Upgrade Operate
docker-compose.yml lb: image: haproxy links: - webtomcat: image: app:buildn links: - db - cachecache: image: memcachedb: image: mysql
lb
tomcat
CacheMySQL
Health Check Interval: 2 secTimeout: 2 secFailure Retry Frequency: 3Success Retry Frequency: 2Recovery Delay: 30 sec
tomcat(build n+1)
Drone, Docker and Rancher
CI here at Rancher Labs
Bill MaxwellDirector of DevOps @CloudNautique
17 #ranchermeetup
© 2015 Rancher Labs, Inc.18
CI / CD @ Rancher Labs
• Primarily Developers
• Distributed• GitHub Focused
The Team
© 2015 Rancher Labs, Inc.19
Solution Space- Low Administrative Overhead- GitHub Integration- Docker Integration
?
© 2015 Rancher Labs, Inc.20
Selection Drone.IO- Built around Docker- State is stored in
Database- Single container
deployment- Simple GitHub
integration
© 2015 Rancher Labs, Inc.
Pipeline View
GitHub Drone
Tests
MasterBuilds
BuildBuild
BuildBuild
refresh
Developer PushesCode
RunTests
WebHook
StatusAPI
© 2015 Rancher Labs, Inc.22
Scaling Drone- Number of
Developers- Build frequency- Multiple projects- Open Source
© 2015 Rancher Labs, Inc.23
RancherWith - Private Networking- Native DockerWe Get- Easily Distributed Drone!
Rancher
DroneWorker
Worker
Worker
Worker
Worker
Worker
© 2015 Rancher Labs, Inc.24
Demo
#ranchermeetup
© 2015 Rancher Labs, Inc.25
Brad RydzewskiFounder of Drone.io @bradrydzewski @droneiowww.drone.io
#ranchermeetup
Latest news about Drone.io
New in 0.4
● Improved Yaml● Improved UX● Matrix Builds● External Plugins
build: image: golang commands: - go test -v
compose: postgres: image: postgres environment: - PG_USER=foo - PG_PASS=bar
Composable Builds
build configuration similar to docker
compose
build: image: golang:$$go_version commands: - go test -v compose: redis: image: redis:$$redis_version
matrix: go_version: - 1.4 - 1.3 redis_version: - 2.8 - latest
Matrix Builds
tests 4 different environment permutations
build: image: golang commands: - go test -v
notify: slack: image: plugins/slack token: ...
External Plugins
docker pull \ plugins/slack
docker run plugins/drone-slack << EOF{ “repo”: { ... }, “build”: { “number”: 1, “status”: “success” } “slack”: { “token”: “...” }}EOF
External Plugins
Plugins accept build details via stdin
Latest Rancher Enhancements
31
Load Balancing• Virtual Host Support for LB Services• Labels/Scheduling for Load Balancers in UI• Internal Load Balancing• Load Balancing External Services• Supporting CNAME for External Services
Service Discovery and Management• Cross Stack Linking• Service Sidekicks• Rancher Compose CLI support for Rolling Upgrades
Other• Support Random IP for DNS• Improved Rancher Server SSL Support
© 2015 Rancher Labs, Inc.32 #ranchermeetup
http://info.rancher.com/beta
© 2015 Rancher Labs, Inc.33
Getting StartedRancher and RancherOS live in GitHub – Get Involved, Join the Beta
#ranchermeetup
http://github.com/rancher
© 2015 Rancher Labs, Inc.© 2015 Rancher Labs, Inc .
Thank yourancher.com