continuous delivery workflow with docker

28
By Igor Moochnick IgorShare Consulting [email protected] Continuous Delivery workflow with Docker

Upload: igor-moochnick

Post on 15-Apr-2017

1.070 views

Category:

Software


6 download

TRANSCRIPT

Page 1: Continuous delivery workflow with Docker

By Igor Moochnick

IgorShare [email protected]

Continuous Delivery workflow with Docker

Page 2: Continuous delivery workflow with Docker

a) What is Docker?b) None at allc) Developmentd) CI/CD infrastructuree) Testingf) Production

How are you using Docker?

Page 3: Continuous delivery workflow with Docker

Where do I come from?

Page 4: Continuous delivery workflow with Docker

Use case: Real-Time high-volume data processing system running on any cloud and on-prem physical machines

Data

pre

-pro

cess

ing:

cl

eanu

p/sa

nitiz

ation

InjectPush/pull

InjectPush/pull

InjectPush/pull

Parti

tioni

ng

Even

t Sou

rcin

g

Sorti

ng

l

l

l

l

l

l

l

l

l

l

l

l

l

l

Processing Graph

Page 5: Continuous delivery workflow with Docker

•NetflixOSS / SpringCloud•Spark•Cassandra•Kafka•ELK Stack•Mesos•Jenkins•Docker

Technology stack (some of the components…)

Page 6: Continuous delivery workflow with Docker

System Components in Production (multi-host)

l

l

l

l

l

l

Multi-host

Page 7: Continuous delivery workflow with Docker

Sandbox environments

ll l

l l

Longevity/Failure/Integration/etc…

Page 8: Continuous delivery workflow with Docker

•Everything:▫Applications▫Services. Ex: Nginx, Consul, Mesos, etc…▫Operational components. Ex: logstash, PuppetD, DataDog, NewRelic, etc…▫CI/CD infrastructure. Ex: Jenkins, Artifactory, Docker Registry, etc…▫And more …

What is going into containers?

Except …

Page 9: Continuous delivery workflow with Docker

•Everything else:▫Data▫Secrets (Ex: tokens, keys, certificates, etc…)▫Configuration/Settings

Unless versioned with the container▫Any other volatile piece of data that can change from environment to

environment

What is not going into containers?

Page 10: Continuous delivery workflow with Docker

Branching model for CD Deploy to Prod

Deploy to Prod

Main / trunk

brunch1

brunch2

Page 11: Continuous delivery workflow with Docker
Page 12: Continuous delivery workflow with Docker
Page 13: Continuous delivery workflow with Docker
Page 14: Continuous delivery workflow with Docker

Release process as artifact

https://github.com/constantcontact/jenkins_pipeline_builder

Jenkins Pipeline Builder

Page 15: Continuous delivery workflow with Docker

Jenkins Workflow Plugin (newcomer)

Page 16: Continuous delivery workflow with Docker

Anatomy of a Continuous Delivery Release process

Commit Acceptance ReleaseProduction

Environment

Code

Artifact

Artifacts

ConfigurationOrchestration

OrchestrationTrigger

Sandbox Environment

Page 17: Continuous delivery workflow with Docker

Commit + Acceptance Stage

version = 0.0.1

./gradlew build

gradle.properties

compile test

Dockerfile.template

Everything is a release candidate !!!

./gradlew e2e

mysvc-0.0.1.jar

./gradlew docker

mysvcdojo/mysvc:0.0.1

ADD {name}-{version}.jar

Orchestration+

health-checkTear-down

Commit stage Acceptance stage

Sandbox environment

Page 18: Continuous delivery workflow with Docker

•Make the contents of containers unified▫As repeatable as possible across different types of applications▫Same ports▫Same file locations (logs, traces, configs …)▫Unified way of applying metadata

Anatomy of a Dockerfile

OS+Security+Environment = service environment service environment + service = operational service

Dockerfile.template

Page 19: Continuous delivery workflow with Docker

Docker layering and inheritance hierarchy

102bb6bb1f36

ce70f38a21dd

d36afbf40e7e 4b74a8a8f2fb

7af2e2ece3a9

c2f61748d146

529b51a5f129

d6bd0ffc4755 FROM img:1.0

EXPOSE 8080

EXPOSE 8080

ADD app-0.3.jar app.jar

ENRYPOINT java –jar app.jar

ENRYPOINT java –jar app.jar

ADD app-0.1.jar app.jar ADD app-0.2.jar app.jar

Page 20: Continuous delivery workflow with Docker

•Environment variables•EXPOSE ports• Labels:

https://docs.docker.com/engine/userguide/labels-custom-metadata/

Metadata

Page 21: Continuous delivery workflow with Docker

Describing sandbox environments

https://github.com/IgorShare/docker_rack DockerRack

Page 22: Continuous delivery workflow with Docker

Why not Docker-compose•Dependency-graph-aware•Validation checks gates•Scriptable / template-driven

Dependency

Check gate

Script

Page 23: Continuous delivery workflow with Docker

•Most likely it’s not a single container• It’s consist of:▫Application container▫Sidecar▫ Log shipping▫Monitoring▫Etc…▫Sometimes:

DB Memcache Etc…

What is a containerized application?

Pod

Page 24: Continuous delivery workflow with Docker

Log management

Container 1

Container 2

Container 3

Container 4 Log Volume

Filebeat Logstash

ElasticSearch

Kibana

1) File-based logging

Container 1 Gelf

Logstash

ElasticSearch

Kibana

2) File-less logging

Container 2 Gelf

Container 3 Gelf

Container 4 Gelf

Worker Host Worker Host

Log Volume

Log ManagementCluster

Log ManagementCluster

Page 25: Continuous delivery workflow with Docker

•Mesos scheduling▫Marathon▫Chronos▫Custom scheduler

Failure and Recovery

Page 26: Continuous delivery workflow with Docker

by Adrian Cockcroft@adrianco

Page 27: Continuous delivery workflow with Docker

Micro-services Dojo

http://accordance.github.io/microservice-dojo/

Use it!Give feedback!Contributions arewelcome!

Page 28: Continuous delivery workflow with Docker

Thank you!@igor_moochnick

[email protected]

http://r44e.wordpress.com/