build, publish, deploy and test docker images and containers with jenkins workflow

44
Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow. Nigel Harniman Senior Solutions Architect, CloudBees Inc

Upload: docker-inc

Post on 12-Jan-2017

10.178 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow.Nigel HarnimanSenior Solutions Architect, CloudBees Inc

Page 2: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

About me

Nigel Harniman

@harnimanBuild Engineer,

Devops and Architect

DevOps, Infra as Code,Continuous Delivery

Page 3: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

“Software is eating the world”

Marc Andreessen

Page 4: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

4

How Do You Deliver Better Software Faster?

ProdDev

Page 5: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

5

Automation is the Key

Photo courtesy of Steve Jurvetson via Flickr

Page 6: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

The Docker Advantage

Page 7: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Docker Has PotentialAn example: Software Configuration Management Space

Page 8: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Docker Has PotentialAn example: Software Configuration Management Space

Page 9: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

The New World Order: Containers Codify OS Config

9

ProdDev QA Staging

DEV Server/VM QA Server/VM STG Server/VMPROD Server/VM

<PROD OS config><STG OS config><QA OS config><DEV OS config>

App<code>

<APP OS config>

App<code>

<APP OS config>

App<code>

<APP OS config>

App<code>

<APP OS config>

Page 10: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

So is this how I build a Docker Image?

10

Page 11: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Jenkins & Docker

Page 12: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

How Can You Use Jenkins & Docker Together?

+

Page 13: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

How Can You Use Jenkins & Docker Together?

1. Run Jenkins Masters & Slaves in Docker

2. Build, Test, & Deploy Docker Images from Jenkins

Page 14: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

1. Run Jenkins Masters & Slaves in DockerDocker (Cloud) – use Docker images as standardized build environments to improve isolation and elasticity

Docker Custom Build Environment – specify customized build environments as Docker containers

CloudBees Docker Shared Config – manage Docker (or Swarm) host configuration centrally in CloudBees Jenkins Operations Center

Page 15: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

2. Build, Test, & Deploy Docker Images from JenkinsBuild and Publish – build projects that have a Dockerfile and push the resultant tagged image to Docker Hub

Docker Traceability – identify which build pushed a particular container and displays the build / image details in Jenkins

Docker Hub Notification – trigger downstream jobs when a tagged container is pushed to Docker Hub

Page 16: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Jenkins Workflow & Docker

Page 17: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Jenkins Workflow PrimerJenkins powered CD pipelines

Jenkins Workflow

ProdDevPerf Test

BuildCommit Selenium Test Stage Deploy

Sonar Test

Pipelines Need: Branching Looping Restarts

Checkpoints Manual Input

??

Page 18: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Key Workflow Features

18

Entire flow is one concise Groovy script using Workflow DSL• For loops, try-finally, fork-join …

Can restart Jenkins while flow is running

Allocate slave nodes and workspaces• As many as you want, when you want

Stages throttle concurrency of builds

Human input/approval integrated into flow

Standard project concepts: SCM, artifacts, plugins

Page 19: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Jenkins Workflow + Docker

Page 20: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

20

Pipeline Stages

Build and Unit Test App

Test Docker Image

Publish Docker Image

SCM Checkoutmvn package

mvn sonar:sonarmvn verify

docker build

docker tag

docker run

notifycucumber

war

img

Sonar Analysi

s

Prepare Release

Build Docker Image

Int Test

docker push

image.inside withServer

Page 21: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

21

Build, unit test and package

Build and Unit Test App

Test Docker Image

Publish Docker Image

SCM Checkoutmvn package

mvn sonar:sonarmvn verify

docker build

docker Tag

docker run

notifycucumber

war

img

Sonar Analysi

s

Prepare Release

Build Docker Image

Int Test

docker push

image.inside withServer

Page 22: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Build, unit test and package

stage 'Build App’

node('docker') {

docker.image(‘maven:3.3.3-jdk-8’).inside(‘-v /data:/data’ {

mkdir –p /data/mvn

writeFile file: 'settings.xml', text: ”(………)"

git 'https://github.com/cloudbees/mobile-deposit-api.git’

sh 'mvn –s settings.xml clean package’

Specify the Stage Name

Specify the slave labelCustom Build Env Mount volume from

slave

.m2 repo locationco and build

Page 23: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Defining a Docker SlaveSpecify Image as

templateAssign labels

Page 24: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

24

Test the app

Build and Unit Test App

Test Docker Image

Publish Docker Image

SCM Checkoutmvn package

mvn sonar:sonarmvn verify

docker build

docker Tag

docker run

notifycucumber

war

img

Sonar Analysi

s

Prepare Release

Build Docker Image

Int Test

docker push

image.inside withServer

Page 25: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Test the app

node('docker') {

docker.image(‘maven:3.3.3-jdk-8’).inside(‘-v /data:/data’ {

stage 'Sonar analysis’

sh 'mvn -s settings.xml sonar:sonar’

stage 'Integration-test’

sh 'mvn -s settings.xml verify’

step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])

}

In same env as build

Sonar tests

Run API Tests

Page 26: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

26

Build, test and publish Docker image

Build and Unit Test App

Test Docker Image

Publish Docker Image

SCM Checkoutmvn package

mvn sonar:sonarmvn verify

docker build

docker Tag

docker run

notifycucumber

war

img

Sonar Analysi

s

Prepare Release

Build Docker Image

Int Test

docker push

image.inside withServer

Page 27: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Build, test and publish Docker image

docker.withServer('tcp://192.168.99.100:2376', 'slave-docker-us-east-1-tls'){stage 'Build Docker image’

def mobileDepositApiImage

dir('.docker') {sh "mv ../target/*-SNAPSHOT.jar mobile-deposit-api.jar”

mobileDepositApiImage = docker.build "harniman/mobile-deposit-api:$

{buildVersion}”

} stage 'Test Docker image’

container=mobileDepositApiImage.run("--name mobile-deposit-api -p 8080:8080”)

sh "curl

http://<user>:<token>@<host>:8080/docker-traceability/submitContainerStatus \......// insert cucumber tests here

stage 'Publish Docker image’

withDockerRegistry(registry: [credentialsId: 'dockerhub-harniman']) {mobileDepositApiImage.push()

}

}

Bind to docker host

Change directory

Launch container

Build docker image

Bind to registryPush image

Submit traceability report

Page 28: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

28

Tagged Version

aTagged Image in Docker Huba

Page 29: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Traceability

Page 30: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

30

Traceability

Builds on existing Jenkins artifact traceabilityAllows the tracking of the creation and use of Docker containers in Jenkins and their future use.Combine with artifact fingerprinting for a comprehensive solutionEach Build shows the image fingerprints created

Identify which build pushed a particular container and display the build / image details in Jenkins

Image fingerprints

Page 31: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

31

Traceability – registering eventsJenkins can track actions against this image such as:

• Creating a container• Container events such as start/stop

To achieve this, it is necessary to call the Traceability API – see $(JENKINS_URL)/docker-traceability/api/There are two endpoints to submit events to:/docker-traceability/submitContainerStatus

Allows to submit the current container status snapshot with a minimal set of parameters. Outputs of docker inspect $(containerId) can be directly submitted to Jenkins server using this command.

/docker-traceability/submitReport

Submits a report using the extended JSON API. This endpoint can be used by scripts to submit the full available info about the container and its environment in a single command.

Page 32: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

32

Traceability – registering events - exampleWorkflow usage example:

container = mobileDepositApiImage.run("--name mobile-deposit-api -p 8080:8080")

sh "curl http://<user>:<token>@<host>:8080/docker-traceability/submitContainerStatus \

--data-urlencode status=deployed \--data-urlencode inspectData=\"\$(docker inspect $container.id)\" \--data-urlencode environment=test \--data-urlencode hostName=mymac \--data-urlencode imageName=harniman/mobile-deposit-api"

Spin up container

Notify Jenkins

Page 33: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

33

Docker Traceability View

Docker TraceabilityContainer

Page 34: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Container Use View

34

Deployment Events

Link to Build

Page 35: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Dockerhub Notifications

Page 36: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

36

Docker Hub NotificationTrigger downstream jobs when a tagged container is pushed to Docker Hub

The Docker Hub Notification Trigger plugin lets you configure Jenkins to trigger builds when an image is pushed to Docker Hub. E.g. to run verification for the container.

What are the steps

Set up a WebHook Account for Notification

Set up your Docker Registry to make callbacks on Image events

Set up your builds

Page 37: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

37

Docker Hub Notification – Docker Registry WebhookIn the format:http://<user>:<token>@<jenkins_url>/dockerhub-webhook/notify

Page 38: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

38

Docker Hub Notification – Job Set up

Configure Trigger

Page 39: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

In Conclusion

Page 40: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

40

Docker and Jenkins with Workflow is the proven CD Platform

+

TESTING

STAGING

PRODUCTIONWorkflow CD Pipeline Triggers:• New application code (i.e. feature, bug, etc.)• Updated certified stack (security fix in Linux, etc.)

… will lead to a new gold image being built and available for…… TESTING

… STAGING… PRODUCTION

All taking place in a standardized/similar/consistent environment

<OS config>

Company“Gold”

Docker Img(~per app)

App<code>

(git, etc.)<OS config>

Certified DockerImages

(Ubuntu, etc.)

Jenkins Workflow

Page 41: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

CloudBees: Leading the Way for Docker and CD

Docker Workflow – Provides first-class support for Jenkins Workflow to build real world CD pipelines for containerized applications using Jenkins and Docker

Build and Publish – Builds projects that have a Dockerfile and pushes the resultant tagged image to Docker Hub

Docker Hub Notification – Triggers downstream jobs when a tagged container is pushed to Docker Hub

Docker Traceability – Identifies which build pushed a particular container that is running in production and displays that on the Jenkins builds page

Docker – Uses Docker containers as standardized build environments to improve isolation and elasticity – Dockerized Build Slaves

Docker Custom Build Environment – Specifies customized build environments as Docker containers

Page 43: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

How Do You Manage CD at Enterprise Scale?

43

CloudBees Jenkins PlatformJenkins at Enterprise Scale for CI and CD

Page 44: Build, Publish, Deploy and Test Docker images and containers with Jenkins Workflow

Thank you!Nigel Harniman@[email protected]