from continuous integration to continuous delivery with jenkins - javaland.de 2014
DESCRIPTION
The concept of DONE have changed in project teams to evolve from The unit tests are green to The software is shippable in production. Continuous Integration mutated into Continuous Delivery and this process was no longer limited to the DEV teams but had to integrate the OPS team to cover the deployment phases of the applications. Come and discover how the Continuous Integration server Jenkins CI became the nexus of Continuous Delivery orchestrating the phases of complex Application Lifecycle processes. Discover how Jenkins is becoming the lingua franca between DEV teams and OPS teams to deliver applications faster.TRANSCRIPT
©2014 CloudBees, Inc. All Rights Reserved
From Continuous Integration To Continuous Delivery With Jenkins Cyrille Le Clerc, Solution Architect, CloudBees
©2014 CloudBees, Inc. All Rights Reserved
About Me
Cyrille Le Clerc
@cyrilleleclerc
Open Source
CTO
DevOps, Infra as Code, Con9nuous Delivery
Solu9on Architect
©2014 CloudBees, Inc. All Rights Reserved
About you
Dev? Ops? Maven? CI? CD? Cloud?
©2014 CloudBees, Inc. All Rights Reserved
Agenda
• Concepts: CI, Jenkins and CD • New roles for Jenkins – Cross team – Deployment – Workflow
• Great power, great responsibili9es • Jenkins reference architecture
©2014 CloudBees, Inc. All Rights Reserved
Concepts
Continuous Integration
©2014 CloudBees, Inc. All Rights Reserved
Continuous Integration
• Prac9ce of SoIware Engineering • Introduced by eXtreme Programming • Prevent the “Integra9on Hell” • Itera9ve
©2014 CloudBees, Inc. All Rights Reserved
Continuous Integration Principles
• Source code repository • Automated build • Self tes9ng build • Commit to the baseline every day
• Build every commit
• Everybody can see the results of the build
• Latest deliverable easy to download
• Fast build
Wikipedia
©2014 CloudBees, Inc. All Rights Reserved
Jenkins
©2014 CloudBees, Inc. All Rights Reserved
Meet Jenkins… • #1 Con9nuous Integra9on
and Con9nuous Deployment server
• Java web applica9on • “java –jar jenkins.war”
• MIT-‐licensed, 8 years old • 500+ releases
jenkins-ci.org
©2014 CloudBees, Inc. All Rights Reserved
#1 Continuous Integration Tool in the Market
Source: Eclipse Community Survey 2012
©2014 CloudBees, Inc. All Rights Reserved
Massive Adoption, Happy Users
• 61,000+ installa9ons around the world; +60% in last year • 700+ plugins by 500 contributors • Huge community
87% are “very satisfied” or “satisfied”!
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery
©2014 CloudBees, Inc. All Rights Reserved
Why Continuous Delivery
• Limits of Con9nuous Integra9on • Lean SoIware Manufacturing
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery
• “Reliable SoAware Releases through Build, Test, and Deployment AutomaJon”
• DONE = Shippable into Produc9on
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery Steps
Con$
nuou
s Delivery
Source Code
Quality Analysis
Unit Tests
Binaries
©2014 CloudBees, Inc. All Rights Reserved
DEV
DEV
OPS
QA
Continuous Delivery Steps
Con$
nuou
s Delivery
Source Code
Quality Analysis
Unit Tests
Binaries
Perfs Environment
Func Test Environment
UAT Environment
Production Environment
GO LIVE!
Release!
Staging Environment
©2014 CloudBees, Inc. All Rights Reserved
Jenkins: pillar of Continuous Delivery
• Connect the dots
• It is the process
©2014 CloudBees, Inc. All Rights Reserved
New Roles for Jenkins
• Cross team pla_orm • Deployment automa9on
– App, dataset … environment provisioning • Test automa9on: load test, acceptance test … • Release management • Workflow
Jenkins as an Applica9on Lifecycle Management pla_orm
©2014 CloudBees, Inc. All Rights Reserved
Cross Team Platform
• Not only used by DEV but also QA and OPS • Various profiles and skills • Mission Cri9cal
©2014 CloudBees, Inc. All Rights Reserved
Deployment Automation
• Same tools and automa9ons on all the environments
xxx Environment
xxx Environment
• Deploy app, dataset … • Plugins ecosystem
©2014 CloudBees, Inc. All Rights Reserved
Test Automation
• Powered by Jenkins ecosystem • Requires full environment setup (app, dataset)
©2014 CloudBees, Inc. All Rights Reserved
Release Management
• Not only “mvn release:prepare release:perform”
• Includes QA valida9on, Staging …
• “Jenkins' Maven job type considered evil” S. Connolly
©2014 CloudBees, Inc. All Rights Reserved
Workflow
Choreograph ac9vi9es that are … • complex • long-‐running • not sequen9al • pausable • retryable
©2014 CloudBees, Inc. All Rights Reserved
Great power, great responsibilities
Cross team collabora9on pla_orm: • High Availability and SLAs • Security
©2014 CloudBees, Inc. All Rights Reserved
High Availability and SLAs
• Jumbo Jenkins masters are fragile – Mul9ple medium sized Jenkins masters
• Careful version upgrades – Tes9ng environment + Jenkins & plugins version control
• Support • It’s not an open bar, security
Jenkins Opera9ons Center (JOC)
Enterprise Update Center + JOC
Professional support
©2014 CloudBees, Inc. All Rights Reserved
Security
• Authen9ca9on • Audit • Role Base Access Control
©2014 CloudBees, Inc. All Rights Reserved
Jenkins Reference Architecture
©2014 CloudBees, Inc. All Rights Reserved
Multiple Medium Sized Masters Jenkins Enterprise Master-1
Jenkins Enterprise Master-N
JenkinsEnterpriseMaster-N
JenkinsEnterpriseMaster-1
…
©2014 CloudBees, Inc. All Rights Reserved
Master Slave Setup Jenkins Enterprise Master-1
Jenkins Enterprise Master-N
JenkinsEnterpriseMaster-N
JenkinsEnterpriseMaster-1
…
Master-1 BuildNodes
Master-N BuildNodes
©2014 CloudBees, Inc. All Rights Reserved
High Availability Jenkins Enterprise Master-1
Jenkins Enterprise Master-N
HA Proxy
JenkinsEnterpriseMaster-N
JenkinsEnterpriseMaster-1
HA Proxy
NAS Storage
NAS Storage
backup
backup
…
©2014 CloudBees, Inc. All Rights Reserved
Manage Multiple Masters
Jenkins Enterprise Master-N
Jenkins Enterprise Master-1
Jenkins Operations Center
JenkinsOperations
Center (JOC)
config
configJenkins
EnterpriseMaster-N
JenkinsEnterpriseMaster-1
…
Master-1 BuildNodes
Master-N BuildNodes
©2014 CloudBees, Inc. All Rights Reserved
Shared Build Nodes Jenkins Enterprise Master-1
Jenkins Operations Center
Shared BuildNodes
Jenkins Enterprise Master-N
JenkinsEnterpriseMaster-N
Master-1 BuildNodes
JenkinsOperations
Center (JOC)
JenkinsEnterpriseMaster-1
confi
g
config
…
©2014 CloudBees, Inc. All Rights Reserved
Testing Environment Jenkins Enterprise Master-1
Jenkins Enterprise Master-N
Jenkins Operations Center
JenkinsEnterpriseMaster-NJenkins
OperationsCenter (JOC)
Jenkins Enterprise Test Environment
JenkinsEnterprise
Test
JenkinsEnterpriseMaster-1
config
config
config
…
©2014 CloudBees, Inc. All Rights Reserved
Global Security Jenkins Enterprise Master-1
Jenkins Enterprise Master-N
Jenkins Operations Center
JenkinsEnterpriseMaster-N
Role Based AccessControl Plugin
HA Proxy
DEV BuildNodes
HA Proxy
JenkinsOperations
Center (JOC)
Jenkins Enterprise Test Environment
JenkinsEnterprise
Test
HA Proxy
JenkinsEnterpriseMaster-1
Access masters
HA Proxy
Security config pushed to masters
confi
g
config
config
Security config pushed to masters
LDAP…
audit
audit
audit
audit
©2014 CloudBees, Inc. All Rights Reserved
Unified Navigation and SSO
Jenkins Enterprise Test Environment
JenkinsEnterprise
TestHA Proxy
Jenkins Enterprise Master-1
HA Proxy JenkinsEnterpriseMaster-1
Jenkins Operations Center
JenkinsOperations
Center (JOC)HA Proxy
config
config
config
Jenkins Enterprise Master-N
HA Proxy
JenkinsEnterpriseMaster-N
http://master-1.jenkins.mycie.com
http://test.jenkins.mycie.com
…http://master-n.jenkins.mycie.com
http://joc.jenkins.mycie.com
Acce
ss m
aste
rs (S
SO)
©2014 CloudBees, Inc. All Rights Reserved
Update Centers Jenkins Enterprise Master-1
Jenkins Enterprise Master-N
Jenkins Operations Center
Master-1 UpdateCenter
JenkinsEnterpriseMaster-NUpdate
CenterJenkins
OperationsCenter (JOC)
Jenkins Enterprise Test Environment
JenkinsEnterprise
Test
Master-1Test UpdateCenter
Test UpdateCenter
Promote plugins
Master specific Update CenterJenkins
EnterpriseMaster-1
confi
g
config
config
Internet
UpdateCenter
http://jenkins-ci.org
…
©2014 CloudBees, Inc. All Rights Reserved
Jenkins Reference Architecture Jenkins Enterprise Master-1
Jenkins Enterprise Master-N
Jenkins Operations Center
Shared BuildNodes
UpdateCenter Jenkins
EnterpriseMaster-N
Role Based AccessControl Plugin
Master-1 UpdateCenter
HA Proxy
Master-1 BuildNodes
HA Proxy
JenkinsOperations
Center (JOC)
Jenkins Enterprise Test Environment
JenkinsEnterprise
TestHA Proxy
Master-1 Test UpdateCenter
Test UpdateCenter
Promote plugins
Master specific Update CenterJenkins
EnterpriseMaster-1
Access masters (SSO)
HA Proxy
Security config pushed to masters
confi
g
config
config
LDAP …
DEV, QA, OPS, admins
©2014 CloudBees, Inc. All Rights Reserved
Continuous Delivery is
• People • Process • Tools
©2014 CloudBees, Inc. All Rights Reserved