continuous delivery with docker and jenkins · production reference: 1230817 ... architecture, and...

ContinuousDeliverywithDockerandJenkins

Deliveringsoftwareatscale

RafałLeszko

BIRMINGHAM-MUMBAI

ContinuousDeliverywithDockerandJenkins

Copyright©2017PacktPublishing

Allrightsreserved.Nopartofthisbookmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformorbyanymeans,withoutthepriorwrittenpermissionofthepublisher,exceptinthecaseofbriefquotationsembeddedincriticalarticlesorreviews.

Everyefforthasbeenmadeinthepreparationofthisbooktoensuretheaccuracyoftheinformationpresented.However,theinformationcontainedinthisbookissoldwithoutwarranty,eitherexpressorimplied.Neithertheauthor,norPacktPublishing,anditsdealersanddistributorswillbeheldliableforanydamagescausedorallegedtobecauseddirectlyorindirectlybythisbook.

PacktPublishinghasendeavoredtoprovidetrademarkinformationaboutallofthecompaniesandproductsmentionedinthisbookbytheappropriateuseofcapitals.However,PacktPublishingcannotguaranteetheaccuracyofthisinformation.

Firstpublished:August2017

Productionreference:1230817

PublishedbyPacktPublishingLtd.LiveryPlace35LiveryStreetBirmingham

B32PB,UK.

ISBN978-1-78712-523-0

www.packtpub.com

Credits

Author

RafałLeszko

CopyEditor

UlkaManjrekar

LaxmiSubramanian

Reviewers

MichaelPailloncyMiteshSoni

ZhiweiChen

ProjectCoordinator

ShwetaHBirwatkar

CommissioningEditor

PratikShah

Proofreader

SafisEditing

AcquisitionEditor

PrachiBisht

Indexer

PratikShirodkar

ContentDevelopmentEditor

DeeptiThore

Graphics

TaniaDutta

TechnicalEditor

SnehaHanchate

ProductionCoordinator

ArvindkumarGupta

AbouttheAuthorRafałLeszkoisapassionatesoftwaredeveloper,trainer,andconferencespeakerlivinginKrakow,Poland.Hehasspenthiscareerwritingcode,designingarchitecture,andtechleadinginanumberofcompaniesandorganizationssuchasGoogle,CERN,andAGHUniversity.Alwaysopentonewchallenges,hehasgiventalksandconductedworkshopsatmorethanafewinternationalconferencessuchasDevoxxandVoxxedDays.

Iwouldliketothankmywife,Maria,forhersupport.Shewastheveryfirstreviewerofthisbook,alwayscheeringmeup,andtakingcareofourbabytogivemetimeandspaceforwriting.IalsogivedeepthanksandgratitudetotheZoopluscompany,whereIcouldfirstexperimentwiththeContinuousDeliveryapproachandespecially,toitsformeremployeeRobertSternforshowingmetheworldofDocker.IwouldalsoliketomakeaspecialmentionofPatroklosPapapetrouforhistrustandhelpinorganizingContinuousDeliveryworkshopsinGreece.Lastbutnottheleast,thankstomymom,dad,andbrotherforbeingsosupportive.

AbouttheReviewerMichaelPailloncyisadevelopertendingtowardthe'Ops'side,constantlytryingtokeepthingssimpleandasmuchautomatedaspossible.MichaelispassionateabouttheDevOpscultureandhasastrongexperienceinContinuousIntegration,ContinuousDelivery,automation,bigsoftwarefactorymanagementandlovestosharetheexperienceswithothers.

MiteshSoniisanavidlearnerwith10yearsofexperienceintheITindustry.HeisanSCJP,SCWCD,VCP,IBMUrbancode,andIBMBluemixcertifiedprofessional.HelovesDevOpsandcloudcomputingandalsohasaninterestinprogramminginJava.Hefindsdesignpatternsfascinatingandbelievesthat"apictureisworthathousandwords."

Heoccasionallycontributestoetutorialsworld.com.Helovestoplaywithkids,fiddlewithhiscamera,andtakephotographsatIndrodaPark.Heisaddictedtotakingpictureswithoutknowingmanytechnicaldetails.HelivesinthecapitalofMahatmaGandhi'shomestate.

MiteshhasauthoredfollowingbookswithPackt:

DevOpsBootcampImplementingDevOpswithMicrosoftAzureDevOpsforWebDevelopmentJenkinsEssentialsLearningChef

www.PacktPub.comForsupportfilesanddownloadsrelatedtoyourbook,pleasevisitwww.PacktPub.com.

DidyouknowthatPacktofferseBookversionsofeverybookpublished,withPDFandePubfilesavailable?YoucanupgradetotheeBookversionatwww.PacktPub.comandasaprintbookcustomer,youareentitledtoadiscountontheeBookcopy.Getintouchwithusatservice@packtpub.comformoredetails.

Atwww.PacktPub.com,youcanalsoreadacollectionoffreetechnicalarticles,signupforarangeoffreenewslettersandreceiveexclusivediscountsandoffersonPacktbooksandeBooks.

https://www.packtpub.com/mapt

Getthemostin-demandsoftwareskillswithMapt.MaptgivesyoufullaccesstoallPacktbooksandvideocourses,aswellasindustry-leadingtoolstohelpyouplanyourpersonaldevelopmentandadvanceyourcareer.

Whysubscribe?FullysearchableacrosseverybookpublishedbyPacktCopyandpaste,print,andbookmarkcontentOndemandandaccessibleviaawebbrowser

CustomerFeedbackThanksforpurchasingthisPacktbook.AtPackt,qualityisattheheartofoureditorialprocess.Tohelpusimprove,pleaseleaveusanhonestreviewonthisbook'sAmazonpageathttps://www.amazon.com/dp/1787125238.

Ifyou'dliketojoinourteamofregularreviewers,youcane-mailusatcustomerreviews@packtpub.com.WeawardourregularreviewerswithfreeeBooksandvideosinexchangefortheirvaluablefeedback.Helpusberelentlessinimprovingourproducts!

TomywonderfulwifeMaria,forallofherlove,wisdom,andsmile.

TableofContents

PrefaceWhatthisbookcovers

Whatyouneedforthisbook

Whothisbookisfor

Conventions

Readerfeedback

CustomersupportDownloadingtheexamplecode

Downloadingthecolorimagesofthisbook

Errata

Piracy

Questions

1. IntroducingContinuousDeliveryWhatisContinuousDelivery?

ThetraditionaldeliveryprocessIntroducingthetraditionaldeliveryprocess

Shortcomingsofthetraditionaldeliveryprocess

BenefitsofContinuousDelivery

Successstories

TheautomateddeploymentpipelineContinuousIntegration

AutomatedacceptancetestingTheAgiletestingmatrix

Thetestingpyramid

Configurationmanagement

PrerequisitestoContinuousDeliveryOrganizationalprerequisites

DevOpsculture

Clientintheprocess

Businessdecisions

Technicalanddevelopmentprerequisites

BuildingtheContinuousDeliveryprocessIntroducingtools

Dockerecosystem

Jenkins

Ansible

GitHub

Java/SpringBoot/Gradle

Theothertools

CreatingacompleteContinuousDeliverysystemIntroducingDocker

ConfiguringJenkins

ContinuousIntegrationPipeline

Automatedacceptancetesting

ConfigurationmanagementwithAnsible/ContinuousDeliverypipeline

ClusteringwithDockerSwarm/AdvancedContinuousDelivery

Summary

2. IntroducingDockerWhatisDocker?

Containerizationversusvirtualization

TheneedforDockerEnvironment

Isolation

Organizingapplications

Portability

Kittensandcattle

Alternativecontainerizationtechnologies

DockerinstallationPrerequisitesforDocker

InstallingonalocalmachineDockerforUbuntu

DockerforLinux

DockerforMac

DockerforWindows

TestingDockerinstallation

InstallingonaserverDedicatedserver

RunningDockerhelloworld>Dockercomponents

Dockerclientandserver

Dockerimagesandcontainers

Dockerapplications

BuildingimagesDockercommit

Dockerfile

CompleteDockerapplicationWritetheapplication

Preparetheenvironment

Buildtheimage

Runtheapplication

Environmentvariables

Dockercontainerstates

DockernetworkingRunningservices

Containernetworks

Exposingcontainerports

Automaticportassignment

UsingDockervolumes

UsingnamesinDockerNamingcontainers

Taggingimages

DockercleanupCleaningupcontainers

Cleaningupimages

Dockercommandsoverview

Exercises

Summary

3. ConfiguringJenkinsWhatisJenkins?

JenkinsinstallationRequirementsforinstallation

InstallingonDocker

InstallingwithoutDocker

Initialconfiguration

Jenkinshelloworld

JenkinsarchitectureMasterandslaves

ScalabilityVerticalscaling

Horizontalscaling

Testandproductioninstances

Samplearchitecture

ConfiguringagentsCommunicationprotocols

SettingagentsPermanentagents

Configuringpermanentagents

Understandingpermanentagents

PermanentDockeragentsConfiguringpermanentDockeragents

UnderstandingpermanentDockeragents

JenkinsSwarmagentsConfiguringJenkinsSwarmagents

UnderstandingJenkinsSwarmagents

DynamicallyprovisionedDockeragentsConfiguringdynamicallyprovisionedDockeragents

UnderstandingdynamicallyprovisionedDockeragents

Testingagents

CustomJenkinsimagesBuildingJenkinsslave

BuildingJenkinsmaster

ConfigurationandmanagementPlugins

Security

Backup

BlueOceanUI

Exercises

Summary

4. ContinuousIntegrationPipelineIntroducingpipelines

Pipelinestructure

Multi-stageHelloWorld

PipelinesyntaxSections

Directives

Steps

CommitpipelineCheckout

CreatingaGitHubrepository

Creatingacheckoutstage

CompileCreatingaJavaSpringBootproject

PushingcodetoGitHub

Creatingacompilestage

UnittestCreatingbusinesslogic

Writingaunittest

Creatingaunitteststage

JenkinsfileCreatingJenkinsfile

RunningpipelinefromJenkinsfile

Codequalitystages

CodecoverageAddingJaCoCotoGradle

Addingacodecoveragestage

Publishingthecodecoveragereport

StaticcodeanalysisAddingtheCheckstyleconfiguration

Addingastaticcodeanalysisstage

Publishingstaticcodeanalysisreports

SonarQube

TriggersandnotificationsTriggers

External

PollingSCM

Scheduledbuild

NotificationsEmail

Groupchat

Teamspace

TeamdevelopmentstrategiesDevelopmentworkflows

Trunk-basedworkflow

Branchingworkflow

Forkingworkflow

AdoptingContinuousIntegrationBranchingstrategies

Featuretoggles

JenkinsMultibranch

Non-technicalrequirements

Exercises

Summary

5. AutomatedAcceptanceTestingIntroducingacceptancetesting

DockerregistryArtifactrepository

InstallingDockerregistryDockerHub

PrivateDockerregistryInstallingtheDockerregistryapplication

Addingadomaincertificate

Addinganaccessrestriction

OtherDockerregistries

UsingDockerregistryBuildinganimage

Pushingtheimage

Pullingtheimage

AcceptancetestinpipelineTheDockerbuildstage

AddingDockerfile

AddingtheDockerbuildtothepipeline

TheDockerpushstage

AcceptancetestingstageAddingastagingdeploymenttothepipeline

Addinganacceptancetesttothepipeline

Addingacleaningstageenvironment

DockerComposeWhatisDockerCompose?

InstallingDockerCompose

Definingdocker-compose.yml

Usingthedocker-composecommand

Buildingimages

Scalingservices

AcceptancetestingwithDockerComposeUsingamulti-containerenvironment

AddingaRedisclientlibrarytoGradle

AddingaRediscacheconfiguration

AddingSpringBootcaching

Checkingthecachingenvironment

Method1–Jenkins-firstacceptancetestingChangingthestagingdeploymentstage

Changingtheacceptanceteststage

Method2–Docker-firstacceptancetestingCreatingaDockerfileforacceptancetest

Creatingdocker-compose.ymlforacceptancetest

Creatinganacceptancetestscript

Runningtheacceptancetest

Changingtheacceptanceteststage

Comparingmethod1andmethod2

WritingacceptancetestsWritinguser-facingtests

UsingtheacceptancetestingframeworkCreatingacceptancecriteria

Creatingstepdefinitions

Runninganautomatedacceptancetest

Acceptancetest-drivendevelopment

Exercises

Summary

6. ConfigurationManagementwithAnsibleIntroducingconfigurationmanagement

Traitsofgoodconfigurationmanagement

Overviewofconfigurationmanagementtools

InstallingAnsibleAnsibleserverrequirements

Ansibleinstallation

Docker-basedAnsibleclient

UsingAnsibleCreatinginventory

Adhoccommands

PlaybooksDefiningaplaybook

Executingtheplaybook

Playbook'sidempotency

Handlers

Variables

RolesUnderstandingroles

AnsibleGalaxy

DeploymentwithAnsibleInstallingRedis

DeployingawebserviceConfiguringaprojecttobeexecutable

ChangingtheRedishostaddress

Addingcalculatordeploymenttotheplaybook

Runningdeployment

AnsiblewithDockerBenefitsofAnsible

AnsibleDockerplaybookInstallingDocker

RunningDockercontainers

UsingDockerCompose

Exercises

Summary

7. ContinuousDeliveryPipelineEnvironmentsandinfrastructure

TypesofenvironmentProduction

Staging

QA

Development

EnvironmentsinContinuousDelivery

Securingenvironments

NonfunctionaltestingTypesofnonfunctionaltest

Performancetesting

Loadtesting

Stresstesting

Scalabilitytesting

Endurancetesting

Securitytesting

Maintainabilitytesting

Recoverytesting

Nonfunctionalchallenges

ApplicationversioningVersioningstrategies

VersioningintheJenkinspipeline

CompleteContinuousDeliverypipelineInventory

Acceptancetestingenvironment

Release

Smoketesting

CompleteJenkinsfile

Exercises

Summary

8. ClusteringwithDockerSwarmServerclustering

Introducingserverclustering

IntroducingDockerSwarm

DockerSwarmfeaturesoverview

DockerSwarminpracticeSettingupaSwarm

Addingworkernodes

Deployingaservice

Scalingservice

Publishingports

AdvancedDockerSwarmRollingupdates

Drainingnodes

Multiplemanagernodes

Schedulingstrategy

DockerComposewithDockerSwarmIntroducingDockerStack

UsingDockerStackSpecifyingdocker-compose.yml

Runningthedockerstackcommand

Verifyingtheservicesandcontainers

Removingthestack

AlternativeclustermanagementsystemsKubernetes

ApacheMesos

Comparingfeatures

ScalingJenkinsDynamicslaveprovisioning

JenkinsSwarm

ComparisonofdynamicslaveprovisioningandJenkinsSwarm

Exercises

Summary

9. AdvancedContinuousDeliveryManagingdatabasechanges

UnderstandingschemaupdatesIntroducingdatabasemigrations

UsingFlywayConfiguringFlyway

DefiningtheSQLmigrationscript

Accessingdatabase

ChangingdatabaseinContinuousDeliveryBackwards-compatiblechanges

Non-backwards-compatiblechangesAddinganewcolumntothedatabase

Changingthecodetousebothcolumns

Mergingthedatainbothcolumns

Removingtheoldcolumnfromthecode

Droppingtheoldcolumnfromthedatabase

Separatingdatabaseupdatesfromcodechanges

Avoidingshareddatabase

PreparingtestdataUnittesting

Integration/acceptancetesting

Performancetesting

PipelinepatternsParallelizingpipelines

ReusingpipelinecomponentsBuildparameters

SharedlibrariesCreatingasharedlibraryproject

ConfigurethesharedlibraryinJenkins

UsesharedlibraryinJenkinsfile

Rollingbackdeployments

Addingmanualsteps

ReleasepatternsBlue-greendeployment

Canaryrelease

WorkingwithlegacysystemsAutomatingbuildanddeployment

Automatingtests

Refactoringandintroducingnewfeatures

Understandingthehumanelement

Exercises

Summary

BestpracticesPractice1–ownprocesswithintheteam!

Practice2–automateeverything!

Practice3–versioneverything!

Practice4–usebusinesslanguageforacceptancetests!

Practice5–bereadytorollback!

Practice6–don'tunderestimatetheimpactofpeople

Practice7–buildintraceability!

Practice8–integrateoften!

Practice9–buildbinariesonlyonce!

Practice10–releaseoften!

PrefaceI'veobservedsoftwaredeliveryprocessesforyears.IwrotethisbookbecauseIknowhowmanypeoplestillstrugglewithreleasesandgetfrustratedafterspendingdaysandnightsongettingtheirproductsintoproduction.Thisallhappenseventhoughalotofautomationtoolsandprocesseshavebeendevelopedthroughouttheyears.AfterIsawforthefirsttimehowsimpleandeffectivetheContinuousDeliveryprocesswas,Iwouldnevercomebacktothetedioustraditionalmanualdeliverycycle.ThisbookisaresultofmyexperienceandanumberofContinuousDeliveryworkshopsIconducted.IsharethemodernapproachusingJenkins,Docker,andAnsible;however,thisbookismorethanjustthetools.ItpresentstheideaandthereasoningbehindContinuousDelivery,andwhat'smostimportant,mymainmessagetoeveryoneImeet:ContinuousDeliveryprocessissimple,useit!

WhatthisbookcoversChapter1,IntroducingContinuousDelivery,presentshowcompaniestraditionallydelivertheirsoftwareandexplainstheideatoimproveitusingtheContinuousDeliveryapproach.Thischapteralsodiscussestheprerequisitesforintroducingtheprocessandpresentsthesystemthatwillbebuiltthroughoutthebook.

Chapter2,IntroducingDocker,explainstheideaofcontainerizationandthefundamentalsoftheDockertool.ThischapteralsoshowshowtouseDockercommands,packageanapplicationasaDockerimage,publishDockercontainer'sports,anduseDockervolumes.

Chapter3,ConfiguringJenkins,presentshowtoinstall,configure,andscaleJenkins.ThischapteralsoshowshowtouseDockertosimplifyJenkinsconfigurationandtoenabledynamicslaveprovisioning.

Chapter4,ContinuousIntegrationPipeline,explainstheideaofpipeliningandintroducestheJenkinsfilesyntax.ThischaptersalsoshowshowtoconfigureacompleteContinuousIntegrationpipeline.

Chapter5,AutomatedAcceptanceTesting,presentstheideaandimplementationofacceptancetesting.Thischaptersalsoexplainsthemeaningofartifactrepositories,theorchestrationusingDockerCompose,andframeworksforwritingBDD-orientedacceptancetests.

Chapter6,ConfigurationManagementwithAnsible,introducestheconceptofconfigurationmanagementanditsimplementationusingAnsible.ThechapteralsoshowshowtouseAnsibletogetherwithDockerandDockerCompose.

Chapter7,ContinuousDeliveryPipeline,combinesalltheknowledgefromthepreviouschaptersinordertobuildthecompleteContinuousDeliveryprocess.Thechapteralsodiscussesvariousenvironmentsandtheaspectsofnonfunctionaltesting.

Chapter8,ClusteringwithDockerSwarm,explainstheconceptofserverclusteringandtheimplementationusingDockerSwarm.Thechapteralsocompares

alternativeclusteringtools(KubernetesandApacheMesos)andexplainshowtouseclusteringfordynamicJenkinsagents.

Chapter9,AdvancedContinuousDelivery,presentsamixtureofdifferentaspectsrelatedtotheContinuousDeliveryprocess:databasemanagement,parallelpipelinesteps,rollbackstrategies,legacysystems,andzero-downtimedeployments.ThechapteralsoincludesbestpracticesfortheContinuousDeliveryprocess.

WhatyouneedforthisbookDockerrequiresthe64-bitLinuxoperatingsystem.AllexamplesinthisbookhavebeendevelopedusingUbuntu16.04,butanyotherLinuxsystemwiththekernelversion3.10oraboveissufficient.

WhothisbookisforThisbookisfordevelopersandDevOpswhowouldliketoimprovetheirdeliveryprocess.Nopriorknowledgeisrequiredtounderstandthisbook.

ConventionsInthisbook,youwillfindanumberofstylesoftextthatdistinguishbetweendifferentkindsofinformation.Herearesomeexamplesofthesestyles,andanexplanationoftheirmeaning.

Codewordsintext,databasetablenames,foldernames,filenames,fileextensions,pathnames,dummyURLs,userinput,andTwitterhandlesareshownasfollows:dockerinfo

Ablockofcodeissetasfollows:

pipeline{

agentany

stages{

stage("Hello"){

steps{

echo'HelloWorld'

}

}

}

}

Whenwewishtodrawyourattentiontoaparticularpartofacodeblock,therelevantlinesoritemsaresetinbold:

FROMubuntu:16.04

RUNapt-getupdate&&\

apt-getinstall-ypython

Anycommand-lineinputoroutputiswrittenasfollows:

$dockerimages

REPOSITORYTAGIMAGEIDCREATEDSIZE

ubuntu_with_pythonlatestd6e85f39f5b7Aboutaminuteago202.6MB

ubuntu_with_git_and_jdklatest8464dc10abbb3minutesago610.9MB

Newtermsandimportantwordsareshowninbold.Wordsthatyouseeonthescreen,inmenusordialogboxesforexample,appearinthetextlikethis:"ClickonNewItem".

Warningsorimportantnotesappearinaboxlikethis.

IfyourDockerdaemonisruninsidethecorporatenetwork,you

havetoconfiguretheHTTPproxy.Thedetaileddescriptioncanbefoundathttps://docs.docker.com/engine/admin/systemd/.

Tipsandtricksappearlikethis.

TheinstallationguidesforallsupportedoperatingsystemsandcloudplatformscanbefoundontheofficialDockerpage,https://docs.docker.com/engine/installation/.

ReaderfeedbackFeedbackfromourreadersisalwayswelcome.Letusknowwhatyouthinkaboutthisbook—whatyoulikedormayhavedisliked.Readerfeedbackisimportantforustodeveloptitlesthatyoureallygetthemostoutof.

Tosendusgeneralfeedback,simplysendane-mailtofeedback@packtpub.com,andmentionthebooktitleviathesubjectofyourmessage.

Ifthereisatopicthatyouhaveexpertiseinandyouareinterestedineitherwritingorcontributingtoabook,seeourauthorguideonwww.packtpub.com/authors.

CustomersupportNowthatyouaretheproudownerofaPacktbook,wehaveanumberofthingstohelpyoutogetthemostfromyourpurchase.

DownloadingtheexamplecodeYoucandownloadtheexamplecodefilesforallPacktbooksyouhavepurchasedfromyouraccountathttp://www.packtpub.com.Ifyoupurchasedthisbookelsewhere,youcanvisithttp://www.packtpub.com/supportandregistertohavethefilese-maileddirectlytoyou.

Youcandownloadthecodefilesbyfollowingthesesteps:

1. Loginorregistertoourwebsiteusingyoure-mailaddressandpassword.2. HoverthemousepointerontheSUPPORTtabatthetop.3. ClickonCodeDownloads&Errata.4. EnterthenameofthebookintheSearchbox.5. Selectthebookforwhichyou'relookingtodownloadthecodefiles.6. Choosefromthedrop-downmenuwhereyoupurchasedthisbookfrom.7. ClickonCodeDownload.

Oncethefileisdownloaded,pleasemakesurethatyouunziporextractthefolderusingthelatestversionof:

WinRAR/7-ZipforWindowsZipeg/iZip/UnRarXforMac7-Zip/PeaZipforLinux

ThecodebundleforthebookisalsohostedonGitHubathttps://github.com/PacktPublishing/Continuous-Delivery-with-Docker-and-Jenkins.Wealsohaveothercodebundlesfromourrichcatalogofbooksandvideosavailableathttps://github.com/PacktPublishing/.Checkthemout!

DownloadingthecolorimagesofthisbookWealsoprovideyouwithaPDFfilethathascolorimagesofthescreenshots/diagramsusedinthisbook.Thecolorimageswillhelpyoubetterunderstandthechangesintheoutput.Youcandownloadthisfilefromhttps://www.packtpub.com/sites/default/files/downloads/ContinuousDeliverywithDockerandJenkins_ColorImages.pdf.

ErrataAlthoughwehavetakeneverycaretoensuretheaccuracyofourcontent,mistakesdohappen.Ifyoufindamistakeinoneofourbooks—maybeamistakeinthetextorthecode—wewouldbegratefulifyouwouldreportthistous.Bydoingso,youcansaveotherreadersfromfrustrationandhelpusimprovesubsequentversionsofthisbook.Ifyoufindanyerrata,pleasereportthembyvisitinghttp://www.packtpub.com/submit-errata,selectingyourbook,clickingontheerratasubmissionformlink,andenteringthedetailsofyourerrata.Onceyourerrataareverified,yoursubmissionwillbeacceptedandtheerratawillbeuploadedonourwebsite,oraddedtoanylistofexistingerrata,undertheErratasectionofthattitle.Anyexistingerratacanbeviewedbyselectingyourtitlefromhttp://www.packtpub.com/support.

PiracyPiracyofcopyrightmaterialontheInternetisanongoingproblemacrossallmedia.AtPackt,wetaketheprotectionofourcopyrightandlicensesveryseriously.Ifyoucomeacrossanyillegalcopiesofourworks,inanyform,ontheInternet,pleaseprovideuswiththelocationaddressorwebsitenameimmediatelysothatwecanpursuearemedy.

Pleasecontactusatcopyright@packtpub.comwithalinktothesuspectedpiratedmaterial.

Weappreciateyourhelpinprotectingourauthors,andourabilitytobringyouvaluablecontent.

QuestionsYoucancontactusatquestions@packtpub.comifyouarehavingaproblemwithanyaspectofthebook,andwewilldoourbesttoaddressit.

IntroducingContinuousDelivery

Thecommonproblemfacedbymostdevelopersishowtoreleasetheimplementedcodequicklyandsafely.Thedeliveryprocessusedtraditionallyis,however,asourceofpitfallsandusuallyleadstothedisappointmentofbothdevelopersandclients.ThischapterpresentstheideaoftheContinuousDeliveryapproachandprovidesthecontextfortherestofthebook.

Thischaptercoversthefollowingpoints:

IntroducingthetraditionaldeliveryprocessanditsdrawbacksDescribingtheideaofContinuousDeliveryandthebenefitsitbringsComparinghowdifferentcompaniesdelivertheirsoftwareExplainingtheautomateddeploymentpipelineanditsphasesClassifyingdifferenttypesoftestsandtheirplaceintheprocessPointingouttheprerequisitestothesuccessfulContinuousDeliveryprocessPresentingtoolsthatwillbeusedthroughoutthebookShowingthecompletesystemthatwillbebuiltthroughoutthebook

WhatisContinuousDelivery?ThemostaccuratedefinitionoftheContinuousDeliveryisstatedbyJezHumbleandreadsasfollows:"ContinuousDeliveryistheabilitytogetchangesofalltypes—includingnewfeatures,configurationchanges,bugfixes,andexperiments—intoproduction,orintothehandsofusers,safelyandquicklyinasustainableway."Thatdefinitioncoversthekeypoints.

Tounderstanditbetter,let'simagineascenario.Youareresponsiblefortheproduct,let'ssay,theemailclientapplication.Userscometoyouwithanewrequirement—theywanttosortemailsbysize.Youdecidethatthedevelopmentwilltakearoundoneweek.Whencantheuserexpecttousethefeature?Usually,afterthedevelopmentisdone,youhandoverthecompletedfeaturefirsttotheQAteamandthentotheoperationsteam,whichtakesadditionaltimerangingfromdaystomonths.Therefore,eventhoughthedevelopmenttookonlyoneweek,theuserreceivesitinacoupleofmonths!TheContinuousDeliveryapproachaddressesthatissuebyautomatingmanualtaskssothattheusercouldreceiveanewfeatureassoonasit'simplemented.

Topresentbetterwhattoautomateandhow,let'sstartbydescribingthedeliveryprocessthatiscurrentlyusedformostsoftwaresystems.

ThetraditionaldeliveryprocessThetraditionaldeliveryprocess,asthenamesuggests,hasbeeninplaceformanyyearsnowandisimplementedinmostITcompanies.Let'sdefinehowitworksandcommentonitsshortcomings.

IntroducingthetraditionaldeliveryprocessAnydeliveryprocessbeginswiththerequirementsdefinedbyacustomerandendsupwiththereleaseontheproduction.Thedifferencesareinbetween.Traditionally,itlooksaspresentedinthefollowingreleasecyclediagram:

ThereleasecyclestartswiththerequirementsprovidedbytheProductOwner,whorepresentstheCustomer(stakeholders).Thentherearethreephases,duringwhichtheworkispassedbetweendifferentteams:

Development:Here,thedevelopers(sometimestogetherwithbusinessanalysts)workontheproduct.TheyoftenuseAgiletechniques(ScrumorKanban)toincreasethedevelopmentvelocityandtoimprovethecommunicationwiththeclient.Demosessionsareorganizedtoobtainacustomer'squickfeedback.Allgooddevelopmenttechniques(liketest-drivendevelopmentorextremeprogrammingpractices)arewelcome.Aftertheimplementationiscompleted,thecodeispassedtotheQAteam.QualityAssurance:ThisphaseisusuallycalledUserAcceptanceTesting(UAT)anditrequiresacodefreezeonthetrunkcodebase,sothatnonewdevelopmentwouldbreakthetests.TheQAteamperformsasuiteofIntegrationTesting,AcceptanceTesting,andNon-functionalTesting(performance,recovery,security,andsoon).Anybugthatisdetectedgoes

backtothedevelopmentteam,sodevelopersusuallyalsohavetheirhandsfullofwork.AftertheUATphaseiscompleted,theQAteamapprovesthefeaturesthatareplannedforthenextrelease.Operations:Thelastphase,usuallytheshortestone,meanspassingthecodetotheOperationsteam,sothattheycanperformthereleaseandmonitortheproduction.Ifanythinggoeswrong,theycontactdeveloperstohelpwiththeproductionsystem.

Thelengthofthereleasecycledependsonthesystemandtheorganization,butitusuallyrangesfromaweektoafewmonths.ThelongestI'veheardaboutwasoneyear.ThelongestIworkedwithwasquarterly-basedandeachparttookasfollows:development-1.5months,UAT-1monthand3weeks,release(andstrictproductionmonitoring)-1week.

ThetraditionaldeliveryprocessiswidelyusedintheITindustryandit'sprobablynotthefirsttimeyou'vereadaboutsuchanapproach.Nevertheless,ithasanumberofdrawbacks.Let'slookatthemexplicitytounderstandwhyweneedtostriveforsomethingbetter.

ShortcomingsofthetraditionaldeliveryprocessThemostsignificantshortcomingsofthetraditionaldeliveryprocessincludethefollowing:

Slowdelivery:Here,thecustomerreceivestheproductlongaftertherequirementswerespecified.Itresultsintheunsatisfactorytimetomarketanddelaysofthecustomer'sfeedback.Longfeedbackcycle:Thefeedbackcycleisnotonlyrelatedtocustomers,butalsotodevelopers.ImaginethatyouaccidentallycreatedabugandyoulearnaboutitduringtheUATphase.Howlongdoesittaketofixsomethingyouworkedontwomonthsago?Evenminorbugscanconsumeweeks.Lackofautomation:Rarereleasesdon'tencouragetheautomation,whichleadstounpredictablereleases.Riskyhotfixes:Hotfixescan'tusuallywaitforthefullUATphase,sotheytendtobetesteddifferently(theUATphaseisshortened)ornottestedatall.Stress:Unpredictablereleasesarestressfulfortheoperationsteam.What'smore,thereleasecycleisusuallytightlyscheduledwhichimposesanadditionalstressondevelopersandtesters.Poorcommunication:Workpassedfromoneteamtoanotherrepresentsthewaterfallapproach,inwhichpeoplestarttocareonlyabouttheirpart,ratherthanthecompleteproduct.Incaseanythinggoeswrong,thatusuallyleadstotheblaminggameinsteadofcooperation.Sharedresponsibility:NoteamtakestheresponsibilityfortheproductfromAtoZ.Fordevelopers:"done"meansthatrequirementsareimplemented.Fortesters:"done"meansthatthecodeistested.Foroperations:"done"meansthatthecodeisreleased.

Lowerjobsatisfaction:Eachphaseisinterestingforadifferentteam,butotherteamsneedtosupporttheprocess.Forexample,thedevelopmentphaseisinterestingfordevelopersbut,duringtwootherphases,theystillneedtofixbugsandsupporttherelease,whichusuallyisnotinterestingfor

thematall.

Thesedrawbacksrepresentjustatipoftheicebergofthechallengesrelatedtothetraditionaldeliveryprocess.Youmayalreadyfeelthattheremustbeabetterwaytodevelopthesoftwareandthisbetterwayis,obviously,theContinuousDeliveryapproach.

BenefitsofContinuousDelivery“Howlongwouldittakeyourorganizationtodeployachangethatinvolvesjustonesinglelineofcode?Doyoudothisonarepeatable,reliablebasis?"ThesearethefamousquestionsfromMaryandTomPoppendieck(authorsofImplementingLeanSoftwareDevelopment),whichhavebeenquotedmanytimesbyJezHumbleandotherauthors.Actually,theanswertothesequestionsistheonlyvalidmeasurementofthehealthofyourdeliveryprocess.

Tobeabletodelivercontinuously,andnottospendafortuneonthearmyofoperationsteamsworking24/7,weneedautomation.Thatiswhy,inshort,ContinuousDeliveryisallaboutchangingeachphaseofthetraditionaldeliveryprocessintoasequenceofscripts,calledtheautomateddeploymentpipelineortheContinuousDeliverypipeline.Then,ifnomanualstepsarerequired,wecanruntheprocessaftereverycodechangeand,therefore,delivertheproductcontinuouslytotheusers.

ContinuousDeliveryletsusgetridofthetediousreleasecycleand,therefore,bringsthefollowingbenefits:

Fastdelivery:Timetomarketissignificantlyreducedascustomerscanusetheproductassoonasthedevelopmentiscompleted.Remember,thesoftwaredeliversnorevenueuntilitisinthehandsofitsusers.Fastfeedbackcycle:Imagineyoucreatedabuginthecode,whichgoesintotheproductionthesameday.Howmuchtimedoesittaketofixsomethingyouworkedonthesameday?Probablynotmuch.This,togetherwiththequickrollbackstrategy,isthebestwaytokeeptheproductionstable.Low-riskreleases:Ifyoureleaseonadailybasis,thentheprocessbecomesrepeatableandthereforemuchsafer.Asthesayinggoes,"Ifithurts,doitmoreoften."Flexiblereleaseoptions:Incaseyouneedtoreleaseimmediately,everythingisalreadyprepared,sothereisnoadditionaltime/costassociatedwiththereleasedecision.

Needlesstosay,wecouldachieveallthebenefitssimplybyeliminatingalldeliveryphasesandproceedingwiththedevelopmentdirectlyontheproduction.Itwould,however,causethequalitytodecline.Actually,thewholedifficultyofintroducingContinuousDeliveryistheconcernthatthequalitywoulddecreasetogetherwitheliminatingmanualsteps.Inthisbook,wewillshowhowtoapproachitinasafemannerandexplainwhy,contrarytocommonbeliefs,theproductsdeliveredcontinuouslyhavefewerbugsandarebetteradjustedtothecustomer'sneeds.

SuccessstoriesMyfavoritestoryonContinuousDeliverywastoldbyRolfRussellatoneofhistalks.Itgoesasfollows.In2005,YahooacquiredFlickranditwasaclashoftwoculturesinthedeveloper'sworld.Flickr,bythattime,wasacompanywiththestart-upapproachinmind.Yahoo,onthecontrary,wasahugecorporationwithstrictrulesandthesafety-firstattitude.Theirreleaseprocessesdifferedalot.WhileYahoousedthetraditionaldeliveryprocess,Flickrreleasedmanytimesaday.Everychangeimplementedbydeveloperswentontheproductionthesameday.Theyevenhadafooteratthebottomoftheirpageshowingthetimeofthelastreleaseandtheavatarsofthedeveloperswhodidthechanges.

Yahoodeployedrarelyandeachreleasebroughtalotofchangeswelltestedandprepared.Flickrworkedinverysmallchunks,eachfeaturewasdividedintosmallincrementalpartsandeachpartwasdeployedquicklytotheproduction.Thedifferenceispresentedinthefollowingdiagram:

Youcanimaginewhathappenedwhenthedevelopersfromtwocompaniesmet.YahooobviouslytreatedFlickr'scolleaguesasjuniorirresponsibledevelopers,"abunchofsoftwarecowboyswhodon'tknowwhattheyaredoing."So,thefirstthingtheywantedtochangewastoaddaQAteamandtheUATphaseintoFlickr'sdeliveryprocess.Beforetheyappliedthechange,however,Flickr'sdevelopershadonlyonewish.TheyaskedtoevaluatethemostreliableproductsinthewholeYahoocompany.WhatasurprisewhenithappenedthatofallthesoftwareinYahoo,Flickrhadthelowestdowntime.TheYahooteamdidn't

understanditatfirst,butletFlickrstaywiththeircurrentprocessanyway.Afterall,theywereengineers,sotheevaluationresultwasconclusive.Onlyaftersometime,theyrealizedthattheContinuousDeliveryprocesscanbebeneficialforallproductsinYahooandtheystartedtograduallyintroduceiteverywhere.

Themostimportantquestionofthestoryremains-howwasitpossiblethatFlickrwasthemostreliablesystem?Actually,thereasonforthatfactwaswhatwealreadymentionedintheprevioussections.Areleaseislessriskyif:

ThedeltaofcodechangesissmallTheprocessisrepeatable

Thatiswhy,eventhoughthereleaseitselfisadifficultactivity,itismuchsaferwhendonefrequently.

ThestoryofYahooandFlickrisonlyanexampleofmanysuccessfulcompaniesforwhichtheContinuousDeliveryprocessprovedtoberight.Someofthemevenproudlysharedetailsfromtheirsystems,asfollows:

Amazon:In2011,theyannouncedreaching11.6seconds(onaverage)betweendeploymentsFacebook:In2013,theyannounceddeploymentofcodechangestwiceadayHubSpot:In2013,theyannounceddeployment300timesadayAtlassian:In2016,theypublishedasurveystatingthat65%oftheircustomerspracticecontinuousdelivery

YoucanreadmoreabouttheresearchontheContinuousDeliveryprocessandindividualcasestudiesathttps://continuousdelivery.com/evidence-case-studies/.

Keepinmindthatthestatisticsgetbettereveryday.However,evenwithoutanynumbers,justimagineaworldinwhicheverylineofcodeyouimplementgoessafelyintotheproduction.Clientscanreactquicklyandadjusttheirrequirements,developersarehappybecausetheydon'thavetosolvethatmanybugs,managersaresatisfiedbecausetheyalwaysknowwhatisthecurrentstateofwork.Afterall,remember,theonlytruemeasureofprogressisthereleasedsoftware.

TheautomateddeploymentpipelineWealreadyknowwhattheContinuousDeliveryprocessisandwhyweuseit.Inthissection,wedescribehowtoimplementit.

Let'sstartbyemphasizingthateachphaseinthetraditionaldeliveryprocessisimportant.Otherwise,itwouldneverhavebeencreatedinthefirstplace.Noonewantstodeliversoftwarewithouttestingitfirst!TheroleoftheUATphaseistodetectbugsandtoensurethatwhatdeveloperscreatediswhatthecustomerwanted.Thesameappliestotheoperationsteam—thesoftwaremustbeconfigured,deployedtotheproduction,andmonitored.That'soutofthequestion.So,howdoweautomatetheprocesssothatwepreserveallthephases?Thatistheroleoftheautomateddeploymentpipeline,whichconsistsofthreestagesaspresentedinthefollowingdiagram:

Theautomateddeploymentpipelineisasequenceofscriptsthatisexecutedaftereverycodechangecommittedtotherepository.Iftheprocessissuccessful,itendsupwiththedeploymenttotheproductionenvironment.

Eachstepcorrespondstoaphaseinthetraditionaldeliveryprocessasfollows:

ContinuousIntegration:ThischeckstomakesurethatthecodewrittenbydifferentdevelopersintegratestogetherAutomatedAcceptanceTesting:ThisreplacesthemanualQAphaseandchecksifthefeaturesimplementedbydevelopersmeettheclient'srequirementsConfigurationManagement:Thisreplacesthemanualoperationsphase-

configurestheenvironmentanddeploysthesoftware

Let'stakeadeeperlookateachphasetounderstandwhatisitsresponsibilityandwhatstepsitincludes.

ContinuousIntegrationTheContinuousIntegrationphaseprovidesthefirstfeedbacktodevelopers.Itchecksoutthecodefromtherepository,compilesit,runsunittests,andverifiesthecodequality.Ifanystepfails,thepipelineexecutionisstoppedandthefirstthingthedevelopersshoulddoisfixtheContinuousIntegrationbuild.Theessentialaspectofthephaseistime;itmustbeexecutedinatimelymanner.Forexample,ifthisphasetookanhourtocompletethenthedeveloperswouldcommitthecodefaster,whichwouldresultintheconstantlyfailingpipeline.

TheContinuousIntegrationpipelineisusuallythestartingpoint.SettingitupissimplebecauseeverythingisdonewithinthedevelopmentteamandnoagreementwiththeQAandoperationsteamsisnecessary.

AutomatedacceptancetestingTheautomatedacceptancetestingphaseisasuiteoftestswrittentogetherwiththeclient(andQAs)thatissupposedtoreplacethemanualUATstage.Itactsasaqualitygatetodecidewhetheraproductisreadyforthereleaseornot.Ifanyoftheacceptancetestsfail,thenthepipelineexecutionisstoppedandnofurtherstepsarerun.ItpreventsmovementtotheConfigurationManagementphaseandthereforetherelease.

Thewholeideaofautomatingtheacceptancephaseistobuildthequalityintotheproductinsteadofverifyingitlater.Inotherwords,whenadevelopercompletestheimplementation,thesoftwareisdeliveredalreadytogetherwithacceptancetestswhichverifythatthesoftwareiswhattheclientwanted.Thatisalargeshiftinthinkingabouttestingsoftware.Thereisnolongerasingleperson(orteam)whoapprovestherelease,buteverythingdependsonpassingtheacceptancetestsuite.ThatiswhycreatingthisphaseisusuallythemostdifficultpartoftheContinuousDeliveryprocess.Itrequiresaclosecooperationwiththeclientandcreatingtestsatthebeginning(notattheend)oftheprocess.

Introducingautomatedacceptancetestsisespeciallychallenginginthecaseoflegacysystems.WedescribemoreonthattopicinChapter9,AdvancedContinuousDelivery.

ThereisusuallyalotofconfusionaboutthetypesoftestsandtheirplaceintheContinuousDeliveryprocess.It'salsooftenunclearhowtoautomateeachtype,whatshouldbethecoverage,andwhatshouldbetheroleoftheQAteaminthewholedevelopmentprocess.Let'sclarifyitusingtheAgiletestingmatrixandthetestingpyramid.

TheAgiletestingmatrixBrianMarick,inaseriesofhisblogposts,madeaclassificationofsoftwaretestsinaformoftheso-calledagiletestingmatrix.Itplacestestsintwodimensions:businessortechnologyfacingandsupportprogrammersorcritiquetheproduct.Let'shavealookatthatclassification:

Let'scommentbrieflyoneachtypeoftest:

AcceptanceTesting(automated):Theseareteststhatrepresentfunctionalrequirementsseenfromthebusinessperspective.Theyarewrittenintheformofstoriesorexamplesbyclientsanddeveloperstoagreeonhowthesoftwareshouldwork.UnitTesting(automated):Theseareteststhathelpdeveloperstoprovidethehigh-qualitysoftwareandminimizethenumberofbugs.ExploratoryTesting(manual):Thisisthemanualblack-boxtesting,whichtriestobreakorimprovethesystem.Non-functionalTesting(automated):Theseareteststhatrepresentsystempropertiesrelatedtotheperformance,scalability,security,andsoon.

ThisclassificationanswersoneofthemostimportantquestionsabouttheContinuousDeliveryprocess:whatistheroleofaQAintheprocess?

ManualQAsperformtheexploratorytesting,sotheyplaywiththesystem,trytobreakit,askquestions,thinkaboutimprovements.AutomationQAshelpwithnonfunctionalandacceptancetesting,forexample,theywritecodetosupport

loadtesting.Ingeneral,QAsdon'thavetheirspecialplaceinthedeliveryprocess,butratheraroleinthedevelopmentteam.

IntheautomatedContinuousDeliveryprocess,thereisnolongeraplaceformanualQAswhoperformrepetitivetasks.

Youmaylookattheclassificationandwonderwhyyouseenointegrationteststhere.WherearetheyuptoBrianMarickandwheretoputthemintheContinuousDeliverypipeline?

Toexplainitwell,wefirstneedtomentionthatthemeaningofanintegrationtestdiffersdependingonthecontext.For(micro)servicearchitecture,theyusuallymeanexactlythesameastheacceptancetesting,asservicesaresmallandneednothingmorethanunitandacceptancetests.Ifyoubuildamodularapplication,thenbyintegrationtestsweusuallymeancomponentteststhatbindmultiplemodules(butnotthewholeapplication)andtestthemtogether.Inthatcase,integrationtestsplacethemselvessomewherebetweenacceptanceandunittests.Theyarewritteninasimilarwayasacceptancetests,butareusuallymoretechnicalandrequiremockingnotonlyexternalservices,butalsointernalmodules.Integrationtests,similartounittests,representthe"code"pointofview,whileacceptancetestsrepresentthe"user"pointofview.ConcerningtheContinuousDeliverypipeline,integrationtestsaresimplyimplementedasaseparatephaseintheprocess.

ThetestingpyramidTheprevioussectionexplainedwhateachtesttyperepresentsintheprocess,butmentionednothingabouthowmanytestsweshoulddevelop.So,whatshouldbethecodecoverageincaseofunittesting?Whataboutacceptancetesting?

Toanswerthesequestions,MikeCohn,inhisbookSucceedingwithAgile:SoftwareDevelopmentUsingScrum,createdaso-calledtestingpyramid.Let'slookatthediagramtounderstanditwell.

Whenwemoveupthepyramid,thetestsbecomeslowerandmoreexpensivetocreate.Theyoftenrequiretouchinguserinterfaceandhiringaseparatetestautomationteam.Thatiswhyacceptancetestsshouldnottarget100%coverage.Onthecontrary,theyshouldbefeature-orientedandverifyonlyselectedtestscenarios.Otherwise,wewouldspendafortuneonthetestdevelopmentandmaintenance,andourContinuousDeliverypipelinebuildwouldtakeagestoexecute.

Thecaseisdifferentatthebottomofthepyramid.Unittestsarecheapandfast,soweshouldstrivefor100%codecoverage.Theyarewrittenbydevelopersandprovidingthemshouldbeastandardprocedureforanymatureteam.

Ihopethattheagiletestingmatrixandthetestingpyramidclarifiedtheroleandtheimportanceofacceptancetesting.

Let'smovetothelastphaseoftheContinuousDeliveryprocess,configurationmanagement.

ConfigurationmanagementTheconfigurationmanagementphaseisresponsiblefortrackingandcontrollingchangesinthesoftwareanditsenvironment.Itconcernstakingcareofpreparingandinstallingthenecessarytools,scalingthenumberofserviceinstancesandtheirdistribution,infrastructureinventory,andalltasksrelatedtotheapplicationdeployment.

Configurationmanagementisasolutiontotheproblemsposedbymanuallydeployingandconfiguringapplicationsontheproduction.Suchcommonpracticeresultsinanissuewherebywenolongerknowswhereeachserviceisrunningandwithwhatproperties.Configurationmanagementtools(suchasAnsible,Chef,orPuppet)enablestoringconfigurationfilesintheversioncontrolsystemandtrackingeverychangethatwasmadeontheproductionservers.

Anadditionalefforttoreplacemanualtasksoftheoperationsteamistotakecareofapplicationmonitoring.Thatisusuallydonebystreaminglogsandmetricsoftherunningsystemstoacommondashboard,whichismonitoredbydevelopers(ortheDevOpsteam,asexplainedinthenextsection).

PrerequisitestoContinuousDeliveryTherestofthebookisdedicatedtotechnicaldetailsonhowtoimplementasuccessfulContinuousDeliverypipeline.Thesuccessoftheprocess,however,dependsnotonlyonthetoolswepresentthroughoutthebook.Inthissection,wetakeaholisticlookatthewholeprocessanddefinetheContinuousDeliveryrequirementsinthreeareas:

Yourorganization'sstructureanditsimpactonthedevelopmentprocessYourproductsandtheirtechnicaldetailsYourdevelopmentteamandthepracticesyouuse

OrganizationalprerequisitesThewayyourorganizationworkshasahighimpactonthesuccessofintroducingtheContinuousDeliveryprocess.It'sabitsimilartointroducingScrum.ManyorganizationswouldliketousetheAgileprocess,buttheydon'tchangetheirculture.Youcan'tuseScruminyourdevelopmentteamunlesstheorganization'sstructureisadjustedtothat.Forexample,youneedaproductowner,stakeholders,andmanagementthatunderstandsthatnorequirementchangesarepossibleduringthesprint.Otherwise,evenwithgoodwill,youwon'tmakeit.ThesameappliestotheContinuousDeliveryprocess;itrequiresanadjustmentofhowtheorganizationisstructured.Let'shavealookatthreeaspects:theDevOpsculture,aclientintheprocess,andbusinessdecisions.

DevOpscultureAlongtimeago,whensoftwarewaswrittenbyindividualsormicroteams,therewasnoclearseparationbetweenthedevelopment,qualityassurance,andoperations.Apersondevelopedthecode,testedit,andthenputitintotheproduction.Ifanythingwentwrong,thesamepersoninvestigatedtheissue,fixedit,andredeployedtotheproduction.Thewaythedevelopmentisorganizednowchangedgradually,whensystemsbecamelargeranddevelopmentteamsgrew.Then,engineersstartedtobecomespecializedinonearea.Thatmadeperfectsense,becausespecializationcausedaboostintheproductivity.However,thesideeffectwasthecommunicationoverhead.Itisespeciallyvisibleifdevelopers,QAs,andoperationsareunderseparatedepartmentsintheorganization,sitindifferentbuildings,orareoutsourcedtodifferentcountries.SuchorganizationstructureisnogoodfortheContinuousDeliveryprocess.Weneedsomethingbetter,weneedtoadapttheso-calledDevOpsculture.

DevOpsculturemeans,inasense,comingbacktotheroots.Asinglepersonorateamisresponsibleforallthreeareas,aspresentedinthefollowingdiagram:

Thereasonwhyit'spossibletomovetotheDevOpsmodelwithoutlosingontheproductivityistheautomation.Mostofthetasksrelatedtothequalityassuranceandoperationsaremovedtotheautomateddeliverypipelineandcanbethereforemanagedbythedevelopmentteam.

ADevOpsteamdoesn'tnecessarilyneedtoconsistonlyof

developers.Averycommonscenarioinmanyorganization'sundertransformationistocreateteamswithfourdevelopers,oneQA,andonepersonfromoperations.Theyneed,however,toworkcloselytogether(sitinonearea,havestand-upstogether,workonthesameproduct).

ThecultureofsmallDevOpsteamsaffectsthesoftwarearchitecture.Functionalrequirementshavetobewellseparatedinto(micro)servicesormodules,sothateachteamcantakecareofanindependentpart.

Theimpactoftheorganization'sstructureonthesoftwarearchitecturewasalreadyobservedin1967andformulatedasConway'sLaw:"Anyorganizationthatdesignsasystem(definedbroadly)willproduceadesignwhosestructureisacopyoftheorganization'scommunicationstructure."

ClientintheprocessTheroleofaclient(oraproductowner)slightlychangesduringtheContinuousDeliveryadoption.Traditionally,clientsareinvolvedindefiningrequirements,answeringquestionsfromdevelopers,attendingdemos,andtakingpartintheUATphasetoagreeifwhatwasbuiltiswhattheyhadinmind.

InContinuousDelivery,thereisnoUAT,andaclientisessentialintheprocessofwritingacceptancetests.Forsomeclients,whoalreadywrotetheirrequirementsinatestablemanner,itisnotabigshift.Fortheothers,itmeansachangeinawayofthinkingtomakerequirementsmoretechnical-oriented.

IntheAgileenvironment,someteamsdon'tevenacceptuserstories(requirements)withoutacceptancetestsattached.Suchtechniques,eventhoughtheymaysoundtoostrict,oftenleadtobetterdevelopmentproductivity.

BusinessdecisionsInmostcompanies,thebusinesshasanimpactonthereleaseschedule.Afterall,thedecisionwhatfeaturesaredelivered,andwhen,isrelatedtodifferentdepartmentsofthecompany(forexample,marketing)andcanbestrategicfortheenterprise.Thatiswhythereleaseschedulinghastobereapproachedanddiscussedbetweenthebusinessandthedevelopmentteams.

Obviously,therearetechniquessuchasfeaturetogglesormanualpipelinesteps,whichhelpwithreleasingfeaturesatthespecifiedtime.Wewilldescribethemlaterinthebook.Tobeprecise,thetermContinuousDeliveryisnotthesameasContinuousDeployment.Theformermeansthateachcommittotherepositoryisautomaticallyreleasedtotheproduction.ContinuousDeliveryislessstrictandmeansthateachcommitendsupwithareleasecandidate,soitallowsthelaststep(releasetotheproduction)tobemanual.

Intherestofthebook,wewillusethetermsContinuousDeliveryandContinuousDeploymentinterchangeably.

TechnicalanddevelopmentprerequisitesFromthetechnicalside,thereareafewrequirementstokeepinmind.Wewilldiscussthemthroughoutthebook,solet'sonlymentionthemherewithoutgoingintodetail:

Automatedbuild,test,package,anddeployoperations:Alloperationsneedtobepossibletoautomate.Ifwedealwiththesystemthatisnon-automatable,forexample,duetosecurityreasonsoritscomplexity,thenit'simpossibletocreateafullyautomateddeliverypipeline.Quickpipelineexecution:Thepipelinemustbeexecutedinatimelymanner,preferablyin5-15minutes.Ifourpipelineexecutiontakeshoursordays,thenitwon'tbepossibletorunitaftereverycommittotherepository.Quickfailurerecovery:Apossibilityofthequickrollbackorsystemrecoveryisamust.Otherwise,werisktheproductionhealthduetofrequentreleases.Zero-downtimedeployment:Thedeploymentcannothaveanydowntimesincewereleasemanytimesaday.Trunk-baseddevelopment:Developersmustcheckinregularlyintoonemasterbranch.Otherwise,ifeveryonedevelopsintheirownbranches,theintegrationisrareandthereforethereleasesarerare,whichisexactlytheoppositeofwhatwewanttoachieve.

Wewillwritemoreontheseprerequisitesandhowtoaddressthemthroughoutthebook.Keepingthatinmind,let'smovetothelastsectionofthischapterandintroducewhatsystemweplantobuildinthisbookandwhattoolswewilluseforthatpurpose.

BuildingtheContinuousDeliveryprocessWeintroducedtheidea,benefits,andprerequisiteswithregardstotheContinuousDeliveryprocess.Inthissection,wedescribethetoolsthatwillbeusedthroughoutthebookandtheirplaceinthecompletesystem.

Ifyou'reinterestedmoreintheideaoftheContinuousDeliveryprocess,thenhavealookatanexcellentbookbyJezHumbleandDavidFarley,ContinuousDelivery:ReliableSoftwareReleasesthroughBuild,Test,andDeploymentAutomation.

IntroducingtoolsFirstofall,thespecifictoolisalwayslessimportantthanunderstandingitsroleintheprocess.Inotherwords,anytoolcanbereplacedwithanotheronewhichplaysthesamerole.Forexample,JenkinscanbereplacedwithAtlassianBambooandChiefcanbeusedinsteadofAnsible.Thatiswhyeachchapterbeginswiththegeneraldescriptionofwhysuchatoolisnecessaryandwhatitsroleisinthewholeprocess.Then,theexacttoolisdescribedwithcomparisontoitssubstitutes.Thatformgivesyoutheflexibilitytochoosetherightoneforyourenvironment.

AnotherapproachcouldbetodescribetheContinuousDeliveryprocessonthelevelofideas;however,Istronglybelievethatgivinganexactexamplewiththecodeextract,somethingthatreaderscanrunbythemselves,resultsinamuchbetterunderstandingoftheconcept.

Therearetwowaystoreadthisbook.ThefirstistoreadandunderstandtheconceptsoftheContinuousDeliveryprocess.Thesecondistocreateyourownenvironmentandexecuteallscriptswhilereadingtounderstandthedetails.

Let'shaveaquicklookatthetoolswewillusethroughoutthebook.Inthissection,however,itisonlyabriefintroductionofeachtechnologyandmuchmoredetailispresentedasthisbookgoeson.

Dockerecosystem

Docker,astheclearleaderofthecontainerizationmovement,hasdominatedthesoftwareindustryintherecentyears.Itallowsthepackagingofanapplicationintheenvironment-agnosticimageandthereforetreatsserversasafarmofresources,ratherthanmachinesthatmustbeconfiguredforeachapplication.Dockerwasaclearchoiceforthisbookbecauseitperfectlyfitsthe(micro)serviceworldandtheContinuousDeliveryprocess.

TogetherwithDockercomesadditionaltechnologies,whichareasfollows:

DockerHub:ThisisaregistryforDockerimagesDockerCompose:ThisisatooltodefinemulticontainerDockerapplicationsDockerSwarm:Thisisaclusteringandschedulingtool

JenkinsJenkinsisbyfarthemostpopularautomationserveronthemarket.IthelpstocreateContinuousIntegrationandContinuousDeliverypipelinesand,ingeneral,anyotherautomatedsequenceofscripts.Highlyplugin-oriented,ithasagreatcommunitywhichconstantlyextendsitwithnewfeatures.What'smore,itallowstowritethepipelineascodeandsupportsdistributedbuildenvironments.

AnsibleAnsibleisanautomationtoolthathelpswithsoftwareprovisioning,configurationmanagement,andapplicationdeployment.Itistrendingfasterthananyotherconfigurationmanagementengineandcansoonovertakeitstwomaincompetitors:ChefandPuppet.ItusesagentlessarchitectureandintegratessmoothlywithDocker.

GitHubGitHubisdefinitelythenumberoneofallhostedversioncontrolsystems.Itprovidesaverystablesystem,agreatweb-basedUI,andafreeserviceforpublicrepositories.Havingsaidthat,anysourcecontrolmanagementserviceortoolwillworkwithContinuousDelivery,nomatterifit'sinthecloudorself-hostedandifit'sbasedonGit,SVN,Mercurial,oranyothertool.

Java/SpringBoot/GradleJavahasbeenthemostpopularprogramminglanguageforyears.Thatiswhyitisbeingusedformostcodeexamplesinthisbook.TogetherwithJava,mostcompaniesdevelopwiththeSpringframework,soweusedittocreateasimplewebserviceneededtoexplainsomeconcepts.Gradleisusedasabuildtool.It'sstilllesspopularthanMaven,however,trendingmuchfaster.Asalways,anyprogramminglanguage,framework,orbuildtoolcanbeexchangedandtheContinuousDeliveryprocesswouldstaythesame,sodon'tworryifyourtechnologystackisdifferent.

TheothertoolsCucumberwaschosenarbitrarilyastheacceptancetestingframework.OthersimilarsolutionsareFitnesseandJBehave.ForthedatabasemigrationweuseFlyway,butanyothertoolwoulddo,forexample,Liquibase.

CreatingacompleteContinuousDeliverysystemYoucanlookathowthisbookisorganizedfromtwoperspectives.

Thefirstoneisbasedonthestepsoftheautomateddeploymentpipeline.EachchaptertakesyouclosertothecompleteContinuousDeliveryprocess.Ifyoulookatthenamesofthechapters,someofthemareevennamedlikethepipelinephases:

ContinuousIntegrationpipelineAutomatedacceptancetestingConfigurationmanagementwithAnsible

Therestofthechaptersgivetheintroduction,summary,oradditionalinformationcomplementarytotheprocess.

Thereisalsoasecondperspectivetothecontentofthisbook.Eachchapterdescribesonepieceoftheenvironment,whichinturniswellpreparedfortheContinuousDeliveryprocess.Inotherwords,thebookpresents,stepbystep,technologybytechnology,howtobuildacompletesystem.Tohelpyougetthefeelingofwhatweplantobuildthroughoutthebook,let'snowhavealookathowthesystemwillevolveineachchapter.

Don'tworryifyoudon'tunderstandtheconceptsandtheterminologyatthispoint.Weexplaineverythingfromscratchinthecorrespondingchapters.

IntroducingDockerInChapter2,IntroducingDocker,westartfromthecenterofoursystemandbuildaworkingapplicationpackagedasaDockerimage.Theoutputofthischapteris

presentedinthefollowingdiagram:

Adockerizedapplication(webservice)isrunasacontaineronaDockerHostandisreachableasitwouldrundirectlyonthehostmachine.Thatispossiblethankstoportforwarding(portpublishingintheDocker'sterminology).

ConfiguringJenkinsInChapter3,ConfiguringJenkins,wepreparetheJenkinsenvironment.Thankstothesupportofmultipleagent(slave)nodes,itisabletohandletheheavyconcurrentload.Theresultispresentedinthefollowingdiagram:

TheJenkinsmasteracceptsabuildrequest,buttheexecutionisstartedatoneoftheJenkinsSlave(agent)machines.SuchanapproachprovideshorizontalscalingoftheJenkinsenvironment.

ContinuousIntegrationPipelineInChapter4,ContinuousIntegrationPipeline,weshowhowtocreatethefirstphaseoftheContinuousDeliverypipeline,thecommitstage.Theoutputofthischapteristhesystempresentedinthefollowingdiagram:

TheapplicationisasimplewebservicewritteninJavawiththeSpringBootframework.GradleisusedasabuildtoolandGitHubasthesourcecoderepository.EverycommittoGitHubautomaticallytriggerstheJenkinsbuild,whichusesGradletocompileJavacode,rununittests,andperformadditionalchecks(codecoverage,staticcodeanalysis,andsoon).AftertheJenkinsbuildiscompleted,anotificationissenttothedevelopers.

Afterthischapter,youwillbeabletocreateacompleteContinuousIntegrationpipeline.

AutomatedacceptancetestingInChapter5,AutomatedAcceptanceTesting,wefinallymergethetwotechnologiesfromthebooktitle:DockerandJenkins.Itresultsinthesystem

presentedinthefollowingdiagram:

Theadditionalelementsinthediagramarerelatedtotheautomatedacceptancetestingstage:

DockerRegistry:AftertheContinuousIntegrationphase,theapplicationispackagedfirstintoaJARfileandthenasaDockerimage.ThatimageisthenpushedtotheDockerRegistry,whichactsasastoragefordockerizedapplications.DockerHost:Beforeperformingtheacceptancetestsuite,theapplicationhastobestarted.JenkinstriggersaDockerHostmachinetopullthedockerizedapplicationfromtheDockerRegistryandstartsit.DockerCompose:IfthecompleteapplicationconsistsofmorethanoneDockercontainer(forexample,twowebservices:Application1usingApplication2),thenDockerComposehelpstorunthemtogether.Cucumber:AftertheapplicationisstartedontheDockerHost,JenkinsrunsasuiteofacceptancetestswrittenintheCucumberframework.

ConfigurationmanagementwithAnsible/ContinuousDeliverypipelineInthenexttwochapters,thatis,Chapter6,ConfigurationManagementwithAnsibleandChapter7,ContinuousDeliveryPipeline,wecompletetheContinuousDeliverypipeline.Theoutputistheenvironmentpresentedinthefollowing

diagram:

Ansibletakescareoftheenvironmentsandenablesthedeploymentofthesameapplicationsonmultiplemachines.Asaresult,wedeploytheapplicationtothestagingenvironment,runtheacceptancetestingsuite,andfinallyreleasetheapplicationtotheproductionenvironment,usuallyinmanyinstances(onmultipleDockerHostmachines).

ClusteringwithDockerSwarm/AdvancedContinuousDeliveryInChapter8,ClusteringwithDockerSwarm,wereplacesinglehostsineachoftheenvironmentswithclustersofmachines.Chapter9,AdvancedContinuousDelivery,additionallyaddsdatabasestotheContinuousDeliveryprocess.Thefinalenvironmentcreatedinthisbookispresentedinthefollowingdiagram:

StagingandproductionenvironmentsareequippedwithDockerSwarmclustersandthereforemultipleinstancesoftheapplicationarerunonthecluster.Wedon'thavetothinkanymoreonwhichexactmachineourapplicationsaredeployed.Allwecareaboutisthenumberoftheirinstances.ThesameappliestoJenkinsslaves,theyarealsorunonacluster.ThelastimprovementistheautomaticmanagementofthedatabaseschemasusingFlywaymigrations

integratedintothedeliveryprocess.

Ihopeyouarealreadyexcitedbywhatweplantobuildthroughoutthisbook.Wewillapproachitstepbystep,explainingeverydetailandallthepossibleoptionsinordertohelpyouunderstandtheproceduresandtools.Afterreadingthisbook,youwillbeabletointroduceorimprovetheContinuousDeliveryprocessinyourprojects.

SummaryInthischapter,wehaveintroducedtheContinuousDeliveryprocessstartingfromtheidea,discussingtheprerequisites,toendupwithtoolsthatareusedintherestofthebook.Thekeytakeawayfromthischapterisasfollows:

ThedeliveryprocessusedcurrentlyinmostcompanieshassignificantshortcomingsandcanbeimprovedusingmoderntoolsforautomationTheContinuousDeliveryapproachprovidesanumberofbenefits,ofwhichthemostsignificantonesare:fastdelivery,fastfeedbackcycle,andlow-riskreleasesTheContinuousDeliverypipelineconsistsofthreestages:ContinuousIntegration,automatedacceptancetesting,andconfigurationmanagementIntroducingContinuousDeliveryusuallyrequiresachangeintheorganization'scultureandstructureThemostimportanttoolsinthecontextofContinuousDeliveryareDocker,Jenkins,andAnsible

Inthenextchapter,weintroduceDockerandpresenthowtobuildadockerizedapplication.

IntroducingDocker

WewilldiscusshowthemodernContinuousDeliveryprocessshouldlookbyintroducingDocker,thetechnologythatchangedtheITindustryandthewaytheserversareused.

Thischaptercoversthefollowingpoints:

IntroducingtheideaofvirtualizationandcontainerizationInstallingDockerfordifferentlocalandserverenvironmentsExplainingthearchitectureoftheDockertoolkitBuildingDockerimageswithDockerfileandbycommittingchangesRunningapplicationsasDockercontainersConfiguringDockernetworksandportforwardingIntroducingDockervolumesasasharedstorage

WhatisDocker?Dockerisanopensourceprojectdesignedtohelpwithapplicationdeploymentusingsoftwarecontainers.ThisquoteisfromtheofficialDockerpage:"Dockercontainerswrapapieceofsoftwareinacompletefilesystemthatcontainseverythingneededtorun:code,runtime,systemtools,systemlibraries-anythingthatcanbeinstalledonaserver.Thisguaranteesthatthesoftwarewillalwaysrunthesame,regardlessofitsenvironment."

Docker,therefore,inasimilarwayasvirtualization,allowspackaginganapplicationintoanimagethatcanberuneverywhere.

ContainerizationversusvirtualizationWithoutDocker,isolationandotherbenefitscanbeachievedwiththeuseofhardwarevirtualization,oftencalledvirtualmachines.ThemostpopularsolutionsareVirtualBox,VMware,andParallels.Avirtualmachineemulatesacomputerarchitectureandprovidesthefunctionalityofaphysicalcomputer.Wecanachievecompleteisolationofapplicationsifeachofthemisdeliveredandrunasaseparatevirtualmachineimage.Thefollowingfigurepresentstheconceptofvirtualization:

Eachapplicationislaunchedasaseparateimagewithalldependenciesandaguestoperatingsystem.Imagesarerunbythehypervisor,whichemulatesthephysicalcomputerarchitecture.Thismethodofdeploymentiswidelysupportedbymanytools(suchasVagrant)anddedicatedtodevelopmentandtestingenvironments.Virtualization,however,hasthreesignificantdrawbacks:

Lowperformance:Thevirtualmachineemulatesthewholecomputerarchitecturetoruntheguestoperatingsystem,sothereisasignificantoverheadassociatedwitheachoperation.Highresourceconsumption:Emulationrequiresalotofresourcesandhastobedoneseparatelyforeachapplication.Thisiswhy,onastandarddesktopmachine,onlyafewapplicationscanberunsimultaneously.Largeimagesize:Eachapplicationisdeliveredwithafulloperatingsystem,sothedeploymentonaserverimpliessendingandstoringalarge

amountofdata.

Theconceptofcontainerizationpresentsadifferentsolution:

Eachapplicationisdeliveredtogetherwithitsdependencies,but,withouttheoperatingsystem.Applicationsinterfacedirectlywiththehostoperatingsystem,sothereisnoadditionallayeroftheguestoperatingsystem.Itresultsinbetterperformanceandnowasteofresources.Moreover,shippedDockerimagesaresignificantlysmaller.

Noticethatinthecaseofcontainerization,theisolationhappensatthelevelofthehostoperatingsystem'sprocesses.Itdoesn'tmean,however,thatthecontainerssharetheirdependencies.Eachofthemhastheirownlibrariesintherightversion,andifanyofthemisupdated,ithasnoimpactontheothers.Toachievethis,DockerEnginecreatesasetofLinuxnamespacesandcontrolgroupsforthecontainer.ThisiswhytheDockersecurityisbasedontheLinuxkernelprocessisolation.Thissolution,althoughmatureenough,couldbeconsideredslightlylesssecurethanthecompleteoperatingsystem-basedisolationofferedbyvirtualmachines.

TheneedforDockerDockercontainerizationsolvesanumberofproblemsseenintraditionalsoftwaredelivery.Let'stakeacloserlook.

EnvironmentInstallingandrunningsoftwareiscomplex.Youneedtodecideabouttheoperatingsystem,resources,libraries,services,permissions,othersoftware,andeverythingyourapplicationdependson.Then,youneedtoknowhowtoinstallit.What'smore,theremaybesomeconflictingdependencies.Whatdoyoudothen?Whatifyoursoftwareneedsanupgradeofalibrarybuttheotherdoesnot?Insomecompanies,suchissuesaresolvedbyhavingclassesofapplications,andeachclassisservedbyadedicatedserver,forexample,aserverforwebserviceswithJava7,anotheroneforbatchjobswithJava8,andsoon.Thissolution,however,isnotbalancedintermsofresourcesandrequiresanarmyofIToperationsteamstotakecareofallproductionandtestservers.

Anotherproblemwiththeenvironmentcomplexityisthatitoftenrequiresaspecialisttorunanapplication.AlesstechnicalpersonmayhaveahardtimesettingupMySQL,ODBC,oranyotherslightlymoresophisticatedtool.Thisisparticularlytrueforapplicationsnotdeliveredasanoperatingsystem-specificbinarybutwhichrequiresourcecodecompilationoranyotherenvironment-specificconfiguration.

IsolationKeeptheworkspacetidy.Oneapplicationcanchangethebehavioroftheotherone.Imaginewhatcanhappen.Applicationsshareonefilesystem,soifapplicationAwritessomethingtothewrongdirectory,applicationBreadstheincorrectdata.Theyshareresources,soifthereisamemoryleakinapplicationA,itcanfreezenotonlyitselfbutalsoapplicationB.Theysharenetworkinterfaces,soifapplicationsAandBbothuseport8080,oneofthemwillcrash.Isolationconcernsthesecurityaspectstoo.Runningabuggyapplicationormalicioussoftwarecancausedamagetootherapplications.Thisiswhyitisamuchsaferapproachtokeepeachapplicationinsideaseparatesandbox,whichlimitsthescopeofdamageimpacttotheapplicationitself.

OrganizingapplicationsServersoftenenduplookingmessywithatonofrunningapplicationsnobodyknowsanythingabout.Howwillyoucheckwhatapplicationsarerunningontheserverandwhatdependencieseachofthemisusing?Theycoulddependonlibraries,otherapplications,ortools.Withouttheexhaustivedocumentation,allwecandoislookattherunningprocessesandstartguessing.Dockerkeepsthingsorganizedbyhavingeachapplicationasaseparatecontainerthatcanbelisted,searched,andmonitored.

Portability"Writeonce,runanywhere,"saidthesloganwhileadvertisingtheearliestversionsofJava.Indeed,Javaaddressestheportabilityissuequitewell;however,Icanstillthinkofafewcaseswhereitfails,forexample,theincompatiblenativedependenciesortheolderversionoftheJavaruntime.Moreover,notallsoftwareiswritteninJava.

Dockermovestheconceptofportabilityonelevelhigher;iftheDockerversioniscompatible,thentheshippedsoftwareworkscorrectlyregardlessoftheprogramminglanguage,operatingsystem,orenvironmentconfiguration.Docker,then,canbeexpressedbytheslogan"Shiptheentireenvironmentinsteadofjustcode."

KittensandcattleThedifferencebetweentraditionalsoftwaredeploymentandDocker-baseddeploymentisoftenexpressedwithananalogyofkittensandcattle.Everybodylikeskittens.Kittensareunique.Eachhasitsownnameandneedsspecialtreatment.Kittensaretreatedwithemotion.Wecrywhentheydie.Onthecontrary,cattleexistonlytosatisfyourneeds.Eventheformcattleissingularsinceit'sjustapackofanimalstreatedtogether.Nonaming,nouniqueness.Surely,theyareunique(thesameaseachserverisunique),butitisirrelevant.ThisiswhythemoststraightforwardexplanationoftheideabehindDockeris"Treatyourserverslikecattle,notpets."

AlternativecontainerizationtechnologiesDockerisnottheonlycontainerizationsystemavailableonthemarket.Actually,thefirstversionsofDockerwerebasedontheopensourceLXC(LinuxContainers)system,whichisanalternativeplatformforcontainers.OtherknownsolutionsareFreeBSDJails,OpenVZ,andSolarisContainers.Docker,however,overtookallothersystemsbecauseofitssimplicity,goodmarketing,andstartupapproach.Itworksundermostoperatingsystems,allowsyoutodosomethingusefulinlessthan15minutes,hasalotofsimple-to-usefeatures,goodtutorials,agreatcommunity,andprobablythebestlogointheITindustry.

DockerinstallationDocker'sinstallationprocessisquickandsimple.Currently,it'ssupportedonmostLinuxoperatingsystemsandawiderangeofthemhavededicatedbinariesprovided.MacandWindowsarealsowellsupportedwithnativeapplications.However,it'simportanttounderstandthatDockerisinternallybasedontheLinuxkernelanditsspecifics,andthisiswhy,inthecaseofMacandWindows,itusesvirtualmachines(xhyveforMacandHyper-VforWindows)toruntheDockerEngineenvironment.

PrerequisitesforDockerDockerrequirementsarespecificforeachoperatingsystem.

Mac:

2010ornewermodel,withIntel’shardwaresupportformemorymanagementunit(MMU)virtualizationmacOS10.10.3YosemiteornewerAtleast4GBofRAMNoVirtualBoxpriortoversion4.3.30installed

Windows:

64-bitWindows10ProTheHyper-Vpackageenabled

Linux:

64-bitarchitectureLinuxkernel3.10orlater

Ifyourmachinedoesnotmeettherequirements,thenthesolutionistouseVirtualBoxwiththeUbuntuoperatingsysteminstalled.Thisworkaround,eventhoughitsoundscomplicated,isnotnecessarilytheworstmethod,especiallytakingintoconsiderationthattheDockerEngineenvironmentisvirtualizedanywayinthecaseofMacandWindows.Furthermore,Ubuntuisoneofthebest-supportedsystemsforusingDocker.

AllexamplesinthisbookhavebeentestedontheUbuntu16.04operatingsystem.

InstallingonalocalmachineDockersinstallationprocessisstraightforwardandverywelldescribedonitsofficialpages.

DockerforUbuntuhttps://docs.docker.com/engine/installation/linux/ubuntulinux/containsaguideonhowtoinstallDockeronanUbuntumachine.

InthecaseofUbuntu16.04,I'veexecutedthefollowingcommands:

$sudoapt-getupdate

$sudoapt-keyadv--keyserverhkp://p80.pool.sks-keyservers.net:80--recv-keys9DC858229FC7DD38854AE2D88D81803C0EBFCD88

$sudoapt-add-repository'deb[arch=amd64]https://download.docker.com/linux/ubuntuxenialmainstable'

$sudoapt-getupdate

$sudoapt-getinstall-ydocker-ce

Afteralloperationsarecompleted,Dockershouldbeinstalled.However,atthemoment,theonlyuserallowedtouseDockercommandsisroot.ThismeansthatthesudokeywordmustprecedeeveryDockercommand.

WecanenableotheruserstouseDockerbyaddingthemtothedockergroup:

$sudousermod-aGdocker<username>

Afterasuccessfullogout,everythingissetup.Withthelatestcommand,however,weneedtotakesomeprecautionsnottogivetheDockerpermissionstoanunwanteduser,andthereforecreateavulnerabilityintheDockerEngine.Thisisparticularlyimportantinthecaseofinstallationontheservermachine.

DockerforLinuxhttps://docs.docker.com/engine/installation/linux/containsinstallationguidesformostLinuxdistributions.

DockerforMachttps://docs.docker.com/docker-for-mac/containsastep-by-stepguideonhowtoinstallDockeronaMacmachine.ItisdeliveredtogetherwithacollectionofDockercomponents:

VirtualmachinewithDockerEngineDockerMachine(atoolusedtocreateDockerhostsonthevirtualmachine)DockerComposeDockerclientandserverKitematic:aGUIapplication

TheDockerMachinetoolhelpsininstallingandmanagingDockerEngineonMac,Windows,oncompanynetworks,indatacenters,andoncloudproviderssuchasAWSorDigitalOcean.

DockerforWindowshttps://docs.docker.com/docker-for-windows/containsastep-by-stepguideonhowtoinstallDockeronaWindowsmachine.ItisdeliveredtogetherwithacollectionofDockercomponentssimilartoMac.

TheinstallationguidesforallsupportedoperatingsystemsandcloudplatformscanbefoundontheofficialDockerpage,https://docs.docker.com/engine/installation/.

<strong>$dockerinfo</strong><br/><strong>Containers:0</strong><br/><strong>Running:0</strong><br/><strong>Paused:0</strong><br/><strong>Stopped:0</strong><br/><strong>Images:0</strong><br/><strong>...</strong>

InstallingonaserverInordertouseDockeroverthenetwork,itispossibletoeithertakeadvantageofcloudplatformprovidersortomanuallyinstallDockeronadedicatedserver.

Inthefirstcase,theDockerconfigurationdiffersfromoneplatformtoanother,butitisalwaysverywelldescribedindedicatedtutorials.MostcloudplatformsenablecreatingDockerhostsviauser-friendlywebinterfacesordescribeexactcommandstoexecuteontheirservers.

Thesecondcase(installingDockermanually)requires,however,afewwordsofcomment.

DedicatedserverInstallingDockermanuallyonaserverdoesnotdiffermuchfromthelocalinstallation.

TwoadditionalstepsarerequiredthatincludesettingtheDockerdaemontolistenonthenetworksocketandsettingsecuritycertificates.

Let'sstartfromthefirststep.Bydefault,duetosecurityreasons,Dockerrunsviaanon-networkedUnixsocketthatonlyallowslocalcommunication.It'snecessarytoaddlisteningonthechosennetworkinterfacesocketsothattheexternalclientscanconnect.https://docs.docker.com/engine/admin/describesindetailalltheconfigurationstepsneededforeachLinuxdistribution.

InthecaseofUbuntu,theDockerdaemonisconfiguredbythesystemd,soinordertochangetheconfigurationofhowit'sstarted,weneedtomodifyonelineinthe/lib/systemd/system/docker.servicefile:

ExecStart=/usr/bin/dockerd-H<server_ip>:2375

Bychangingthisline,weenabledtheaccesstotheDockerdaemonviathespecifiedIPaddress.Allthedetailsonthesystemdconfigurationcanbefoundathttps://docs.docker.com/engine/admin/systemd/.

ThesecondstepofserverconfigurationconcernstheDockersecuritycertificates.Thisenablesonlyclientsauthenticatedbyacertificatetoaccesstheserver.ThecomprehensivedescriptionoftheDockercertificatesconfigurationcanbefoundathttps://docs.docker.com/engine/security/https/.Thisstepisn'tstrictlyrequired;however,unlessyourDockerdaemonserverisinsidethefirewallednetwork,itisessential.

IfyourDockerdaemonisruninsidethecorporatenetwork,youhavetoconfiguretheHTTPproxy.Thedetaileddescriptioncanbefoundathttps://docs.docker.com/engine/admin/systemd/.

RunningDockerhelloworld>TheDockerenvironmentissetupandready,sowecanstartthefirstexample.

Enterthefollowingcommandinyourconsole:$dockerrunhello-worldUnabletofindimage'hello-world:latest'locallylatest:Pullingfromlibrary/hello-world78445dd45222:PullcompleteDigest:sha256:c5515758d4c5e1e838e9cd307f6c6a0d620b5e07e6f927b07d05f6d12a1ac8d7Status:Downloadednewerimageforhello-world:latest

HellofromDocker!Thismessageshowsthatyourinstallationappearstobeworkingcorrectly....

Congratulations,you'vejustrunyourfirstDockercontainer.IhopeyoualreadyfeelhowsimpleDockeris.Let'sexaminestep-by-stepwhathappenedunderthehood:

1. YourantheDockerclientwiththeruncommand.2. TheDockerclientcontactedtheDockerdaemonaskingtocreatea

containerfromtheimagecalledhello-world.3. TheDockerdaemoncheckedifitcontainedthehello-worldimagelocally

and,sinceitdidn't,requestedthehello-worldimagefromtheremoteDockerHubregistry.

4. TheDockerHubregistrycontainedthehello-worldimage,soitwaspulledintotheDockerdaemon.

5. TheDockerdaemoncreatedanewcontainerfromthehello-worldimagethatstartedtheexecutableproducingtheoutput.

6. TheDockerdaemonstreamedthisoutputtotheDockerclient.7. TheDockerclientsentittoyourterminal.

Theprojectedflowcanbepresentedinthefollowingdiagram:

Let'slookateachDockercomponentthatwasillustratedinthissection.

DockercomponentsTheofficialDockerpagesaysthis:

"DockerEngineisaclient-serverapplicationthatcreatesandmanagesDockerobjects,suchasimagesandcontainers."

Let'ssortoutwhatthismeans.

DockerclientandserverLet'slookatadiagramthatpresentstheDockerEnginearchitecture:

DockerEngineconsistsofthreecomponents:

DockerDaemon(server)runninginthebackgroundDockerClientrunningasacommandtoolRESTAPI

InstallingDockerEnginemeansinstallingallthecomponentssothattheDockerdaemonrunsonourcomputerallthetimeasaservice.Inthecaseofthehello-worldexample,weusedtheDockerclienttointeractwiththeDockerdaemon;however,wecoulddoexactlythesameusingRESTAPI.Also,inthecaseofthehello-worldexample,weconnectedtothelocalDockerdaemon;however,wecouldusethesameclienttointeractwiththeDockerdaemonrunningonaremotemachine.

ToruntheDockercontaineronaremotemachine,youcanusethe-Hoption:docker-H<server_ip>:2375runhello-world

DockerimagesandcontainersAnimageisastatelessbuildingblockintheDockerworld.Youcanimagineanimageasacollectionofallfilesnecessarytorunyourapplicationtogetherwiththerecipeonhowtorunit.Theimageisstateless,soyoucansenditoverthenetwork,storeitintheregistry,nameit,versionit,andsaveitasafile.Imagesarelayered,whichmeansthatyoucanbuildanimageontopofanotherimage.

Acontainerisarunninginstanceofanimage.Wecancreatemanycontainersfromthesameimageifwewanttohavemanyinstancesofthesameapplication.Sincecontainersarestateful,wecaninteractwiththemandmakechangestotheirstates.

Let'slookattheexampleofacontainerandtheimagelayersstructure:

Atthebottom,thereisalwaysthebaseimage.Inmostcases,itrepresentsanoperatingsystem,andwebuildourimagesontopoftheexistingbaseimages.It'stechnicallypossibletocreateownbaseimageshowever,thisisrarelyneeded.

Inourexample,theubuntubaseimageprovidesallthecapabilitiesoftheUbuntuoperatingsystem.TheaddgitimageaddstheGittoolkit.Then,thereisanimageaddingtheJDKenvironment.Finally,onthetop,thereisacontainercreatedfromtheaddJDKimage.Suchcontainerisable,forexample,todownloadaJavaprojectfromtheGitHubrepositoryandcompileittoaJARfile.Asaresult,wecanusethiscontainertocompileandrunJavaprojectswithoutinstallinganytoolsonouroperatingsystem.

Itisimportanttonoticethatlayeringisaverysmartmechanismtosavebandwidthandstorage.Imaginethatwehaveanapplicationthatisalsobasedonubuntu:

ThistimewewillusethePythoninterpreter.Whileinstallingtheaddpythonimage,theDockerdaemonwillnotethattheubuntuimageisalreadyinstalled,andwhatitneedstodoisonlytoaddthepythonlayer,whichisverysmall.So,theubuntuimageisadependencythatisreused.Thesameifwewouldliketodeployourimageinthenetwork.WhenwedeploytheGitandJDKapplication,weneedtosendthewholeubuntuimage.However,whilesubsequentlydeployingthepythonapplication,weneedtosendjustthesmalladdpythonlayer.

DockerapplicationsAlotofapplicationsareprovidedintheformofDockerimagesthatcanbedownloadedfromtheinternet.Ifweknewtheimagename,thenitwouldbeenoughtorunitinthesamewaywedidwiththehelloworldexample.HowcanwefindthedesiredapplicationimageontheDockerHub?

Let'stakeMongoDBasanexample.IfweliketofinditontheDockerHub,wehavetwooptions:

SearchontheDockerHubExplorepage(https://hub.docker.com/explore/)Usethedockersearchcommand

Inthesecondcase,wecanperformthefollowingoperation:

$dockersearchmongo

NAMEDESCRIPTIONSTARSOFFICIALAUTOMATED

mongoMongoDBdocumentdatabasesprovidehighav...2821[OK]

mongo-expressWeb-basedMongoDBadmininterface,written...106[OK]

mvertes/alpine-mongolightMongoDBcontainer39[OK]

mongoclient/mongoclientOfficialdockerimageforMongoclient,fea...19[OK]

...

Therearemanyinterestingoptions.Howdowechoosethebestimage?Usually,themostappealingoneistheonewithoutanyprefix,sinceitmeansthatit'sanofficialDockerHubimageandshouldbethereforestableandmaintained.Theimageswithprefixesareunofficial,usuallymaintainedasopensourceprojects.Inourcase,thebestchoiceseemstobemongo,soinordertoruntheMongoDBserver,wecanrunthefollowingcommand:

$dockerrunmongo

Unabletofindimage'mongo:latest'locally

latest:Pullingfromlibrary/mongo

5040bd298390:Pullcomplete

ef697e8d464e:Pullcomplete

67d7bf010c40:Pullcomplete

bb0b4f23ca2d:Pullcomplete

8efff42d23e5:Pullcomplete

11dec5aa0089:Pullcomplete

e76feb0ad656:Pullcomplete

5e1dcc6263a9:Pullcomplete

2855a823db09:Pullcomplete

Digest:sha256:aff0c497cff4f116583b99b21775a8844a17bcf5c69f7f3f6028013bf0d6c00c

Status:Downloadednewerimageformongo:latest

2017-01-28T14:33:59.383+0000ICONTROL[initandlisten]MongoDBstarting:pid=1port=27017dbpath=/data/db64-bithost=0f05d9df0dc2

...

That'sall,MongoDBhasstarted.RunningapplicationsasDockercontainersisthatsimplebecausewedon'tneedtothinkofanydependencies;theyarealldeliveredtogetherwiththeimage.

OntheDockerHubservice,youcanfindalotofapplications;theystoremorethan100,000differentimages.

BuildingimagesDockercanbetreatedasausefultooltorunapplications;however,therealpowerliesinbuildingownDockerimagesthatwraptheprogramstogetherwiththeenvironment.Inthissection,wewillseehowtodothisusingtwodifferentmethods,theDockercommitcommandandtheDockerfileautomatedbuild.

DockercommitLet'sstartwithanexampleandprepareanimagewiththeGitandJDKtoolkits.WewilluseUbuntu16.04asabaseimage.Thereisnoneedtocreateit;mostbaseimagesareavailableintheDockerHubregistry:

1. Runacontainerfromtheubuntu:16.04andconnectittoitscommandline:

$dockerrun-i-tubuntu:16.04/bin/bash

We'vepulledtheubuntu:16.04imageandrunitasacontainerandthencalledthe/bin/bashcommandinaninteractiveway(-iflag).Youshouldseetheterminalofthecontainer.Sincecontainersarestatefulandwritable,wecandoanythingwewantinitsterminal.

2. InstalltheGittoolkit:

root@dee2cb192c6c:/#apt-getupdate

root@dee2cb192c6c:/#apt-getinstall-ygit

3. CheckiftheGittoolkitisinstalled:

root@dee2cb192c6c:/#whichgit

/usr/bin/git

4. Exitthecontainer:

root@dee2cb192c6c:/#exit

5. Checkwhathaschangedinthecontainercomparingittotheubuntuimage:

$dockerdiffdee2cb192c6c

Thecommandshouldprintalistofallfileschangedinthecontainer.

6. Committhecontainertotheimage:

$dockercommitdee2cb192c6cubuntu_with_git

We'vejustcreatedourfirstDockerimage.Let'slistalltheimagesofourDockerhosttoseeiftheimageispresent:

$dockerimages

REPOSITORYTAGIMAGEIDCREATEDSIZE

ubuntu_with_gitlatestf3d674114fe2Aboutaminuteago259.7MB

ubuntu16.04f49eec89601e7daysago129.5MB

mongolatest0dffc7177b0610daysago402MB

hello-worldlatest48b5124b27682weeksago1.84kB

Asexpected,weseehello-world,mongo(installedbefore),ubuntu(baseimagepulledfromDockerHub),andfreshlybuiltubuntu_with_git.Bytheway,wecanobservethesizeofeachimagethatcorrespondstowhatwe'veinstalledontheimage.

Now,ifwecreateacontainerfromtheimage,itwillhavetheGittoolinstalled:

$dockerrun-i-tubuntu_with_git/bin/bash

root@3b0d1ff457d4:/#whichgit

/usr/bin/git

root@3b0d1ff457d4:/#exit

Usingtheexactsamemethod,wecanbuildubuntu_with_git_and_jdkontopoftheubuntu_with_gitimage:

$dockerrun-i-tubuntu_with_git/bin/bash

root@6ee6401ed8b8:/#apt-getinstall-yopenjdk-8-jdk

root@6ee6401ed8b8:/#exit

$dockercommit6ee6401ed8b8ubuntu_with_git_and_jdk

DockerfileCreatingeachDockerimagemanuallywiththecommitcommandcouldbelaborious,especiallyinthecaseofbuildautomationandtheContinuousDeliveryprocess.Luckily,thereisabuilt-inlanguagetospecifyalltheinstructionsthatshouldbeexecutedtobuildtheDockerimage.

Let'sstartwithanexamplesimilartotheonewithGitandJDK.Thistime,wewillpreparetheubuntu_with_pythonimage.

1. CreateanewdirectoryandafilecalledDockerfilewiththefollowingcontent:

FROMubuntu:16.04

RUNapt-getupdate&&\

apt-getinstall-ypython

2. Runthecommandtocreatetheubuntu_with_pythonimage:

$dockerbuild-tubuntu_with_python.

3. Checkthattheimagewascreated:

$dockerimages

REPOSITORYTAGIMAGEIDCREATEDSIZE

ubuntu_with_pythonlatestd6e85f39f5b7Aboutaminuteago202.6MB

ubuntu_with_git_and_jdklatest8464dc10abbb3minutesago610.9MB

ubuntu_with_gitlatestf3d674114fe29minutesago259.7MB

ubuntu16.04f49eec89601e7daysago129.5MB

mongolatest0dffc7177b0610daysago402MB

hello-worldlatest48b5124b27682weeksago1.84kB

WecannowcreateacontainerfromtheimageandcheckthatthePythoninterpreterexistsinexactlythesamewaywedidafterexecutingthedockercommitcommand.Notethattheubuntuimageislistedonlyonceeventhoughit'sthebaseimageforbothubuntu_with_gitandubuntu_with_python.

Inthisexample,weusedthefirsttwoDockerfileinstructions:

FROMdefinestheimageontopofwhichthenewimagewillbebuiltRUNspecifiesthecommandstoruninsidethecontainer

AllDockerfileinstructionscanbefoundontheofficialDockerpageathttps://docs.docker.com/engine/reference/builder/.Themostwidely-usedinstructionsareasfollows:

MAINTAINERdefinesthemetainformationabouttheauthorCOPYcopiesafileoradirectoryintothefilesystemoftheimageENTRYPOINTdefineswhichapplicationshouldberunintheexecutablecontainer

AcompleteguideofallDockerfileinstructionscanbefoundontheofficialDockerpageathttps://docs.docker.com/engine/reference/builder/.

CompleteDockerapplicationWealreadyhavealltheinformationnecessarytobuildafullyworkingapplicationasaDockerimage.Asanexample,wewillprepare,stepbystep,asimplePythonhelloworldprogram.Thesamestepsexistalways,nomatterwhatenvironmentorprogramminglanguageweuse.

WritetheapplicationCreateanewdirectoryandinsidethisdirectory,ahello.pyfilewiththefollowingcontent:

print"HelloWorldfromPython!"

Closethefile.Thisisthesourcecodeofourapplication.

FROMubuntu:16.04<br/>MAINTAINERRafalLeszko<br/>RUNapt-getupdate&&\<br/>apt-getinstall-ypython<br/>COPYhello.py.<br/>ENTRYPOINT["python","hello.py"]

BuildtheimageNow,wecanbuildtheimageexactlythesamewaywedidbefore:

$dockerbuild-thello_world_python.

RuntheapplicationWeruntheapplicationbyrunningthecontainer:

$dockerrunhello_world_python

YoushouldseethefriendlyHelloWorldfromPython!message.ThemostinterestingthinginthisexampleisthatweareabletoruntheapplicationwritteninPythonwithouthavingthePythoninterpreterinstalledinourhostsystem.Thisispossiblebecausetheapplicationpackedasanimagehasalltheenvironmentneededinside.

AnimagewiththePythoninterpreteralreadyexistsintheDockerHubservice,sointhereal-lifescenario,itwouldbeenoughtouseit.

EnvironmentvariablesWe'verunourfirsthome-madeDockerapplication.However,whatiftheexecutionoftheapplicationshoulddependonsomeconditions?

Forexample,inthecaseoftheproductionserver,wewouldliketoprintHellotothelogs,nottotheconsole,orwemaywanttohavedifferentdependentservicesduringthetestingphaseandtheproductionphase.OnesolutionwouldbetoprepareaseparateDockerfileforeachcase;however,thereisabetterway,environmentvariables.

Let'schangeourhelloworldapplicationtoprintHelloWorldfrom<name_passed_as_environment_variable>!.Inordertodothis,weneedtoproceedwiththefollowingsteps:

1. ChangethePythonscripttousetheenvironmentvariable:

importos

print"HelloWorldfrom%s!"%os.environ['NAME']

2. Buildtheimage:

$dockerbuild-thello_world_python_name.

3. Runthecontainerpassingenvironmentvariable:

$dockerrun-eNAME=Rafalhello_world_python_name

HelloWorldfromRafal!

4. Alternatively,wecandefinetheenvironmentvariablevalueinDockerfile,forexample:

ENVNAMERafal

5. Then,wecanrunthecontainerwithoutspecifyingthe-eoption.

$dockerbuild-thello_world_python_name_default.

$dockerrunhello_world_python_name_default

HelloWorldfromRafal!

Environmentvariablesareespeciallyusefulwhenweneedtohavedifferent

versionsoftheDockercontainerdependingonitspurpose,forexample,tohaveseparateprofilesforproductionandtestingservers.

IftheenvironmentvariableisdefinedbothinDockerfileandasaflag,thenthecommandflagtakesprecedence.

DockercontainerstatesEveryapplicationwe'verunsofarwassupposedtodosomeworkandstop.Forexample,we'veprintedHellofromDocker!andexited.Thereare,however,applicationsthatshouldruncontinuouslysuchasservices.Torunacontainerinthebackground,wecanusethe-d(--detach)option.Let'stryitwiththeubuntuimage:

$dockerrun-d-tubuntu:16.04

ThiscommandstartedtheUbuntucontainerbutdidnotattachtheconsoletoit.Wecanseethatit'srunningusingthefollowingcommand:

$dockerps

CONTAINERIDIMAGECOMMANDSTATUSPORTSNAMES

95f29bfbaadcubuntu:16.04"/bin/bash"Up5secondskickass_stonebraker

Thiscommandprintsallcontainersthatareintherunningstate.Whataboutourold,already-exitedcontainers?Wecanfindthembyprintingallcontainers:

$dockerps-a

CONTAINERIDIMAGECOMMANDSTATUSPORTSNAMES

95f29bfbaadcubuntu:16.04"/bin/bash"Up33secondskickass_stonebraker

34080d914613hello_world_python_name_default"pythonhello.py"Exitedlonely_newton

7ba49e8ee677hello_world_python_name"pythonhello.py"Exitedmad_turing

dd5eb1ed81c3hello_world_python"pythonhello.py"Exitedthirsty_bardeen

6ee6401ed8b8ubuntu_with_git"/bin/bash"Exitedgrave_nobel

3b0d1ff457d4ubuntu_with_git"/bin/bash"Exiteddesperate_williams

dee2cb192c6cubuntu:16.04"/bin/bash"Exitedsmall_dubinsky

0f05d9df0dc2mongo"/entrypoint.shmongo"Exitedtrusting_easley

47ba1c0ba90ehello-world"/hello"Exitedtender_bell

Notethatalltheoldcontainersareintheexitedstate.Therearetwomorestateswehaven'tobservedyet:pausedandrestarting.

Allofthestatesandthetransitionsbetweenthemarepresentedinthefollowingdiagram:

PausingDockercontainersisveryrare,andtechnically,it'srealizedbyfreezingtheprocessesusingtheSIGSTOPsignal.Restartingisatemporarystatewhenthecontainerisrunwiththe--restartoptiontodefinetherestartingstrategy(theDockerdaemonisabletoautomaticallyrestartthecontainerincaseoffailure).

ThediagramalsoshowstheDockercommandsusedtochangetheDockercontainerstatefromonetoanother.Forexample,wecanstoptherunningUbuntucontainer:

$dockerstop95f29bfbaadc

$dockerps

CONTAINERIDIMAGECOMMANDCREATEDSTATUSPORTSNAMES

Wealwaysusedthedockerruncommandtocreateandstartthecontainer;however,it'spossibletojustcreatethecontainerwithoutstartingit.

DockernetworkingMostapplicationsthesedaysdonotruninisolationbutneedtocommunicatewithothersystemsoverthenetwork.Ifwewanttorunawebsite,webservice,database,oracacheserverinsideaDockercontainer,thenweneedtounderstandatleastthebasicsofDockernetworking.

RunningservicesLet'sstartwithasimpleexample,andrunaTomcatserverdirectlyfromDockerHub:

$dockerrun-dtomcat

Tomcatisawebapplicationserverwhoseuserinterfacecanbeaccessedbytheport8080.Therefore,ifweinstalledTomcatonourmachine,wecouldbrowseitathttp://localhost:8080.

Inourcase,however,TomcatisrunninginsidetheDockercontainer.WestarteditthesamewaywedidwiththefirstHelloWorldexample.Wecanseeit'srunning:

$dockerps

CONTAINERIDIMAGECOMMANDSTATUSPORTSNAMES

d51ad8634factomcat"catalina.shrun"UpAboutaminute8080/tcpjovial_kare

Sinceit'srunasadaemon(withthe-doption),wedon'tseethelogsintheconsolerightaway.Wecan,however,accessitbyexecutingthefollowingcode:

$dockerlogsd51ad8634fac

Iftherearenoerrors,weshouldseealotoflogsthatconcludesTomcathasbeenstartedandisaccessibleviaport8080.Wecantrygoingtohttp://localhost:8080,butwewon'tbeabletoconnect.ThereasonforthisisthatTomcathasbeenstartedinsidethecontainerandwe'retryingtoreachitfromtheoutside.Inotherwords,wecanreachitonlyifweconnectwiththecommandtotheconsoleinthecontainerandcheckitthere.HowtomaketherunningTomcataccessiblefromoutside?

Weneedtostartthecontainerspecifyingtheportmappingwiththe-p(--publish)flag:

-p,--publish<host_port>:<container_port>

So,let'sfirststoptherunningcontainerandstartanewone:

$dockerstopd51ad8634fac

$dockerrun-d-p8080:8080tomcat

Afterwaitingafewseconds,Tomcatmusthavestartedandweshouldbeabletoopenitspage,http://localhost:8080.

SuchasimpleportmappingcommandissufficientinmostcommonDockerusecases.Weareabletodeploy(micro)servicesasDockercontainersandexposetheirportstoenablecommunication.However,let'sdivealittledeeperintowhathappenedunderthehood.

Dockerallowspublishingtothespecifiedhostnetworkinterfacewith-p<ip>:<host_port>:<container_port>.

ContainernetworksWehaveconnectedtotheapplicationrunninginsidethecontainer.Infact,theconnectionistwo-way,becauseifyourememberourpreviousexamples,weexecutedtheapt-getinstallcommandsfrominsideandthepackagesweredownloadedfromtheinternet.Howisthispossible?

Ifyoucheckthenetworkinterfacesonyourmachine,youcanseethatoneoftheinterfacesiscalleddocker0:

$ifconfigdocker0

docker0Linkencap:EthernetHWaddr02:42:db:d0:47:db

inetaddr:172.17.0.1Bcast:0.0.0.0Mask:255.255.0.0

...

Thedocker0interfaceiscreatedbytheDockerdaemoninordertoconnectwiththeDockercontainer.Now,wecanseewhatinterfacesarecreatedinsidetheDockercontainerwiththedockerinspectcommand:

$dockerinspect03d1e6dc4d9e

ItprintsalltheinformationaboutthecontainerconfigurationintheJSONformat.Amongothers,wecanfindthepartrelatedtothenetworksettings:

"NetworkSettings":{

"Bridge":"",

"Ports":{

"8080/tcp":[

{

"HostIp":"0.0.0.0",

"HostPort":"8080"

}

]

},

"Gateway":"172.17.0.1",

"IPAddress":"172.17.0.2",

"IPPrefixLen":16,

}

Inordertofilterthedockerinspectresponse,wecanusethe--formatoption,forexample,dockerinspect--format'{{.NetworkSettings.IPAddress}}'<container_id>.

WecanobservethattheDockercontainerhastheIPaddress172.17.0.2andit

communicateswiththeDockerhostwiththeIPaddress172.17.0.1.Thismeansthatinourpreviousexample,wecouldaccesstheTomcatserverevenwithouttheportforwarding,usingtheaddresshttp://172.17.0.2:8080.Nevertheless,inmostcases,weruntheDockercontaineronaservermachineandwanttoexposeitoutside,soweneedtousethe-poption.

Notethatbydefault,thecontainersareprotectedbythehost'sfirewallsystemanddon'topenanyroutesfromexternalsystems.Wecanchangethisdefaultbehaviorbyplayingwiththe--networkflagandsettingitasfollows:

bridge(default):networkviathedefaultDockerbridgenone:nonetworkcontainer:networkjoinedwiththeother(specified)containerhost:hostnetwork(nofirewall)

Thedifferentnetworkscanbelistedandmanagedbythedockernetworkcommand:

$dockernetworkls

NETWORKIDNAMEDRIVERSCOPE

b3326cb44121bridgebridgelocal

84136027df04hosthostlocal

80c26af0351cnonenulllocal

Ifwespecifynoneasthenetwork,thenwewillnotbeabletoconnecttothecontainer,andviceversa;thecontainerhasnonetworkaccesstotheexternalworld.Thehostoptionmakesthecontainernetworkinterfacesidenticaltothehost.TheysharethesameIPaddresses,soeverythingstartedonthecontainerisvisibleoutside.Themostoftenusedoptionisthedefaultone(bridge)becauseitletsusdefineexplicitlywhichportsshouldbepublished.Itisbothsecureandaccessible.

EXPOSE8080

ThisDockerfileinstructionexpressesthattheport8080shouldbeexposedfromthecontainer.However,aswehavealreadyseen,thisdoesn'tmeanthattheportisautomaticallypublished.TheEXPOSEinstructiononlyinformstheuserswhichportstheyshouldpublish.

AutomaticportassignmentLet'strytorunthesecondTomcatcontainerwithoutstoppingthefirstone:

$dockerrun-d-p8080:8080tomcat

0835c95538aeca79e0305b5f19a5f96cb00c5d1c50bed87584cfca8ec790f241

docker:Errorresponsefromdaemon:driverfailedprogrammingexternalconnectivityonendpointdistracted_heyrovsky(1b1cee9896ed99b9b804e4c944a3d9544adf72f1ef3f9c9f37bc985e9c30f452):Bindfor0.0.0.0:8080failed:portisalreadyallocated.

Thiserrormaybecommon.Insuchcases,wehavetoeithertakecareoftheuniquenessoftheportsonourownorletDockerassigntheportsautomaticallyusingoneofthefollowingversionsofthepublishcommand:

-p<container_port>:publishthecontainerporttotheunusedhostport-P(--publish-all):publishallportsexposedbythecontainertotheunusedhostports:

$dockerrun-d-Ptomcat

078e9d12a1c8724f8aa27510a6390473c1789aa49e7f8b14ddfaaa328c8f737b

$dockerport078e9d12a1c8

8080/tcp->0.0.0.0:32772

WecanseethatthesecondTomcathasbeenpublishedtoport32772,soitcanbebrowsedathttp://localhost:32772.

UsingDockervolumesImaginethatyouwouldliketorunthedatabaseasacontainer.Youcanstartsuchacontainerandenterthedata.Whereisitstored?Whathappenswhenyoustopthecontainerorremoveit?Youcanstartthenewone,butthedatabasewillbeemptyagain.Unlessit'syourtestingenvironment,youdon'texpectsuchascenario.

DockervolumeistheDockerhost'sdirectorymountedinsidethecontainer.Itallowsthecontainertowritetothehost'sfilesystemasitwaswritingtoitsown.Themechanismispresentedinthefollowingdiagram:

Dockervolumeenablesthepersistenceandsharingofthecontainer'sdata.Volumesalsoclearlyseparatetheprocessingfromthedata.

Let'sstartwithanexampleandspecifythevolumewiththe-v<host_path>:<container_path>optionandconnecttothecontainer:

$dockerrun-i-t-v~/docker_ubuntu:/host_directoryubuntu:16.04/bin/bash

Now,wecancreateanemptyfileinhost_directoryinthecontainer:

root@01bf73826624:/#touchhost_directory/file.txt

Let'scheckifthefilewascreatedintheDockerhost'sfilesystem:

root@01bf73826624:/#exit

exit

$ls~/docker_ubuntu/

file.txt

Wecanseethatthefilesystemwassharedandthedatawasthereforepersistedpermanently.Wecannowstopthecontainerandrunanewonetoseethatourfilewillstillbethere:$dockerstop01bf73826624

$dockerrun-i-t-v~/docker_ubuntu:/host_directoryubuntu:16.04/bin/bashroot@a9e0df194f1f:/#lshost_directory/file.txt

root@a9e0df194f1f:/#exit

Insteadofspecifyingthevolumewiththe-vflag,it'spossibletospecifythevolumeasaninstructionintheDockerfile,forexample:

VOLUME/host_directory

Inthiscase,ifwerunthedockercontainerwithoutthe-vflag,thenthecontainer's/host_directorywillbemappedintothehost'sdefaultdirectoryforvolumes,/var/lib/docker/vfs/.Thisisagoodsolutionifyoudeliveranapplicationasanimageandyouknowitneedsthepermanentstorageforsomereason(forexample,storingapplicationlogs).

IfthevolumeisdefinedbothinDockerfileandasaflag,thenthecommandflagtakesprecedence.

Dockervolumescanbemuchmorecomplicated,especiallyinthecaseofdatabases.MorecomplexusecasesoftheDockervolumeare,however,outofthescopeofthisbook.

AverycommonapproachtodatamanagementwithDockeristointroduceanadditionallayerintheformofdatavolumecontainers.AdatavolumecontainerisaDockercontainerwhoseonlypurposeistodeclarethevolume.Then,othercontainerscanuseit(withthe--volumes-from<container>option)insteadofdeclaringthevolumedirectly.Readmoreathttps://docs.docker.com/engine/tutorials/dockervolumes/#creating-and-mounting-a-data-volume-container.

UsingnamesinDockerSofar,whenweoperatedonthecontainers,wealwaysusedautogeneratednames.Thisapproachhassomeadvantages,suchasthenamesbeingunique(nonamingconflicts)andautomatic(noneedtodoanything).Inmanycases,however,it'sbettertogivearealuser-friendlynameforthecontainerortheimage.

NamingcontainersTherearetwogoodreasonstonamethecontainer:convenienceandthepossibilityofautomation:

Convenience,becauseit'ssimplertomakeanyoperationsonthecontaineraddressingitbynamethancheckingthehashesortheautogeneratednameAutomation,becausesometimeswewouldliketodependonthespecificnamingofthecontainer

Forexample,wewouldliketohavecontainersthatdependoneachotherandtohaveonelinkedtoanother.Therefore,weneedtoknowtheirnames.

Tonameacontainer,weusethe--nameparameter:

$dockerrun-d--nametomcattomcat

Wecancheck(bydockerps)thatthecontainerhasameaningfulname.Also,asaresult,anyoperationcanbeperformedusingthecontainer'sname,forexample:

$dockerlogstomcat

Pleasenotethatwhenthecontainerisnamed,itdoesnotloseitsidentity.WecanstilladdressthecontainerbyitsautogeneratedhashIDlikewedidbefore.

ThecontaineralwayshasbothIDandname.Itcanbeaddressedbyanyofthemandbothofthemareunique.

TaggingimagesImagescanbetagged.We'vealreadydonethiswhilecreatingourownimages,forexample,inthecaseofbuildingthehello-world_pythonimage:

$dockerbuild-thello-world_python.

The-tflagdescribesthetagoftheimage.Ifwedidn'tuseit,thentheimagewouldbebuiltwithoutanytagsand,asaresult,wewouldhavetoaddressitbyitsID(hash)inordertorunthecontainer.

Theimagecanhavemultipletags,andtheyshouldfollowthenamingconvention:

<registry_address>/<image_name>:<version>

Thetagconsistsofthefollowingparts:

registry_address:IPandportoftheregistryorthealiasnameimage_name:nameoftheimagethatisbuilt,forexample,ubuntuversion:aversionoftheimageinanyform,forexample,16.04,20170310

WewillcoverDockerregistriesinChapter5,AutomatedAcceptanceTesting.IftheimageiskeptontheofficialDockerHubregistry,thenwecanskiptheregistryaddress.Thisiswhywe'verunthetomcatimagewithoutanyprefix.Thelastversionisalwaystaggedasthelatestanditcanalsobeskipped,sowe'verunthetomcatimagewithoutanysuffix.

Imagesusuallyhavemultipletags,forexample,allfourtagsarethesameimage:ubuntu:16.04,ubuntu:xenial-20170119,ubuntu:xenial,andubuntu:latest.

DockercleanupThroughoutthischapter,wehavecreatedanumberofcontainersandimages.Thisis,however,onlyasmallpartofwhatyouwillseeinreal-lifescenarios.Evenwhenthecontainersarenotrunningatthemoment,theyneedtobestoredontheDockerhost.Thiscanquicklyresultinexceedingthestoragespaceandstoppingthemachine.Howcanweapproachthisconcern?

CleaningupcontainersFirst,let'slookatthecontainersthatarestoredonourmachine.Toprintallthecontainers(nomatteroftheirstate),wecanusethedockerps-acommand:$dockerps-aCONTAINERIDIMAGECOMMANDSTATUSPORTSNAMES95c2d6c4424etomcat"catalina.shrun"Up5minutes8080/tcptomcata9e0df194f1fubuntu:16.04"/bin/bash"Exitedjolly_archimedes01bf73826624ubuntu:16.04"/bin/bash"Exitedsuspicious_feynman078e9d12a1c8tomcat"catalina.shrun"Up14minutes0.0.0.0:32772->8080/tcpnauseous_fermi0835c95538aetomcat"catalina.shrun"Createddistracted_heyrovsky03d1e6dc4d9etomcat"catalina.shrun"Up50minutes0.0.0.0:8080->8080/tcpdrunk_ritchied51ad8634factomcat"catalina.shrun"Exitedjovial_kare95f29bfbaadcubuntu:16.04"/bin/bash"Exitedkickass_stonebraker34080d914613hello_world_python_name_default"pythonhello.py"Exitedlonely_newton7ba49e8ee677hello_world_python_name"pythonhello.py"Exitedmad_turingdd5eb1ed81c3hello_world_python"pythonhello.py"Exitedthirsty_bardeen6ee6401ed8b8ubuntu_with_git"/bin/bash"Exitedgrave_nobel3b0d1ff457d4ubuntu_with_git"/bin/bash"Exiteddesperate_williamsdee2cb192c6cubuntu:16.04"/bin/bash"Exitedsmall_dubinsky0f05d9df0dc2mongo"/entrypoint.shmongo"Exitedtrusting_easley47ba1c0ba90ehello-world"/hello"Exitedtender_bell

Inordertodeleteastoppedcontainer,wecanusethedockerrmcommand(ifthecontainerisrunning,weneedtostopitfirst):$dockerrm47ba1c0ba90e

Ifwewanttoremoveallstoppedcontainers,wecanusethefollowingcommand:

$dockerrm$(dockerps--no-trunc-aq)

The-aqoptionspecifiestopassonlyIDs(noadditionaldata)forallcontainers.

Additionally,--no-truncasksDockernottotruncatetheoutput.

Wecanalsoadoptadifferentapproachandaskthecontainertoremoveitself,whenit'sstoppedusingthe--rmflag,forexample:$dockerrun--rmhello-world

Inmostreal-lifescenarios,wedon'tusethestoppedcontainers,andtheyareleftonlyforthedebuggingpurpose.

CleaningupimagesImagesarejustasimportantascontainers.Theycanoccupyalotofspace,especiallyinthecaseoftheContinuousDeliveryprocess,wheneachbuildendsupinanewDockerimage.Thiscanquicklyresultinthenospaceleftondeviceerror.TocheckalltheimagesintheDockercontainer,wecanusethedockerimagescommand:$dockerimagesREPOSITORYTAGIMAGEIDCREATEDSIZEhello_world_python_name_defaultlatest9a056ca928412hoursago202.6MBhello_world_python_namelatest72c8c50ffa892hoursago202.6MBhello_world_pythonlatest3e1fa5c29b442hoursago202.6MBubuntu_with_pythonlatestd6e85f39f5b72hoursago202.6MBubuntu_with_git_and_jdklatest8464dc10abbb2hoursago610.9MBubuntu_with_gitlatestf3d674114fe23hoursago259.7MBtomcatlatestc822d296d2322daysago355.3MBubuntu16.04f49eec89601e7daysago129.5MBmongolatest0dffc7177b0611daysago402MBhello-worldlatest48b5124b27682weeksago1.84kB

Toremoveanimage,wecancallthefollowingcommand:

$dockerrmi48b5124b2768

Inthecaseofimages,theautomaticcleanupprocessisslightlymorecomplex.Imagesdon'thavestates,sowecannotaskthemtoremovethemselveswhennotused.ThecommonstrategywouldbetosetuptheCroncleanupjob,whichremovesalloldandunusedimages.Wecoulddothisusingthefollowingcommand:$dockerrmi$(dockerimages-q)

Inordertopreventremovingtheimageswithtags(forexample,tonotremoveallthelatestimages),it'sverycommontousethedanglingparameter:

$dockerrmi$(dockerimages-f"dangling=true"-q)

Ifwehavecontainersthatusevolumes,then,inadditiontoimagesandcontainers,it'sworthtothinkaboutcleaningupvolumes.The

easiestwaytodothisistousethedockervolumels-qfdangling=true|xargs-rdockervolumermcommand.

DockercommandsoverviewAllDockercommandscanbefoundbyexecutingthefollowinghelpcommand:$dockerhelp

ToseealltheoptionsofanyparticularDockercommand,wecanusedockerhelp<command>,forexample:$dockerhelprun

ThereisalsoaverygoodexplanationofallDockercommandsontheofficialDockerpagehttps://docs.docker.com/engine/reference/commandline/docker/.It'sreallyworthreadingoratleastskimmingthrough.

Inthischapter,we'vecoveredthemostusefulcommandsandtheiroptions.Asaquickreminder,let'swalkthroughthem:

Command Explanation

dockerbuild BuildanimagefromaDockerfile

dockercommit Createanimagefromthecontainer

dockerdiff Showchangesinthecontainer

dockerimages Listimages

dockerinfo DisplayDockerinformation

dockerinspect ShowtheconfigurationoftheDockerimage/container

dockerlogs Showlogsofthecontainer

dockernetwork Managenetworks

dockerport Showallexposedportsbythecontainer

dockerps Listcontainers

dockerrm Removecontainer

dockerrmi Removeimage

dockerrun Runacontainerfromtheimage

dockersearch SearchfortheDockerimageinDockerHub

dockerstart/stop/pause/unpause Managethecontainer'sstate

Exercises

We'vecoveredalotofmaterialinthischapter.Tomakewell-remembered,werecommendtwoexercises.

1. RunCouchDBasaDockercontainerandpublishitsport:

YoucanusethedockersearchcommandtofindtheCouchDBimage.

RunthecontainerPublishtheCouchDBportOpenthebrowserandcheckthatCouchDBisavailable

2. CreateaDockerimagewiththeRESTservicereplyingHelloWorld!tolocalhost:8080/hello.Useanylanguageandframeworkyouprefer:

TheeasiestwaytocreateaRESTserviceistousePythonwiththeFlaskframework,http://flask.pocoo.org/.Notethatalotofwebframeworksstarttheapplicationonthelocalhostinterfaceonlybydefault.Inordertopublishaport,it'snecessarytostartitonallinterfaces(app.run(host='0.0.0.0')inthecaseofaFlaskframework).

CreateawebserviceapplicationCreateaDockerfiletoinstalldependenciesandlibrariesBuildtheimageRunthecontainerpublishingtheportCheckifit'srunningcorrectlyusingthebrowser

SummaryInthischapter,wehavecoveredtheDockerbasicsthatareenoughtobuildimagesandrunapplicationsascontainers.Thekeytakeaway,fromthechapterarethefollowingpoints:

ThecontainerizationtechnologyaddressestheissuesofisolationandenvironmentdependenciesusingtheLinuxkernelfeatures.Thisisbasedontheprocessseparationmechanism,thereforenorealperformancedropisobserved.DockercanbeinstalledonmostofthesystemsbutissupportednativelyonlyonLinux.Dockerallowsrunningapplicationsfromtheimagesavailableontheinternetandtobuildownimages.Animageisanapplicationpackedtogetherwithalldependencies.Dockerprovidestwomethodsforbuildingtheimages:Dockerfileorcommittingthecontainer.Inmostcases,thefirstoptionisused.Dockercontainerscancommunicateoverthenetworkbypublishingtheportstheyexpose.Dockercontainerscansharethepersistentstorageusingvolumes.Forthepurposeofconvenience,Dockercontainersshouldbenamed,andDockerimagesshouldbetagged.IntheDockerworld,thereisaspecificconventionofhowtotagimages.Dockerimagesandcontainersshouldbecleanedfromtimetotimeinordertosavetheserverspaceandavoidthenospaceleftondeviceerror.

Inthenextchapter,wewillcoverJenkinsconfigurationandthewayJenkinscanbeusedtogetherwithDocker.

ConfiguringJenkins

WehaveseenhowtoconfigureanduseDocker.Inthischapter,wewillpresentJenkins,whichcanbeusedseparatelyortogetherwithDocker.Wewillshowthatthecombinationofthesetwotoolsoutcomesinthesurprisinglygoodresults:automatedconfigurationandflexiblescalability.

Thischaptercoversthefollowingtopics:

IntroducingJenkinsanditsadvantagesInstallingandstartingJenkinsCreatingthefirstpipelineScalingJenkinswithagentsConfiguringDocker-basedagentsBuildingcustommasterandslaveDockerimagesConfiguringstrategiesforthesecurityandbackup

WhatisJenkins?

JenkinsisanopensourceautomationserverwritteninJava.Withtheveryactivecommunity-basedsupportandahugenumberofplugins,itisthemostpopulartooltoimplementtheContinuousIntegrationandContinuousDeliveryprocesses.FormerlyknownasHudson,itwasrenamedafterOracleboughtHudsonanddecidedtodevelopitastheproprietarysoftware.JenkinsremainedundertheMITlicenseandishighlyvaluedforitssimplicity,flexibility,andversatility.

JenkinsoutstandsotherContinuousIntegrationtoolsandisthemostwidelyusedsoftwareofitskind.Thatisallpossiblebecauseofitsfeaturesandcapabilities.

Let'swalkthroughthemostinterestingpartsoftheJenkins'characteristics.

Languageagnostic:Jenkinshasalotofplugins,whichsupportmostoftheprogramminglanguagesandframeworks.Moreover,sinceitcanuseanyshellcommandandanysoftwareinstalled,itissuitableforeveryautomationprocessthatcanbeimagined.Extensiblebyplugins:Jenkinshasagreatcommunityandalotofpluginsavailable(1000plus).ItalsoallowsyoutowriteyourownpluginsinordertocustomizeJenkinsforyourneeds.Portable:JenkinsiswritteninJava,soitcanberunonanyoperatingsystem.Forconvenience,itisalsodeliveredinalotofversions:webapplicationarchive(WAR),Dockerimage,Windowsbinary,Macbinary,andLinuxbinaries.SupportsmostSCM:Jenkinsintegrateswithvirtuallyeverysourcecodemanagementorbuildtoolthatexists.Again,becauseofitswidecommunityandplugins,thereisnoothercontinuousintegrationtoolthatsupportssomanyexternalsystems.Distributed:Jenkinshasabuilt-inmechanismforthemaster/slavemode,whichdistributesitsexecutionsacrossmultiplenodeslocatedonmultiple

machines.Itcanalsouseheterogeneousenvironment,forexample,differentnodescanhavedifferentoperatingsysteminstalled.Simplicity:Theinstallationandconfigurationprocessissimple.Thereisnoneedtoconfigureanyadditionalsoftware,northedatabase.ItcanbeconfiguredcompletelyviaGUI,XML,orGroovyscripts.Code-oriented:Jenkinspipelinesaredefinedascode.Also,theJenkinsitselfcanbeconfiguredusingXMLfilesorGroovyscripts.ThatallowskeepingtheconfigurationinthesourcecoderepositoryandhelpsintheautomationoftheJenkinsconfiguration.

JenkinsinstallationTheJenkinsinstallationprocessisquickandsimple.Therearedifferentmethodstodoit,butsincewe'realreadyfamiliarwiththeDockertoolandthebenefitsitgives,wewillstartfromtheDocker-basedsolution.Thisisalsotheeasiest,themostpredictable,andthesmartestwaytogo.However,let'smentiontheinstallationrequirementsfirst.

RequirementsforinstallationTheminimalsystemrequirementsarerelativelylow:

Java8256MBfreememory1GB+freediskspace

However,it'sessentialtounderstandthattherequirementsstrictlydependonwhatyouplantodowithJenkins.IfJenkinsisusedtoservethewholeteamastheContinuousIntegrationserver,thenevenincaseofasmallteam,it'sadvisedtohave1GBplusfreememoryand50GBplusfreediskspace.Needlesstosay,Jenkinsalsoperformssomecomputationsandtransfersalotofdataacrossthenetwork,soCPUandbandwidtharecrucial.

Togetafeelingwhatcouldbetherequirementsincaseofabigcompany,theNetflixexampleispresentedintheJenkinsarchitecturesection.

InstallingonDockerLet’sseethestep-by-stepproceduretoinstallJenkinsusingDocker.

TheJenkinsimageisavailableintheofficialDockerHubregistry,soinordertoinstallitweshouldexecutethefollowingcommand:

$dockerrun-p<host_port>:8080-v<host_volume>:/var/jenkins_homejenkins:2.60.1

Weneedtospecifythefirsthost_portparameter—theportunderwhichJenkinsisvisibleoutsideofthecontainer.Thesecondparameter,host_volume,specifiesthedirectorywheretheJenkinshomeismapped.Itneedstobespecifiedasvolume,andthereforepersistedpermanently,becauseitcontainstheconfiguration,pipelinebuilds,andlogs.

Asanexample,let'sseewhattheinstallationstepswouldlooklikeincaseoftheDockerhostonLinux/Ubuntu.

1. Preparethevolumedirectory:WeneedaseparatedirectorywiththeadminownershiptokeeptheJenkinshome.Let'sprepareonewiththefollowingcommands:

$mkdir$HOME/jenkins_home

$chown1000$HOME/jenkins_home

2. RuntheJenkinscontainer:Let'srunthecontainerasadaemonandgiveitapropername:

$dockerrun-d-p49001:8080

-v$HOME/jenkins_home:/var/jenkins_home--name

jenkinsjenkins:2.60.1

3. CheckifJenkinsisrunning:Afteramoment,wecancheckwhetherJenkinshasstartedcorrectlybyprintingthelogs:

$dockerlogsjenkins

Runningfrom:/usr/share/jenkins/jenkins.war

webroot:EnvVars.masterEnvVars.get("JENKINS_HOME")

Feb04,20179:01:32AMMaindeleteWinstoneTempContents

WARNING:FailedtodeletethetemporaryWinstonefile

/tmp/winstone/jenkins.war

Feb04,20179:01:32AMorg.eclipse.jetty.util.log.JavaUtilLoginfo

INFO:Logginginitialized@888ms

Feb04,20179:01:32AMwinstone.LoggerlogInternal

...

Intheproductionenvironment,youmayalsowanttosetupthereverseproxyinordertohidetheJenkinsinfrastructurebehindtheproxyserver.TheshortdescriptionhowtodoitusingtheNginxservercanbefoundathttps://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+with+Docker.

Afterperformingthesefewsteps,Jenkinsisreadytouse.TheDocker-basedinstallationhastwomajoradvantages:

Failurerecovery:IfJenkinscrashes,thenit'senoughtorunanewcontainerwiththesamevolumespecified.Customimages:YoucanconfigureJenkinsasperyourneedsandstoreitastheJenkinsimage.Thenitcanbesharedwithinyourorganizationorteam,andthereisnoneedtorepeatthesameconfigurationstepsallthetime,manytimes.

Everywhereinthisbook,weuseJenkinsinversion2.60.1.

<strong>$wget-q-O-https://pkg.jenkins.io/debian/jenkins.io.key|sudoapt-keyadd-</strong><br/><strong>$sudosh-c'echodebhttp://pkg.jenkins.io/debian-stablebinary/>/etc/apt/sources.list.d/jenkins.list'</strong><br/><strong>$sudoapt-getupdate</strong><br/><strong>$sudoapt-getinstalljenkins</strong>

Allinstallationguides(Ubuntu,Mac,Windows,andothers)canbefoundontheofficialJenkinspagehttps://jenkins.io/doc/book/getting-started/installing/.

InitialconfigurationNomatterwhichinstallationyouchoose,thefirststartofJenkinsrequiresafewconfigurationsteps.Let'swalkthroughthemstepbystep:

1. OpenJenkinsinthebrowser:http://localhost:49001(forbinaryinstallations,thedefaultportis8080).

2. Jenkinsshouldaskfortheadministratorpassword.ItcanbefoundintheJenkinslogs:

$dockerlogsjenkins

...

Jenkinsinitialsetupisrequired.Anadminuserhasbeencreated

andapasswordgenerated.

Pleaseusethefollowingpasswordtoproceedtoinstallation:

c50508effc6843a1a7b06f6491ed0ca6

...

3. Afteracceptingtheinitialpassword,Jenkinsaskswhethertoinstallthesuggestedplugins,whichareadjustedtothemostcommonusecases.Youranswerdepends,ofcourse,onyourneeds.However,asthefirstJenkinsinstallation,it'sreasonabletoletJenkinsinstallalltherecommendedplugins.

4. Aftertheplugininstallation,Jenkinsaskstosetupusername,password,andotherbasicinformation.Ifyouskipit,thetokenfromstep2willbeusedastheadminpassword.

TheinstallationiscompleteandyoushouldseetheJenkinsdashboard:

WearereadytouseJenkinsandcreatethefirstpipeline.

JenkinshelloworldEverythingintheentireITworldstartsfromtheHelloWorldexample.

Let'sfollowthisruleandseethestepstocreatethefirstJenkinspipeline:

1. ClickonNewItem.2. Enterhelloworldastheitemname,choosePipeline,andclickonOK.3. Therearealotofoptions.Wewillskipthemfornowandgodirectlytothe

Pipelinesection.4. There,intheScripttextbox,wecanenterthepipelinescript:

pipeline{

agentany

stages{

stage("Hello"){

steps{

echo'HelloWorld'

}

}

}

}

5. ClickonSave.6. ClickonBuildNow.

Weshouldsee#1undertheBuildHistory.Ifweclickonit,andthenonConsoleOutput,wewillseethelogfromthePipelinebuild.

WehavejustseenthefirstexampleanditssuccessfuloutputmeansthatJenkins

isinstalledcorrectly.Now,let'smovetoslightlymoreadvancedJenkinsconfiguration.

WewilldescribemoreonthepipelinesyntaxinChapter4,ContinuousIntegrationPipeline.

JenkinsarchitectureThehelloworldjobexecutedinalmostnotimeatall.However,thepipelinesareusuallymorecomplexandspendtimeontaskssuchasdownloadingfilesfromtheinternet,compilingthesourcecode,orrunningtests.Onebuildcantakefromminutestohours.

Incommonscenarios,therearealsomanyconcurrentpipelines.Usually,thewholeteam,oreventhewholeorganization,usesthesameJenkinsinstance.Howtoensurethatthebuildswillrunquicklyandsmoothly?

MasterandslavesJenkinsbecomesoverloadedsoonerthanitseems.Evenincaseofasmall(micro)service,thebuildcantakeafewminutes.ThatmeansthatoneteamcommittingfrequentlycaneasilykilltheJenkinsinstance.

Forthatreason,unlesstheprojectisreallysmall,Jenkinsshouldnotexecutebuildsatall,butdelegatethemtotheslave(agent)instances.Tobeprecise,theJenkinswe'recurrentlyrunningiscalledtheJenkinsmasteranditcandelegatetotheJenkinsagents.

Let'slookatthediagrampresentingthemaster-slaveinteraction:

Inadistributedbuildsenvironment,theJenkinsmasterisresponsiblefor:

Receivingbuildtriggers(forexample,afteracommittoGitHub)Sendingnotifications(forexample,emailorHipChatmessagesentafterthebuildfailure)HandlingHTTPrequests(interactionwithclients)

Managingthebuildenvironment(orchestratingthejobexecutionsonslaves)

Thebuildagentisamachinethattakecareofeverythingthathappensafterthebuildisstarted.

Sincetheresponsibilitiesofthemasterandtheslavesaredifferent,theyhavedifferentenvironmentalrequirements:

Master:Thisisusually(unlesstheprojectisreallysmall)adedicatedmachinewithRAMrangingfrom200MBforsmallprojectsto70GBplusforhugesingle-masterprojects.Slave:Therearenogeneralrequirements(otherthanthefactthatitshouldbecapableofexecutingasinglebuild,forexample,iftheprojectisahugemonoliththatrequires100GBofRAM,thentheslavemachineneedstosatisfytheseneeds).

Agentsshouldalsobeasgenericaspossible.Forinstance,ifwehavedifferentprojects:oneinJava,oneinPython,andoneinRuby,thenitwouldbeperfectifeachagentcouldbuildanyoftheseprojects.Insuchacase,theagentscanbeinterchanged,whichhelpstooptimizetheusageofresources.

Ifagentscannotbegenericenoughtomatchallprojects,thenit'spossibletolabel(tag)agentsandprojects,sothatthegivenbuildwouldbeexecutedonagiventypeofagent.

ScalabilityWecanuseJenkinsslavestobalancetheloadandscaleuptheJenkinsinfrastructure.Suchaprocessiscalledthehorizontalscaling.Theotherpossibilitywouldbetouseonlyonemasternodeandincreaseresourcesofitsmachine.Thatprocessiscalledtheverticalscaling.Let'slookcloseratthesetwoconcepts.

VerticalscalingVerticalscalingmeansthat,whenthemaster'sloadgrows,thenmoreresourcesareappliedtothemaster'smachine.So,whennewprojectsappearinourorganization,webuymoreRAM,addCPUcores,andextendtheHDDdrives.Thismaysoundlikeano-gosolution;however,it'softenused,evenbywell-knownorganizations.HavingasingleJenkinsmastersetontheultra-efficienthardwarehasoneverystrongadvantage:maintenance.Anyupgrades,scripts,securitysettings,rolesassignment,orplugininstallationshavetobedoneinoneplaceonly.

Horizontalscaling

Horizontalscalingmeansthat,whenanorganizationgrows,thenmoremasterinstancesarelaunched.Thisrequiresasmartallocationofinstancestoteamsand,intheextremecase,eachteamcanhaveitsownJenkinsmaster.Inthatcase,itmightevenhappenthatnoslavesareneeded.

Thedrawbacksarethatitmaybedifficulttoautomatecross-projectintegrationsandthatapartoftheteam'sdevelopmenttimeisspentontheJenkinsmaintenance.However,thehorizontalscalinghassomesignificantadvantages:

Mastermachinesdon'tneedtobespecialintermsofhardwareDifferentteamscanhavedifferentJenkinssettings(forexample,differentsetofplugins)TeamsusuallyfeelbetterandworkwithJenkinsmoreefficientlyiftheinstanceistheirownIfonemasterinstanceisdown,itdoesnotimpactthewholeorganizationTheinfrastructurecanbesegregatedintostandardandmission-criticalSomemaintenanceaspectscanbesimplified,forexample,theteamoffivecouldreusethesameJenkinspassword,sowemayskiptherolesandsecuritysettings(surelythatispossibleonlyifthecorporatenetworkiswellfirewalled)

TestandproductioninstancesApartfromthescalingapproach,thereisonemoreissue:howtotesttheJenkinsupgrades,newplugins,orpipelinedefinitions?Jenkinsiscriticaltothewholecompany.Itguaranteesthequalityofthesoftwareand(incaseofContinuousDelivery)deploystotheproductionservers.Thatiswhyitneedstobehighlyavailable,soitisdefinitelynotforthepurposeoftesting.ItmeansthereshouldalwaysbetwoinstancesofthesameJenkinsinfrastructure:testandproduction.

Testenvironmentshouldalwaysbeassimilaraspossibletotheproduction,soitalsorequiresthesimilarnumberofagentsattached.

SamplearchitectureWealreadyknowthatthereshouldbeslaves,(possiblymultiple)master(s),andthateverythingshouldbeduplicatedintothetestandproductionenvironments.However,whatwouldthecompletepicturelooklike?

Luckily,therearealotofcompaniesthatpublishedhowtheyusedJenkinsandwhatkindofarchitecturestheycreated.Itwouldbedifficulttomeasureifmoreofthempreferredverticalorhorizontalscaling,butitrangedfromhavingonlyonemasterinstancetohavingonemasterforeachteam.

Let'slookattheexampleofNetflixtohaveacompletepictureoftheJenkinsinfrastructure(theyshareditastheplannedinfrastructureatJenkinsUserConferenceSanFrancisco2012):

Theyhavetestandproductionmasterinstances,eachofthemowningapollofslavesandadditionaladhocslaves.Altogether,itservesaround2000buildsperday.NotealsothatapartoftheirinfrastructureishostedonAWSandapartisontheirownservers.

WeshouldalreadyhaveatleastaroughideaofwhattheJenkinsinfrastructure

canlooklikedependingonthetypeoftheorganization.

Let'snowfocusonthepracticalaspectsofsettingtheagents.

ConfiguringagentsWe'veseenwhattheagentsareandwhentheycanbeused.However,howtosetupanagentandletitcommunicatewiththemaster?Let'sstartwiththesecondpartofthequestionanddescribethecommunicationprotocolsbetweenthemasterandtheagent.

CommunicationprotocolsInorderforthemasterandtheagenttocommunicate,thebi-directionalconnectionhastobeestablished.

Therearedifferentoptionshowitcanbeinitiated:

SSH:MasterconnectstoslaveusingthestandardSSHprotocol.JenkinshasanSSH-clientbuilt-in,sotheonlyrequirementistheSSHDserverconfiguredonslaves.ThisisthemostconvenientandstablemethodbecauseitusesstandardUnixmechanisms.JavaWebStart:JavaapplicationisstartedoneachagentmachineandtheTCPconnectionisestablishedbetweentheJenkinsslaveapplicationandthemasterJavaapplication.Thismethodisoftenusediftheagentsareinsidethefirewallednetworkandthemastercannotinitiatetheconnection.Windowsservice:ThemasterregistersanagentontheremotemachineasaWindowsservice.Thismethodisdiscouragedsincethesetupistrickyandtherearelimitationsonthegraphicalinterfacesusage.

Ifweknowthecommunicationprotocols,let'sseehowwecanusethemtosettheagents.

SettingagentsAtthelowlevel,agentscommunicatewiththeJenkinsmasteralwaysusingoneoftheprotocolsdescribedabove.However,atthehigherlevel,wecanattachslavestothemasterinvariousways.Thedifferencesconcerntwoaspects:

staticversusdynamic:ThesimplestoptionistoaddslavespermanentlyintheJenkinsmaster.Thedrawbackofsuchsolutionisthatwealwaysneedtomanuallychangesomethingifweneedmore(orless)slavenodes.Abetteroptionistodynamicallyprovisionslavesastheyareneeded.specificversusgeneral-purpose:Agentscanbespecific(forexample,differentagentsfortheprojectsbasedonJava7anddifferentagentsforJava8)orgeneral-purpose(anagentactsasaDockerhostandapipelineisbuiltinsideaDockercontainer).

Thesedifferencesresultedinfourcommonstrategieshowagentsareconfigured:

PermanentagentsPermanentDockeragentsJenkinsSwarmagentsDynamicallyprovisionedDockeragents

Let'sexamineeachofthesolutions.

PermanentagentsWestartfromthesimplestoptionwhichistopermanentlyaddspecificagentnodes.ItcanbedoneentirelyviatheJenkinswebinterface.

ConfiguringpermanentagentsIntheJenkinsmaster,whenweopenManageJenkinsandthenManageNodes,wecanviewalltheattachedagents.Then,byclickingonNewNode,givingitaname,andconfirmingwiththeOKbutton,weshouldfinallyseetheagent's

setuppage:

Let'swalkthroughtheparametersweneedtofill:

Name:ThisistheuniquenameoftheagentDescription:Thisisanyhuman-readabledescriptionoftheagent#ofexecutors:ThisisthenumberofconcurrentbuildsthatcanberunontheslaveRemoterootdirectory:Thisisthededicateddirectoryontheslavemachinethattheagentcanusetorunbuildjobs(forexample,/var/jenkins);themostimportantdataistransferredbacktothemaster,sothedirectoryisnotcriticalLabels:Thisincludesthetagstomatchonlythespecificbuilds(taggedthesame),forexample,onlyprojectsbasedonJava8

Usage:Thisistheoptiontodecidewhethertheagentshouldbeusedonlyformatchedlabels(forexample,onlyforAcceptanceTestingbuilds)orforanybuilds

Launchmethod:Thisincludesthefollowing:

LaunchagentviaJavaWebStart:Here,theconnectionwillbeestablishedbytheagent;itispossibletodownloadtheJARfileandtheinstructionsonhowtorunitontheslavemachine

Launchagentviaexecutionofcommandonthemaster:Thisisthecustomcommandrunonthemastertostarttheslave;inmostcasesitwillsendtheJavaWebStartJARapplicationandstartitontheslave(forexample,ssh<slave_hostname>java-jar~/bin/slave.jar)LaunchslaveagentsviaSSH:Here,themasterwillconnecttotheslaveusingtheSSHprotocolLetJenkinscontrolthisWindowsslaveasaWindowsservice:Here,themasterwillstartaremotemanagementfacilitybuiltintoWindows

Availability:Thisistheoptiontodecidewhethertheagentshouldbeupallthetimeorthemastershouldturnitofflineundercertainconditions

Whentheagentsaresetupcorrectly,it'spossibletoswitchthemasternodeoffline,sothatnobuildswouldbeexecutedonitanditwouldserveonlyastheJenkinsUIandthebuilds'coordinator.

UnderstandingpermanentagentsAsalreadymentioned,thedrawbackofsuchasolutionisthatweneedtomaintainmultipleslavetypes(labels)fordifferentprojecttypes.Suchasituationispresentedinthefollowingdiagram:

Inourexample,ifwehavethreetypesofprojects(java7,java8,andruby),thenweneedtomaintainthreeseparatelylabeled(setsof)slaves.Thatisthesameissuewehadwhilemaintainingmultipleproductionservertypes,asdescribedinChapter2,IntroducingDocker.WeaddressedtheissuebyhavingtheDockerEngineinstalledontheproductionservers.Let'strytodothesamewithJenkinsslaves.

PermanentDockeragentsTheideabehindthissolutionistopermanentlyaddgeneral-purposeslaves.Eachslaveisidenticallyconfigured(DockerEngineinstalled)andeachbuildisdefinedtogetherwiththeDockerimageinsidewhichthebuildisrun.

pipeline{<br/>agent{<br/>docker{<br/>image'openjdk:8-jdk-alpine'<br/>}<br/>}<br/>...<br/>}

Whenthebuildisstarted,theJenkinsslavestartsacontainerfromtheDockerimageopenjdk:8-jdk-alpineandthenexecutesallpipelinestepsinsidethatcontainer.Thisway,wealwaysknowtheexecutionenvironmentanddon'thavetoconfigureeachslaveseparatelydependingontheparticularprojecttype.

UnderstandingpermanentDockeragentsLookingatthesamescenariowetookforthepermanentagents,thediagramlookslikethis:

EachslaveisexactlythesameandifwewouldliketobuildaprojectthatdependsonJava8,thenwedefinetheappropriateDockerimageinthepipelinescript(insteadofspecifyingtheslavelabel).

JenkinsSwarmagentsSofar,wealwayshadtopermanentlydefineeachoftheagentsintheJenkinsmaster.Suchasolution,eventhoughgoodenoughinmanycases,canbeaburdenifweneedtofrequentlyscalethenumberofslavemachines.JenkinsSwarmallowsyoutodynamicallyaddslaveswithouttheneedtoconfigurethemintheJenkinsmaster.

ConfiguringJenkinsSwarmagentsThefirststeptouseJenkinsSwarmistoinstalltheSelf-OrganizingSwarmPlug-inModulesplugininJenkins.WecandoitviatheJenkinswebUIunderManageJenkinsandManagePlugins.Afterthisstep,theJenkinsmasterispreparedforJenkinsslavestobedynamicallyattached.

ThesecondstepistoruntheJenkinsSwarmslaveapplicationoneverymachinethatwouldactasaJenkinsslave.Wecandoitusingtheswarm-client.jarapplication.

Theswarm-client.jarapplicationcanbedownloadedfromtheJenkinsSwarmpluginpage:https://wiki.jenkins-ci.org/display/JENKINS/Swarm+Plugin.Onthatpage,youcanalsofindallthepossibleoptionsofitsexecution.

InordertoattachtheJenkinsSwarmslavenode,it'senoughtorunthefollowingcommand:

$java-jarswarm-client.jar-master<jenkins_master_url>-username<jenkins_master_user>-password<jenkins_master_password>-namejenkins-swarm-slave-1

Bythetimeofwritingthisbook,therewasanopenbugthatclient-slave.jardidn'tworkviathesecuredHTTPSprotocol,soitwasnecessarytoaddthe-disableSslVerificationoptiontothecommandexecution.

Afterthesuccessfulexecution,weshouldnoticethatanewslaveappearedontheJenkinsmasteraspresentedonthescreenshot:

Now,whenwerunabuild,itwillbestartedonthisagent.

TheotherpossibilitytoaddtheJenkinsSwarmagentistousethe

Dockerimagebuiltfromtheswarm-client.jartool.ThereareafewofthemavailableontheDockerHub.Wecanusethecsanchez/jenkins-swarm-slaveimage.

UnderstandingJenkinsSwarmagentsJenkinsSwarmallowstodynamicallyaddagents,butitsaysnothingaboutwhethertousespecificorDocker-basedslaves,sowecanuseitforboth.Atfirstglance,JenkinsSwarmmaynotseemveryuseful.Afterall,wemovedsettingagentsfrommastertoslave,butstillhavetodoitmanually.However,aswewillseelaterinChapter8,ClusteringwithDockerSwarm,JenkinsSwarmenablesdynamicscalingofslavesonaclusterofservers.

DynamicallyprovisionedDockeragentsAnotheroptionistosetupJenkinstodynamicallycreateanewagenteachtimeabuildisstarted.Suchasolutionisobviouslythemostflexibleonesincethenumberofslavesdynamicallyadjusttothenumberofbuilds.Let'shavealookathowtoconfigureJenkinsthisway.

ConfiguringdynamicallyprovisionedDockeragentsWeneedtofirstinstalltheDockerplugin.AsalwayswithJenkinsplugins,wecandoinManageJenkinsandManagePlugins.Afterthepluginisinstalled,wecanstartthefollowingconfigurationsteps:

1. OpentheManageJenkinspage.2. ClickontheConfigureSystemlink.3. Atthebottomofthepage,thereistheCloudsection.4. ClickonAddanewcloudandchooseDocker.5. FilltheDockeragentdetails.

6. Mostparametersdonotneedtobechanged;however,weneedtosettwoofthemasfollows:

DockerURL:TheaddressoftheDockerhostmachinewhereagentswillberunCredentials:ThecredentialsincasetheDockerhostrequires

authentication

IfyouplantousethesameDockerhostwherethemasterisrunning,thentheDockerdaemonneedstolistenonthedocker0networkinterface.YoucandoitinasimilarwayasdescribedintheInstallingonaserversectionoftsof)slaves.Thatisthesameissuewehadwhilemaintainingmultipleproductionservertypes,asdescribedinChapter2,IntroducingDocker,bychangingonelineinthe/lib/systemd/system/docker.servicefiletoExecStart=/usr/bin/dockerd-H0.0.0.0:2375-Hfd://

7. ClickonAddDockerTemplateandselectDockerTemplate.8. FillthedetailsabouttheDockerslaveimage:

Wecanusethefollowingparameters:

DockerImage:ThemostpopularslaveimagefromtheJenkinscommunity

isevarga/jenkins-slaveCredentials:Thecredentialstotheevarga/jenkins-slaveimageare:

username:jenkinspassword:jenkins

Instancecapacity:Thisdefinesthemaximumnumberofagentsrunningatthesametime;forthebeginning,itcanbesetto10

Insteadofevarga/jenkins-slave,it'spossibletobuildanduseyourownslaveimages.Thisisnecessarywhentherearespecificenvironmentrequirements,forexample,thePythoninterpreterinstalled.Inallexamplesforthisbookweusedleszko/jenkins-docker-slave.

Aftersaving,everythingissetup.WecouldrunthepipelinetoobservethattheexecutionreallytakesplaceontheDockeragent,butfirstlet'sdigalittledeepertounderstandhowtheDockeragentswork.

UnderstandingdynamicallyprovisionedDockeragentsDynamicallyprovisionedDockeragentscanbetreatedasalayeroverthestandardagentmechanism.Itchangesneitherthecommunicationprotocolnorhowtheagentiscreated.So,whatdoesJenkinsdowiththeDockeragentconfigurationweprovided?

ThefollowingdiagrampresentstheDockermaster-slavearchitecturewe'veconfigured:

Let'sdescribestepbystephowtheDockeragentmechanismisused:

1. WhentheJenkinsjobisstarted,themasterrunsanewcontainerfromthejenkins-slaveimageontheslaveDockerhost.

2. Thejenkins-slavecontaineris,actually,theubuntuimagewiththeSSHDserverinstalled.

3. TheJenkinsmasterautomaticallyaddsthecreatedagenttotheagentlist(sameaswhatwedidmanuallyintheSettingagentssection).

4. TheagentisaccessedusingtheSSHcommunicationprotocoltoperformthebuild.

5. Afterthebuild,themasterstopsandremovestheslavecontainer.

RunningJenkinsmasterasaDockercontainerisindependentfrom

runningJenkinsagentsasDockercontainers.It'sreasonabletodoboth,butanyofthemwillworkseparately.

ThesolutionissomehowsimilartothepermanentDockeragentssolution,becauseinresult,werunthebuildinsideaDockercontainer.Thedifferenceis,however,intheslavenodeconfiguration.Here,thewholeslaveisdockerized,notonlythebuildenvironment.Therefore,ithastwogreatadvantagesasfollows:

Automaticagentlifecycle:Theprocessofcreating,adding,andremovingtheagentisautomated.Scalability:Actually,theslaveDockerhostcouldbenotasinglemachine,butaclustercomposedofmultiplemachines(we'llcoverclusteringusingDockerSwarminChapter8,ClusteringwithDockerSwarm).Inthatcase,addingmoreresourcesisassimpleasaddinganewmachinetotheclusteranddoesnotrequireanychangesinJenkins.

Jenkinsbuildusuallyneedstodownloadalotofprojectdependencies(forexample,Gradle/Mavendependencies),whichmaytakealotoftime.IfDockerslavesareautomaticallyprovisionedforeachbuild,thenitmaybeworthtosetupaDockervolumeforthemtoenablecachingbetweenthebuilds.

TestingagentsNomatterwhichagentconfigurationyouchose,weshouldnowcheckifitworkscorrectly.

Let'sgobacktothehelloworldpipeline.Usually,thebuildslastlongerthanthehello-worldexample,sowecansimulateitbyaddingsleepingtothepipelinescript:

pipeline{

agentany

stages{

stage("Hello"){

steps{

sleep300//5minutes

echo'HelloWorld'

}

}

}

}

AfterclickingonBuildNowandgoingtotheJenkinsmainpage,weshouldseethatthebuildisexecutedonanagent.Now,ifweclickonbuildmanytimes,thendifferentagentsshouldbeexecutingdifferentbuilds(asshowninthefollowingscreenshot):

Topreventjobexecutionsonmaster,rememberaboutturningthemasternodeofflineorsetting#ofexecutorsto0intheManageNodesconfiguration.

Havingseenthattheagentsareexecutingourbuilds,we'veconfirmedthattheyareconfiguredcorrectly.Now,let'sseehowandforwhatreasonwecouldcreateourownJenkinsimages.

CustomJenkinsimagesSofar,wehaveusedtheJenkinsimagespulledfromtheinternet.Weusedjenkinsforthemastercontainerandevarga/jenkins-slavefortheslavecontainer.However,wemaywanttobuildourownimagestosatisfythespecificbuildenvironmentrequirements.Inthissection,wecoverhowtodoit.

BuildingJenkinsslaveLet'sstartfromtheslaveimage,becauseit'smoreoftencustomized.Thebuildexecutionisperformedontheagent,soit'stheagentthatneedstohavetheenvironmentadjustedtotheprojectwewouldliketobuild.Forexample,itmayrequirethePythoninterpreterifourprojectiswritteninPython.Thesameisappliedtoanylibrary,tool,testingframework,oranythingthatisneededbytheproject.

Youcancheckwhatisalreadyinstalledinsidetheevarga/jenkins-slaveimagebylookingatitsDockerfileathttps://github.com/evarga/docker-images.

Therearethreestepstobuildandusethecustomimage:

1. CreateaDockerfile.2. Buildtheimage.3. Changetheagentconfigurationonmaster.

Asanexample,let'screateaslavethatservesthePythonproject.Wecanbuilditontopoftheevarga/jenkins-slaveimageforthesakeofsimplicity.Let'sdoitusingthefollowingthreesteps:

1. Dockerfile:Let'screateanewdirectoryinsidetheDockerfilewiththefollowingcontent:

FROMevarga/jenkins-slave

RUNapt-getupdate&&\

apt-getinstall-ypython

ThebaseDockerimageevarga/jenkins-slaveissuitableforthedynamicallyprovisionedDockeragentssolution.IncaseofpermanentDockeragents,it'senoughtousealpine,ubuntu,oranyotherimage,sinceit'snottheslavethatisdockerized,butonlythebuildexecutionenvironment.

2. Buildtheimage:Wecanbuildtheimagebyexecutingthefollowingcommand:

$dockerbuild-tjenkins-slave-python.

3. Configurethemaster:Thelaststep,ofcourse,istosetjenkins-slave-pythoninsteadofevarga/jenkins-slaveintheJenkinsmaster'sconfiguration(asdescribedintheSettingDockeragentsection).

TheslaveDockerfileshouldbekeptinthesourcecoderepositoryandtheimagebuildcanbeperformedautomaticallybyJenkins.ThereisnothingwrongwithbuildingthenewJenkinsslaveimageusingtheoldJenkinsslave.

WhatifweneedJenkinstobuildtwodifferentkindsofprojects,forexample,onebasedonPythonandanotherbasedonRuby?Inthatcase,wecouldprepareanagent,whichisgenericenoughtosupportboth:PythonandRuby.However,incaseofDocker,it'srecommendedtocreatethesecondslaveimage(jenkins-slave-rubybyanalogy).Then,intheJenkinsconfigurationweneedtocreatetwoDockertemplatesandlabelthemaccordingly.

BuildingJenkinsmasterWealreadyhaveacustomslaveimage.Whywouldwealsowanttobuildourownmasterimage?Oneofthereasonsmightbethatwedon'twanttouseslavesatall,and,sincetheexecutionwouldbedoneonmaster,itsenvironmenthastobeadjustedtotheproject'sneeds.Thatis,however,averyrarecase.Moreoften,wewillwanttoconfigurethemasteritself.

Imaginethefollowingscenario,yourorganizationscalesJenkinshorizontallyandeachteamhasitsowninstance.Thereis,however,somecommonconfiguration,forexample:asetofbaseplugins,backupstrategies,orthecompanylogo.Then,repeatingthesameconfigurationforeachoftheteamsisawasteoftime.So,wecanpreparethesharedmasterimageandlettheteamsuseit.

JenkinsisconfiguredusingXMLfilesanditprovidestheGroovy-basedDSLlanguagetomanipulateoverthem.ThatiswhywecanaddtheGroovyscripttotheDockerfileinordertomanipulatetheJenkinsconfiguration.Whatismore,therearespecialscriptstohelpwiththeJenkinsconfigurationifitrequiressomethingmorethanXMLchanges,forinstance,plugininstallation.

AllpossibilitiesoftheDockerfileinstructionsarewelldescribedontheGitHubpagehttps://github.com/jenkinsci/docker.

Asanexample,let'screateamasterimagewiththedocker-pluginalreadyinstalledandanumberofexecutorssetto5.Inordertodoit,weneedto:

1. CreatetheGroovyscripttomanipulateonconfig.xmlandsetthenumberofexecutorsto5.

2. CreatetheDockerfiletoinstalldocker-pluginandexecutetheGroovyscript.

3. Buildtheimage.

Let'susethethreestepsmentionedandbuildtheJenkinsmasterimage.

1. Groovyscript:Let'screateanewdirectoryinsidetheexecutors.groovyfilewiththefollowingcontent:

importjenkins.model.*

Jenkins.instance.setNumExecutors(5)

ThecompleteJenkinsAPIcanbefoundontheofficialpagehttp://javadoc.jenkins.io/.

2. Dockerfile:Inthesamedirectory,let'screatetheDockerfile:

FROMjenkins

COPYexecutors.groovy

/usr/share/jenkins/ref/init.groovy.d/executors.groovy

RUN/usr/local/bin/install-plugins.shdocker-plugin

3. Buildtheimage:Wecanfinallybuildtheimage:

$dockerbuild-tjenkins-master.

Aftertheimageiscreated,eachteamintheorganizationcanuseittolaunchtheirownJenkinsinstance.

Havingourownmasterandslaveimagesletsusprovidetheconfigurationandthebuildenvironmentfortheteamsinourorganization.Inthenextsection,we'llseewhatelseisworthbeingconfiguredinJenkins.

ConfigurationandmanagementWehavealreadycoveredthemostcrucialpartoftheJenkinsconfiguration:agentsprovisioning.SinceJenkinsishighlyconfigurable,youcanexpectmuchmorepossibilitiestoadjustittoyourneeds.Thegoodnewsisthattheconfigurationisintuitiveandaccessibleviathewebinterface,soitdoesnotrequireanydetaileddescription.EverythingcanbechangedundertheManageJenkinssubpage.Inthissection,wewillfocusonlyonafewaspectsthataremostlikelytobechanged:plugins,security,andbackup.

PluginsJenkinsishighlyplugin-oriented,whichmeansthatalotoffeaturesaredeliveredbytheuseofplugins.TheycanextendJenkinsalmostintheunlimitedway,which,takingintoconsiderationthelargecommunity,isoneofthereasonswhyJenkinsissuchasuccessfultool.WithJenkins'openness,comestheriskandit'sbettertodownloadpluginsonlyfromthereliablesourceorchecktheirsourcecode.

Thereareliterallytonsofpluginstochoosefrom.Someofthemwerealreadyinstalledautomaticallyduringtheinitialconfiguration.Anotherone(Dockerplugin)wasinstalledwhilesettingtheDockeragents.Therearepluginsforcloudintegration,sourcecontroltools,codecoverage,andmuchmore.Youcanalsowriteyourownplugin,butit'sbettertocheckfirstiftheoneyouneedisnotalreadywritten.

ThereisanofficialJenkinspagetobrowsepluginsfromhttps://plugins.jenkins.io/.

SecurityThewayyoushouldapproachtheJenkinssecuritydependsontheJenkinsarchitectureyouhavechosenwithinyourorganization.IfyouhaveaJenkinsmasterforeverysmallteam,thenyoumaynotneeditatall(undertheassumptionthatthecorporatenetworkisfirewalled).However,ifyouhaveasingleJenkinsmasterinstanceforthewholeorganization,thenyou'dbetterbesureyousecureditwell.

Jenkinscomeswithitsownuserdatabase-wehavealreadycreatedauserduringtheinitialconfigurationprocess.Youcancreate,delete,andmodifyusersbyopeningtheManageUserssettingpage.Thebuilt-indatabasecanbeasolutionincaseofsmallorganizations;however,forthelargegroupofusers,youwillprobablywanttouseLDAPinstead.YoucanchooseitontheConfigureGlobalSecuritypage.There,youcanalsoassignroles,groups,andusers.Bydefault,theLogged-inuserscandoanythingoptionisset,butinalarge-scaleorganizationyoushouldprobablythinkofmoredetailedgranularity.

BackupAstheoldsayinggoes:"Therearetwotypesofpeople:thosewhobackup,andthosewhowillbackup".Believeitornot,thebackupissomethingyouprobablywanttoconfigure.Whatfilestobackupandfromwhichmachines?Luckily,agentsautomaticallysendalltherelevantdatabacktothemaster,sowedon'tneedtobotheraboutthem.IfyourunJenkinsinthecontainer,thenthecontaineritselfisalsonotinteresting,sinceitdoesnotholdanypersistentstate.TheonlyplaceweareinterestedinistheJenkinshomedirectory.

WecaneitherinstallaJenkinsplugin(whichwillhelpustosetperiodicbackups)orsimplysetacronjobtoarchivethedirectoryintoasafeplace.Toreducethesize,wecanexcludethesubfolderswhicharenotinteresting(thatwilldependonyourneeds;however,almostcertainly,youdon'tneedtocopy:"war","cache","tools",and"workspace").

Therearequiteafewplugins,whichcanhelpwiththebackupprocess;themostcommononeiscalledBackupPlugin.

BlueOceanUIThefirstversionofHudson(formerJenkins)wasreleasedin2005.It'sbeenonthemarketformorethan10yearsnow.However,itslookandfeelhasn'tchangedmuch.We'vealreadyuseditforawhileandit'shardtodenythatitlooksoutdated.BlueOceanistheplugin,whichredefinestheuserexperienceofJenkins.IfJenkinsisaestheticallydispleasingtoyou,thenit'sdefinitelyworthgivingatry.

YoucanreadmoreontheBlueOceanpageathttps://jenkins.io/projects/blueocean/.

Exercises

WehavelearnedalotaboutJenkinsconfigurationthroughoutthischapter.Toconsolidatetheknowledge,werecommendtwoexercisesonpreparingtheJenkinsimagesandtestingtheJenkinsenvironment.

1. CreateJenkinsmasterandslaveDockerimagesandusethemtoruntheJenkinsinfrastructurecapableofbuildingtheRubyprojects:

CreatethemasterDockerfile,whichautomaticallyinstallstheDockerplugin.BuildthemasterimageandruntheJenkinsinstanceCreatetheslaveDockerfile(suitableforthedynamicslaveprovisioning),whichinstallstheRubyinterpreterBuildtheslaveimageChangetheconfigurationintheJenkinsinstancetousetheslaveimage

2. Createapipeline,whichrunsaRubyscriptprintingHelloWorldfromRuby:

CreateanewpipelineUsethefollowingshellcommandtocreatethehello.rbscriptonthefly:sh"echo"puts'HelloWorldfromRuby'">hello.rb"

Addthecommandtorunhello.rbusingtheRubyinterpreterRunthebuildandobservetheconsoleoutput

SummaryInthischapter,wehavecoveredtheJenkinsenvironmentanditsconfiguration.TheknowledgegainedissufficienttosetupthecompleteDocker-basedJenkinsinfrastructure.Thekeytakeawayfromthechapterisasfollows:

Jenkinsisageneral-purposeautomationtoolthatcanbeusedwithanylanguageorframework.Jenkinsishighlyextensiblebyplugins,whichcanbewrittenorfoundontheinternet.JenkinsiswritteninJava,soitcanbeinstalledonanyoperatingsystem.It'salsoofficiallydeliveredasaDockerimage.Jenkinscanbescaledusingthemaster-slavearchitecture.Themasterinstancescanbescaledhorizontallyorverticallydependingontheorganization'sneeds.Jenkins'agentscanbeimplementedwiththeuseofDocker,whichhelpsinautomaticconfigurationanddynamicslavesallocation.CustomDockerimagescanbecreatedforboth:JenkinsmasterandJenkinsslave.Jenkinsishighlyconfigurableandtheaspectsthatshouldalwaysbeconsideredare:securityandbackups.

Inthenextchapter,wewillfocusonthepartthatwe'vealreadytouchedwiththe"helloworld"example,pipelines.WewilldescribetheideaandthemethodtobuildacompleteContinuousIntegrationpipeline.

ContinuousIntegrationPipeline

WealreadyknowhowtoconfigureJenkins.Inthischapter,youwillseehowtouseiteffectively,focusingonthefeaturethatlaysattheheartofJenkins,pipelines.BybuildingacompleteContinuousIntegrationprocessfromscratch,wewilldescribeallaspectsofmodernteam-orientedcodedevelopment.

Thischaptercoversthefollowingpoints:

ExplainingtheideaofpipeliningIntroducingtheJenkinspipelinesyntaxCreatingaContinuousIntegrationpipelineExplainingtheideaofJenkinsfileCreatingcodequalitychecksAddingpipelinetriggersandnotificationsExplainingdevelopmentworkflowsandbranchingstrategiesIntroducingJenkinsMultibranch

IntroducingpipelinesApipelineisasequenceofautomatedoperationsthatusuallyrepresentsapartofsoftwaredeliveryandthequalityassuranceprocess.Itcanbesimplyseenasachainofscriptsprovidingthefollowingadditionalbenefits:

Operationgrouping:Operationsaregroupedtogetherintostages(alsoknownasgatesorqualitygates)thatintroduceastructureintotheprocessandclearlydefinestherule:ifonestagefails,nofurtherstagesareexecutedVisibility:Allaspectsoftheprocessarevisualized,whichhelpinquickfailureanalysisandpromotesteamcollaborationFeedback:Teammemberslearnaboutanyproblemsassoonastheyoccur,sotheycanreactquickly

TheconceptofpipeliningissimilarformostContinuousIntegrationtools,however,thenamingcandiffer.Inthisbook,westicktotheJenkinsterminology.

PipelinestructureAJenkinspipelineconsistsoftwokindsofelements:stagesandsteps.Thefollowingfigureshowshowtheyareused:

Thefollowingarethebasicpipelineelements:

Step:Asingleoperation(tellsJenkinswhattodo,forexample,checkoutcodefromrepository,executeascript)Stage:Alogicalseparationofsteps(groupsconceptuallydistinctsequencesofsteps,forexample,Build,Test,andDeploy)usedtovisualizetheJenkinspipelineprogress

Technically,it'spossibletocreateparallelsteps;however,it'sbettertotreatitasanexceptionwhenreallyneededforoptimizationpurposes.

Multi-stageHelloWorldAsanexample,let'sextendtheHelloWorldpipelinetocontaintwostages:

pipeline{

agentany

stages{

stage('FirstStage'){

steps{

echo'Step1.HelloWorld'

}

}

stage('SecondStage'){

steps{

echo'Step2.SecondtimeHello'

echo'Step3.ThirdtimeHello'

}

}

}

}

Thepipelinehasnospecialrequirementsintermsofenvironment(anyslaveagent),anditexecutesthreestepsinsidetwostages.WhenweclickonBuildNow,weshouldseethevisualrepresentation:

Thepipelinesucceeded,andwecanseethestepexecutiondetailsbyclickingontheconsole.Ifanyofthestepsfailed,theprocessingwouldstopandnofurtherstepswouldrun.Actually,theentirereasonforapipelineistopreventallfurtherstepsfromexecutionandvisualizethepointoffailure.

PipelinesyntaxWehavediscussedthepipelineelementsandalreadyusedafewofthepipelinesteps,forexample,echo.Whatotheroperationscanweuseinsidethepipelinedefinition?

Inthisbook,weusethedeclarativesyntaxthatisrecommendedforallnewprojects.ThedifferentoptionsareGroovy-basedDSLand(priortoJenkins2)XML(createdviathewebinterface).

Thedeclarativesyntaxwasdesignedtomakeitassimpleaspossibletounderstandthepipeline,evenbythepeoplewhodonotwritecodeonadailybasis.Thisiswhythesyntaxislimitedonlytothemostimportantkeywords.

Let'sprepareanexperimentand,beforewedescribeallthedetails,readthefollowingpipelinedefinitionandtrytoguesswhatitdoes:

pipeline{

agentany

triggers{cron('*****')}

options{timeout(time:5)}

parameters{

booleanParam(name:'DEBUG_BUILD',defaultValue:true,

description:'Isitthedebugbuild?')

}

stages{

stage('Example'){

environment{NAME='Rafal'}

when{expression{returnparams.DEBUG_BUILD}}

steps{

echo"Hellofrom$NAME"

script{

defbrowsers=['chrome','firefox']

for(inti=0;i<browsers.size();++i){

echo"Testingthe${browsers[i]}browser."

}

}

}

}

}

post{always{echo'IwillalwayssayHelloagain!'}}

}

Hopefully,thepipelinedidn'tscareyou.Itisquitecomplex.Actually,itissocomplexthatitcontainsallpossibleJenkinsinstructions.Toanswertheexperimentpuzzle,let'sseewhatthepipelinedoesinstructionbyinstruction:

1. Useanyavailableagent.2. Executeautomaticallyeveryminute.3. Stopiftheexecutiontakesmorethan5minutes.4. AskfortheBooleaninputparameterbeforestarting.5. SetRafalastheenvironmentvariableNAME.6. Onlyinthecaseofthetrueinputparameter:

PrintHellofromRafalPrintTestingthechromebrowserPrintTestingthefirefoxbrowser

7. PrintIwillalwayssayHelloagain!nomatterifthereareanyerrorsduringtheexecution.

Let'sdescribethemostimportantJenkinskeywords.Adeclarativepipelineisalwaysspecifiedinsidethepipelineblockandcontainssections,directives,andsteps.Wewillwalkthrougheachofthem.

ThecompletepipelinesyntaxdescriptioncanbefoundontheofficialJenkinspageathttps://jenkins.io/doc/book/pipeline/syntax/.

SectionsSectionsdefinethepipelinestructureandusuallycontainoneormoredirectivesorsteps.Theyaredefinedwiththefollowingkeywords:

Stages:ThisdefinesaseriesofoneormorestagedirectivesSteps:ThisdefinesaseriesofoneormorestepinstructionsPost:Thisdefinesaseriesofoneormorestepinstructionsthatarerunattheendofthepipelinebuild;markedwithacondition(forexample,always,success,orfailure),usuallyusedtosendnotificationsafterthepipelinebuild(wewillcoverthisindetailintheTriggersandnotificationssection.)

DirectivesDirectivesexpresstheconfigurationofapipelineoritsparts:

Agent:ThisspecifieswheretheexecutiontakesplaceandcandefinethelabeltomatchtheequallylabeledagentsordockertospecifyacontainerthatisdynamicallyprovisionedtoprovideanenvironmentforthepipelineexecutionTriggers:Thisdefinesautomatedwaystotriggerthepipelineandcanusecrontosetthetime-basedschedulingorpollScmtochecktherepositoryforchanges(wewillcoverthisindetailintheTriggersandnotificationssection)Options:Thisspecifiespipeline-specificoptions,forexample,timeout(maximumtimeofpipelinerun)orretry(numberoftimesthepipelineshouldbererunafterfailure)Environment:ThisdefinesasetofkeyvaluesusedasenvironmentvariablesduringthebuildParameters:Thisdefinesalistofuser-inputparametersStage:ThisallowsforlogicalgroupingofstepsWhen:Thisdetermineswhetherthestageshouldbeexecuteddependingonthegivencondition

StepsStepsarethemostfundamentalpartofthepipeline.Theydefinetheoperationsthatareexecuted,sotheyactuallytellJenkinswhattodo.

sh:Thisexecutestheshellcommand;actually,it'spossibletodefinealmostanyoperationusingshcustom:Jenkinsoffersalotofoperationsthatcanbeusedassteps(forexample,echo);manyofthemaresimplywrappersovertheshcommandusedforconvenience;pluginscanalsodefinetheirownoperationsscript:ThisexecutesablockoftheGroovy-basedcodethatcanbeusedforsomenon-trivialscenarios,whereflowcontrolisneeded

Thecompletespecificationoftheavailablestepscanbefoundat:https://jenkins.io/doc/pipeline/steps/.

Noticethatthepipelinesyntaxisverygenericandtechnically,canbeusedforalmostanyautomationprocess.Thisiswhythepipelineshouldbetreatedasamethodofstructurizationandvisualization.Themostcommonusecaseis,however,implementingtheContinuousIntegrationserverthatwewilllookatinthefollowingsection.

CommitpipelineThemostbasicContinuousIntegrationprocessiscalledacommitpipeline.Thisclassicphase,asitsnamesays,startswithacommit(orpushinGit)tothemainrepositoryandresultsinareportaboutthebuildsuccessorfailure.Sinceitrunsaftereachchangeinthecode,thebuildshouldtakenomorethan5minutesandshouldconsumeareasonableamountofresources.ThecommitphaseisalwaysthestartingpointoftheContinuousDeliveryprocess,anditprovidesthemostimportantfeedbackcycleinthedevelopmentprocess,constantinformationifthecodeisinahealthystate.

Thecommitphaseworksasfollows.Adeveloperchecksinthecodetotherepository,theContinuousIntegrationserverdetectsthechange,andthebuildstarts.Themostfundamentalcommitpipelinecontainsthreestages:

Checkout:ThisstagedownloadsthesourcecodefromtherepositoryCompile:ThisstagecompilesthesourcecodeUnittest:Thisstagerunsasuiteofunittests

Let'screateasampleprojectandseehowtoimplementthecommitpipeline.

ThisisanexampleofapipelinefortheprojectthatusestechnologiessuchasGit,Java,Gradle,andSpringBoot.Nevertheless,thesameprinciplesapplytoanyothertechnology.

CheckoutCheckingoutcodefromtherepositoryisalwaysthefirstoperationinanypipeline.Inordertoseethis,weneedtohavearepository.Then,wewillbeabletocreateapipeline.

CreatingaGitHubrepositoryCreatingarepositoryontheGitHubservertakesjustafewsteps:

1. Gotothehttps://github.com/page.2. Createanaccountifyoudon'thaveoneyet.3. ClickonNewrepository.4. Giveitaname,calculator.5. TickInitializethisrepositorywithaREADME.6. ClickonCreaterepository.

Now,youshouldseetheaddressoftherepository,forexample,https://github.com/leszko/calculator.git.

CreatingacheckoutstageWecancreateanewpipelinecalledcalculatorand,asPipelinescript,putthecodewithastagecalledCheckout:

pipeline{

agentany

stages{

stage("Checkout"){

steps{

giturl:'https://github.com/leszko/calculator.git'

}

}

}

}

Thepipelinecanbeexecutedonanyoftheagents,anditsonlystepdoesnothingmorethandownloadingcodefromtherepository.WecanclickonBuildNowandseeifitwasexecutedsuccessfully.

NotethattheGittoolkitneedstobeinstalledonthenodewherethebuildisexecuted.

Whenwehavethecheckout,we'rereadyforthesecondstage.

CompileInordertocompileaproject,weneedto:

1. Createaprojectwiththesourcecode.2. Pushittotherepository.3. AddtheCompilestagetothepipeline.

CreatingaJavaSpringBootprojectLet'screateaverysimpleJavaprojectusingtheSpringBootframeworkbuiltbyGradle.

SpringBootisaJavaframeworkthatsimplifiesbuildingenterpriseapplications.GradleisabuildautomationsystemthatisbasedontheconceptsofApacheMaven.

ThesimplestwaytocreateaSpringBootprojectistoperformthefollowingsteps:

1. Gotothehttp://start.spring.io/page.2. SelectGradleprojectinsteadofMavenproject(youcanalsoleaveMavenif

youpreferittoGradle).3. FillGroupandArtifact(forexample,com.leszkoandcalculator).4. AddWebtoDependencies.5. ClickonGenerateProject.6. Thegeneratedskeletonprojectshouldbedownloaded(thecalculator.zip

file).

Thefollowingscreenshotpresentsthehttp://start.spring.io/page:

PushingcodetoGitHubWewillusetheGittooltoperformthecommitandpushoperations:Inordertorunthegitcommand,youneedtohavetheGittoolkitinstalled(itcanbedownloadedfromhttps://git-scm.com/downloads).

Let'sfirstclonetherepositorytothefilesystem:

$gitclonehttps://github.com/leszko/calculator.git

Extracttheprojectdownloadedfromhttp://start.spring.io/intothedirectorycreatedbyGit.

Ifyouprefer,youcanimporttheprojectintoIntelliJ,Eclipse,oryourfavoriteIDEtool.

Asaresult,thecalculatordirectoryshouldhavethefollowingfiles:$ls-a...build.gradle.git.gitignoregradlegradlewgradlew.batREADME.mdsrc

InordertoperformtheGradleoperationslocally,youneedtohaveJavaJDKinstalled(inUbuntu,youcandoitbyexecutingsudoapt-getinstall-ydefault-jdk).

Wecancompiletheprojectlocallyusingthefollowingcode:

$./gradlewcompileJava

InthecaseofMaven,youcanrun./mvnwcompile.BothGradleandMavencompiletheJavaclasseslocatedinthesrcdirectory.

YoucanfindallpossibleGradleinstructions(fortheJavaproject)athttps://docs.gradle.org/current/userguide/java_plugin.html.

Now,wecancommitandpushtotheGitHubrepository:$gitadd.$gitcommit-m"AddSpringBootskeleton"

$gitpush-uoriginmaster

Afterrunningthegitpushcommand,youwillbepromptedtoentertheGitHubcredentials(usernameandpassword).

ThecodeisalreadyintheGitHubrepository.Ifyouwanttocheckit,youcangototheGitHubpageandseethefiles.

stage("Compile"){<br/>steps{<br/>sh"./gradlewcompileJava"<br/>}<br/>}

NotethatweusedexactlythesamecommandlocallyandintheJenkinspipeline,whichisaverygoodsignbecausethelocaldevelopmentprocessisconsistentwiththeContinuousIntegrationenvironment.Afterrunningthebuild,youshouldseetwogreenboxes.Youcanalsocheckthattheprojectwascompiledcorrectlyintheconsolelog.

UnittestIt'stimetoaddthelaststagethatisUnittest,whichchecksifourcodedoeswhatweexpectittodo.Wehaveto:

AddthesourcecodeforthecalculatorlogicWriteunittestforthecodeAddastagetoexecutetheunittest

CreatingbusinesslogicThefirstversionofthecalculatorwillbeabletoaddtwonumbers.Let'saddthebusinesslogicasaclassinthesrc/main/java/com/leszko/calculator/Calculator.javafile:packagecom.leszko.calculator;importorg.springframework.stereotype.Service;

@ServicepublicclassCalculator{intsum(inta,intb){returna+b;}}

Toexecutethebusinesslogic,wealsoneedtoaddthewebservicecontrollerinaseparatefilesrc/main/java/com/leszko/calculator/CalculatorController.java:

packagecom.leszko.calculator;

importorg.springframework.beans.factory.annotation.Autowired;

importorg.springframework.web.bind.annotation.RequestMapping;

importorg.springframework.web.bind.annotation.RequestParam;

importorg.springframework.web.bind.annotation.RestController;

@RestController

classCalculatorController{

@Autowired

privateCalculatorcalculator;

@RequestMapping("/sum")

Stringsum(@RequestParam("a")Integera,

@RequestParam("b")Integerb){

returnString.valueOf(calculator.sum(a,b));

}

}

Thisclassexposesthebusinesslogicasawebservice.Wecanruntheapplicationandseehowitworks:

$./gradlewbootRun

Itshouldstartourwebserviceandwecancheckthatitworksbynavigatingtothebrowserandopeningthepagehttp://localhost:8080/sum?a=1&b=2.Thisshouldsumtwonumbers(1and2)andshow3inthebrowser.

WritingaunittestWealreadyhavetheworkingapplication.Howcanweensurethatthelogicworksasexpected?Wehavetrieditonce,butinordertoknowconstantly,weneedaunittest.Inourcase,itwillbetrivial,maybeevenunnecessary;however,inrealprojects,unittestscansavefrombugsandsystemfailures.

Let'screateaunittestinthefilesrc/test/java/com/leszko/calculator/CalculatorTest.java:

packagecom.leszko.calculator;

importorg.junit.Test;

importstaticorg.junit.Assert.assertEquals;

publicclassCalculatorTest{

privateCalculatorcalculator=newCalculator();

@Test

publicvoidtestSum(){

assertEquals(5,calculator.sum(2,3));

}

}

Wecanrunthetestlocallyusingthe./gradlewtestcommand.Then,let'scommitthecodeandpushittotherepository:

$gitadd.

$gitcommit-m"Addsumlogic,controllerandunittest"

$gitpush

CreatingaunitteststageNow,wecanaddaUnitteststagetothepipeline:stage("Unittest"){steps{sh"./gradlewtest"}}

InthecaseofMaven,wewouldhavetouse./mvnwtest.

Whenwebuildthepipelineagain,weshouldseethreeboxes,whichmeansthatwe'vecompletedtheContinuousIntegrationpipeline:

JenkinsfileAllthetime,sofar,wecreatedthepipelinecodedirectlyinJenkins.Thisis,however,nottheonlyoption.WecanalsoputthepipelinedefinitioninsideafilecalledJenkinsfileandcommitittotherepositorytogetherwiththesourcecode.Thismethodisevenmoreconsistentbecausethewayyourpipelinelooksisstrictlyrelatedtotheprojectitself.

Forexample,ifyoudon'tneedthecodecompilationbecauseyourprogramminglanguageisinterpreted(andnotcompiled),thenyouwon'thavetheCompilestage.Thetoolsyouusealsodifferdependingontheenvironment.WeusedGradle/Mavenbecausewe'vebuilttheJavaproject;however,inthecaseofaprojectwritteninPython,youcouldusePyBuilder.Itleadstotheideathatthepipelinesshouldbecreatedbythesamepeoplewhowritethecode,developers.Also,thepipelinedefinitionshouldbeputtogetherwiththecode,intherepository.

Thisapproachbringsimmediatebenefits,asfollows:

IncaseofJenkins'failure,thepipelinedefinitionisnotlost(becauseit'sstoredinthecoderepository,notinJenkins)ThehistoryofthepipelinechangesisstoredPipelinechangesgothroughthestandardcodedevelopmentprocess(forexample,theyaresubjectedtocodereviews)Accesstothepipelinechangesisrestrictedexactlyinthesamewayastheaccesstothesourcecode

CreatingJenkinsfileWecancreatetheJenkinsfileandpushittoourGitHubrepository.Itscontentisalmostthesameasthecommitpipelinewewrote.TheonlydifferenceisthatthecheckoutstagebecomesredundantbecauseJenkinshastocheckoutthecode(togetherwithJenkinsfile)firstandthenreadthepipelinestructure(fromJenkinsfile).ThisiswhyJenkinsneedstoknowtherepositoryaddressbeforeitreadsJenkinsfile.

Let'screateafilecalledJenkinsfileintherootdirectoryofourproject:pipeline{agentanystages{stage("Compile"){steps{sh"./gradlewcompileJava"}}stage("Unittest"){steps{sh"./gradlewtest"}}}}

WecannowcommittheaddedfilesandpushtotheGitHubrepository:$gitadd.$gitcommit-m"AddsumJenkinsfile"$gitpush

RunningpipelinefromJenkinsfileWhenJenkinsfileisintherepository,thenallwehavetodoistoopenthepipelineconfigurationandinthePipelinesection:

ChangeDefinitionfromPipelinescripttoPipelinescriptfromSCMSelectGitinSCMPuthttps://github.com/leszko/calculator.gitinRepositoryURL

Aftersaving,thebuildwillalwaysrunfromthecurrentversionofJenkinsfileintotherepository.

Wehavesuccessfullycreatedthefirstcompletecommitpipeline.Itcanbetreatedasaminimumviableproduct,andactually,inmanycases,it'ssufficientastheContinuousIntegrationprocess.Inthenextsections,wewillseewhat

improvementscanbedonetomakethecommitpipelineevenbetter.

CodequalitystagesWecanextendtheclassicthreestepsofContinuousIntegrationwithadditionalsteps.Themostwidelyusedarecodecoverageandstaticanalysis.Let'slookateachofthem.

CodecoverageThinkaboutthefollowingscenario:youhaveawell-configuredContinuousIntegrationprocess;however,nobodyinyourprojectwritesunittests.Itpassesallthebuilds,butitdoesn'tmeanthatthecodeisworkingasexpected.Whattodothen?Howtoensurethatthecodeistested?

Thesolutionistoaddthecodecoveragetoolthatrunsalltestsandverifieswhichpartsofthecodehavebeenexecuted.Then,itcreatesareportshowingnot-testedsections.Moreover,wecanmakethebuildfailwhenthereistoomuchuntestedcode.

Therearealotoftoolsavailabletoperformthetestcoverageanalysis;forJava,themostpopularareJaCoCo,Clover,andCobertura.

Let'suseJaCoCoandshowhowthecoveragecheckworksinpractice.Inordertodothis,weneedtoperformthefollowingsteps:

1. AddJaCoCototheGradleconfiguration.2. Addthecodecoveragestagetothepipeline.3. Optionally,publishJaCoCoreportsinJenkins.

AddingJaCoCotoGradleInordertorunJaCoCofromGradle,weneedtoaddthejacocoplugintothebuild.gradlefilebyaddingthefollowinglineinthepluginsection:

applyplugin:"jacoco"

Next,ifwewouldliketomaketheGradlefailincaseoftoolowcodecoverage,wecanaddthefollowingconfigurationtothebuild.gradlefileaswell:

jacocoTestCoverageVerification{

violationRules{

rule{

limit{

minimum=0.2

}

}

}

}

Thisconfigurationsetstheminimumcodecoverageto20%.Wecanrunitwiththefollowingcommand:

$./gradlewtestjacocoTestCoverageVerification

Thecommandchecksifthecodecoverageisatleast20%.Youcanplaywiththeminimumvaluetoseethelevelatwhichthebuildfails.Wecanalsogenerateatestcoveragereportusingthefollowingcommand:$./gradlewtestjacocoTestReport

Youcanalsohavealookatthefullcoveragereportinthebuild/reports/jacoco/test/html/index.htmlfile:

stage("Codecoverage"){<br/>steps{<br/>sh"./gradlewjacocoTestReport"<br/>sh"./gradlewjacocoTestCoverageVerification"<br/>}<br/>}

Afteraddingthisstage,ifanyonecommitscodethatisnotwell-coveredwithtests,thebuildwillfail.

PublishingthecodecoveragereportWhenthecoverageislowandthepipelinefails,itwouldbeusefultolookatthecodecoveragereportandfindwhatpartsarenotyetcoveredwithtests.WecouldrunGradlelocallyandgeneratethecoveragereport;however,itismoreconvenientifJenkinsshowsthereportforus.

InordertopublishthecodecoveragereportinJenkins,weneedthefollowingstagedefinition:

stage("Codecoverage"){

steps{

sh"./gradlewjacocoTestReport"

publishHTML(target:[

reportDir:'build/reports/jacoco/test/html',

reportFiles:'index.html',

reportName:"JaCoCoReport"

])

sh"./gradlewjacocoTestCoverageVerification"

}

}

ThisstagecopiesthegeneratedJaCoCoreporttotheJenkinsoutput.Whenwerunthebuildagain,weshouldseealinktothecodecoveragereports(inthemenuontheleftside,below"BuildNow").

ToperformthepublishHTMLstep,youneedtohavetheHTMLPublisherplugininstalledinJenkins.Youcanreadmoreaboutthepluginathttps://jenkins.io/doc/pipeline/steps/htmlpublisher/#publishhtml-publish-html-reports.

Wehavecreatedthecodecoveragestage,whichshowsthecodethatisnottestedandthereforevulnerabletobugs.Let'sseewhatelsecanbedoneinordertoimprovethecodequality.

Ifyouneedcodecoveragethatismorestrict,youcanchecktheconceptofmutationtestingandaddthePITframeworkstagetothepipeline.Readmoreathttp://pitest.org/.

StaticcodeanalysisYourcodemayworkperfectlyfine,however,whataboutthequalityofthecodeitself?Howdoweensureitismaintainableandwritteninagoodstyle?

Staticcodeanalysisisanautomaticprocessofcheckingthecodewithoutactuallyexecutingit.Inmostcases,itimpliescheckinganumberofrulesonthesourcecode.Theserulesmayapplytoawiderangeofaspects;forexample,allpublicclassesneedtohaveaJavadoccomment;themaximumlengthofalineis120characters,orifaclassdefinestheequals()method,ithastodefinethehashCode()methodaswell.

ThemostpopulartoolstoperformthestaticanalysisontheJavacodeareCheckstyle,FindBugs,andPMD.Let'slookatanexampleandaddthestaticcodeanalysisstageusingCheckstyle.Wewilldothisinthreesteps:

1. AddtheCheckstyleconfiguration.2. AddtheCheckstylestage.3. Optionally,publishtheCheckstylereportinJenkins.

AddingtheCheckstyleconfigurationInordertoaddtheCheckstyleconfiguration,weneedtodefinetherulesagainstwhichthecodeischecked.Wecandothisbyspecifyingtheconfig/checkstyle/checkstyle.xmlfile:

<?xmlversion="1.0"?>

<!DOCTYPEmodulePUBLIC

"-//PuppyCrawl//DTDCheckConfiguration1.2//EN"

"http://www.puppycrawl.com/dtds/configuration_1_2.dtd">

<modulename="Checker">

<modulename="TreeWalker">

<modulename="JavadocType">

<propertyname="scope"value="public"/>

</module>

</module>

</module>

Theconfigurationcontainsonlyonerule:Checkingifpublicclasses,interfaces,andenumsaredocumentedwithJavadoc.Iftheyarenot,thebuildfails.

ThecompleteCheckstyledescriptioncanbefoundathttp://checkstyle.sourceforge.net/config.html.

Wealsoneedtoaddthecheckstyleplugintothebuild.gradlefile:

applyplugin:'checkstyle'

Then,wecanrunthecheckstylewiththefollowingcode:

$./gradlewcheckstyleMain

Inthecaseofourproject,itshouldresultinafailurebecausenoneofourpublicclasses(Calculator.java,CalculatorApplication.java,CalculatorTest.java,CalculatorApplicationTests.java)hasaJavadoccomment.Weneedtofixitbyaddingthedocumentation,forexample,incaseofthesrc/main/java/com/leszko/calculator/CalculatorApplication.javafile:

/**

*MainSpringApplication.

*/

@SpringBootApplication

publicclassCalculatorApplication{

publicstaticvoidmain(String[]args){

SpringApplication.run(CalculatorApplication.class,args);

}

}

Now,thebuildshouldbesuccessful.

AddingastaticcodeanalysisstageWecanaddaStaticcodeanalysisstagetothepipeline:

stage("Staticcodeanalysis"){

steps{

sh"./gradlewcheckstyleMain"

}

}

Now,ifanyonecommitsafilewithapublicclasswithoutJavadoc,thebuildwillfail.

PublishingstaticcodeanalysisreportsVerysimilartoJaCoCo,wecanaddtheCheckstylereporttoJenkins:

publishHTML(target:[

reportDir:'build/reports/checkstyle/',

reportFiles:'main.html',

reportName:"CheckstyleReport"

])

ItgeneratesalinktotheCheckstylereport.

Wehaveaddedthestaticcodeanalysisstagethatcanhelpinfindingbugsandinstandardizingthecodestyleinsidetheteamororganization.

SonarQubeSonarQubeisthemostwidespreadsourcecodequalitymanagementtool.Itsupportsmultipleprogramminglanguagesandcanbetreatedasanalternativetothecodecoverageandstaticcodeanalysisstepswelookedat.Actually,itisaseparateserverthataggregatesdifferentcodeanalysisframeworks,suchasCheckstyle,FindBugs,andJaCoCo.IthasitsowndashboardsandintegrateswellwithJenkins.

Insteadofaddingcodequalitystepstothepipeline,wecaninstallSonarQube,addpluginsthere,andadda"sonar"stagetothepipeline.TheadvantageofthissolutionisthatSonarQubeprovidesauser-friendlywebinterfacetoconfigurerulesandshowcodevulnerabilities.

YoucanreadmoreaboutSonarQubeonitsofficialpagehttps://www.sonarqube.org/.

TriggersandnotificationsSofar,wehavealwaysbuiltthepipelinemanuallybyclickingontheBuildNowbutton.Itworksbutisnotveryconvenient.Allteammemberswouldhavetorememberthataftercommittingtotherepository,theyneedtoopenJenkinsandstartthebuild.Thesameworkswithpipelinemonitoring;sofar,wemanuallyopenedJenkinsandcheckedthebuildstatus.Inthissection,wewillseehowtoimprovetheprocesssothatthepipelinewouldstartautomaticallyand,whencompleted,notifytheteammembersaboutitsstatus.

TriggersAnautomaticactiontostartthebuildiscalledthepipelinetrigger.InJenkins,therearemanyoptionstochoosefrom;however,theyallboildowntothreetypes:

ExternalPollingSCM(SourceControlManagement)Scheduledbuild

Let'stakealookateachofthem.

ExternalExternaltriggersarenaturaltounderstand.TheymeanthatJenkinsstartsthebuildafterit'scalledbythenotifier,whichcanbetheotherpipelinebuild,theSCMsystem(forexample,GitHub),oranyremotescript.

Thefollowingfigurepresentsthecommunication:

GitHubtriggersJenkinsafterapushtotherepositoryandthebuildisstarted.

Toconfigurethesystemthisway,weneedthefollowingsetupsteps:

1. InstalltheGitHubplugininJenkins.2. GenerateasecretkeyforJenkins.3. SettheGitHubwebhookandspecifytheJenkinsaddressandkey.

InthecaseofthemostpopularSCMproviders,dedicatedJenkinspluginsarealwaysprovided.

ThereisalsoamoregenericwaytotriggerJenkinsviatheRESTcalltotheendpoint<jenkins_url>/job/<job_name>/build?token=<token>.Forsecurityreasons,itrequiressettingtokeninJenkinsandthenusingitintheremotescript.

JenkinsmustbeaccessiblefromtheSCMserver.Inotherwords,ifweusethepublicGitHubtotriggerJenkins,thenourJenkinsservermustbepublicaswell.Thisalsoappliestothegenericsolution;the<jenkins_url>addressmustbeaccessible.

PollingSCMPollingSCMtriggerisalittlelessintuitive.Thefollowingfigurepresentsthecommunication:

JenkinsperiodicallycallsGitHubandchecksiftherewasanypushtotherepository.Then,itstartsthebuild.Itmaysoundcounter-intuitive,however,thereareatleasttwogoodcasesforusingthismethod:

Jenkinsisinsidethefirewallednetwork(whichGitHubdoesnothaveaccessto)Commitsarefrequentandthebuildtakesalongtime,soexecutingabuildaftereverycommitwouldcauseanoverload

TheconfigurationofpollSCMisalsosomehowsimplerbecausethewaytoconnectfromJenkinstoGitHubisalreadysetup(JenkinschecksoutthecodefromGitHub,soitneedstohaveaccess).Inthecaseofourcalculatorproject,wecansetupanautomatictriggerbyaddingthetriggersdeclaration(justafteragent)tothepipeline:

triggers{

pollSCM('*****')

}

Afterrunningthepipelinemanuallyforthefirsttime,theautomatictriggerisset.Then,itchecksGitHubeveryminute,andfornewcommits,itstartsabuild.Totestthatitworksasexpected,youcancommitandpushanythingtotheGitHubrepositoryandseethatthebuildstarts.

Weusedthemysterious*****asanargumenttopollSCM.ItspecifieshowoftenJenkinsshouldcheckfornewsourcechangesandisexpressedinthecron-stylestringformat.

Thecronstringformatisdescribed(togetherwiththecrontool)athttps://en.wikipedi

a.org/wiki/Cron.

ScheduledbuildScheduledtriggermeansthatJenkinsrunsthebuildperiodically,nomatteriftherewasanycommittotherepositoryornot.

Asthefollowingfigurepresents,thereisnocommunicationwithanysystem

needed:

TheimplementationofScheduledbuildisexactlythesameaspollingSCM.TheonlydifferenceisthatthekeywordcronisusedinsteadofpollSCM.Thistriggermethodisrarelyusedforthecommitpipelinebutapplieswelltonightlybuilds(forexample,complexintegrationtestingexecutedatnights).

NotificationsJenkinsprovidesalotofwaystoannounceitsbuildstatus.What'smore,aswitheverythinginJenkins,newnotificationtypescanbeaddedusingplugins.

Let'swalkthroughthemostpopulartypessothatyoucanchoosetheonethatfitsyourneeds.

EmailThemostclassicwaytonotifyabouttheJenkinsbuildstatusistosendemails.Theadvantageofthissolutionisthateverybodyhasamailbox;everybodyknowshowtousethemailbox;andeverybodyisusedtoreceivinginformationbythemailbox.ThedrawbackisthatusuallytherearesimplytoomanyemailsandtheonesfromJenkinsquicklybecomefilteredoutandneverread.

Theconfigurationoftheemailnotificationisverysimple;it'senoughto:

HavetheSMTPserverconfiguredSetitsdetailsinJenkins(inManageJenkins|ConfigureSystem)Usemailtoinstructioninthepipeline

Thepipelineconfigurationcanbeasfollows:post{always{mailto:'team@company.com',subject:"CompletedPipeline:${currentBuild.fullDisplayName}",body:"Yourbuildcompleted,pleasecheck:${env.BUILD_URL}"}}

Notethatallnotificationsareusuallycalledinthepostsectionofthepipeline,whichisexecutedafterallsteps,nomatterwhetherthebuildsucceededorfailed.Weusedthealwayskeyword;however,therearedifferentoptions:

always:Executeregardlessofthecompletionstatuschanged:Executeonlyifthepipelinechangeditsstatusfailure:Executeonlyifthepipelinehasthefailedstatussuccess:Executeonlyifthepipelinehasthesuccessstatusunstable:Executeonlyifthepipelinehastheunstablestatus(usuallycausedbytestfailuresorcodeviolations)

post{<br/>failure{<br/>slackSendchannel:'#dragons-team',<br/>color:'danger',<br/>message:"Thepipeline${currentBuild.fullDisplayName}failed."<br/>}<br/>}

TeamspaceTogetherwiththeagileculturecametheideathatit'sbettertohaveeverythinghappeningintheteamspace.Insteadofwritingemails,meettogether;insteadofonlinemessaging,comeandtalk;insteadoftask-trackingtool,haveawhiteboard.ThesameideacametoContinuousDeliveryandJenkins.Currently,it'sverycommontoinstallbigscreens(alsocalledbuildradiators)intheteamspace.Then,whenyoucometotheoffice,thefirstthingyouseeisthecurrentstatusofthepipeline.Buildradiatorsareconsideredoneofthemosteffectivenotificationstrategies.Theyensurethateveryoneisawareoffailingbuildsand,asaside-effectbenefit,theyboostteamspiritandfavorin-personcommunication.

Sincedevelopersarecreativebeings,theyinventedalotofotherideasthatplaythesameroleastheradiators.Someteamshanglargespeakersthatbeepwhenthepipelinefailed.Someothershavetoysthatblinkwhenthebuildisdone.OneofmyfavoritesisPipelineStateUFO,whichisprovidedasanopensourceprojectonGitHub.Onitspage,youcanfindthedescriptionofhowtoprintandconfigureaUFOthathangsundertheceilingandsignalsthepipelinestate.Youcanfindmoreathttps://github.com/Dynatrace/ufo.

SinceJenkinsisextensiblebyplugins,itscommunitywrotealotofdifferentwaystoinformaboutthebuildstatuses.Amongthem,youcanfindRSSfeeds,SMSnotifications,mobileapplications,desktopnotifiers,andmuchmore.

TeamdevelopmentstrategiesWehavealreadydescribedeverythingabouthowtheContinuousIntegrationpipelineshouldlook.However,whenexactlyshoulditberun?Ofcourse,itistriggeredafterthecommittotherepositorybutafterthecommittowhichbranch?Onlytothetrunkortoeverybranch?Ormaybeitshouldrunbefore,notaftercommittingsothattherepositorywouldalwaysbehealthy?Or,howaboutthecrazyideatohavenobranchesatall?

Thereisnosinglebestanswertothesequestions.Actually,thewayyouusetheContinuousIntegrationprocessdependsonyourteamdevelopmentworkflow.So,beforewegoanyfurther,let'sdescribewhatthepossibleworkflowsare.

DevelopmentworkflowsAdevelopmentworkflowisthewayyourteamputsthecodeintotherepository.Itdepends,ofcourse,onmanyfactorssuchasthesourcecontrolmanagementtool,theprojectspecifics,ortheteamsize.

Asaresult,eachteamdevelopsthecodeinaslightlydifferentmanner.Wecan,however,classifythemintothreetypes:trunk-basedworkflow,branchingworkflow,andforkingworkflow.

Allworkflowsaredescribedindetailwithexamplesathttps://www.atlassian.com/git/tutorials/comparing-workflows.

Trunk-basedworkflowTrunk-basedworkflowisthesimplestpossiblestrategy.Itsoverviewispresentedinthefollowingfigure:

Thereisonecentralrepositorywithasingleentryforallchangestotheproject,whichiscalledthetrunkormaster.Everymemberoftheteamclonesthecentralrepositorytohavetheirownlocalcopies.Thechangesarecommitteddirectlytothecentralrepository.

BranchingworkflowBranchingworkflow,asitsnamesuggests,meansthatthecodeiskeptinmanydifferentbranches.Theideaispresentedinthefollowingfigure:

Whendevelopersstarttoworkonanewfeature,theycreateadedicatedbranchfromthetrunkandcommitallfeature-relatedchangesthere.Thismakesiteasyformultipledeveloperstoworkonafeaturewithoutbreakingthemaincodebase.Thisiswhy,inthecaseofbranchingworkflow,thereisnoprobleminkeepingthemasterhealthy.Whenthefeatureiscompleted,adeveloperrebasesthefeaturebranchfrommasterandcreatesapullrequestthatcontainsallfeature-relatedcodechanges.Itopensthecodereviewdiscussionsandmakesspacetocheckifthechangesdon'tdisturbthemaster.Whenthecodeisacceptedbyotherdevelopersandautomaticsystemchecks,thenitismergedintothemaincodebase.Then,thebuildisrunagainonmasterbutshouldalmostneverfailsinceitdidn'tfailonthebranch.

ForkingworkflowForkingworkflowisverypopularamongtheopensourcecommunity.Itsideaispresentedinthefollowingfigure:

Eachdeveloperhashisownserver-siderepository.Theymayormaynotbetheofficialrepository,buttechnically,eachrepositoryisexactlythesame.

Forkingmeansliterallycreatinganewrepositoryfromtheotherrepository.Developerspushtotheirownrepositoriesandwhentheywanttointegratethecode,theycreateapullrequesttotheotherrepository.

Themainadvantageoftheforkingworkflowisthattheintegrationisnotnecessarilyviaacentralrepository.Italsohelpswiththeownershipbecauseitallowsacceptingpullrequestsfromotherswithoutgivingthemwriteaccess.

Inthecaseofrequirement-orientedcommercialprojects,theteamusuallyworksononeproductandthereforehasacentralrepository,sothismodelboilsdowntothebranchingworkflowwiththegoodownershipassignment,forexample,onlyprojectleadscanmergepullrequestsintothecentralrepository.

AdoptingContinuousIntegrationWedescribeddifferentdevelopmentworkflows,buthowdotheyinfluencetheContinuousIntegrationconfiguration?

BranchingstrategiesEachdevelopmentworkflowimpliesadifferentContinuousIntegrationapproach:

Trunk-basedworkflow:impliesconstantlystrugglingagainstthebrokenpipeline.Ifeveryonecommitstothemaincodebase,thenthepipelineoftenfails.Inthiscase,theoldContinuousIntegrationrulesays,"Ifthebuildisbroken,thenthedevelopmentteamstopswhatevertheyaredoingandfixestheproblemimmediately."Branchingworkflow:solvesthebrokentrunkissuebutintroducesanotherone:ifeveryonedevelopsintheirownbranches,thenwhereistheintegration?Afeatureusuallytakesweeksormonthstodevelop,andforallthistime,thebranchisnotintegratedintothemaincode,thereforeitcannotbereallycalled"continuous"integration;nottomentionthatthereisaconstantneedformergingandresolvingconflicts.Forkingworkflow:impliesmanagingtheContinuousIntegrationprocessbyeveryrepositoryowner,whichisn'tusuallyaproblem.Itshares,however,thesameissuesasthebranchingworkflow.

Thereisnosilverbullet,anddifferentorganizationschoosedifferentstrategies.Thesolutionthatistheclosesttoperfectionisusingthetechniqueofthebranchingworkflowandthephilosophyofthetrunk-basedworkflow.Inotherwords,wecancreateverysmallbranchesandintegratethemfrequentlyintomaster.Thisseemstotakethebestofboth,however,requireseitherhavingtinyfeaturesorusingfeaturetoggles.SincetheconceptoffeaturetogglesfitsverywellintoContinuousIntegrationandContinuousDelivery,let'stakeamomenttoexploreit.

if(feature_toggle){<br/>//dosomething<br/>}

4. Duringthefeaturedevelopment:Codingisdoneinmasterwithfeature_toggle=true(insteadofcodinginthefeaturebranch)Releaseisdonefrommasterwithfeature_toggle=false

5. Whenthefeaturedevelopmentiscompleted,allifstatementsareremovedandfeature_toggleisremovedfromtheconfiguration(insteadofmergingfeaturetomasterandremovingthefeaturebranch).

Thebenefitoffeaturetoggleisthatalldevelopmentisdoneinthetrunk,whichenablestherealContinuousIntegrationandmitigatesproblemswithmergingcode.

JenkinsMultibranchIfyoudecidetousebranchesinanyform,thelongfeaturebranchesortherecommendedshort-livedbranches,thenitisconvenienttoknowthatthecodeishealthybeforemergingitintomaster.Thisapproachresultsinalwayskeepingthemaincodebasegreenand,luckily,thereisaneasywaytodoitwithJenkins.

InordertouseMultibranchinourcalculatorproject,let'sproceedwiththefollowingsteps:

1. OpenthemainJenkinspage.2. ClickonNewItem.3. Entercalculator-branchesastheitemname,selectMultibranchPipeline,and

clickonOK.4. IntheBranchSourcessection,clickonAddsource,andselectGit.5. EntertherepositoryaddressintoProjectRepository.

6. TickPeriodicallyifnototherwiserunandset1minuteasInterval.7. ClickonSave.

Everyminute,thisconfigurationchecksiftherewereanybranchesadded(orremoved)andcreates(ordeletes)thededicatedpipelinedefinedbyJenkinsfile.

Wecancreateanewbranchandseehowitworks.Let'screateanewbranchcalledfeatureandpushitintotherepository:$gitcheckout-bfeature$gitpushoriginfeature

Afteramoment,youshouldseeanewbranchpipelineautomaticallycreatedand

run:

Now,beforemergingthefeaturebranchtomaster,wecancheckifit'sgreen.Thisapproachshouldneverbreakthemasterbuild.

InthecaseofGitHub,thereisanevenbetterapproach,usingtheGitHubOrganizationFolderplugin.Itautomaticallycreatespipelineswithbranchesandpullrequestsforallprojects.

Averysimilarapproachistobuildapipelineperpullrequestinsteadofapipelineperbranch,whichgivesthesameresult;themaincodebaseisalwayshealthy.

Non-technicalrequirementsLastbutnotleast,ContinuousIntegrationisnotallaboutthetechnology.Onthecontrary,technologycomessecond.JamesShoreinhisarticleContinuousIntegrationonaDollaraDaydescribedhowtosetuptheContinuousIntegrationprocesswithoutanyadditionalsoftware.Allheusedwasarubberchickenandabell.Theideaistomaketheteamworkinoneroomandsetupaseparatecomputerwithanemptychair.Puttherubberchickenandthebellinfrontofthatcomputer.Now,whenyouplantocheckinthecode,taketherubberchicken,checkinthecode,gototheemptycomputer,checkoutthefreshcode,runallteststhere,andifeverythingpasses,putbacktherubberchickenandringthebellsothateveryoneknowsthatsomethinghasbeenaddedtotherepository.

ContinuousIntegrationonaDollaraDaybyJamesShorecanbefoundat:http://www.jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html.

Theideaisalittleoversimplified,andautomatedtoolsareuseful;however,themainmessageisthatwithouteachteammember'sengagement,eventhebesttoolswon'thelp.JezHumbleinhisgreatbook,ContinuousDelivery,mentionstheprerequisitesforContinuousIntegrationthatcanberephrasedwiththefollowingpoints:

Checkinregularly:QuotingMikeRoberts,""Continuouslyismoreoftenthanyouthink"",theminimumisonceaday.Createcomprehensiveunittests:It'snotonlyaboutthehightestcoverage,it'spossibletohavenoassertionsandstillkeep100%coverage.Keeptheprocessquick:ContinuousIntegrationmusttakeashorttime,preferablyunder5minutes.10minutesisalreadyalot.Monitorthebuilds:Itcanbeasharedresponsibilityoryoucanadaptthebuildmasterrolethatrotatesweekly.

Exercises

You'velearnedalotabouthowtoconfiguretheContinuousIntegrationprocess.Sincepracticemakesmanperfect,werecommenddoingthefollowingexercises:

1. CreateaPythonprogramthatmultipliestwonumberspassedasthecommand-lineparameters.AddunittestsandpublishtheprojectonGitHub:

Createtwofilescalculator.pyandtest_calculator.pyYoucanusetheunittestlibraryathttps://docs.python.org/library/unittest.htmlRuntheprogramandtheunittest

2. BuildtheContinuousIntegrationpipelineforthePythoncalculatorproject:

UseJenkinsfileforspecifyingthepipelineConfigurethetriggersothatthepipelinerunsautomaticallyincaseofanycommittotherepositoryThepipelinedoesn'tneedtheCompilestepsincePythonisaninterpretablelanguageRunthepipelineandobservetheresultsTrytocommitthecodethatbreakseachstageofthepipelineandobservehowitisvisualizedinJenkins

SummaryInthischapter,wecoveredallaspectsoftheContinuousIntegrationpipeline,whichisalwaysthefirststepforContinuousDelivery.Thekeytakeawayfromthechapter:

Pipelineprovidesageneralmechanismfororganizinganyautomationprocesses;however,themostcommonusecasesareContinuousIntegrationandContinuousDeliveryJenkinsacceptsdifferentwaysofdefiningpipelinesbuttherecommendedoneisthedeclarativesyntaxCommitpipelineisthemostbasicContinuousIntegrationprocessand,asitsnamesuggests,itshouldberunaftereverycommittotherepositoryThepipelinedefinitionshouldbestoredintherepositoryasaJenkinsfileCommitpipelinecanbeextendedwiththecodequalitystagesNomattertheprojectbuildtool,JenkinscommandsshouldalwaysbeconsistentwiththelocaldevelopmentcommandsJenkinsoffersawiderangeoftriggersandnotificationsThedevelopmentworkflowshouldbecarefullychoseninsidetheteamororganizationbecauseitaffectstheContinuousIntegrationprocessanddefinesthewaythecodeisdeveloped

Inthenextchapter,wewillfocusonthenextphaseoftheContinuousDeliveryprocess,automatedacceptancetesting.Itcanbeconsideredasthemostimportantand,inmanycases,themostdifficultsteptoimplement.WewillexploretheideaofacceptancetestingandasampleimplementationusingDocker.

AutomatedAcceptanceTesting

WealreadyconfiguredthecommitphaseoftheContinuousDeliveryprocessandnowit'stimetoaddresstheacceptancetestingphase,whichisusuallythemostchallengingpart.Bygraduallyextendingthepipeline,wewillseedifferentaspectsofawell-doneacceptancetestingautomation.

Thischaptercoversthefollowingpoints:

IntroducingtheacceptancetestingprocessanditsdifficultiesExplainingtheideaoftheartifactrepositoryCreatingtheDockerregistryonDockerHubInstallingandsecuringprivateDockerregistryImplementingacceptancetestingintheJenkinspipelineIntroducingandexploringDockerComposeUsingDockerComposeintheacceptancetestingprocessWritingacceptancetestswithusers

IntroducingacceptancetestingAcceptancetestingisatestperformedtodetermineifthebusinessrequirementsorcontractsaremet.Itinvolvesblack-boxtestingagainstacompletesystemfromauserperspectiveanditspositiveresultshouldimplytheacceptanceofthesoftwaredelivery.Sometimes,alsocalledUAT(useracceptancetesting),endusertesting,orbetatesting,itisaphaseofthedevelopmentprocesswhensoftwaremeetsthereal-worldaudience.

ManyprojectsrelyonmanualstepsperformedbyQAsoruserstoverifythefunctionalandnonfunctionalrequirements,butstill,it'swaymorereasonabletorunthemasprogrammedrepeatableoperations.

Automatedacceptancetests,however,canbeconsidereddifficultduetotheirspecifics:

User-facing:Theyneedtobewrittentogetherwithauser,whichrequiresanunderstandingbetweentwoworlds,technicalandnon-technical.Dependenciesintegration:Thetestedapplicationshouldberuntogetherwithitsdependenciesinordertocheckwhetherthesystemasawholeworksproperly.Environmentidentity:Staging(testing)andproductionenvironmentsshouldbeidenticaltoensurethatwhenruninproduction,theapplicationalsobehavesasexpected.Applicationidentity:Applicationshouldbebuiltonlyonceandthesamebinaryshouldbetransferredtoproduction.Thatguaranteesnochangesincodebetweentestingandreleasingandeliminatestheriskofdifferentbuildingenvironments.Relevanceandconsequences:Ifacceptancetestpasses,itshouldbeclearthattheapplicationisreadyforreleasefromtheuserperspective.

Weaddressallthesedifficultiesindifferentsectionsofthischapter.ApplicationidentitycanbeachievedbybuildingtheDockerimageonlyonceandusingDockerregistryforitsstorageandversioning.DockerComposehelpswiththedependenciesintegrationprovidingawaytobuildagroupofcontainerized

applicationsworkingtogether.Creatingtestsinauser-facingmannerisexplainedintheWritingacceptancetestssection,andtheenvironmentidentityisaddressedbytheDockertoolitselfandcanbealsoimprovedbyothertoolsdescribedinthenextchapter.Concerningtherelevanceandconsequences,theonlygoodansweristokeepinmindthatacceptancetestsmustalwaysbeofahighquality.

Acceptancetestingcanhavemultiplemeanings;inthisbook,wetreatacceptancetestingasacompleteintegrationtestfromauserperspective,excludingnonfunctionaltesting,suchasperformance,load,andrecovery.

DockerregistryDockerregistryisastorageforDockerimages.Tobeprecise,itisastatelessserverapplicationthatallowstheimagestobepublished(pushed)andlaterretrieved(pulled)whenneeded.WehavealreadyseenanexampleoftheregistrywhilerunningtheofficialDockerimages,suchasjenkins.WepulledtheimagesfromDockerHub,whichisanofficialcloud-basedDockerregistry.Havingaseparateservertostore,load,andsearchsoftwarepackagesisamoregeneralconceptcalledthesoftwarerepositoryor,evenmoregeneral,theartifactrepository.Let'slookcloseratthisidea.

ArtifactrepositoryWhilethesourcecontrolmanagementstoresthesourcecode,theartifactrepositoryisdedicatedforstoringsoftwarebinaryartifacts,forexample,compiledlibrariesorcomponents,laterusedtobuildacompleteapplication.Whydoweneedtostorebinariesonaseparateserverusingaseparatetool?

Filesize:Artifactfilescanbelarge,sothesystemsneedtobeoptimizedfortheirdownloadandupload.Versions:Eachuploadedartifactneedstohaveaversionthatmakesiteasytobrowseanduse.Notallversions,however,havetobestoredforever;forexample,iftherewasabugdetected,wemaynotbeinterestedintherelatedartifactandremoveit.Revisionmapping:Eachartifactshouldpointtoexactlyonerevisionofthesourcecontroland,what'smore,thebinarycreationprocessshouldberepeatable.Packages:Artifactsarestoredinthecompiledandcompressedformsothatthesetime-consumingstepsneednotberepeated.Accesscontrol:Userscanberestricteddifferentlytothesourcecodeandartifactbinaryaccess.Clients:Usersoftheartifactrepositorycanbedevelopersoutsidetheteamororganization,whowanttousethelibraryviaitspublicAPI.Usecases:Artifactbinariesareusedtoguaranteethatexactlythesamebuiltversionisdeployedtoeveryenvironmenttoeasetherollbackprocedureincaseoffailure.

ThemostpopularartifactrepositoriesareJFrogArtifactoryandSonatypeNexus.

TheartifactrepositoryplaysaspecialroleintheContinuousDeliveryprocessbecauseitguaranteesthatthesamebinaryisusedthroughoutallpipelinesteps.

Let'slookatthefollowingfigurepresentinghowitworks:

TheDeveloperpushesachangetothesourcecoderepository,whichtriggersthepipelinebuild.AsthelaststepoftheCommitStage,abinaryiscreatedandstoredintheartifactrepository.Afterward,duringallotherstagesofthedeliveryprocess,thesamebinaryispulledandused.

Thebuiltbinaryisoftencalledthereleasecandidateandtheprocessofmovingbinarytothenextstageiscalledpromotion.

Dependingontheprogramminglanguageandtechnologies,thebinaryformatscandiffer.

Forexample,inthecaseofJava,usually,JARfilesarestoredand,inthecaseofRuby,gemfiles.WeworkwithDocker,sowewillstoreDockerimagesasartifacts,andthetooltostoreDockerimagesiscalledDockerregistry.

Someteamsmaintaintworepositoriesatthesametime,artifactrepositoryforJARfilesandDockerregistryforDockerimages.WhileitmaybeusefulduringthefirstphaseoftheDockerintroduction,thereisnogoodreasontomaintainbothforever.

InstallingDockerregistryFirst,weneedtoinstallaDockerregistry.Thereareanumberofoptionsavailable,buttwoofthemaremorecommonthanothers,cloud-basedDockerHubregistryandyourownprivateDockerregistry.Let'sdigintothem.

DockerHubDockerHubisacloud-basedservicethatprovidesDockerregistryandotherfeaturessuchasbuildingimages,testingthem,andpullingcodedirectlyfromthecoderepository.DockerHubiscloud-hosted,soitdoesnotreallyneedanyinstallationprocess.AllyouneedtodoiscreateaDockerHubaccount:

1. Openhttps://hub.docker.com/inabrowser.2. Fillinthepassword,emailaddress,andDockerID.3. Afterreceivinganemailandclickingtheactivationlink,theaccountis

created.

DockerHubisdefinitelythesimplestoptiontostartwith,anditallowsstoringbothprivateandpublicimages.

PrivateDockerregistry

DockerHubmaynotalwaysbeacceptable.Itisnotfreeforenterprisesand,what'sevenmoreimportant,alotofcompanieshavepoliciesnottostoretheirsoftwareoutsidetheirownnetwork.Inthiscase,theonlyoptionistoinstallaprivateDockerregistry.

TheDockerregistryinstallationprocessisquickandsimple,however,makingitsecureandavailableinpublicrequiressettingupaccessrestrictionandthedomaincertificate.Thisiswhywesplitthissectionintothreeparts:

InstallingtheDockerregistryapplicationAddingdomaincertificateAddingaccessrestriction

InstallingtheDockerregistryapplicationDockerregistryisavailableasaDockerimage.Tostartthis,wecanrunthefollowingcommand:

$dockerrun-d-p5000:5000--restart=always--nameregistryregistry:2

Bydefault,theregistrydataisstoredasadockervolumeinthedefaulthostfilesystem'sdirectory.Tochangeit,youcanadd-v<host_directory>:/var/lib/registry.Anotheralternativeistouseavolumecontainer.

Thecommandstartstheregistryandmakesitaccessibleviaport5000.Theregistrycontainerisstartedfromtheregistryimage(version2).The--restart=alwaysoptioncausesthecontainertoautomaticallyrestartwheneverit'sdown.

ConsidersettingupaloadbalancerandstartingafewDockerregistrycontainersincaseofalargenumberofusers.

AddingadomaincertificateIftheregistryisrunonthelocalhost,theneverythingworksfineandnootherinstallationstepsarerequired.However,inmostcases,wewanttohaveadedicatedserverfortheregistry,sothattheimagesarewidelyavailable.Inthatcase,DockerrequiressecuringtheregistrywithSSL/TLS.Theprocessisverysimilartothepublicwebserverconfigurationand,similarly,it'shighlyrecommendedhavingthecertificatesignedbyCA(certificateauthority).IfobtainingtheCA-signedcertificateisnotanoption,thenwecanself-signacertificateorusethe--insecure-registryflag.

Youcanreadaboutcreatingandusingself-signedcertificatesathttps://docs.docker.com/registry/insecure/#using-self-signed-certificates.

HavingthecertificateseithersignedbyCAorself-signed,wecanmovedomain.crtanddomain.keytothecertsdirectoryandstarttheregistry.

$dockerrun-d-p5000:5000--restart=always--nameregistry-v`pwd`/certs:/certs-eREGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt-eREGISTRY_HTTP_TLS_KEY=/certs/domain.keyregistry:2

Incaseofaself-signedcertificate,clientshavetoexplicitlytrustthecertificate.Inordertodothis,theycancopythedomain.crtfileto/etc/docker/certs.d/<docker_host_domain>:5000/ca.crt.

Usingthe--insecure-registryflagisnotrecommendedsinceitprovidesnosecurityatall.

AddinganaccessrestrictionUnlessweusetheregistryinsideawell-securedprivatenetwork,weshouldconfiguretheauthentication.

Thesimplestwaytodothisistocreateauserwithapasswordusingthehtpasswdtoolfromtheregistryimage:

$mkdirauth

$dockerrun--entrypointhtpasswdregistry:2-Bbn<username><password>>auth/passwords

Thecommandrunsthehtpasswdtooltocreatetheauth/passwordsfile(withoneuserinside).Then,wecanruntheregistrywiththatoneuserauthorizedtoaccessit:

$dockerrun-d-p5000:5000--restart=always--nameregistry-v`pwd`/auth:/auth-e"REGISTRY_AUTH=htpasswd"-e"REGISTRY_AUTH_HTPASSWD_REALM=RegistryRealm"-eREGISTRY_AUTH_HTPASSWD_PATH=/auth/passwords-v`pwd`/certs:/certs-eREGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt-eREGISTRY_HTTP_TLS_KEY=/certs/domain.keyregistry:2

Thecommand,inadditiontosettingthecertificates,createstheaccessrestrictionlimitedtotheusersspecifiedintheauth/passwordsfile.

Asaresult,beforeusingtheregistry,aclientneedstospecifytheusernameandpassword.

Accessrestrictiondoesn'tworkinthecaseofthe--insecure-registryflag.

OtherDockerregistries

DockerHubandprivateregistryarenottheonlypossibilitieswhenitcomestoDocker-basedartifactrepositories.

Theotheroptionsareasfollows:

General-purposerepositories:Widely-usedgeneral-purposerepositories,suchasJFrogArtifactoryorSonatypeNexus,implementtheDockerregistryAPI.TheiradvantageisthatoneservercanstorebothDockerimagesandotherartifacts(forexample,JARfiles).Thesesystemsarealsomatureandprovideenterpriseintegration.Cloud-basedregistries:DockerHubisnottheonlycloudprovider.Mostcloud-orientedservicesofferDockerregistriesinthecloud,forexample,GoogleCloudorAWS.Customregistries:TheDockerregistryAPIisopen,soit'spossibletoimplementcustomsolutions.What'smore,imagescanbeexportedtofiles,soit'sfeasibletostoreimagessimplyasfiles.

UsingDockerregistryWhentheregistryisconfigured,wecanshowhowtoworkwithitinthreesteps:

BuildinganimagePushingtheimagetotheregistryPullingtheimagefromtheregistry

FROMubuntu:16.04<br/>RUNapt-getupdate&&\<br/>apt-getinstall-ypython

<strong>$dockerbuild-tubuntu_with_python.</strong>

PushingtheimageInordertopushthecreatedimage,weneedtotagitaccordingtothenamingconvention:

<registry_address>/<image_name>:<tag>

The"registry_address"canbe:

UsernameincaseofDockerHubDomainnameorIPaddresswithportforaprivateregistry(forexample,localhost:5000)

Inmostcases,<tag>isintheformofimage/applicationversion.

Let'stagtheimagetouseDockerHub:

$dockertagubuntu_with_pythonleszko/ubuntu_with_python:1

Wecouldhavealsotaggedtheimageinthebuildcommand:"dockerbuild-tleszko/ubuntu_with_python:1.".

Iftherepositoryhasaccessrestrictionconfigured,weneedtoauthorizeitfirst:

$dockerlogin--username<username>--password<password>

It'spossibletousethedockerlogincommandwithoutparametersandDockerwouldaskinteractivelyfortheusernameandpassword.

Now,wecanstoretheimageintheregistryusingthepushcommand:

$dockerpushleszko/ubuntu_with_python:1

NotethatthereisnoneedtospecifytheregistryaddressbecauseDockerusesthenamingconventiontoresolveit.Theimageisstored,andwecancheckitusing

theDockerHubwebinterfaceavailableathttps://hub.docker.com.

PullingtheimageTodemonstratehowtheregistryworks,wecanremovetheimagelocallyandretrieveitfromtheregistry:$dockerrmiubuntu_with_pythonleszko/ubuntu_with_python:1

Wecanseethattheimagehasbeenremovedusingthedockerimagescommand.Then,let'sretrievetheimagebackfromtheregistry:$dockerpullleszko/ubuntu_with_python:1

IfyouusethefreeDockerHubaccount,youmayneedtochangetheubuntu_with_pythonrepositorytopublicbeforepullingit.

Wecanconfirmtheimageisbackwiththedockerimagescommand.

Whenwehavetheregistryconfiguredandunderstandhowitworks,wecanseehowtouseitinsidetheContinuousDeliverypipelineandbuildtheacceptancetestingstage.

AcceptancetestinpipelineWealreadyunderstoodtheideabehindacceptancetestingandknowhowtoconfigureDockerregistry,sowearereadyforitsfirstimplementationinsidetheJenkinspipeline.

Let'slookatthefigurethatpresentstheprocesswewilluse:

Theprocessgoesasfollows:

1. ThedeveloperpushesacodechangetoGitHub.2. Jenkinsdetectsthechange,triggersthebuild,andchecksoutthecurrent

code.3. JenkinsexecutesthecommitphaseandbuildstheDockerimage.4. JenkinspushestheimagetoDockerregistry.5. JenkinsrunstheDockercontainerinthestagingenvironment.6. StagingtheDockerhostneedstopulltheimagefromtheDockerregistry.7. Jenkinsrunstheacceptancetestsuiteagainsttheapplicationrunninginthe

stagingenvironment.

Forthesakeofsimplicity,wewillruntheDockercontainerlocally

(andnotonaseparatestagingserver).Inordertorunitremotely,weneedtousethe-HoptionortoconfiguretheDOCKER_HOSTenvironmentvariable.Wewillcoverthispartinthenextchapter.

Let'scontinuethepipelinewestartedinthepreviouschapterandaddthreemorestages:

Dockerbuild

Dockerpush

Acceptancetest

KeepinmindthatyouneedtohavetheDockertoolinstalledontheJenkinsexecutor(agentslaveormaster,inthecaseofslave-lessconfiguration)sothatitisabletobuildDockerimages.

IfyouusedynamicallyprovisionedDockerslaves,thenthereisnomatureDockerimageprovidedyet.Youcanbuildityourselforusetheleszko/jenkins-docker-slaveimage.YoualsoneedtomarktheprivilegedoptionintheDockeragentconfiguration.Thissolution,however,hassomedrawbacks,sobeforeusingitinproduction,readthehttp://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/.

TheDockerbuildstageWewouldliketorunthecalculatorprojectasaDockercontainer,soweneedtocreateDockerfileandaddthe"Dockerbuild"stagetoJenkinsfile.

AddingDockerfileLet'screateDockerfileintherootdirectoryofthecalculatorproject:FROMfrolvlad/alpine-oraclejdk8:slimCOPYbuild/libs/calculator-0.0.1-SNAPSHOT.jarapp.jarENTRYPOINT["java","-jar","app.jar"]

ThedefaultbuilddirectoryforGradleisbuild/libs/,andcalculator-0.0.1-SNAPSHOT.jaristhecompleteapplicationpackagedintooneJARfile.NotethatGradleautomaticallyversionedtheapplicationusingtheMaven-styleversion0.0.1-SNAPSHOT.

DockerfileusesabaseimagethatcontainsJDK8(frolvlad/alpine-oraclejdk8:slim).ItalsocopiestheapplicationJAR(createdbyGradle)andrunsit.Let'scheckiftheapplicationbuildsandruns:$./gradlewbuild$dockerbuild-tcalculator.$dockerrun-p8080:8080--namecalculatorcalculator

Usingtheprecedingcommands,we'vebuilttheapplication,builttheDockerimage,andruntheDockercontainer.Afterawhile,weshouldbeabletoopenthebrowsertohttp://localhost:8080/sum?a=1&b=2andsee3asaresult.

WecanstopthecontainerandpushtheDockerfiletotheGitHubrepository:$gitaddDockerfile$gitcommit-m"AddDockerfile"$gitpush

AddingtheDockerbuildtothepipelineThelaststepweneedisaddingthe"Dockerbuild"stagetoJenkinsfile.Usually,theJARpackagingisalsodeclaredasaseparatePackagestage:stage("Package"){steps{sh"./gradlewbuild"}}

stage("Dockerbuild"){steps{sh"dockerbuild-tleszko/calculator."}}

Wedon'texplicitlyversiontheimage,buteachimagehasauniquehashID.Wewillcovertheexplicitversioninginthenextchapter.

NotethatweusedtheDockerregistrynameintheimagetag.Thereisnoneedtohavetheimagetaggedtwicecalculatorandleszko/calculator.

WhenwecommitandpushJenkinsfile,thepipelinebuildshouldstartautomaticallyandweshouldseeallboxesgreen.ThismeansthattheDockerimagehasbeenbuiltsuccessfully.

ThereisalsoaGradlepluginforDockerthatallowsexecutingtheDockeroperationswithinGradlescripts.Youcanseeanexampleat:https://spring.io/guides/gs/spring-boot-docker/.

TheDockerpushstageWhentheimageisready,wecanstoreitintheregistry.TheDockerpushstageisverysimple.It'senoughtoaddthefollowingcodetoJenkinsfile:

stage("Dockerpush"){

steps{

sh"dockerpushleszko/calculator"

}

}

IfDockerregistryhastheaccessrestricted,thenfirstweneedtologinusingthedockerlogincommand.Needlesstosay,thecredentialsmustbewellsecured,forexample,usingadedicatedcredentialstoreasdescribedontheofficialDockerpage:https://docs.docker.com/engine/reference/commandline/login/#credentials-store.

Asalways,pushingchangestotheGitHubrepositorytriggersJenkinstostartthebuildand,afterawhile,weshouldhavetheimageautomaticallystoredintheregistry.

AcceptancetestingstageToperformacceptancetesting,first,weneedtodeploytheapplicationtothestagingenvironmentandthenruntheacceptancetestsuiteagainstit.

stage("Deploytostaging"){<br/>steps{<br/>sh"dockerrun-d--rm-p8765:8080--namecalculatorleszko/calculator"<br/>}<br/>}

Afterrunningthisstage,thecalculatorcontainerisrunningasadaemon,publishingitsportas8765andbeingremovedautomaticallywhenstopped.

AddinganacceptancetesttothepipelineAcceptancetestingusuallyrequiresrunningadedicatedblack-boxtestsuitethatchecksthebehaviorofthesystem.WewillcoveritintheWritingacceptancetestssection.Atthemoment,forthesakeofsimplicity,let'sperformacceptancetestingsimplybycallingthewebserviceendpointwiththecurltoolandcheckingtheresultusingthetestcommand.

Intherootdirectoryoftheproject,let'screatetheacceptance_test.shfile:

#!/bin/bash

test$(curllocalhost:8765/sum?a=1\&b=2)-eq3

Wecallthesumendpointwithparametersa=1andb=2andexpecttoreceive3inresponse.

Then,theAcceptanceteststagecanlookasfollows:

stage("Acceptancetest"){

steps{

sleep60

sh"./acceptance_test.sh"

}

}

Sincethedockerrun-dcommandisasynchronous,weneedtowaitusingthesleepoperationtomakesuretheserviceisalreadyrunning.

Thereisnogoodwaytocheckiftheserviceisalreadyrunning.Analternativetosleepingcouldbeascriptcheckingeverysecondwhethertheservicehasalreadystarted.

post{<br/>always{<br/>sh"dockerstopcalculator"<br/>}<br/>}

ThisstatementmakessurethatthecalculatorcontainerisnolongerrunningontheDockerhost.

DockerComposeLifeiseasywithoutdependencies.Inreal-life,however,almosteveryapplicationlinkstoadatabase,cache,messagingsystem,oranotherapplication.Inthecaseofa(micro)servicearchitecture,eachserviceneedsabunchofotherservicestodoitswork.Themonolithicarchitecturedoesnoteliminatetheissue,anapplicationusuallyhassomedependencies,atleasttothedatabase.

Imagineanewcomerjoiningyourdevelopmentteam;howmuchtimedoesittaketosetuptheentiredevelopmentenvironmentandruntheapplicationwithallitsdependencies?

Whenitcomestoautomatedacceptancetesting,thedependenciesissueisnolongeronlyamatterofconvenience,butitbecomesanecessity.While,duringunittesting,wecouldmockthedependencies,theacceptancetestingsuiterequiresacompleteenvironment.Howdowesetitupquicklyandinarepeatablemanner?Luckily,DockerComposeisatoolthatcanhelp.

WhatisDockerCompose?DockerComposeisatoolfordefining,running,andmanagingmulti-containerDockerapplications.Servicesaredefinedinaconfigurationfile(aYAMLformat)andcanbecreatedandrunalltogetherwithasinglecommand.

DockerComposeorchestratescontainersusingstandardDockermechanismsandprovidesaconvenientwaytospecifytheentireenvironment.

DockerComposecomeswithalotoffeatures,themostinterestingare:

BuildingasetofservicesLaunchingasetofservicestogetherManagingthestateofindividualservicesPreservingvolumedatabetweenrunsScalingservicesupanddownShowinglogsofindividualservicesCachingconfigurationandrecreatingchangedcontainersbetweenruns

AdetaileddescriptionofDockerComposeanditsfeaturescanbefoundontheofficialpageat:https://docs.docker.com/compose/.

WepresenttheDockerComposetoolstartingwiththeinstallationprocess,goingthroughthedocker-compose.ymlconfigurationfileandthedocker-composecommand,toendupwiththebuildingandscalingfeatures.

InstallingDockerComposeThesimplestmethodtoinstallDockerComposeistousethepippackagemanager:

Youcanfindthepiptoolinstallationguideathttps://pip.pypa.io/en/stable/installing/,orforUbuntu,atsudoapt-getinstallpython-pip.

$pipinstalldocker-compose

TocheckthatDockerComposeisinstalled,wecanrun:

$docker-compose--version

Installationguidelinesforalloperatingsystemscanbefoundathttps://docs.docker.com/compose/install/.

Definingdocker-compose.ymlThedocker-compose.ymlfileisusedtodefinetheconfigurationforcontainers,theirrelations,andruntimeproperties.

Inotherwords,whenDockerfilespecifieshowtocreateasingleDockerimage,thendocker-compose.ymlspecifieshowtosetuptheentireenvironmentoutofDockerimages.

Therearethreeversionsofthedocker-compose.ymlfileformat.Inthisbook,weuseversion3,whichisthemostcurrentandrecommended.Readmoreat:https://docs.docker.com/compose/compose-file/compose-versioning/.

Thedocker-compose.ymlfilehasalotoffeaturesandallofthemcanbefoundattheofficialpage:https://docs.docker.com/compose/compose-file/.WewillcoverthemostimportantonesinthecontextoftheContinuousDeliveryprocess.

Let'sstartwithanexampleandimaginethatourcalculatorprojectusestheRedisserverforcaching.Inthiscase,weneedanenvironmentwithtwocontainers,calculatorandredis.Inanewdirectory,let'screatethedocker-compose.ymlfile.

version:"3"

services:

calculator:

image:calculator:latest

ports:

-8080

redis:

image:redis:latest

Theenvironmentconfigurationispresentedinthefollowingfigure:

Let'sseethedefinitionofthetwocontainers:

redis:AcontainerfromthelatestversionoftheredisimagepulledfromtheofficialDockerHub.calculator:Acontainerfromthelatestversionofthecalculatorimagebuiltlocally.Itpublishesthe8080porttotheDockerhost(whichisasubstituteforthe-poptionofthedockercommand).Thecontainerlinkstotherediscontainer,whichmeansthattheysharethesamenetworkandtheredisIPaddressisvisibleundertheredishostnamefrominsidethecalculatorcontainer.

Ifwelikeaservicetobeaddressedbyadifferenthostnamethanitsservicename(forexample,byredis-cacheapartfromredis),thenwecancreatealiasesusingthelinkskeyword.

Usingthedocker-composecommandThedocker-composecommandreadsthedefinitionfileandcreatestheenvironment:

$docker-composeup-d

Thecommandstartedtwocontainers,calculatorandredisinthebackground(-doption).Wecancheckthatthecontainersarerunning:

$docker-composeps

NameCommandStatePorts

---------------------------------------------------------------------------

project_calculator_1java-jarapp.jarUp0.0.0.0:8080->8080/tcp

project_redis_1docker-entrypoint.shredis...Up6379/tcp

Thecontainernamesareprefixedwiththeprojectnameproject,whichistakenfromthenameofthedirectoryinwhichthedocker-compose.ymlfileisplaced.Wecouldspecifytheprojectnamemanuallyusingthe-p<project_name>option.SinceDockerComposeisrunontopofDocker,wecanalsousethedockercommandtoconfirmthatthecontainersarerunning:

$dockerps

CONTAINERIDIMAGECOMMANDPORTS

360518e46bd3calculator:latest"java-jarapp.jar"0.0.0.0:8080->8080/tcp

2268b9f1e14bredis:latest"docker-entrypoint..."6379/tcp

Whenwe'redone,wecanteardowntheenvironment:

$docker-composedown

Theexampleisverysimple,butthetoolitselfisextremelypowerful.Withashortconfigurationandabunchofcommands,wecontroltheorchestrationofallservices.BeforeweuseDockerComposeforacceptancetesting,let'slookattwootherDockerComposefeatures:buildingimagesandscalingcontainers.

BuildingimagesIntheprecedingexample,wehadtofirstbuildthecalculatorimageusingthedockerbuildcommand,andthenitcouldbespecifiedinsidedocker-compose.yml.ThereisalsoanotherapproachtoletDockerComposebuildtheimage.Inthatcase,weneedtospecifythebuildpropertyinsteadofimageintheconfiguration.

Let'sputthedocker-compose.ymlfileinthecalculatorproject'sdirectory.WhenDockerfileandDockerComposeconfigurationsareinthesamedirectory,theformercanlookasfollows:version:"3"services:calculator:build:.ports:-8080redis:image:redis:latest

Thedocker-composebuildcommandbuildstheimage.WecanalsoaskDockerComposetobuildimagesbeforerunningthecontainerswiththeuseofthedocker-compose--buildupcommand.

ScalingservicesDockerComposeprovidesthefunctionalitytoautomaticallycreatemultipleinstancesofthesamecontainer.Wecaneitherspecifythereplicas:<number>parameterinsidedocker-compose.ymlorusethedocker-composescalecommand.

Asanexample,let'sruntheenvironmentagainandreplicatethecalculatorcontainer:

$docker-composeup-d

$docker-composescalecalculator=5

Wecancheckwhichcontainersarerunning:

$docker-composeps

NameCommandStatePorts

---------------------------------------------------------------------------

calculator_calculator_1java-jarapp.jarUp0.0.0.0:32777->8080/tcp

calculator_calculator_2java-jarapp.jarUp0.0.0.0:32778->8080/tcp

calculator_calculator_3java-jarapp.jarUp0.0.0.0:32779->8080/tcp

calculator_calculator_4java-jarapp.jarUp0.0.0.0:32781->8080/tcp

calculator_calculator_5java-jarapp.jarUp0.0.0.0:32780->8080/tcp

calculator_redis_1docker-entrypoint.shredis...Up6379/tcp

Fivecalculatorcontainersareexactlythesame,apartfromthecontainerID,containername,andpublishedportnumbers.

TheyallusethesameinstanceoftheRediscontainer.Wecannowstopandremoveallthecontainers:

$docker-composedown

ScalingcontainersisoneofthemostimpressiveDockerComposefeatures.Withonecommand,wecanscaleupanddownthenumberofcloneinstances.DockerComposetakescareofcleaningupthecontainersthatarenolongerused.

WehaveseenthemostinterestingfunctionalitiesoftheDockerComposetool.

Inthenextsection,wewillfocusonhowtouseitinthecontextofautomatedacceptancetesting.

AcceptancetestingwithDockerComposeDockerComposefitstheacceptancetestingprocessperfectlybecauseitenablessettinguptheentireenvironmentwithonecommand.What'smore,afterthetestingisperformed,theenvironmentcanalsobecleanedupwithonecommand.IfwedecidetouseDockerComposeonproduction,thentheotherbenefitisthattheacceptancetestusesexactlythesameconfiguration,tools,andcommandsasthereleasedapplication.

ToseehowtoapplyDockerComposefortheJenkinsacceptancetestingstage,let'scontinuethecalculatorprojectexampleandaddtheRedis-basedcachingtotheapplication.Then,wewillseetwodifferentapproachestorunacceptancetesting:theJenkins-firstmethodandtheDocker-firstmethod.

Usingamulti-containerenvironmentDockerComposeprovidesdependenciesbetweenthecontainers;inotherwords,itlinksonecontainertoanothercontainer.Technically,thismeansthatcontainerssharethesamenetworkandthatonecontainerisvisiblefromtheother.Tocontinueourexample,weneedtoaddthisdependencyinthecode,andwewilldothisinafewsteps.

AddingaRedisclientlibrarytoGradleInthebuild.gradlefile,addthefollowingconfigurationtothedependenciessection:

compile"org.springframework.data:spring-data-redis:1.8.0.RELEASE"

compile"redis.clients:jedis:2.9.0"

ItaddstheJavalibrariesthattakecareofthecommunicationwithRedis.

packagecom.leszko.calculator;<br/>importorg.springframework.cache.CacheManager;<br/>importorg.springframework.cache.annotation.CachingConfigurerSupport;<br/>importorg.springframework.cache.annotation.EnableCaching;<br/>importorg.springframework.context.annotation.Bean;<br/>importorg.springframework.context.annotation.Configuration;<br/>importorg.springframework.data.redis.cache.RedisCacheManager;<br/>importorg.springframework.data.redis.connection.RedisConnectionFactory;<br/>importorg.springframework.data.redis.connection.jedis.JedisConnectionFactory;<br/>importorg.springframework.data.redis.core.RedisTemplate;<br/><br/>/**Cacheconfig.*/<br/>@Configuration<br/>@EnableCaching<br/>publicclassCacheConfigextendsCachingConfigurerSupport{<br/>privatestaticfinalStringREDIS_ADDRESS="redis";<br/><br/>@Bean<br/>publicJedisConnectionFactoryredisConnectionFactory(){<br/>JedisConnectionFactoryredisConnectionFactory=new<br/>JedisConnectionFactory();<br/>redisConnectionFactory.setHostName(REDIS_ADDRESS);<br/>redisConnectionFactory.setPort(6379);<br/>returnredisConnectionFactory;<br/>}<br/><br/>@Bean<br/>publicRedisTemplate<String,String>redisTemplate(RedisConnectionFactorycf){<br/>RedisTemplate<String,String>redisTemplate=newRedisTemplate<String,<br/>String>();<br/>redisTemplate.setConnectionFactory(cf);<br/>returnredisTemplate;<br/>}<br/><br/>@Bean<br/>publicCacheManagercacheManager(RedisTemplateredisTemplate){<br/>returnnewRedisCacheManager(redisTemplate);<br/>}<br/>}

ThisastandardSpringcacheconfiguration.NotethatfortheRedisserveraddress,weusetheredishostnamethatisautomaticallyavailablethankstotheDockerComposelinkingmechanism.

packagecom.leszko.calculator;<br/>importorg.springframework.cache.annotation.Cacheable;<br/>importorg.springframework.stereotype.Service;<br/><br/>/**Calculatorlogic*/<br/>@Service<br/>publicclassCalculator{<br/>@Cacheable("sum")<br/>publicintsum(inta,intb){<br/>returna+b;<br/>}<br/>}

Fromnowon,thesumcalculationsarecachedinRedis,andwhenwecallthe/sumendpointofthecalculatorwebservice,itwillfirsttrytoretrievetheresultfromthecache.

CheckingthecachingenvironmentAssumingthatwehaveourdocker-compose.ymlinthecalculatorproject'sdirectory,wecannowstartthecontainers:

$./gradlewcleanbuild

$docker-composeup--build-d

Wecanchecktheportonwhichthecalculatorserviceispublished:

$docker-composeportcalculator8080

0.0.0.0:32783

Ifweopenthebrowseronlocalhost:32783/sum?a=1&b=2,thecalculatorserviceshouldreply3and,inthemeantime,accesstheredisserviceandstorethecachedvaluethere.ToseethatthecachevaluewasreallystoredinRedis,wecanaccesstherediscontainerandlookinsidetheRedisdatabase:

$docker-composeexecredisredis-cli

127.0.0.1:6379>keys*

1)"\xac\xed\x00\x05sr\x00/org.springframework.cache.interceptor.SimpleKeyL\nW\x03km\x93\xd8\x02\x00\x02I\x00\bhashCode[\x00\x06paramst\x00\x13[Ljava/lang/Object;xp\x00\x00\x03\xe2ur\x00\x13[Ljava.lang.Object;\x90\xceX\x9f\x10s)l\x02\x00\x00xp\x00\x00\x00\x02sr\x00\x11java.lang.Integer\x12\xe2\xa0\xa4\xf7\x81\x878\x02\x00\x01I\x00\x05valuexr\x00\x10java.lang.Number\x86\xac\x95\x1d\x0b\x94\xe0\x8b\x02\x00\x00xp\x00\x00\x00\x01sq\x00~\x00\x05\x00\x00\x00\x02"

2)"sum~keys"

Thedocker-composeexeccommandexecutedtheredis-cli(theRedisclienttobrowseitsdatabasecontent)commandinsidetherediscontainer.Then,wecanrunkeys*toprinteverythingthatisstoredinRedis.

YoucanplaymorewiththecalculatorapplicationandopenthebrowserwithdifferentvaluestoseethattheRedisservicecontentincreases.Afterthis,it'simportanttoteardowntheenvironmentwiththedocker-composedowncommand.

Inthenextsections,wewillseetwomethodsofacceptancetestsforthemulti-containerproject.Obviously,beforewetakeanyactiononJenkins,weneedtocommitandpushallthechangedfiles(includingdocker-compose.yml)toGitHub.

Notethat,forfurthersteps,DockerComposehastobeinstalledonJenkinsexecutors.

Method1–Jenkins-firstacceptancetestingThefirstmethodistoperformacceptancetestinginthesamewaywedidinthecaseofasinglecontainerapplication.Theonlydifferenceisthatnowwehavetwocontainersrunningaspresentedinthefollowingfigure:

Therediscontainerisnotvisiblefromauserperspective,soasaresult,theonlydifferencebetweensingle-containerandmulti-containeracceptancetestingisthatweusethedocker-composeupcommandinsteadofdockerrun.

OtherDockercommandscanbealsoreplacedwiththeirDockerComposeequivalents:docker-composebuildfordockerbuildanddocker-composepushfordockerpush.Nevertheless,ifwebuildjustoneimage,thenleavingDockercommandsisfineaswell.

ChangingthestagingdeploymentstageLet'schangetheDeploytostagingstagetouseDockerCompose:

stage("Deploytostaging"){

steps{

sh"docker-composeup-d"

}

}

Wemustchangethecleanupinexactlythesameway:

post{

always{

sh"docker-composedown"

}

}

ChangingtheacceptanceteststageForthepurposeofusingdocker-composescale,wedidn'tspecifytheportnumberunderwhichourwebservicewouldbepublished.Ifwedid,thenthescalingprocesswouldfailbecauseallcloneswouldtrytopublishunderthesameportnumber.Onthecontrary,weletDockerchoosetheport.Therefore,weneedtochangetheacceptance_test.shscripttofirstfindwhattheportnumberisandthenruncurlwiththecorrectportnumber.

#!/bin/bash

CALCULATOR_PORT=$(docker-composeportcalculator8080|cut-d:-f2)

test$(curllocalhost:$CALCULATOR_PORT/sum?a=1\&b=2)-eq3

Let'sfigureouthowwefoundtheportnumber:

1. Thedocker-composeportcalculator8080commandchecksunderwhichIPandportaddressthewebserviceispublished(itreturns,forexample,127.0.0.1:57648).

2. cut-d:-f2selectsonlyport(forexample,for127.0.0.1:57648,itreturns57648).

WecanpushthechangetoGitHubandobservetheJenkinsresults.Theideaisstillthesameaswiththesingle-containerapplication,setuptheenvironment,runtheacceptancetestsuite,andteardowntheenvironment.Eventhoughthisacceptancetestingmethodisgoodandworkswell,let'slookatthealternativesolution.

Method2–Docker-firstacceptancetestingIntheDocker-firstapproach,wecreateanadditionaltestcontainerthatperformstestingfrominsidetheDockerhost,aspresentedinthefollowingfigure:

ThisapproachfacilitatestheacceptancetestscriptintermsofretrievingtheportnumberandcanbeeasilyrunwithoutJenkins.It'salsomuchmoreintheDockerstyle.

ThedrawbackisthatweneedtocreateaseparateDockerfileandDockerComposeconfigurationforthepurposeoftesting.

FROMubuntu:trusty<br/>RUNapt-getupdate&&\<br/>apt-getinstall-yqcurl<br/>COPYtest.sh.<br/>CMD["bash","test.sh"]

Itcreatesanimagethatrunstheacceptancetest.

version:"3"<br/>services:<br/>test:<br/>build:./acceptance

Itcreatesanewcontainerthatislinkedtothecontainerbeingtested:calculator.What'smore,internallyit'salways8080sothateliminatestheneedforthetrickypartofportfinding.

#!/bin/bash<br/>sleep60<br/>test$(curlcalculator:8080/sum?a=1\&b=2)-eq3

It'sverysimilartothepreviousacceptancetestscript,theonlydifferenceisthatwecanaddressthecalculatorservicebythecalculatorhostnameandthattheportnumberisalways8080.Also,inthiscase,wewaitinsidethescript,notintheJenkinsfile.

RunningtheacceptancetestWecanrunthetestlocallyusingtheDockerComposecommandfromtherootprojectdirectory:$docker-compose-fdocker-compose.yml-facceptance/docker-compose-acceptance.yml-pacceptanceup-d--build

ThecommandusestwoDockerComposeconfigurationstoruntheacceptanceproject.Oneofthestartedcontainersshouldbecalledacceptance_test_1andbeinterestedinitsresult.Wecancheckitslogswiththefollowingcommand:$dockerlogsacceptance_test_1%Total%Received%XferdAverageSpeedTime1001100100100:00:01

Thelogshowsthatthecurlcommandhasbeensuccessfullycalled.Ifwewanttocheckwhetherthetestsucceededorfailed,wecanchecktheexitcodeofthecontainer:$dockerwaitacceptance_test_10

The0exitcodemeansthatthetestsucceeded.Anycodeotherthan0wouldmeanthatthetestfailed.Afterthetestisdone,weshould,asalways,teardowntheenvironment:$docker-compose-fdocker-compose.yml-facceptance/docker-compose-acceptance.yml-pacceptancedown

ChangingtheacceptanceteststageAsthelaststep,wecanaddtheacceptancetestexecutiontothepipeline.Let'sreplacethelastthreestagesinJenkinsfilewithonenewAcceptanceteststage:

stage("Acceptancetest"){

steps{

sh"docker-compose-fdocker-compose.yml

-facceptance/docker-compose-acceptance.ymlbuildtest"

sh"docker-compose-fdocker-compose.yml

-facceptance/docker-compose-acceptance.yml

-pacceptanceup-d"

sh'test$(dockerwaitacceptance_test_1)-eq0'

}

}

Thistime,wefirstbuildthetestservice.Thereisnoneedtobuildthecalculatorimage;it'salreadydonebythepreviousstages.Intheend,weshouldcleanuptheenvironment:

post{

always{

sh"docker-compose-fdocker-compose.yml

-facceptance/docker-compose-acceptance.yml

-pacceptancedown"

}

}

AfteraddingthistoJenkinsfile,we'redonewiththesecondmethod.WecantestthisbypushingallthechangestoGitHub.

Comparingmethod1andmethod2Tosumup,let'scomparebothsolutions.Thefirstapproachistherealblack-boxtestingfromtheuserperspectiveinwhichJenkinsplaystheroleofauser.Theadvantageisthatit'sveryclosetowhatwillbedoneinproduction;intheend,wewillaccesscontainersviaitsDockerhost.Thesecondapproachteststheapplicationfromtheinsideofanothercontainer.Thesolutionissomehowmoreelegantandcanberunlocallyinasimpleway;however,itrequiresmorefilestocreateanddoesnotcalltheapplicationviaitsDockerhostlikeitwillbelaterdoneinproduction.

Inthenextsection,westepawayfromDockerandJenkinsandtakeacloserlookattheprocessofwritingtheacceptanceteststhemselves.

WritingacceptancetestsSofar,weusedthecurlcommandtoperformasuiteofacceptancetests.Thatisobviouslyaconsiderablesimplification.Technicallyspeaking,ifwewriteaRESTwebservice,thenwecouldwriteallblack-boxtestsasabigscriptwithanumberof"curl"calls.Thissolutionwouldbe,however,verydifficulttoread,understand,andmaintain.What'smore,thescriptwouldbecompletelyincomprehensiblebynon-technical,business-relatedusers.Howtoaddressthisissueandcreatetestswithagoodstructure,readablebyusers,andmeetitsfundamentalgoal:automaticallycheckingifthesystemisasexpected?Iwillanswerthisquestionthroughoutthissection.

Writinguser-facingtestsAcceptancetestsarewrittenwithusersandshouldbecomprehensibletousers.Thisiswhythechoiceofamethodforwritingthemdependsonwhothecustomeris.

Forexample,imagineapurelytechnicalperson.Ifyouwriteawebservicethatoptimizesthedatabasestoring,andyoursystemisusedonlybyothersystemsandreadonlybyotherdevelopers,thenyourtestscanbeexpressedinthesamewayasunittests.Asarule,thetestisgoodifunderstoodbyboth,developeranduser.

Inreallife,mostsoftwareiswrittentodeliveraspecificbusinessvalue,andthatbusinessvalueisdefinedbynon-developers.Therefore,weneedacommonlanguagetocollaborate.Ononeside,thereisthebusinesswhounderstandswhatisneededbutnothowtodoit;ontheotherside,thedevelopmentteamwhoknowshowbutdoesn'tknowwhat.Luckily,thereareanumberofframeworksthathelptoconnectthesetwoworlds,forinstance,Cucumber,FitNesse,JBehave,Capybara,andmanymore.Theydifferfromeachother,andeachofthemmaybeasubjectforaseparatebook;however,thegeneralideaofwritingacceptancetestsisthesameandcanbepresentedinthefollowingfigure:

TheAcceptanceCriteriaarewrittenbyusers(oraproductownerastheirrepresentative)withthehelpofdevelopers.Theyareusuallywrittenintheformofthefollowingscenarios:

GivenIhavetwonumbers:1and2

Whenthecalculatorsumsthem

ThenIreceive3asaresult

Developerswritethetestingimplementationcalledfixturesorstepdefinitionsthatintegratesthehuman-friendlyDSLspecificationwiththeprogramminglanguage.Asaresult,wehaveanautomatedtestthatcanbewell-integratedintotheContinuousDeliverypipeline.

Needlesstoadd,writingacceptancetestsisacontinuousagileprocess,notawaterfallone.Itrequiresconstantcollaborationduringwhichthetestspecificationsareimprovedandmaintainedbyboth,developersandbusiness.

Inthecaseofanapplicationwithauserinterface,itcanbetemptingtoperformtheacceptancetestingdirectlyviatheinterface(forexample,byrecordingSeleniumscripts);however,thisapproachwhennotdoneproperlycanleadtoteststhatareslowandtightlycoupledtotheinterfacelayer.

Let'sseehowwritingacceptancetestslookinpracticeandhowtobindthemtotheContinuousDeliverypipeline.

UsingtheacceptancetestingframeworkLet'susetheCucumberframeworkandcreateanacceptancetestforthecalculatorproject.Aspreviouslydescribed,wewilldothisinthreesteps:

CreatingacceptancecriteriaCreatingstepdefinitionsRunninganautomatedacceptancetest

Feature:Calculator<br/>Scenario:Sumtwonumbers<br/>GivenIhavetwonumbers:1and2<br/>Whenthecalculatorsumsthem<br/>ThenIreceive3asaresult

Thisfileshouldbecreatedbyuserswiththehelpofdevelopers.Notethatitiswritteninawaythatnon-technicalpeoplecanunderstandit.

packageacceptance;<br/><br/>importcucumber.api.java.en.Given;<br/>importcucumber.api.java.en.Then;<br/>importcucumber.api.java.en.When;<br/>importorg.springframework.web.client.RestTemplate;<br/><br/>importstaticorg.junit.Assert.assertEquals;<br/><br/>/**Stepsdefinitionsforcalculator.feature*/<br/>publicclassStepDefinitions{<br/>privateStringserver=System.getProperty("calculator.url");<br/><br/>privateRestTemplaterestTemplate=newRestTemplate();<br/><br/>privateStringa;<br/>privateStringb;<br/>privateStringresult;<br/><br/>@Given("^Ihavetwonumbers:(.*)and(.*)$")<br/>publicvoidi_have_two_numbers(Stringa,Stringb)throwsThrowable{<br/>this.a=a;<br/>this.b=b;<br/>}<br/><br/>@When("^thecalculatorsumsthem$")<br/>publicvoidthe_calculator_sums_them()throwsThrowable{<br/>Stringurl=String.format("%s/sum?a=%s&b=%s",server,a,b);<br/>result=restTemplate.getForObject(url,String.class);<br/>}<br/><br/>@Then("^Ireceive(.*)asaresult$")<br/>publicvoidi_receive_as_a_result(StringexpectedResult)throwsThrowable{<br/>assertEquals(expectedResult,result);<br/>}<br/>}

Eachline(Given,When,andThen)fromthefeaturespecificationfileismatchedbyregularexpressionswiththecorrespondingmethodintheJavacode.Thewildcards(.*)arepassedasparameters.NotethattheserveraddressispassedastheJavapropertycalculator.url.Themethodperformsthefollowingactions:

i_have_two_numbers:Savesparametersasfieldsthe_calculator_sums_them:Callstheremotecalculatorserviceandstorestheresultinafieldi_receive_as_a_result:Assertsthattheresultisasexpected

RunninganautomatedacceptancetestTorunanautomatedtest,weneedtomakeafewconfigurations:

1. AddJavacucumberlibraries:Inthebuild.gradlefile,addthefollowingcodetothedependenciessection:

testCompile("info.cukes:cucumber-java:1.2.4")

testCompile("info.cukes:cucumber-junit:1.2.4")

2. AddGradletarget:Inthesamefile,addthefollowingcode:

taskacceptanceTest(type:Test){

include'**/acceptance/**'

systemPropertiesSystem.getProperties()

}

test{

exclude'**/acceptance/**'

}

Thissplitsthetestsintounit(runwith./gradlewtest)andacceptance(runwith./gradlewacceptanceTest).

3. AddJUnitrunner:Addanewfilesrc/test/java/acceptance/AcceptanceTest.java:

packageacceptance;

importcucumber.api.CucumberOptions;

importcucumber.api.junit.Cucumber;

importorg.junit.runner.RunWith;

/**AcceptanceTest*/

@RunWith(Cucumber.class)

@CucumberOptions(features="classpath:feature")

publicclassAcceptanceTest{}

Thisistheentrypointtotheacceptancetestsuite.

Afterthisconfiguration,iftheserverisrunningonthelocalhost,wecantestitbyexecutingthefollowingcode:

$./gradlewacceptanceTest-Dcalculator.url=http://localhost:8080

Obviously,wecanaddthiscommandtoouracceptance_test.shinsteadofthecurlcommand.ThiswouldmaketheCucumberacceptancetestrunintheJenkinspipeline.

Acceptancetest-drivendevelopmentAcceptancetests,likemostaspectsoftheContinuousDeliveryprocess,arelessabouttechnologyandmoreaboutpeople.Thetestqualitydependson,ofcourse,theengagementofusersanddevelopers,butalso,whatismaybelessintuitive,thetimewhenthetestsarecreated.

Thelastquestiontoaskis,duringwhichphaseofthesoftwaredevelopmentlifecycleshouldtheacceptancetestsbeprepared?Ortorephraseit,shouldwecreateacceptancetestsbeforeorafterwritingthecode?

Technicallyspeaking,theresultisthesame;thecodeiswell-coveredwithboth,unitandacceptancetests.However,it'stemptingtoconsiderwritingtestsfirst.TheideaofTDD(test-drivendevelopment)canbewelladaptedforacceptancetesting.Ifunittestsarewrittenbeforethecode,theresultcodeiscleanerandbetterstructured.Analogously,ifacceptancetestsarewrittenbeforethesystemfeature,theresultingfeaturecorrespondsbettertothecustomer'srequirements.Thisprocess,oftencalledacceptancetest-drivendevelopment,ispresentedinthefollowingfigure:

Users,withdevelopers,writetheacceptancecriteriaspecificationinthehuman-friendlyDSLformat.Developerswritethefixturesandthetestsfail.Then,thefeaturedevelopmentstartsusingtheTDDmethodologyinternally.Afterthe

featureiscompleted,theacceptancetestshouldpass,andthisisasignthatthefeatureiscompleted.

AverygoodpracticeistoattachtheCucumberfeaturespecificationtotherequestticketintheissuetrackingtool(forexample,JIRA)sothatthefeaturewouldbealwaysrequestedtogetherwithitsacceptancetest.Somedevelopmentteamstakeanevenmoreradicalapproachandrefusetostartthedevelopmentprocessifnoacceptancetestsareprepared.Thereisalotofsenseinthat,afterall,howcanyoudevelopsomethingthattheclientcan'ttest?

Exercises

Wecoveredalotofnewmaterialthroughoutthischapter,sotobetterunderstand,werecommenddoingtheexercisesandcreatingyourownprojectwithacceptancetests:

1. CreateaRuby-basedwebservicebook-librarytostorebooks:

TheacceptancecriteriaisdeliveredintheformofthefollowingCucumberfeature:

Scenario:Storebookinthelibrary

Given:Book"TheLordoftheRings"by"J.R.R.Tolkien"withISBNnumber

"0395974682"

When:Istorethebookinlibrary

Then:IamabletoretrievethebookbytheISBNnumber

WritestepdefinitionsfortheCucumbertestWritethewebservice(thesimplestistousetheSinatraframework:http://www.sinatrarb.com/,butyoucanalsouseRubyonRails).Thebookshouldhavethefollowingattributes:name,author,andISBN.Thewebserviceshouldhavethefollowingendpoints:

POST"/books/"toaddabookGET"books/<isbn>"toretrievethebook

Thedatacanbestoredinthememory.Intheend,checkiftheacceptancetestisgreen.

2. Add"book-library"asaDockerimagetotheDockerregistry:CreateanaccountonDockerHub.CreateDockerfilefortheapplication.BuildtheDockerimageandtagitaccordingtothenamingconvention.PushtheimagetoDockerHub.

3. CreatetheJenkinspipelinetobuildDockerimage,pushittotheDockerregistry,andperformacceptancetesting:

Createa"Dockerbuild"stage.CreatetheDockerloginandDockerpushstages.CreateatestcontainerthatperformsacceptancetestinganduseDockerComposetoperformthetest.AddanAcceptanceteststagetothepipeline.Runthepipelineandobservetheresult.

SummaryInthischapter,youlearnedhowtobuildacompleteandfunctionalacceptanceteststage,whichisanessentialpartoftheContinuousDeliveryprocess.Thekeytakeawayfromthechapter:

Acceptancetestscanbedifficulttocreatebecausetheycombinetechnicalchallenges(applicationdependencies,environmentsetup)withpersonalchallenges(developers-businesscollaboration).Acceptancetestingframeworksprovideawaytowritetestsinahuman-friendlylanguagethatmakesthemcomprehensibletonon-technicalpeople.DockerregistryisanartifactrepositoryforDockerimages.DockerregistryfitswellwiththeContinuousDeliveryprocessbecauseitprovidesawaytouseexactlythesameDockerimagethroughoutthestagesandenvironments.DockerComposeorchestratesagroupofDockercontainerinteractingtogether.Itcanalsobuildimagesandscalecontainers.DockerComposecanhelpwithsettingupacompleteenvironmentbeforerunningasuiteofacceptancetests.AcceptancetestscanbewrittenasaDockerimage,andDockerComposecanrunthecompleteenvironmenttogetherwiththetestsandprovidetheresults.

Inthenextchapter,wewillcoverthemissingstagesnecessarytocompletetheContinuousDeliverypipeline.

ConfigurationManagementwithAnsible

WehavealreadycoveredthetwomostcrucialphasesoftheContinuousDeliveryprocess:thecommitphaseandtheautomatedacceptancetesting.Inthischapter,wewillfocusontheconfigurationmanagement,whichconnectsthevirtualcontainerizedenvironmenttotherealserverinfrastructure.

Thischaptercoversthefollowingpoints:

IntroducingtheconceptofconfigurationmanagementExplainingthemostpopularconfigurationmanagementtoolsDiscussingAnsiblerequirementsandtheinstallationprocessUsingAnsiblewithadhoccommandsShowingthepowerofAnsibleautomationwithplaybooksExplainingAnsiblerolesandAnsibleGalaxyImplementingausecaseofthedeploymentprocessUsingAnsibletogetherwithDockerandDockerCompose

IntroducingconfigurationmanagementConfigurationmanagementisaprocessofcontrollingconfigurationchangesinawaythatthesystemmaintainsintegrityovertime.EventhoughthetermdidnotoriginateintheITindustry,currentlyitisbroadlyusedtorefertothesoftwareandthehardware.Inthiscontext,itconcernsthefollowingaspects:

Applicationconfiguration:Thisinvolvessoftwarepropertiesthatdecidehowthesystemworks,whichareusuallyexpressedintheformofflagsorpropertiesfilespassedtotheapplication,forexample,thedatabaseaddress,themaximumchunksizeforfileprocessing,orthelogginglevel.Theycanbeappliedduringdifferentdevelopmentphases:build,package,deploy,orrun.Infrastructureconfiguration:Thisinvolvesserverinfrastructureandenvironmentconfiguration,whichtakescareofthedeploymentprocess.Itdefineswhatdependenciesshouldbeinstalledoneachserverandspecifiesthewayapplicationsareorchestrated(whichapplicationisrunonwhichserverandinhowmanyinstances).

Asanexample,wecanthinkofthecalculatorwebservice,whichusestheRedisserver.Let'slookatthediagrampresentinghowtheconfigurationmanagementtoolworks.

Theconfigurationmanagementtoolreadstheconfigurationfileandpreparestheenvironmentrespectively(installsdependenttoolsandlibraries,deploystheapplicationstomultipleinstances).

Intheprecedingexample,theInfrastructureConfigurationspecifiesthattheCalculatorserviceshouldbedeployedintwoinstancesonServer1andServer2andthattheRedisserviceshouldbeinstalledonServer3.CalculatorApplicationConfigurationspecifiestheportandtheaddressoftheRedisserversothattheservicescancommunicate.

Configurationcandifferdependingonthetypeoftheenvironment(QA,staging,production),forexample,serveraddressescanbedifferent.

Therearemanyapproachestoconfigurationmanagement,butbeforewelookintoconcretesolutions,let'scommentonwhatcharacteristicsagoodconfigurationmanagementtoolshouldhave.

TraitsofgoodconfigurationmanagementWhatshouldthemodernconfigurationmanagementsolutionlooklike?Let'swalkthroughthemostimportantfactors:

Automation:Eachenvironmentshouldbeautomaticallyreproducible,includingtheoperatingsystem,thenetworkconfiguration,thesoftwareinstalled,andtheapplicationsdeployed.Insuchanapproach,fixingproductionissuesmeansnothingmorethananautomaticrebuildoftheenvironment.What'smore,thatsimplifiesserverreplicationsandensuresthatthestagingandproductionenvironmentsareexactlythesame.Versioncontrol:Everychangeintheconfigurationshouldbetracked,sothatweknowwhomadeit,why,andwhen.Usually,thatmeanskeepingtheconfigurationinthesourcecoderepositoryeithertogetherwiththecodeorinaseparateplace.Theformersolutionisrecommendedbecauseconfigurationpropertieshaveadifferentlifecyclethantheapplicationitself.Versioncontrolalsohelpswithfixingproductionissues—theconfigurationcanalwaysberolledbacktothepreviousversionandtheenvironmentautomaticallyrebuilt.Theonlyexceptiontotheversioncontrol-basedsolutionisstoringcredentialsandothersensitiveinformation-theseshouldbenevercheckedin.Incrementalchanges:Applyingachangeintheconfigurationshouldnotrequirerebuildingthewholeenvironment.Onthecontrary,asmallchangeintheconfigurationshouldchangeonlytherelatedpartoftheinfrastructure.Serverprovisioning:Thankstoautomation,addinganewservershouldbeasquickasaddingitsaddresstotheconfiguration(andexecutingonecommand).Security:Theaccesstoboth,theconfigurationmanagementtoolandthemachinesunderitscontrol,shouldbewellsecured.WhenusingtheSSHprotocolforcommunication,theaccesstothekeysorcredentialsneedstobewellprotected.Simplicity:Everymemberoftheteamshouldbeabletoreadthe

configuration,makeachange,andapplyittotheenvironment.Thepropertiesthemselvesshouldalsobekeptassimpleaspossibleandtheonesthatarenotsubjectedtochangearebetteroffkepthardcoded.

Itisimportanttokeepthesepointsinmindwhilecreatingtheconfigurationand,evenbefore,whilechoosingtherightconfigurationmanagementtool.

OverviewofconfigurationmanagementtoolsThemostpopularconfigurationmanagementtoolsareAnsible,Puppet,andChef.Eachofthemisagoodchoice;theyareallopensourceproductswithfreebasicversionsandpaidenterpriseeditions.Themostimportantdifferencesbetweenthemare:

ConfigurationLanguage:ChefusesRuby,PuppetusesitsownDSL(basedonRuby),andAnsibleusesYAML.Agent-based:PuppetandChefuseagentsforcommunication,whichmeansthateachmanagedserverneedstohaveaspecialtoolinstalled.Ansible,onthecontrary,isagentlessandusesthestandardSSHprotocolforcommunication.

Theagentlessfeatureisasignificantadvantagebecauseitimpliesnoneedtoinstallanythingonservers.What'smore,Ansibleisquicklytrendingupwards,whichiswhyitwaschosenforthisbook.Nevertheless,othertoolscanalsobesuccessfullyusedfortheContinuousDeliveryprocess.

InstallingAnsibleAnsibleisanopensource,agentlessautomationengineforsoftwareprovisioning,configurationmanagement,andapplicationdeployment.Itsfirstreleasewasin2012anditsbasicversionisfreeforboth,personalandcommercialuse.Theenterpriseversion,calledAnsibleTower,providesGUImanagementanddashboards,RESTAPI,role-basedaccesscontrol,andsomemorefeatures.

WepresenttheinstallationprocessandadescriptionofhowitcanbeusedseparatelyaswellastogetherwithDocker.

AnsibleserverrequirementsAnsibleusestheSSHprotocolforcommunicationandhasnospecialrequirementsregardingthemachineitmanages.Thereisalsonocentralmasterserver,soit'senoughtoinstalltheAnsibleclienttoolanywhereandwecanalreadyuseittomanagethewholeinfrastructure.

TheonlyrequirementforthemachinesbeingmanagedistohavethePythontooland,obviously,theSSHserverinstalled.Thesetoolsare,however,almostalwaysavailablebydefaultonanyserver.

AnsibleinstallationTheinstallationinstructionsdifferdependingontheoperatingsystem.InthecaseofUbuntu,it'senoughtorunthefollowingcommands:

$sudoapt-getinstallsoftware-properties-common

$sudoapt-add-repositoryppa:ansible/ansible

$sudoapt-getupdate

$sudoapt-getinstallansible

YoucanfindtheinstallationguidesforalloperatingsystemsontheofficialAnsiblepageat:http://docs.ansible.com/ansible/intro_installation.html.

Aftertheinstallationprocessiscompleted,wecanexecutetheAnsiblecommandtocheckthateverythingwasinstalledsuccessfully.

$ansible--version

ansible2.3.2.0

configfile=/etc/ansible/ansible.cfg

configuredmodulesearchpath=Defaultw/ooverrides

<strong>$dockerrunwilliamyeh/ansible:ubuntu14.04</strong><br/><strong>ansible-playbook2.3.2.0</strong><br/><strong>configfile=/etc/ansible/ansible.cfg</strong><br/><strong>configuredmodulesearchpath=Defaultw/ooverrides</strong>

TheAnsibleDockerimageisnolongerofficiallysupported,sotheonlysolutionistousethecommunity-drivenversion.YoucanreadmoreonitsusageontheDockerHubpage.

UsingAnsibleInordertouseAnsible,firstweneedtodefinetheinventory,whichrepresentstheavailableresources.Then,wewillbeabletoeitherexecuteasinglecommandordefineasetoftasksusingtheAnsibleplaybook.

CreatinginventoryAninventoryisalistofalltheserversthataremanagedbyAnsible.EachserverrequiresnothingmorethanthePythoninterpreterandtheSSHserverinstalled.Bydefault,AnsibleassumesthattheSSHkeysareusedforauthentication;however,itisalsopossibletousetheusernameandthepasswordbyaddingthe--ask-passoptiontotheAnsiblecommands.

SSHkeyscanbegeneratedwiththessh-keygentoolandareusuallystoredinthe~/.sshdirectory.

Theinventoryisdefinedinthe/etc/ansible/hostsfileandithasthefollowingstructure:

[group_name]

<server1_address>

<server2_address>

...

Theinventorysyntaxalsoacceptsrangesofservers,forexample,www[01-22].company.com.TheSSHportshouldalsobespecifiedifit'sanythingotherthan22(thedefaultone).YoucanreadmoreontheofficialAnsiblepageat:http://docs.ansible.com/ansible/intro_inventory.html.

Theremaybe0ormanygroupsintheinventoryfile.Asanexample,let'sdefinetwomachinesinonegroupofservers.

[webservers]

192.168.0.241

192.168.0.242

Wecanalsocreatetheconfigurationwithserveraliasesandspecifytheremoteuser:

[webservers]

web1ansible_host=192.168.0.241ansible_user=admin

web2ansible_host=192.168.0.242ansible_user=admin

Theprecedingfiledefinesagroupcalledwebservers,whichconsistsoftwoservers.TheAnsibleclientwillloginastheuseradmintobothofthem.Whenwe

havetheinventorycreated,let'sdiscoverhowwecanuseittoexecutethesamecommandonmanyservers.

Ansibleoffersapossibilitytodynamicallypulltheinventoryfromthecloudprovider(forexample,AmazonEC2/Eucalyptus),LDAP,orCobbler.Readmoreaboutdynamicinventoriesat:http://docs.ansible.com/ansible/intro_dynamic_inventory.html.

AdhoccommandsThesimplestcommandwecanrunisapingonallservers.

$ansibleall-mping

web1|SUCCESS=>{

"changed":false,

"ping":"pong"

}

web2|SUCCESS=>{

"changed":false,

"ping":"pong"

}

Weusedthe-m<module_name>option,whichallowsspecifyingthemodulethatshouldbeexecutedontheremotehosts.Theresultissuccessful,whichmeansthattheserversarereachableandtheauthenticationisconfiguredcorrectly.

AfulllistofmodulesavailableinAnsiblecanbefoundonthepage:http://docs.ansible.com/ansible/modules.html.

Notethatweusedall,sothatallserverswouldbeaddressed,butwecouldalsocallthembythegroupnamewebserversorbythesinglehostalias.Asasecondexample,let'sexecuteashellcommandonlyononeoftheservers.

$ansibleweb1-a"/bin/echohello"

web1|SUCCESS|rc=0>>

hello

The-a<arguments>optionspecifiestheargumentsthatarepassedtotheAnsiblemodule.Inthiscase,wedidn'tspecifythemodule,sotheargumentsareexecutedasashellUnixcommand.Theresultwassuccessfulandhellowasprinted.

Iftheansiblecommandisconnectingtotheserverforthefirsttime(ortheserverisreinstalled),thenwearepromptedwiththekeyconfirmationmessage(SSHmessagewhenthehostisnotpresentinknown_hosts).Sinceitmayinterruptanautomatedscript,wecandisablethepromptmessagebyuncommentinghost_key_checking=Falseinthe/etc/ansible/ansible.cfgfileorbysettingtheenvironment

variableANSIBLE_HOST_KEY_CHECKING=False.

Initssimplisticform,theAnsibleadhoccommandsyntaxlooksasfollows:

ansible<target>-m<module_name>-a<module_arguments>

Thepurposeofadhoccommandsistodosomethingquicklywhenitisnotnecessarytorepeatit.Forexample,wemaywanttocheckifaserverisaliveortopoweroffallthemachinesfortheChristmasbreak.Thismechanismcanbeseenasacommandexecutiononagroupofmachineswiththeadditionalsyntaxsimplificationprovidedbythemodules.TherealpowerofAnsibleautomation,however,liesinplaybooks.

PlaybooksAnAnsibleplaybookisaconfigurationfile,whichdescribeshowserversshouldbeconfigured.Itprovidesawaytodefineasequenceoftasksthatshouldbeperformedoneachofthemachines.AplaybookisexpressedintheYAMLconfigurationlanguage,whichmakesithuman-readableandeasytounderstand.Let'sstartwithasampleplaybookandthenseehowwecanuseit.

DefiningaplaybookAplaybookiscomposedofoneormanyplays.Eachplaycontainsahostgroupname,taskstoperform,andconfigurationdetails(forexample,remoteusernameoraccessrights).Anexampleplaybookmightlooklikethis:----hosts:web1become:yesbecome_method:sudotasks:-name:ensureapacheisatthelatestversionapt:name=apache2state=latest-name:ensureapacheisrunningservice:name=apache2state=startedenabled=yes

Thisconfigurationcontainsoneplaywhich:

Isexecutedonlyonthehostweb1GainsrootaccessusingthesudocommandExecutestwotasks:

Installingthelatestversionofapache2:TheAnsiblemoduleapt(calledwithtwoparametersname=apache2andstate=latest)checkswhethertheapache2packageisinstalledontheserver,andifnot,thenitusestheapt-gettooltoinstallapache2Runningtheapache2service:TheAnsiblemoduleservice(calledwiththreeparametersname=apache2,state=started,andenabled=yes)checkswhethertheUnixserviceapache2isstarted,andifnot,itusestheservicecommandtostartit

Whileaddressingthehosts,youcanalsousepatterns,forexample,wecoulduseweb*toaddressbothweb1andweb2.YoucanreadmoreaboutAnsiblepatternsat:http://docs.ansible.com/ansible/intro_patterns.html.

Notethateachtaskhasahuman-readablename,whichisusedintheconsoleoutputsuchthataptandserviceareAnsiblemodulesandname=apache2,state=latest,andstate=startedaremodulearguments.WehavealreadyseenAnsiblemodules

andargumentswhileusingadhoccommands.Intheprecedingplaybook,wedefinedonlyoneplay,buttherecanbemanyofthemandeachcanberelatedtodifferentgroupsofhosts.

Forexample,wecoulddefinetwogroupsofserversintheinventory:databaseandwebservers.Then,intheplaybook,wecouldspecifytasksthatshouldbeexecutedonalldatabase-hostingmachinesandsomedifferenttasksthatshouldbeexecutedonallthewebservers.Byusingonecommand,wecouldsetupthewholeenvironment.

ExecutingtheplaybookWhenplaybook.ymlisdefined,wecanexecuteitusingtheansible-playbookcommand.

$ansible-playbookplaybook.yml

PLAY[web1]***************************************************************

TASK[setup]**************************************************************

ok:[web1]

TASK[ensureapacheisatthelatestversion]*****************************

changed:[web1]

TASK[ensureapacheisrunning]*******************************************

ok:[web1]

PLAYRECAP****************************************************************

web1:ok=3changed=1unreachable=0failed=0

Iftheserverrequiresenteringthepasswordforthesudocommand,thenweneedtoaddthe--ask-sudo-passoptiontotheansible-playbookcommand.It'salsopossibletopassthesudopassword(ifrequired)bysettingtheextravariable-eansible_become_pass=<sudo_password>.

Theplaybookconfigurationwasexecuted,and,therefore,theapache2toolwasinstalledandstarted.Notethatifthetaskchangedsomethingontheserver,itismarkedaschanged.Onthecontrary,iftherewasnochange,it'smarkedasok.

Itispossibletoruntasksinparallelusingthe-f<num_of_threads>option.

Playbook'sidempotencyWecanexecutethecommandagain.

$ansible-playbookplaybook.yml

PLAY[web1]***************************************************************

TASK[setup]**************************************************************

ok:[web1]

TASK[ensureapacheisatthelatestversion]*****************************

ok:[web1]

TASK[ensureapacheisrunning]*******************************************

ok:[web1]

PLAYRECAP****************************************************************

web1:ok=3changed=0unreachable=0failed=0

Notethattheoutputisslightlydifferent.Thistimethecommanddidn'tchangeanythingontheserver.ThatisbecauseeachAnsiblemoduleisdesignedtobeidempotent.Inotherwords,executingthesamemodulemanytimesinasequenceshouldhavethesameeffectasexecutingitonlyonce.

Thesimplestwaytoachieveidempotencyistoalwaysfirstcheckifthetaskhasn'tbeenexecutedyet,andexecuteitonlyifithasn't.IdempotencyisapowerfulfeatureandweshouldalwayswriteourAnsibletasksthisway.

Ifalltasksareidempotent,thenwecanexecutethemasmanytimesaswewant.Inthatcontext,wecanthinkoftheplaybookasadescriptionofthedesiredstateofremotemachines.Then,theansible-playbookcommandtakescareofbringingthemachine(orgroupofmachines)intothatstate.

HandlersSomeoperationsshouldbeexecutedonlyifsomeothertaskchanged.Forexample,imaginethatyoucopytheconfigurationfiletotheremotemachineandtheApacheservershouldberestartedonlyiftheconfigurationfilehaschanged.Howtoapproachsuchacase?

Forexample,imaginethatyoucopytheconfigurationfiletotheremotemachineandtheApacheservershouldberestartedonlyiftheconfigurationfilehaschanged.Howtoapproachsuchacase?

Ansibleprovidesanevent-orientedmechanismtonotifyaboutthechanges.Inordertouseit,weneedtoknowtwokeywords:

handlers:Thisspecifiesthetasksexecutedwhennotifiednotify:Thisspecifiesthehandlersthatshouldbeexecuted

Let'slookatanexampleofhowwecouldcopytheconfigurationtotheserverandrestartApacheonlyiftheconfigurationhaschanged.

tasks:

-name:copyconfiguration

copy:

src:foo.conf

dest:/etc/foo.conf

notify:

-restartapache

handlers:

-name:restartapache

service:

name:apache2

state:restarted

Now,wecancreatethefoo.conffileandruntheansible-playbookcommand.

$touchfoo.conf

$ansible-playbookplaybook.yml

...

TASK[copyconfiguration]************************************************

changed:[web1]

RUNNINGHANDLER[restartapache]*****************************************

changed:[web1]

PLAYRECAP***************************************************************

web1:ok=5changed=2unreachable=0failed=0

Handlersareexecutedalwaysattheendoftheplayandonlyonce,eveniftriggeredbymultipletasks.

AnsiblecopiedthefileandrestartedtheApacheserver.It'simportanttounderstandthatifwerunthecommandagain,nothingwillhappen.However,ifwechangethecontentofthefoo.conffileandthenruntheansible-playbookcommand,thefilewillbecopiedagain(andtheApacheserverwillberestarted).

$echo"something">foo.conf

$ansible-playbookplaybook.yml

...

TASK[copyconfiguration]*************************************************

changed:[web1]

RUNNINGHANDLER[restartapache]******************************************

changed:[web1]

PLAYRECAP****************************************************************

web1:ok=5changed=2unreachable=0failed=0

Weusedthecopymodule,whichissmartenoughtodetectifthefilehaschanged,andtheninsuchacase,makeachangeontheserver.

Thereisalsoapublish-subscribemechanisminAnsible.Usingitmeansassigningatopictomanyhandlers.Then,atasknotifiesthetopictoexecuteallrelatedhandlers.Youcanreadmoreaboutitat:http://docs.ansible.com/ansible/playbooks_intro.html.

VariablesWhiletheAnsibleautomationmakesthingsidenticalandrepeatableformultiplehosts,itisinevitablethatserversmayrequiresomedifferences.Forexample,thinkoftheapplicationportnumber.Itcanbedifferentdependingonthemachine.Luckily,Ansibleprovidesvariables,whichisagoodmechanismtodealwithserverdifferences.Let'screateanewplaybookanddefineavariable.

Forexample,thinkoftheapplicationportnumber.Itcanbedifferentdependingonthemachine.Luckily,Ansibleprovidesvariables,whichisagoodmechanismtodealwithserverdifferences.Let'screateanewplaybookanddefineavariable.

---

-hosts:web1

vars:

http_port:8080

Theconfigurationdefinesthehttp_portvariablewiththevalue8080.Now,wecanuseitusingtheJinja2syntax.

tasks:

-name:printportnumber

debug:

msg:"Portnumber:{{http_port}}"

TheJinja2languageallowsdoingwaymorethanjustgettingavariable.Wecanuseittocreateconditions,loops,andmanymore.YoucanfindmoredetailsontheJinjapageat:http://jinja.pocoo.org/.

Thedebugmoduleprintsthemessagewhileexecuting.Ifweruntheansible-playbookcommand,wecanseethevariableusage.

$ansible-playbookplaybook.yml

...

TASK[printportnumber]**************************************************

ok:[web1]=>{

"msg":"Portnumber:8080"

}

Variablescanalsobedefinedintheinventoryfilingusingthe

[group_name:vars]section.Youcanreadmoreaboutitat:http://docs.ansible.com/ansible/intro_inventory.html#host-variables.

Apartfromuser-definedvariables,therearealsopredefinedautomaticvariables.Forexample,thehostvarsvariablestoresamapwiththeinformationregardingallhostsfromtheinventory.UsingtheJinja2syntax,wecoulditerateandprinttheIPaddressesofallhostsintheinventory.

---

-hosts:web1

tasks:

-name:printIPaddress

debug:

msg:"{%forhostingroups['all']%}{{

hostvars[host]['ansible_host']}}{%endfor%}"

Then,wecanexecutetheansible-playbookcommand.

$ansible-playbookplaybook.yml

...

TASK[printIPaddress]**************************************************

ok:[web1]=>{

"msg":"192.168.0.241192.168.0.242"

}

NotethatwiththeuseoftheJinja2language,wecanspecifytheflowcontroloperationsinsidetheAnsibleplaybookfile.

AnalternativetotheJinja2templatinglanguage,fortheconditionalsandloops,istousetheAnsiblebuilt-inkeywords:whenandwith_items.Youcanreadmoreaboutitat:http://docs.ansible.com/ansible/playbooks_conditionals.html.

RolesWecaninstallanytoolontheremoteserverusingAnsibleplaybooks.ImaginewewouldliketohaveaserverwithMySQL.Wecouldeasilyprepareaplaybooksimilartotheonewiththeapache2package.However,ifyouthinkaboutit,aserverwithMySQLisquiteacommoncaseand,forsure,someonehasalreadypreparedaplaybookforit,somaybewecouldjustreuseit?HerecomesAnsiblerolesandAnsibleGalaxy.

UnderstandingrolesTheAnsibleroleisawell-structuredplaybookpartpreparedtobeincludedintheplaybooks.Rolesareseparateunitsthatalwayshavethefollowingdirectorystructure:

templates/

tasks/

handlers/

vars/

defaults/

meta/

YoucanreadmoreaboutrolesandwhateachdirectorymeansontheofficialAnsiblepageat:http://docs.ansible.com/ansible/playbooks_roles.html.

Ineachofthedirectories,wecandefinethemain.ymlfile,whichcontainstheplaybookpartsthatcanbeincludedintheplaybook.ymlfile.ContinuingtheMySQLcase,thereisaroledefinedonGitHub:https://github.com/geerlingguy/ansible-role-mysql.Thisrepositorycontainskind-oftasktemplatesthatcanbeusedinourplaybook.Let'slookatapartofthetasks/main.ymlfile,whichinstallsthemysqlpackage.

...

-name:EnsureMySQLPythonlibrariesareinstalled.

apt:"name=python-mysqldbstate=installed"

-name:EnsureMySQLpackagesareinstalled.

apt:"name={{item}}state=installed"

with_items:"{{mysql_packages}}"

register:deb_mysql_install_packages

...

Thisisonlyoneofthetasksdefinedinthetasks/main.ymlfile.OthersareresponsiblefortheMySQLconfiguration.

Thewith_itemskeywordisusedtocreatealoopoveralltheitems.Thewhenkeywordmeansthatthetaskisexecutedonlyunderacertaincondition.

Ifweusethisrole,theninordertoinstalltheMySQLontheserver,it'senoughtocreatethefollowingplaybook.yml:

---

-hosts:all

become:yes

become_method:sudo

roles:

-role:geerlingguy.mysql

become:yes

SuchconfigurationinstallstheMySQLdatabasetoallserversusingthegeerlingguy.mysqlrole.

AnsibleGalaxyAnsibleGalaxyistoAnsiblewhatDockerHubisforDocker-itstorescommonroles,sothattheycanbereusedbyothers.YoucanbrowsetheavailablerolesontheAnsibleGalaxypageat:https://galaxy.ansible.com/.

ToinstalltherolefromAnsibleGalaxy,wecanusetheansible-galaxycommand.

$ansible-galaxyinstallusername.role_name

Thiscommandautomaticallydownloadstherole.InthecaseoftheMySQLexample,wecoulddownloadtherolebyexecuting:$ansible-galaxyinstallgeerlingguy.mysql

Thecommanddownloadsthemysqlrole,whichcanbelaterusedintheplaybookfile.

Ifyouneedtoinstallalotofrolesatthesametime,youcandefinethemintherequirements.ymlfileanduseansible-galaxyinstall-rrequirements.yml.ReadmoreaboutthatapproachandaboutAnsibleGalaxyat:http://docs.ansible.com/ansible/galaxy.html.

DeploymentwithAnsibleWehavecoveredthemostfundamentalfeaturesofAnsible.Let'snowforget,justforalittlewhile,aboutDockerandconfigureacompletedeploymentstepusingAnsible.WewillrunthecalculatorserviceononeserverandtheRedisserviceonthesecondserver.

InstallingRedisWecanspecifyaplayinthenewplaybook.Let'screatetheplaybook.ymlfilewiththefollowingcontent:

---

-hosts:web1

become:yes

become_method:sudo

tasks:

-name:installRedis

apt:

name:redis-server

state:present

-name:startRedis

service:

name:redis-server

state:started

-name:copyRedisconfiguration

copy:

src:redis.conf

dest:/etc/redis/redis.conf

notify:restartRedis

handlers:

-name:restartRedis

service:

name:redis-server

state:restarted

Theconfigurationisexecutedononeserverweb1.Itinstallstheredis-serverpackage,copiestheRedisconfiguration,andstartsRedis.Notethateachtimewechangethecontentoftheredis.conffileandre-runtheansible-playbookcommand,theconfigurationisupdatedontheserverandtheRedisserviceisrestarted.

Wealsoneedtocreatetheredis.conffilewiththefollowingcontent:

daemonizeyes

pidfile/var/run/redis/redis-server.pid

port6379

bind0.0.0.0

ThisconfigurationrunsRedisasadaemonandexposesittoallnetworkinterfacesunderportnumber6379.Let'snowdefinethesecondplay,whichsetsupthecalculatorservice.

DeployingawebserviceWepreparethecalculatorwebserviceinthreesteps:

1. Configuretheprojecttobeexecutable.2. ChangetheRedishostaddress.3. Addcalculatordeploymenttotheplaybook.

bootRepackage{<br/>executable=true<br/>}

ChangingtheRedishostaddressPreviously,we'vehardcodedtheRedishostaddressasredis,sonowweshouldchangeitinthesrc/main/java/com/leszko/calculator/CacheConfig.javafileto192.168.0.241.

Inreal-lifeprojects,theapplicationpropertiesareusuallykeptinthepropertiesfile.Forexample,fortheSpringBootframework,it'safilecalledapplication.propertiesorapplication.yml.

-hosts:web2<br/>become:yes<br/>become_method:sudo<br/>tasks:<br/>-name:ensureJavaRuntimeEnvironmentisinstalled<br/>apt:<br/>name:default-jre<br/>state:present<br/>-name:createdirectoryforCalculator<br/>file:<br/>path:/var/calculator<br/>state:directory<br/>-name:configureCalculatorasaservice<br/>file:<br/>path:/etc/init.d/calculator<br/>state:link<br/>force:yes<br/>src:/var/calculator/calculator.jar<br/>-name:copyCalculator<br/>copy:<br/>src:build/libs/calculator-0.0.1-SNAPSHOT.jar<br/>dest:/var/calculator/calculator.jar<br/>mode:a+x<br/>notify:<br/>-restartCalculator<br/>handlers:<br/>-name:restartCalculator<br/>service:<br/>name:calculator<br/>enabled:yes<br/>state:restarted

Let'swalkthroughthestepswedefined:

Preparetheenvironment:ThistaskensuresthattheJavaRuntimeEnvironmentisinstalled.Basically,itpreparestheserverenvironment,sothatthecalculatorapplicationwouldhaveallthenecessarydependencies.Withmorecomplexapplications,thelistofdependenttoolsandlibrariescanbewaylonger.Configureapplicationasaservice:WewouldliketohavethecalculatorapplicationrunningasaUnixservice,sothatitwillbemanageableinthestandardway.Inthiscase,it'senoughtocreatealinktoourapplicationinthe/etc/init.d/directory.Copythenewversion:Thenewversionoftheapplicationiscopiedintotheserver.Notethatifthesourcefiledidn'tchange,thenthefilewon'tbecopiedandthereforetheservicewon'tberestarted.Restarttheservice:Asahandler,everytimethenewversionoftheapplicationiscopied,theserviceisrestarted.

RunningdeploymentAsalways,wecanexecutetheplaybookusingtheansible-playbookcommand.Beforethat,weneedtobuildthecalculatorprojectwithGradle.

$./gradlewbuild

$ansible-playbookplaybook.yml

Afterthesuccessfuldeployment,theserviceshouldbeavailableandwecancheckit'sworkingathttp://192.168.0.242:8080/sum?a=1&b=2.Asexpected,itshouldreturn3astheoutput.

Notethatwehaveconfiguredthewholeenvironmentbyexecutingonecommand.What'smore,ifweneedtoscaletheservice,thenit'senoughtoaddanewservertotheinventoryandre-runtheansible-playbookcommand.

WehaveshowedhowtouseAnsibleforenvironmentconfigurationandapplicationdeployment.ThenextstepistouseAnsibletogetherwithDocker.

AnsiblewithDockerAsyoumayhavenoticed,AnsibleandDockeraddresssimilarsoftwaredeploymentissues:

Environmentconfiguration:BothAnsibleandDockerprovideawaytoconfiguretheenvironment;however,theyusedifferentmeans.WhileAnsibleusesscripts(encapsulatedinsidetheAnsiblemodules),Dockerencapsulatesthewholeenvironmentinsideacontainer.Dependencies:Ansibleprovidesawaytodeploydifferentservicesonthesameordifferenthostsandletthembedeployedtogether.DockerComposehasasimilarfunctionality,whichallowsrunningmultiplecontainersatthesametime.Scalability:Ansiblehelpstoscaleservicesprovidingtheinventoryandhostgroups.DockerComposehasasimilarfunctionalitytoautomaticallyincreaseordecreasethenumberofrunningcontainers.Automationwithconfigurationfiles:BothDockerandAnsiblestorethewholeenvironmentconfigurationandservicedependenciesinfiles(storedinthesourcecontrolrepository).ForAnsible,thisfileiscalledplaybook.yml.InthecaseofDocker,wehaveDockerfilefortheenvironmentanddocker-compose.ymlforthedependenciesandscaling.Simplicity:Bothtoolsareverysimpletouseandprovideawaytosetupthewholerunningenvironmentwithaconfigurationfileandjustonecommandexecution.

Ifwecomparethetools,thenDockerdoesalittlemore,sinceitprovidestheisolation,portability,andsomekindofsecurity.WecouldevenimagineusingDockerwithoutanyotherconfigurationmanagementtool.Then,whydoweneedAnsibleatall?

BenefitsofAnsibleAnsiblemayseemredundant;however,itbringsadditionalbenefitstothedeliveryprocess:

Dockerenvironment:TheDockerhostitselfhastobeconfiguredandmanaged.EverycontainerisultimatelyrunningonLinuxmachines,whichneedskernelpatching,Dockerengineupdates,networkconfiguration,andsoon.What'smore,theremaybedifferentservermachineswithdifferentLinuxdistributionsandtheresponsibilityofAnsibleistomakesuretheDockerengineisupandrunning.Non-Dockerizedapplications:Noteverythingisruninsideacontainer.Ifpartoftheinfrastructureiscontainerizedandpartisdeployedinthestandardwayorinthecloud,thenAnsiblecanmanageitallwiththeplaybookconfigurationfile.Theremaybedifferentreasonsfornotrunninganapplicationasacontainer,forexampleperformance,security,specifichardwarerequirements,Windows-basedsoftware,orworkingwiththelegacysoftware.Inventory:Ansibleoffersaveryfriendlywaytomanagethephysicalinfrastructureusinginventories,whichstoretheinformationaboutallservers.Itcanalsosplitthephysicalinfrastructureintodifferentenvironments:production,testing,development.GUI:Ansibleoffersa(commercial)GUImanagercalledAnsibleTower,whichaimstoimprovetheinfrastructuremanagementfortheenterprises.Improvetestingprocess:AnsiblecanhelpwiththeintegrationandacceptancetestingandcanencapsulatethetestingscriptsinasimilarwaythatDockerComposedoes.

WecanlookatAnsibleasthetoolthattakescareoftheinfrastructure,whileDockerasatoolthattakescareoftheenvironmentconfiguration.Theoverviewispresentedinthefollowingdiagram:

Ansiblemanagestheinfrastructure:Dockerservers,Dockerregistry,serverswithoutDocker,andcloudproviders.Italsotakescareofthephysicallocationoftheservers.Usingtheinventoryhostgroups,itcanlinkthewebservicestothedatabasesthatareclosetotheirgeographiclocation.

AnsibleDockerplaybookAnsibleintegratessmoothlywithDockerbecauseitprovidesasetofDocker-dedicatedmodules.IfwecreateanAnsibleplaybookforDocker-baseddeployment,thenthefirsttasksneedtomakesurethattheDockerengineisinstalledoneverymachine.Then,itshouldrunacontainerusingDockerorasetofinteractingcontainersusingDockerCompose.

ThereareafewveryusefulDocker-relatedmodulesprovidedbyAnsible:docker_image(build/manageimages),docker_container(runcontainers),docker_image_facts(inspectimages),docker_login(logintoDockerregistry),docker_network(manageDockernetworks),anddocker_service(manageDockerCompose).

InstallingDockerWecaninstalltheDockerengineusingthefollowingtaskintheAnsibleplaybook.

tasks:

-name:adddockeraptkeys

apt_key:

keyserver:hkp://p80.pool.sks-keyservers.net:80

id:9DC858229FC7DD38854AE2D88D81803C0EBFCD88

-name:updateapt

apt_repository:

repo:deb[arch=amd64]https://download.docker.com/linux/ubuntuxenialmainstable

state:present

-name:installDocker

apt:

name:docker-ce

update_cache:yes

state:present

-name:addadmintodockergroup

user:

name:admin

groups:docker

append:yes

-name:installpython-pip

apt:

name:python-pip

state:present

-name:installdocker-py

pip:

name:docker-py

-name:installDockerCompose

pip:

name:docker-compose

version:1.9.0

Theplaybooklooksslightlydifferentforeachoperatingsystem.TheonepresentedhereisforUbuntu16.04.

ThisconfigurationinstallstheDockerengine,enablestheadminusertoworkwithDocker,andinstallsDockerComposewithdependenttools.

Alternatively,youmayalsousethedocker_ubunturoleasdescribedhere:https://www.ansible.com/2014/02/12/installing-and-building-docker-with-ansible.

WhenDockerisinstalled,wecanaddatask,whichwillrunaDockercontainer.

RunningDockercontainersRunningDockercontainersisdonewiththeuseofthedocker_containermoduleanditlooksverysimilartowhatwepresentedfortheDockerComposeconfiguration.Let'saddittotheplaybook.ymlfile.

-name:runRediscontainer

docker_container:

name:redis

image:redis

state:started

exposed_ports:

-6379

Youcanreadmoreaboutalloftheoptionsofthedocker_containermoduleontheofficialAnsiblepageat:https://docs.ansible.com/ansible/docker_container_module.html.

WecannowexecutetheplaybooktoobservethatDockerhasbeeninstalledandtheRediscontainerstarted.Notethatit'saveryconvenientwayofusingDocker,sincewedon'tneedtomanuallyinstallDockerengineoneverymachine.

UsingDockerComposeTheAnsibleplaybookisverysimilartotheDockerComposeconfiguration.TheyevenbothsharethesameYAMLfileformat.What'smore,itispossibletousedocker-compose.ymldirectlyfromAnsible.Wewillshowhowtodoit,butfirst,let'sdefinethedocker-compose.ymlfile.

version:"2"

services:

calculator:

image:leszko/calculator:latest

ports:

-8080

redis:

image:redis:latest

Itisalmostthesameaswhatwedefinedinthepreviouschapter.ThistimewegetthecalculatorimagedirectlyfromtheDockerHubregistry,anddonotbuilditindocker-compose.yml,sincewewanttobuildtheimageonce,pushittotheregistry,andthenreuseitineverydeploymentstep(oneveryenvironment),tomakesurethesameimageisdeployedoneachDockerhostmachine.Whenwehavedocker-compose.yml,wearereadytoaddnewtaskstoplaybook.yml.

-name:copydocker-compose.yml

copy:

src:./docker-compose.yml

dest:./docker-compose.yml

-name:rundocker-compose

docker_service:

project_src:.

state:present

Wefirstcopythedocker-compose.ymlfileintotheserverandthenexecutedocker-compose.Asaresult,Ansiblecreatestwocontainers:calculatorandredis.

WehaveseenthemostimportantfeaturesofAnsible.Inthenextsections,wewritealittlebitabouttheinfrastructureandapplicationversioning.Attheendofthischapter,wewillpresenthowtouseAnsibleinordertocompletetheContinuousDeliverypipeline.

Exercises

Inthischapter,wehavecoveredthefundamentalsofAnsibleandthewaytouseittogetherwithDocker.Asanexercise,weproposethefollowingtasks:

1. CreatetheserverinfrastructureanduseAnsibletomanageit.

ConnectaphysicalmachineorrunaVirtualBoxmachinetoemulatetheremoteserverConfigureSSHaccesstotheremotemachine(SSHkeys)InstallPythonontheremotemachineCreateanAnsibleinventorywiththeremotemachineRuntheAnsibleadhoccommand(withthepingmodule)tocheckthattheinfrastructureisconfiguredcorrectly

2. CreateaPython-based"helloworld"webserviceanddeployitinaremotemachineusingAnsibleplaybook.

TheservicecanlookexactlythesameasdescribedintheexercisesforthechapterCreateaplaybook,whichdeploystheserviceintotheremotemachineRuntheansible-playbookcommandandcheckwhethertheservicewasdeployed

SummaryWehavecoveredtheconfigurationmanagementprocessanditsrelationtoDocker.Thekeytakeawayfromthechapterisasfollows:

ConfigurationmanagementisaprocessofcreatingandapplyingtheconfigurationsoftheinfrastructureandtheapplicationAnsibleisoneofthebesttrendingconfigurationmanagementtools.ItisagentlessandthereforerequiresnospecialserverconfigurationAnsiblecanbeusedwithadhoccommands,buttherealpowerliesinAnsibleplaybooksTheAnsibleplaybookisadefinitionofhowtheenvironmentshouldbeconfiguredThepurposeofAnsiblerolesistoreusepartsofplaybooks.AnsibleGalaxyisanonlineservicetoshareAnsiblerolesAnsibleintegrateswellwithDockerandbringsadditionalbenefitscomparedtousingDockerandDockerComposealone

Inthenextchapter,wewillwrapuptheContinuousDeliveryprocessandcompletethefinalJenkinspipeline.

ContinuousDeliveryPipeline

WehavealreadycoveredthemostcrucialpartsoftheContinuousDeliveryprocess:thecommitphase,theartifactrepository,automatedacceptancetesting,andconfigurationmanagement.

Inthischapter,wewillfocusonthemissingpartsofthefinalpipeline,whicharetheenvironmentsandinfrastructure,applicationversioning,andnonfunctionaltesting.

Thischaptercoversthefollowingpoints:

DesigningdifferentsoftwareenvironmentsandtheirinfrastructuresSecuringtheconnectionbetweenJenkinsagentsandserversIntroducingvariouskindsofnonfunctionaltestsPresentingthechallengesofnonfunctionaltestsintheContinuousDeliveryprocessExplainingdifferenttypesofapplicationversioningCompletingtheContinuousDeliverypipelineIntroducingtheconceptofsmoketestingandaddingittothefinalpipeline

EnvironmentsandinfrastructureSofar,wehavealwaysusedoneDockerhostforeverythingandtreateditasthevirtualizationofendlessresourceswherewecandeployeverything.Obviously,theDockerhostcanactuallybeaclusterofmachinesandwewillshowhowtocreateitusingDockerSwarminthenextchapters.However,eveniftheDockerhostwereunlimitedintermsofresources,westillneedtothinkabouttheunderlyinginfrastructureforatleasttworeasons:

PhysicallocationofthemachinesmattersNotestingshouldbedoneontheproductionphysicalmachines

Takingthesefactsintoconsideration,inthissection,wewilldiscussdifferenttypesofenvironment,theirroleintheContinuousDeliveryprocess,andinfrastructuresecurityaspects.

TypesofenvironmentTherearefourmostcommonenvironmenttypes:production,staging,QA(testing),anddevelopment.Let'sdiscusseachofthemanditsinfrastructure.

ProductionProductionistheenvironmentthatisusedbytheenduser.Itexistsineverycompanyand,ofcourse,itisthemostimportantenvironment.

Let'slookatthefollowingdiagramandseehowmostproductionenvironmentsareorganized:

Usersaccesstheservicethroughtheloadbalancer,whichchoosestheexactmachine.Iftheapplicationisreleasedinmultiplephysicallocations,thenthe(first)deviceisusuallyaDNS-basedgeographicloadbalancer.Ineachlocation,wehaveaclusterofservers.IfweuseDocker,thenthatclusterofserverscanbehiddenbehindoneormultipleDockerhosts(whichareinternallycomposedofmanymachinesusingDockerSwarm).

Thephysicallocationofmachinesmattersbecausetherequest-responsetimecandiffersignificantlydependingonthephysicaldistance.Moreover,thedatabaseandotherdependentservicesshouldbelocatedonamachinethatisclosetowheretheserviceisdeployed.What'sevenmoreimportantisthatthedatabaseshouldbeshardedinawaythatthereplicationoverheadbetweendifferentlocationsisminimized.Otherwise,wemayendupwaitingalotforthedatabasestoreachconsensusbetweenitsinstanceslocatedfarawayfromeachother.Moredetailsonthephysicalaspectsarebeyondthescopeofthisbook,butit'simportanttorememberthatDockerisnotalwaysasilverbullet.

Containerizationandvirtualizationallowyoutothinkaboutserversasaninfiniteresource;however,somephysicalaspectssuchaslocationarestillrelevant.

StagingThestagingenvironmentistheplacewherethereleasecandidateisdeployedinordertoperformthefinaltestsbeforegoinglive.Ideally,thisenvironmentisamirroroftheproduction.

Let'slookatthefollowingtoseehowsuchanenvironmentshouldlookinthecontextofthedeliveryprocess:

Notethatthestagingenvironmentanexactaproductionclone.Iftheapplicationisdeployedinmultiplelocations,thenthestagingshouldalsohavemultiplelocations.

IntheContinuousDeliveryprocess,allautomatedacceptancefunctionalandnonfunctionaltestsarerunagainstthisenvironment.Whilemostfunctionaltestsdon'tusuallyrequireidenticalproduction-likeinfrastructure,inthecaseofnonfunctional(especiallyperformance)tests,it'samust.

Itisnotuncommonthat,forthepurposeofcostsaving,thestaginginfrastructurediffersfromtheproduction(usuallyitcontainsfewermachines).Suchanapproachcan,however,leadtomanyproductionissues.MichaelT.Nygard,inhisgreatbookReleaseIt!,givesanexampleofareal-lifescenarioinwhichfewermachineswereusedinthestagingenvironmentthaninproduction.

Thestorygoeslikethis:inonecompany,thesystemwasstableuntilacertaincodechange,whichcausedtheproductiontobeextremelyslow,eventhoughallstresstestspassed.Howwasitpossible?Ithappenedthattherewasasynchronizationpoint,inwhicheachservercommunicatedwitheachother.In

thecaseofthestaging,therewasoneserver,soactuallytherewasnoblocker.Inproduction,however,thereweremanyservers,whichresultedinserverswaitingforeachother.Thisexampleisjustthetipoftheicebergandmanyproductionissuesmayfailtobetestedbyacceptancetestsifthestagingenvironmentisdifferentfromtheproductionenvironment.

QATheQAenvironment(alsocalledthetestingenvironment)isintendedfortheQAteamtoperformexploratorytestingandforexternalapplications(whichdependonourservice)toperformintegrationtesting.TheusecasesandtheinfrastructureoftheQAenvironmentarepresentedinthefollowingdiagram:

Whilestagingdoesnotneedtobestable(inthecaseofContinuousDelivery,itischangedaftereverycodechangecommittedtotherepository),theQAinstanceneedstoprovideacertainstabilityandexposethesame(orbackwardcompatible)APIastheproduction.Contrarytothestagingenvironment,theinfrastructurecanbedifferentfromtheproduction,sinceitspurposeisnottoensurethatthereleasecandidateworksproperly.

Averycommoncaseistoallocatefewermachines(forexample,onlyfromonelocation)forthepurposeoftheQAinstance.

DeployingtotheQAenvironmentisusuallydoneinaseparatepipeline,sothatitwillbeindependentfromtheautomaticreleaseprocess.Suchanapproachisconvenient,becausetheQAinstancehasadifferentlifecyclethanproduction(forinstance,theQAteammaywanttoperformtestingontheexperimentalcodebranchedfromthetrunk).

DevelopmentThedevelopmentenvironmentcanbecreatedasasharedserverforalldevelopersoreachdevelopercanhavehis/herowndevelopmentenvironment.Asimplediagramispresentedhere:

Thedevelopmentenvironmentalwayscontainsthelatestversionofthecode.ItisusedtoenableintegrationbetweendevelopersandcanbetreatedthesamewayastheQAenvironment,butisusedbydevelopers,notQAs.

EnvironmentsinContinuousDeliveryForthepurposeoftheContinuousDeliveryprocess,thestagingenvironmentisindispensable.Insomeveryrarecases,whentheperformanceisnotimportantandtheprojectdoesnothavemanydependencies,wecouldperformtheacceptancetestsonthelocal(development)Dockerhost(likewedidinthepreviouschapter),butthatshouldbeanexception,notarule.Insuchacase,wealwaysrisksomeproductionissuesrelatedtotheenvironment.

TheotherenvironmentsareusuallynotimportantwithregardtoContinuousDelivery.IfwewouldliketodeploytotheQAordevelopmentenvironmentwitheverycommit,thenwecancreateseparatepipelinesforthatpurpose(beingcarefulnottoobscurethemainreleasepipeline).Inmanycases,deploymenttotheQAenvironmentistriggeredmanually,becauseitcanhavedifferentlifecyclesfromproduction.

SecuringenvironmentsAllenvironmentsneedtobewellsecured.That'sclear.What'sevenmoreobviousisthatthemostimportantrequirementistokeeptheproductionsecure,becauseourbusinessdependsonitandtheconsequencesofanysecurityflawcanbehighestthere.

Securityisabroadtopic.Inthissection,wefocusonlyonthetopicsrelatedtotheContinuousDeliveryprocess.Nevertheless,settingupacompleteserverinfrastructurerequiresmuchmoreknowledgeaboutsecurity.

IntheContinuousDeliveryprocess,theslavemusthaveaccesstoservers,sothatitcandeploytheapplication.

Therearedifferentapproachesforprovidingslaveswiththeserver'scredentials:

PutSSHkeyintoslave:Ifwedon'tusedynamicDockerslaveprovisioning,thenwecanconfigureJenkinsslavemachinestocontainprivateSSHkeys.PutSSHkeyintoslaveimage:IfweusedynamicDockerslaveprovisioning,wecouldaddtheSSHprivatekeyintotheDockerslaveimage.However,itcreatesapossiblesecurityhole,sinceanyonewhohasaccesstothatimagewouldhaveaccesstotheproductionservers.Jenkinscredentials:WecanconfigureJenkinstostorecredentialsandusetheminthepipeline.CopytoSlaveJenkinsplugin:WecancopytheSSHkeydynamicallyintotheslavewhilestartingtheJenkinsbuild.

Eachsolutionhassomeadvantagesanddrawbacks.Whileusinganyofthemwehavetotakeextracaution,since,whenaslavehasaccesstotheproduction,thenanyonebreakingintotheslavebreaksintotheproduction.

ThemostriskysolutionistoputSSHprivatekeysintotheJenkinsslaveimage,sincethenalltheplaceswheretheimageisstored(theDockerregistryorDockerhostwithJenkins)needtobewellsecured.

NonfunctionaltestingWelearnedalotaboutfunctionalrequirementsandautomatedacceptancetestinginthepreviouschapter.However,whatshouldwedowithnonfunctionalrequirements?Orevenmorechallenging,whatiftherearenorequirements?ShouldweskipthematallintheContinuousDeliveryprocess?Let'sanswerthesequestionsthroughoutthissection.

Nonfunctionalaspectsofthesoftwarearealwaysimportant,becausetheycancauseasignificantrisktotheoperationofthesystem.

Forexample,manyapplicationsfail,becausetheyarenotabletobeartheloadofasuddenincreaseinthenumberofusers.InthebookUsabilityEngineering,JakobNielsen,writesthat1.0secondisaboutthelimitfortheuser'sflowofthoughttostayuninterrupted.Imaginethatoursystem,withthegrowingload,startstoexceedthatlimit.Userscanstopusingtheservicejustbecauseofitsperformance.Takingitintoconsideration,nonfunctionaltestingisasimportantasfunctionaltesting.

Tocutalongstoryshort,weshouldalwaystakethefollowingstepsfornonfunctionaltesting:

DecidewhichnonfunctionalaspectsarecrucialtoourbusinessForeachofthem:

SpecifytheteststhesamewaywedidforacceptancetestingAddastagetotheContinuousDeliverypipeline(afteracceptancetesting,whiletheapplicationisstilldeployedonthestagingenvironment)

Theapplicationcomestothereleasestageonlyafterallnonfunctionaltestspass

Irrespectiveofthetypeofthenonfunctionaltest,theideaisalwaysthesame.Theapproach,however,mayslightlydiffer.Let'sexaminedifferenttesttypesandthechallengestheypose.

TypesofnonfunctionaltestFunctionaltestarealwaysrelatedtothesameaspect—thebehaviorofthesystem.Onthecontrary,nonfunctionaltestsconcernalotofdifferentaspects.Let'sdiscussthemostcommonsystempropertiesandhowtheycanbetestedinsidetheContinuousDeliveryprocess.

PerformancetestingPerformancetestsarethemostwidelyusednonfunctionaltests.Theymeasuretheresponsivenessandstabilityofthesystem.Thesimplestperformancetestwecouldcreateistosendarequesttothewebserviceandmeasureitsround-triptime(RTT).

Therearedifferentdefinitionsofperformancetesting.Inmanyplaces,theyaremeanttoincludeload,stress,andscalabilitytesting.Sometimestheyarealsodescribedaswhite-boxtests.Inthisbook,wedefineperformancetestingasthemostbasicformofblack-boxtesttomeasurethelatencyofthesystem.

Forthepurposeofperformancetesting,wecanuseadedicatedframework(forJavathemostpopularisJMeter)orjustusethesametoolweusedforacceptancetests.AsimpleperformancetestisusuallyaddedasapipelinestagejustafterAcceptancetests.SuchatestshouldfailiftheRTTexceedsthegivenlimitanditdetectsbugsthatdefinitelyslowdowntheservice.

TheJMeterpluginforJenkinscanshowperformancetrendsoverthetime.

LoadtestingLoadtestsareusedtocheckhowthesystemfunctionswhentherearealotofconcurrentrequests.Whileasystemcanbeveryfastwithasinglerequest,itdoesnotmeanthatitworksfastenoughwith1,000requestsatthesametime.Duringloadtesting,wemeasuretheaveragerequest-responsetimeofmanyconcurrentcalls,usuallyperformedfrommanymachines.LoadtestingisaverycommonQAphaseinthereleasecycle.Toautomateit,wecanusethesametoolsaswiththesimpleperformancetest;however,inthecaseoflargersystems,wemayneedaseparateclientenvironmenttoperformalargenumberofconcurrentrequests.

StresstestingStresstesting,alsocalledcapacitytestingorthroughputtesting,isatestthatdetermineshowmanyconcurrentuserscanaccessourservice.Itmaysoundthesameasloadtesting;however,inthecaseofloadtesting,wesetthenumberofconcurrentusers(throughput)toagivennumber,checktheresponsetime(latency),andmakethebuildfailifthelimitisexceeded.Duringstresstesting,however,wekeepthelatencyconstantandincreasethethroughputtodiscoverthemaximumnumberofconcurrentcallswhenthesystemisstilloperable.Sotheresultofastresstestmaybenotificationthatoursystemcanhandle10,000concurrentusers,whichhelpsusprepareforthepeakusagetime.

StresstestingisnotwellsuitedfortheContinuousDeliveryprocess,becauseitrequireslongtestswithanincreasingnumberofconcurrentrequests.ItshouldbepreparedasaseparatescriptofaseparateJenkinspipelineandtriggeredondemand,whenweknowthatthecodechangecancauseperformanceissues.

ScalabilitytestingScalabilitytestingexplainshowlatencyandthroughputchangewhenweaddmoreserversorservices.Theperfectcharacteristicwouldbelinear,whichmeansifwehaveoneserverandtheaveragerequest-responsetimeis500mswhenusedby100parallelusers,thenaddinganotherserverwouldkeeptheresponsetimethesameandallowustoaddanother100parallelusers.Inreality,it'softenhardtoachievethsibecauseofkeepingdataconsistencybetweenservers.

Scalabilitytestingshouldbeautomatedandshouldprovidethegraphpresentingtherelationshipbetweenthenumberofmachinesandthenumberofconcurrentusers.Suchdataishelpfulindeterminingthelimitsofthesystemandthepointatwhichaddingmoremachinesdoesnothelp.

Scalabilitytests,similartostresstests,arehardtoputintotheContinuousDeliverypipelineandshouldratherbekeptseparate.

EndurancetestingEndurancetests,alsocalledlongevitytests,runthesystemforalongtimetoseeiftheperformancedropsafteracertainperiodoftime.Theydetectmemoryleaksandstabilityissues.Sincetheyrequireasystemrunningforalongtime,itdoesn'tmakesensetoruntheminsidetheContinuousDeliverypipeline.

SecuritytestingSecuritytestingdealswithdifferentaspectsrelatedtosecuritymechanismsanddataprotection.Somesecurityaspectsarepurelyfunctionalrequirementssuchasauthentication,authorization,orroleassignment.Thesepartsshouldbecheckedthesamewayasanyotherfunctionalrequirement—duringtheacceptancetestphase.Therearealsoothersecurityaspectsthatarenonfunctional;forexample,thesystemshouldbeprotectedagainstSQLinjection.Noclientwouldprobablyspecifysucharequirement,butit'simplicit.

SecuritytestsshouldbeincludedinContinuousDeliveryasapipelinestage.Theycanbewrittenusingthesameframeworksastheacceptancetestsorwithdedicatedsecuritytestingframeworks,forexample,BDDsecurity.

Securityshouldalsoalwaysbeaparttheexplanatorytestingprocess,inwhichtestersandsecurityexpertsdetectsecurityholesandaddnewtestingscenarios.

MaintainabilitytestingMaintainabilitytestsexplainhowsimpleasystemistomaintain.Inotherwords,theyjudgecodequality.Wealreadyhaverelatedstagesinthecommitphasethatcheckthetestcoverageandperformstaticcodeanalysis.TheSonartoolcanalsogivesomeoverviewofthecodequalityandthetechnicaldebt.

RecoverytestingRecoverytestingisatechniquetodeterminehowquicklythesystemcanrecoverafteritcrashedbecauseofasoftwareorhardwarefailure.Thebestcasewouldbeifthesystemdoesnotfailatall,evenifapartofitsservicesisdown.Somecompaniesevenperformproductionfailuresonpurposetocheckiftheycansurviveadisaster.ThebestknownexampleisNetflixandtheirChaosMonkeytool,whichrandomlyterminatesrandominstancesoftheproductionenvironment.Suchanapproachforcesengineerstowritecodethatmakessystemsresilienttofailures.

RecoverytestingisobviouslynotpartoftheContinuousDeliveryprocess,butratheraperiodiceventtochecktheoverallhealth.

YoucanreadmoreaboutChaosMonkeyathttps://github.com/Netflix/chaosmonkey.

Therearemanymorenonfunctionaltesttypes,whichareclosertoorfurtherfromthecodeandtheContinuousDeliveryprocess.Someofthemrelatetothelawsuchascompliancetesting;othersarerelatedtothedocumentationorinternationalization.Therearealsousabilitytestingsandvolumetestings(whichcheckwhetherthesystembehaveswellinthecaseoflargeamountsofdata).Mostofthesetests,however,havenopartintheContinuousDeliveryprocess.

Nonfunctionalchallenges

Nonfunctionalaspectsposenewchallengestothesoftwaredevelopmentanddelivery:

Longtestrun:Thetestscantakealongtimetorunandmayneedaspecialexecutionenvironment.Incrementalnature:It'shardtosetthelimitvaluewhenthetestshouldfail(unlessSLAiswelldefined).Eveniftheedgelimitisset,theapplicationwouldprobablyincrementallyapproachthelimit.Inmostcases,actually,noonecodechangecausedthetestfailure.Vaguerequirements:Usersusuallydon'thavemuchinputconcerningnonfunctionalrequirements.Theymayprovidesomeguidelinesconcerningtherequest-responsetimeorthenumberofusers;however,theywon'tprobablyknowmuchaboutmaintainability,security,orscalability.Multiplicity:Therearealotofdifferentnonfunctionaltestsandchoosingwhichshouldbeimplementedrequiresmakingsomecompromises.

Thebestapproachtoaddressnonfunctionalaspectsistotakethefollowingsteps:

1. Makealistofallnonfunctionaltesttypes.2. Crossoutexplicitlythetestyoudon'tneedforyoursystem.Theremaybea

lotofreasonsyoudon'tneedonekindoftest,forexample:

TheserviceissupersmallandasimpleperformancetestisenoughThesystemisinternalonlyandavailableonlyforread-only,soitmaynotneedanysecuritychecksThesystemisdesignedforonemachineonlyanddoesnotneedanyscalingThecostofcreatingcertaintestsistoohigh

3. Splityourtestsintotwogroups:

ContinuousDelivery:ItispossibletoaddittothepipelineAnalysis:Itisnotpossibletoaddtothepipelinebecauseoftheirexecutiontime,theirnature,ortheassociatedcost

4. FortheContinuousDeliverygroup,implementtherelatedpipelinestages.5. FortheAnalysisgroup:

CreateautomatedtestsSchedulewhentheyshouldberunSchedulemeetingstodiscusstheirresultsandtakeactionpoints

Averygoodapproachistohaveanightlybuildwiththelongteststhatdon'tfittheContinuousDeliverypipeline.Then,it'spossibletoscheduleaweeklymeetingtomonitorandanalyzethetrendsofsystemperformance.

Aspresented,therearemanytypesofnonfunctionaltestandtheyposeadditionalchallengestothedeliveryprocess.Nevertheless,forthesakeofthestabilityofoursystem,thesetestsshouldneverbeblanklyskipped.Thetechnicalimplementationdiffersdependingonthetesttype,butinmostcasestheycanbeimplementedinasimilarmannertofunctionalacceptancetestsandshouldberunagainstthestagingenvironment.

Ifyou'reinterestedinthetopicofnonfunctionaltesting,systemproperties,andsystemstability,thenreadthebookReleaseIt!byMichaelT.Nygard.

ApplicationversioningSofar,duringeveryJenkinsbuild,wehavecreatedanewDockerimage,pusheditintotheDockerregistry,andusedthelatestversionthroughouttheprocess.However,suchasolutionhasatleastthreedisadvantages:

If,duringtheJenkinsbuild,aftertheacceptancetests,someonepushesanewversionoftheimage,thenwecanendupreleasingtheuntestedversionWealwayspushanimagenamedinthesameway;thus,soeffectively,itisoverwrittenintheDockerregistryIt'sveryhardtomanageimageswithoutversionsjustbytheirhashed-styleIDs

WhatistherecommendedwayofmanagingDockerimageversionstogetherwiththeContinuousDeliveryprocess?Inthissection,wegettoseedifferentversioningstrategiesandlearndifferentwaysofcreatingversionsintheJenkinspipeline.

VersioningstrategiesTherearedifferentwaystoversionapplications.

Let'sdiscussthesemostpopularsolutions,whichcanbeappliedtogetherwiththeContinuousDeliveryprocess(wheneachcommitcreatesanewversion).

Semanticversioning:Themostpopularsolutionistousesequence-basedidentifiers(usuallyintheformofx.y.z).ThismethodrequiresacommittotherepositorydonebyJenkinsinordertoincreasethecurrentversionnumber,whichisusuallystoredinthebuildfile.ThissolutioniswellsupportedbyMaven,Gradle,andotherbuildtools.Theidentifierusuallyconsistsofthreenumbers:

x:Thisisthemajorversion;thesoftwaredoesnotneedtobebackward-compatiblewhenthisversionisincrementedy:Thisistheminorversion;thesoftwareneedstobebackwardcompatiblewhentheversionisincrementedz:Thisisthebuildnumber;thisissometimesalsoconsideredasabackwardandforward-compatiblechange

Timestamp:Usingthedateandtimeofthebuildfortheapplicationversionislessverbosethansequentialnumbers,butveryconvenientinthecaseoftheContinuousDeliveryprocess,becauseitdoesnotrequirecommittingbacktotherepositorybyJenkins.Hash:Arandomlygeneratedhashversionsharesthebenefitofthedatetimeandisprobablythesimplestsolutionpossible.Thedrawbackisthatit'snotpossibletolookattwoversionsandtellwhichisthelatestone.Mixed:Therearemanyvariationsofthesolutionsdescribedearlier,forexample,majorandminorversionswiththedatetime.

AllsolutionsarefinetousewiththeContinuousDeliveryprocess.Semanticversioningrequires,however,acommittotherepositoryfromthebuildexecution,sothattheversionisincreasedinthesourcecoderepository.

Maven(andtheotherbuildtools)popularizedversionsnapshotting,whichaddedasuffixSNAPSHOTtotheversionsthat

arenotreleased,butkeptjustforthedevelopmentprocess.SinceContinuousDeliverymeansreleasingeverychange,therearenosnapshots.

VersioningintheJenkinspipelineAsdescribedearlier,therearedifferentpossibilitieswhenitcomestousingsoftwareversioningandeachofthemcanbeimplementedinJenkins.

Asanexample,let'susethedatetime.

InordertousethetimestampinformationfromJenkins,youneedtoinstalltheBuildTimestampPluginandsetthetimestampformatintheJenkinsconfiguration(forexample,to"yyyyMMdd-HHmm").

IneveryplacewhereweusetheDockerimage,weneedtoaddthetagsuffix:${BUILD_TIMESTAMP}.

Forexample,theDockerbuildstageshouldlooklikethis:sh"dockerbuild-tleszko/calculator:${BUILD_TIMESTAMP}."

Afterthechanges,whenweruntheJenkinsbuild,weshouldhavetheimagetaggedwiththetimestampversioninourDockerregistry.

Notethatafterexplicitlytaggingtheimage,it'snolongerimplicitlytaggedasthelatest.

Withversioningcompleted,wearefinallyreadytocompletetheContinuousDeliverypipeline.

CompleteContinuousDeliverypipeline

AfterdiscussingalltheaspectsofAnsible,environments,nonfunctionaltesting,andversioning,wearereadytoextendtheJenkinspipelineandfinalizeasimple,butcomplete,ContinuousDeliverypipeline.

Wewilldoitinafewstepsasfollows:

CreatetheinventoryofstagingandproductionenvironmentsUpdateacceptanceteststousetheremotehost(insteadoflocal)ReleasetheapplicationtotheproductionenvironmentAddasmoketestwhichmakessuretheapplicationwassuccessfullyreleased

InventoryIntheirsimplestform,wecanhavetwoenvironments:stagingandproduction,eachhavingoneDockerhostmachine.Inreallife,wemaywanttoaddmorehostgroupsforeachenvironmentifwewanttohaveserversindifferentlocationsorhavingdifferentrequirements.

Let'screatetwoAnsibleinventoryfiles.Startingfromthestaging,wecandefinetheinventory/stagingfile.Assumingthestagingaddressis192.168.0.241,itwouldhavethefollowingcontent:[webservers]web1ansible_host=192.168.0.241ansible_user=admin

Byanalogy,iftheproductionIPaddressis192.168.0.242,thentheinventory/productionshouldlooklikethis:

[webservers]

web2ansible_host=192.168.0.242ansible_user=admin

Itmaylookoversimplifiedtohavejustonemachineforeachenvironment;however,usingDockerSwarm(whichweshowlaterinthisbook),aclusterofhostscanbehiddenbehindoneDockerhost.

Havingtheinventorydefined,wecanchangeacceptancetestingtousethestagingenvironment.

AcceptancetestingenvironmentDependingonourneeds,wecouldtesttheapplicationbyrunningitonthelocalDockerhost(likewedidinthepreviouschapter)orusingtheremotestagingenvironment.Theformersolutionisclosertowhathappensinproduction,soitcanbeconsideredasabetterone.ThisisveryclosetowhatwaspresentedintheMethod1:Jenkins-firstacceptancetestingsectionofthepreviouschapter.TheonlydifferenceisthatnowwedeploytheapplicationonaremoteDockerhost.

Inordertodothis,wecouldusedocker(orthedocker-composecommand)withthe-Hparameter,whichspecifiestheremoteDockerhostaddress.Thiswouldbeagoodsolutionandifyoudon'tplantouseAnsibleoranyotherconfigurationmanagementtool,thenthatisthewaytogo.Nevertheless,forthereasonsalreadymentionedinthischapter,itisbeneficialtouseAnsible.Inthatcase,wecanusetheansible-playbookcommandinsidetheContinuousDeliverypipeline.

stage("Deploytostaging"){

steps{

sh"ansible-playbookplaybook.yml-iinventory/staging"

}

}

Ifplaybook.ymlanddocker-compose.ymllookthesameasintheAnsiblewithDockersection,thenitshouldbeenoughtodeploytheapplicationwithdependenciesintothestagingenvironment.

TheAcceptanceteststagelooksexactlythesameasinthepreviouschapter.Theonlyadjustmentcanbethehostnameofthestagingenvironment(oritsloadbalancer).It'salsopossibletoaddstagesforperformancetestingorothernonfunctionaltestsagainsttheapplicationrunningonthestagingenvironment.

Afteralltestsarepassed,it'shightimetoreleasetheapplication.

ReleaseTheproductionenvironmentshouldbeasclosetothestagingenvironmentaspossible.TheJenkinsstepforthereleaseshouldalsobeverysimilartothestagethatdeploystheapplicationtothestagingenvironment.

Inthesimplestscenario,theonlydifferencesaretheinventoryfileandtheapplicationconfiguration(forexample,incaseofaSpringBootapplication,wewouldsetadifferentSpringprofile,whichresultsintakingadifferentpropertiesfile).Inourcase,therearenoapplicationproperties,sotheonlydifferenceistheinventoryfile.

stage("Release"){

steps{

sh"ansible-playbookplaybook.yml-iinventory/production"

}

}

Inreality,thereleasestepcanbealittlemorecomplexifwewanttoprovidezerodowntimedeployment.Moreonthattopicispresentedinthenextchapters.

Afterthereleaseisdone,wemightthinkthateverythingiscompleted;however,thereisonemoremissingstage,asmoketest.

SmoketestingAsmoketestisaverysmallsubsetofacceptancetestswhoseonlypurposeistocheckthatthereleaseprocessiscompletedsuccessfully.Otherwise,wecouldhaveasituationinwhichtheapplicationisperfectlyfine;however,thereisanissueinthereleaseprocess,sowemayendupwithanon-workingproduction.

Thesmoketestisusuallydefinedinthesamewayastheacceptancetest.SotheSmoketeststageinthepipelineshouldlooklikethis:stage("Smoketest"){steps{sleep60sh"./smoke_test.sh"}}

Aftereverythingissetup,theContinuousDeliverybuildshouldrunautomaticallyandtheapplicationshouldbereleasedtoproduction.Withthisstep,wehavecompletedtheContinuousDeliverypipelineinitssimplest,butfullyproductive,form.

CompleteJenkinsfileTosumup,throughouttherecentchapterswehavecreatedquiteafewstages,whichresultsinacompleteContinuousDeliverypipelinethatcouldbesuccessfullyusedinmanyprojects.

NextweseethecompleteJenkinsfileforthecalculatorproject:

pipeline{

agentany

triggers{

pollSCM('*****')

}

stages{

stage("Compile"){steps{sh"./gradlewcompileJava"}}

stage("Unittest"){steps{sh"./gradlewtest"}}

stage("Codecoverage"){steps{

sh"./gradlewjacocoTestReport"

publishHTML(target:[

reportDir:'build/reports/jacoco/test/html',

reportFiles:'index.html',

reportName:"JaCoCoReport"])

sh"./gradlewjacocoTestCoverageVerification"

}}

stage("Staticcodeanalysis"){steps{

sh"./gradlewcheckstyleMain"

publishHTML(target:[

reportDir:'build/reports/checkstyle/',

reportFiles:'main.html',

reportName:"CheckstyleReport"])

}}

stage("Build"){steps{sh"./gradlewbuild"}}

stage("Dockerbuild"){steps{

sh"dockerbuild-tleszko/calculator:${BUILD_TIMESTAMP}."

}}

stage("Dockerpush"){steps{

sh"dockerpushleszko/calculator:${BUILD_TIMESTAMP}"

}}

stage("Deploytostaging"){steps{

sh"ansible-playbookplaybook.yml-iinventory/staging"

sleep60

}}

stage("Acceptancetest"){steps{sh"./acceptance_test.sh"}}

//Performanceteststages

stage("Release"){steps{

sh"ansible-playbookplaybook.yml-iinventory/production"

sleep60

}}

stage("Smoketest"){steps{sh"./smoke_test.sh"}}

}

}

YoucanfindthisJenkinsfileonGitHubathttps://github.com/leszko/calculator/blob/master/Jenkinsfile.

Exercises

Inthischapter,wehavecoveredalotofnewaspectsfortheContinuousDeliverypipeline;tobetterunderstandtheconcept,werecommendyouperformthefollowingexercises:

1. Addaperformancetest,whichteststhe"helloworld"service:

The"helloworld"servicecanbetakenfromthepreviouschapterCreateaperformance_test.shscript,whichmakes100callsinparallelandcheckswhethertheaveragerequest-responsetimeisbelow1secondYoucanuseCucumberorthecurlcommandforthescript

2. CreateaJenkinspipelinethatbuildsthe"helloworld"webserviceasaversionedDockerimageandperformsperformancetest:

CreatetheDockerbuildstage,whichbuildstheDockerimagewiththe"helloworld"serviceandaddsatimestampasaversiontagCreateanAnsibleplaybookthatusestheDockerimageAddtheDeploytostagingstage,whichdeploystheimageintotheremotemachineAddthePerformancetestingstage,whichexecutesperformance_test.shRunthepipelineandobservetheresults

SummaryInthischapter,wehavecompletedtheContinuousDeliverypipeline,whichfinallyreleasestheapplication.Thefollowingarethekeytakeawaysfromthechapter:

ForthepurposeofContinuousDelivery,twoenvironmentsareindispensable:stagingandproduction.NonfunctionaltestsareanessentialpartoftheContinuousDeliveryprocessandshouldalwaysbeconsideredaspipelinestages.Nonfunctionalteststhatdon'tfittheContinuousDeliveryprocessshouldbeconsideredasperiodictasksinordertomonitortheoverallperformancetrends.Applicationsshouldalwaysbeversioned;however,theversioningstrategydependsonthetypeoftheapplication.TheminimalContinuousDeliverypipelinecanbeimplementedasasequenceofscriptsthatendswithtwostages:releaseandsmoketest.ThesmoketestshouldalwaysbeaddedasthelaststageoftheContinuousDeliverypipelineinordertocheckwhetherthereleasewassuccessful.

Inthenextchapter,wewillhavealookattheDockerSwarmtool,whichhelpsustocreateaclusterofDockerhosts.

ClusteringwithDockerSwarm

WehavealreadycoveredallthefundamentalaspectsoftheContinuousDeliverypipeline.Inthischapter,wewillseehowtochangetheDockerenvironmentfromasingleDockerhostintoaclusterofmachinesandhowtouseitalltogetherwithJenkins.

Thischaptercoversthefollowingpoints:

ExplainingtheconceptofserverclusteringIntroducingDockerSwarmanditsmostimportantfeaturesPresentinghowtobuildaswarmclusterfrommultipleDockerhostsRunningandscalingDockerimagesonaclusterExploringadvancedswarmfeatures:rollingupdates,drainingnodes,multiplemanagernodes,andtuningtheschedulingstrategyDeployingtheDockerComposeconfigurationonaclusterIntroducingKubernetesandApacheMesosasalternativestoDockerSwarmDynamicallyscalingJenkinsagentsonacluster

ServerclusteringSofar,wehavehaveinteractedwitheachofthemachinesindividually.EvenwhenweusedAnsibletorepeatthesameoperationsonmultipleservers,wehadtoexplicitlyspecifyonwhichhostthegivenserviceshouldbedeployed.Inmostcases,however,ifserverssharethesamephysicallocation,wearenotinterestedonwhichparticularmachinetheserviceisdeployed.Allweneedistohaveitaccessibleandreplicatedinmanyinstances.Howcanweconfigureasetofmachinestoworktogethersothataddinganewonewouldrequirenoadditionalsetup?Thisistheroleofclustering.

Inthissection,youwillbeintroducedtotheconceptofserverclusteringandtheDockerSwarmtoolkit.

IntroducingserverclusteringAserverclusterisasetofconnectedcomputersthatworktogetherinawaythattheycanbeusedsimilarlytoasinglesystem.Serversareusuallyconnectedthroughthelocalnetworkwithaconnectionfastenoughtoensureasmallinfluenceofthefactthatservicesaredistributed.Asimpleserverclusterispresentedinthefollowingimage:

Auseraccessestheclusterviaahostcalledthemanager,whoseinterfaceshouldbesimilartoausualDockerhost.Insidethecluster,therearemultipleworkernodesthatreceivetasks,executethem,andnotifythemanageroftheircurrentstate.Themanagerisresponsiblefortheorchestrationprocess,includingtaskdispatching,servicediscovery,loadbalancing,andworkerfailuredetection.

Themanagercanalsoexecutetasks,whichisthedefaultconfigurationinDockerSwarm.However,forlargeclusters,themanagershouldbeconfiguredformanagementpurposesonly.

IntroducingDockerSwarmDockerSwarmisanativeclusteringsystemforDockerthatturnsasetofDockerhostsintooneconsistentcluster,calledaswarm.Eachhostconnectedtotheswarmplaystheroleofamanageroraworker(theremustbeatleastonemanagerinacluster).Technically,thephysicallocationofthemachinesdoesnotmatter;however,it'sreasonabletohaveallDockerhostsinsideonelocalnetwork,otherwise,managingoperations(orreachingconsensusbetweenmultiplemanagers)cantakeasignificantamountoftime.

SinceDocker1.12,DockerSwarmisnativelyintegratedintoDockerEngineasswarmmode.Inolderversions,itwasnecessarytoruntheswarmcontaineroneachofthehoststoprovidetheclusteringfunctionality.

Regardingtheterminology,inswarmmode,arunningimageiscalledaservice,asopposedtoacontainer,whichisrunonasingleDockerhost.Oneservicerunsaspecifiednumberoftasks.Ataskisanatomicschedulingunitoftheswarmthatholdstheinformationaboutthecontainerandthecommandthatshouldberuninsidethecontainer.Areplicaiseachcontainerthatisrunonthenode.Thenumberofreplicasistheexpectednumberofallcontainersforthegivenservice.

Let'slookatanimagepresentingtheterminologyandtheDockerSwarmclusteringprocess:

Westartbyspecifyingaservice,theDockerimageandthenumberofreplicas.Themanagerautomaticallyassignstaskstoworkernodes.Obviously,eachreplicatedcontainerisrunfromthesameDockerimage.Inthecontextofthepresentedflow,DockerSwarmcanbeviewedasalayerontopoftheDockerEnginemechanismthatisresponsibleforcontainerorchestration.

Intheprecedingsampleimage,wehavethreetasks,andeachofthemisrunonaseparateDockerhost.Nevertheless,itmayalsohappenthatallcontainerswouldbestartedonthesameDockerhost.Everythingdependsonthemanagernodethatallocatestaskstoworkernodesusingtheschedulingstrategy.Wewillshowhowtoconfigurethatstrategylater,inaseparatesection.

DockerSwarmfeaturesoverviewDockerSwarmprovidesanumberinterestingfeatures.Let'swalkthroughthemostimportantones:

Loadbalancing:DockerSwarmtakescareoftheloadbalancingandassigninguniqueDNSnamessothattheapplicationdeployedontheclustercanbeusedinthesamewayasdeployedonasingleDockerhost.Inotherwords,aswarmcanpublishportsinasimilarmannerastheDockercontainer,andthentheswarmmanagerdistributesrequestsamongtheservicesinthecluster.Dynamicrolemanagement:Dockerhostscanbeaddedtotheswarmatruntime,sothereisnoneedforaclusterrestart.What'smore,theroleofthenode(managerorworker)canalsobedynamicallychanged.Dynamicservicescaling:EachservicecanbedynamicallyscaledupordownwiththeDockerclient.Themanagernodetakescareofaddingorremovingcontainersfromthenodes.Failurerecovery:Nodesareconstantlymonitoredbythemanagerand,ifanyofthemfails,newtasksarestartedondifferentmachinessothatthedeclarednumberofreplicaswouldbeconstant.It'salsopossibletocreatemultiplemanagernodesinordertopreventabreakdownincaseoneofthemfails.Rollingupdates:Anupdatetoservicescanbeappliedincrementally;forexample,ifwehave10replicasandwewouldliketomakeachange,wecandefineadelaybetweenthedeploymenttoeachreplica.Insuchacase,whenanythinggoeswrong,weneverendupwithascenariowherenoreplicaisworkingcorrectly.Twoservicemodes:Therearetwomodesinwhichcanberun:

Replicatedservices:ThespecifiednumberofreplicatedcontainersaredistributedamongthenodesbasedontheschedulingstrategyalgorithmGlobalservices:Onecontainerisrunoneveryavailablenodeinthecluster

Security:AseverythingisinDocker,DockerSwarmenforcestheTLS

authenticationandthecommunicationencryption.It'salsopossibletouseCA(orself-signed)certificates.

Let'sseehowallofthislooksinpractice.

DockerSwarminpracticeDockerEngineincludestheSwarmmodebydefault,sothereisnoadditionalinstallationprocessrequired.SinceDockerSwarmisanativeDockerclusteringsystem,managingclusternodesisdonebythedockercommandandisthereforeverysimpleandintuitive.Let'sstartbycreatingamanagernodewithtwoworkernodes.Then,wewillrunandscaleaservicefromaDockerimage.

SettingupaSwarmInordertosetupaSwarm,weneedtoinitializethemanagernode.Wecandothisusingthefollowingcommandonamachinethatissupposedtobecomethemanager:

$dockerswarminit

Swarminitialized:currentnode(qfqzhk2bumhd2h0ckntrysm8l)isnowamanager.

Toaddaworkertothisswarm,runthefollowingcommand:

dockerswarmjoin\

--tokenSWMTKN-1-253vezc1pqqgb93c5huc9g3n0hj4p7xik1ziz5c4rsdo3f7iw2-df098e2jpe8uvwe2ohhhcxd6w\

192.168.0.143:2377

Toaddamanagertothisswarm,run'dockerswarmjoin-tokenmanager'andfollowtheinstructions.

Averycommonpracticeistousethe--advertise-addr<manager_ip>parameter,becauseifthemanagermachinehasmorethanonepotentialnetworkinterfaces,thendockerswarminitcanfail.

Inourcase,themanagermachinehastheIPaddress192.168.0.143and,obviously,ithastobereachablefromtheworkernodes(andviceversa).Notethatthecommandtoexecuteonworkermachineswasprintedtotheconsole.Alsonotethataspecialtokenhasbeengenerated.Fromnowon,itwillbeusedtoconnectamachinetotheclusterandshouldbekeptsecret.

WecancheckthattheSwarmhasbeencreatedusingthedockernodecommand:

$dockernodels

IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS

qfqzhk2bumhd2h0ckntrysm8l*ubuntu-managerReadyActiveLeader

Whenthemanagerisupandrunning,wearereadytoaddworkernodestotheSwarm.

AddingworkernodesInordertoaddamachinetotheSwarm,wehavetologintothegivenmachineandexecutethefollowingcommand:

$dockerswarmjoin\

--tokenSWMTKN-1-253vezc1pqqgb93c5huc9g3n0hj4p7xik1ziz5c4rsdo3f7iw2-df098e2jpe8uvwe2ohhhcxd6w\

192.168.0.143:2377

Thisnodejoinedaswarmasaworker.

WecancheckthatthenodehasbeenaddedtotheSwarmwiththedockernodelscommand.Assumingthatwe'veaddedtwonodemachines,theoutputshouldlookasfollows:

$dockernodels

IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS

cr7vin5xzu0331fvxkdxla22nubuntu-worker2ReadyActive

md4wx15t87nn0c3pyv24kewtzubuntu-worker1ReadyActive

qfqzhk2bumhd2h0ckntrysm8l*ubuntu-managerReadyActiveLeader

Atthispoint,wehaveaclusterthatconsistsofthreeDockerhosts,ubuntu-manager,ubuntu-worker1,andubuntu-worker2.Let'sseehowwecanrunaserviceonthiscluster.

DeployingaserviceInordertorunanimageonacluster,wedon'tusedockerrunbuttheSwarm-dedicateddockerservicecommand(whichisexecutedonthemanagernode).Let'sstartasingletomcatapplicationandgiveitthenametomcat:$dockerservicecreate--replicas1--nametomcattomcat

Thecommandcreatedtheserviceandthereforesentatasktostartacontainerononeofthenodes.Let'slisttherunningservices:

$dockerservicels

IDNAMEMODEREPLICASIMAGE

x65aeojumj05tomcatreplicated1/1tomcat:latest

Thelogconfirmsthatthetomcatserviceisrunning,andithasonereplica(oneDockercontainerisrunning).Wecanexaminetheserviceevenmoreclosely:

$dockerservicepstomcat

IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE

kjy1udwcnwmitomcat.1tomcat:latestubuntu-managerRunningRunningaboutaminuteago

Ifyouareinterestedinthedetailedinformationaboutaservice,youcanusethedockerserviceinspect<service_name>command.

Fromtheconsoleoutput,wecanseethatthecontainerisrunningonthemanagernode(ubuntu-manager).Itcouldhavebeenstartedonanyothernodeaswell;themanagerautomaticallychoosestheworkernodeusingtheschedulingstrategyalgorithm.Wecanconfirmthatthecontainerisrunningusingthewell-knowndockerpscommand:$dockerpsCONTAINERIDIMAGECOMMANDCREATEDSTATUSPORTSNAMES6718d0bcba98tomcat@sha256:88483873b279aaea5ced002c98dde04555584b66de29797a4476d5e94874e6de"catalina.shrun"AboutaminuteagoUpAboutaminute8080/tcptomcat.1.kjy1udwcnwmiosiw2qn71nt1r

Ifwedon'twantatasktobeexecutedonthemanagernode,wecanconstraintheservicewiththe--constraintnode.role==workeroption.

Theotherpossibilityistodisablethemanagercompletelyfromexecutingtaskswithdockernodeupdate--availabilitydrain<manager_name>.

ScalingserviceWhentheserviceisrunning,wecanscaleitupordownsothatitwillberunninginmanyreplicas:

$dockerservicescaletomcat=5

tomcatscaledto5

Wecancheckthattheservicehasbeenscaled:

$dockerservicepstomcat

IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE

kjy1udwcnwmitomcat.1tomcat:latestubuntu-managerRunningRunning2minutesago

536p5zc3kaxztomcat.2tomcat:latestubuntu-worker2RunningPreparing18secondsagonpt6ui1g9bdptomcat.3tomcat:latestubuntu-managerRunningRunning18secondsagozo2kger1rmqctomcat.4tomcat:latestubuntu-worker1RunningPreparing18secondsago1fb24nf94488tomcat.5tomcat:latestubuntu-worker2RunningPreparing18secondsago

Notethatthistime,twocontainersarerunningonthemanagernode,oneontheubuntu-worker1node,andtheotherontheubuntu-worker2node.Wecancheckthattheyarereallyrunningbyexecutingdockerpsoneachofthemachines.

Ifwewanttoremovetheservices,it'senoughtoexecutethefollowingcommand:

$dockerservicermtomcat

Youcancheckwiththedockerservicelscommandthattheservicehasbeenremoved,andthereforeallrelatedtomcatcontainerswerestoppedandremovedfromallthenodes.

PublishingportsDockerservices,similartothecontainers,haveaportforwardingmechanism.Weuseitbyaddingthe-p<host_port>:<container:port>parameter.Here'swhatstartingaservicecouldlooklike:$dockerservicecreate--replicas1--publish8080:8080--nametomcattomcat

Now,wecanopenabrowserandseetheTomcat'smainpageundertheaddresshttp://192.168.0.143:8080/.

Theapplicationisavailableonthemanagerhostthatactsasaloadbalanceranddistributesrequeststoworkernodes.WhatmaysoundalittlelessintuitiveisthefactthatwecanaccessTomcatusingtheIPaddressofanyworker,forexample,ifworkernodesareavailableunder192.168.0.166and192.168.0.115,wecanaccessthesamerunningcontainerwithhttp://192.168.0.166:8080/andhttp://192.168.0.115:8080/.ThisispossiblebecauseDockerSwarmcreatesaroutingmesh,inwhicheachnodehastheinformationhowtoforwardthepublishedport.

YoucanreadmoreabouthowtheloadbalancingandroutingaredonebyDockerSwarmathttps://docs.docker.com/engine/swarm/ingress/.

Bydefault,theinternalDockerSwarmloadbalancingisused.Therefore,it'senoughtoaddressallrequeststothemanager'smachine,anditwilltakecareofitsdistributionbetweennodes.Theotheroptionistoconfigureanexternalloadbalancer(forexample,HAProxyorTraefik).

WehavediscussedthebasicusageofDockerSwarm.Let'snowdiveintomorechallengingfeatures.

AdvancedDockerSwarmDockerSwarmoffersalotofinterestingfeaturesthatareusefulintheContinuousDeliveryprocess.Inthissection,wewillwalkthroughthemostimportantones.

RollingupdatesImagineyoudeployanewversionofyourapplication.Youneedtoupdateallreplicasinthecluster.OneoptionwouldbetostopthewholeDockerSwarmserviceandtorunanewonefromtheupdatedDockerimage.Suchapproach,however,causesdowntimebetweenthemomentwhentheserviceisstoppedandthemomentwhenthenewoneisstarted.IntheContinuousDeliveryprocess,downtimeisnotacceptable,sincethedeploymentcantakeplaceaftereverysourcecodechange,whichissimplyoften.Then,howcanweprovidezero-downtimedeploymentinacluster?Thisistheroleofrollingupdates.

Arollingupdateisanautomaticmethodforreplacingaservice,replicabyareplica,inawaythatsomeofthereplicasareworkingallthetime.DockerSwarmusesrollingupdatesbydefault,andtheycanbesteeredwithtwoparameters:

update-delay:Delaybetweenstartingonereplicaandstoppingthenextone(0secondsbydefault)update-parallelism:Maximumnumberofreplicasupdatedatthesametime(onebydefault)

TheDockerSwarmrollingupdateprocesslooksasfollows:

1. Stopthe<update-parallelism>numberoftasks(replicas).2. Intheirplace,runthesamenumberofupdatedtasks.3. IfataskreturnstheRUNNINGstate,thenwaitforthe<update-delay>period.4. If,atanytime,anytaskreturnstheFAILEDstate,thenpausetheupdate.

Thevalueoftheupdate-parallelismparametershouldbeadaptedtothenumberofreplicaswerun.Ifthenumberissmallandbootingtheserviceisfast,it'sreasonabletokeepthedefaultvalueof1.Theupdate-delayparametershouldbesettotheperiodlongerthantheexpectedboottimeofourapplicationsothatwewillnoticethefailure,andthereforepausetheupdate.

Let'slookatanexampleandchangetheTomcatapplicationfromversion8to

version9.Supposewehavethetomcat:8servicewithfivereplicas:

$dockerservicecreate--replicas5--nametomcat--update-delay10stomcat:8

Wecancheckthatallreplicasarerunningwiththedockerservicepstomcatcommand.Anotherusefulcommandthathelpsexaminetheserviceisthedockerserviceinspectcommand:

$dockerserviceinspect--prettytomcat

ID:au1nu396jzdewyq2y8enm0b6i

Name:tomcat

ServiceMode:Replicated

Replicas:5

Placement:

UpdateConfig:

Parallelism:1

Delay:10s

Onfailure:pause

Maxfailureratio:0

ContainerSpec:

Image:tomcat:8@sha256:835b6501c150de39d2b12569fd8124eaebc53a899e2540549b6b6f8676538484

Resources:

EndpointMode:vip

Wecanseethattheservicehasfivereplicascreatedoutoftheimagetomcat:8.Thecommandoutputalsoincludestheinformationabouttheparallelismandthedelaytimebetweenupdates(assetbytheoptionsinthedockerservicecreatecommand).

Now,wecanupdatetheserviceintothetomcat:9image:

$dockerserviceupdate--imagetomcat:9tomcat

Let'scheckwhathappens:

$dockerservicepstomcat

IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE

4dvh6ytn4lsqtomcat.1tomcat:8ubuntu-managerRunningRunning4minutesago

2mop96j5q4ajtomcat.2tomcat:8ubuntu-managerRunningRunning4minutesago

owurmusr1c48tomcat.3tomcat:9ubuntu-managerRunningPreparing13secondsago

r9drfjpizuxf\_tomcat.3tomcat:8ubuntu-managerShutdownShutdown12secondsago

0725ha5d8p4vtomcat.4tomcat:8ubuntu-managerRunningRunning4minutesago

wl25m2vrqgc4tomcat.5tomcat:8ubuntu-managerRunningRunning4minutesago

Notethatthefirstreplicaoftomcat:8hasbeenshutdownandthefirsttomcat:9isalreadyrunning.Ifwekeptoncheckingtheoutputofthedockerservicepstomcatcommand,wewouldnoticethatafterevery10seconds,anotherreplicaisintheshutdownstateandanewoneisstarted.Ifwealsomonitoredthedockerinspect

command,wewouldseethatthevalueUpdateStatus:Statewillchangetoupdatingandthen,whentheupdateisdone,tocompleted.

ArollingupdateisaverypowerfulfeaturethatallowszerodowntimedeploymentanditshouldalwaysbeusedintheContinuousDeliveryprocess.

DrainingnodesWhenweneedtostopaworkernodeformaintenance,orwewouldjustliketoremoveitfromthecluster,wecanusetheSwarmdrainingnodefeature.Drainingthenodemeansaskingthemanagertomovealltasksoutofagivennodeandexcludeitfromreceivingnewtasks.Asaresult,allreplicasarerunningonlyontheactivenodesandthedrainednodesareidle.

Let'slookhowthisworksinpractice.SupposewehavethreeclusternodesandaTomcatservicewithfivereplicas:

$dockernodels

IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS

4mrrmibdrpa3yethhmy13mwzqubuntu-worker2ReadyActive

kzgm7erw73tu2rjjninxdb4wp*ubuntu-managerReadyActiveLeader

yllusy42jp08w8fmze43rmqqsubuntu-worker1ReadyActive

$dockerservicecreate--replicas5--nametomcattomcat

Let'scheckonwhichnodesthereplicasarerunning:

$dockerservicepstomcat

IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE

zrnawwpupuqltomcat.1tomcat:latestubuntu-managerRunningRunning17minutesago

x6rqhyn7mrottomcat.2tomcat:latestubuntu-worker1RunningRunning16minutesago

rspgxcfv3is2tomcat.3tomcat:latestubuntu-worker2RunningRunning5weeksago

cf00k61vo7xhtomcat.4tomcat:latestubuntu-managerRunningRunning17minutesago

otjo08e06qbxtomcat.5tomcat:latestubuntu-worker2RunningRunning5weeksago

Therearetworeplicasrunningontheubuntu-worker2node.Let'sdrainthatnode:

$dockernodeupdate--availabilitydrainubuntu-worker2

Thenodeisputintothedrainavailability,soallreplicasshouldbemovedoutofit:

$dockerservicepstomcat

IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE

zrnawwpupuqltomcat.1tomcat:latestubuntu-managerRunningRunning18minutesago

x6rqhyn7mrottomcat.2tomcat:latestubuntu-worker1RunningRunning17minutesagoqrptjztd777itomcat.3tomcat:latestubuntu-worker1RunningRunninglessthanasecondago

rspgxcfv3is2\_tomcat.3tomcat:latestubuntu-worker2ShutdownShutdownlessthanasecondago

cf00k61vo7xhtomcat.4tomcat:latestubuntu-managerRunningRunning18minutesagok4c14tyo7leqtomcat.5tomcat:latestubuntu-worker1RunningRunninglessthanasecondago

otjo08e06qbx\_tomcat.5tomcat:latestubuntu-worker2ShutdownShutdownlessthanasecondago

Wecanseethatnewtaskswerestartedontheubuntu-worker1nodeandtheold

replicaswereshutdown.Wecancheckthestatusofthenodes:

$dockernodels

IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS

4mrrmibdrpa3yethhmy13mwzqubuntu-worker2ReadyDrain

kzgm7erw73tu2rjjninxdb4wp*ubuntu-managerReadyActiveLeader

yllusy42jp08w8fmze43rmqqsubuntu-worker1ReadyActive

Asexpected,theubuntu-worker2nodeisavailable(statusReady),butitsavailabilityissettodrain,whichmeansitdoesn'thostanytasks.Ifwewouldliketogetthenodeback,wecancheckitsavailabilitytoactive:

$dockernodeupdate--availabilityactiveubuntu-worker2

Averycommonpracticeistodrainthemanagernodeand,asaresult,itwillnotreceiveanytasks,butdomanagementworkonly.

Analternativemethodtodrainingthenodewouldbetoexecutethedockerswarmleavecommandfromtheworker.Thisapproach,however,hastwodrawbacks:

Foramoment,therearefewerreplicasthanexpected(afterleavingtheswarmandbeforethemasterstartsnewtasksonothernodes)Themasterdoesnotcontrolifthenodeisstillinthecluster

Forthesereasons,ifweplantostoptheworkerforsometimeandthengetitback,it'srecommendedtousethedrainingnodefeature.

MultiplemanagernodesHavingasinglemanagernodeisriskybecausewhenthemanagermachineisdown,thewholeclusterisdown.Thissituationis,obviously,notacceptableinthecaseofbusiness-criticalsystems.Inthissection,wepresenthowtomanagemultiplemasternodes.

Inordertoaddanewmanagernodetothesystem,weneedtofirstexecutethefollowingcommandonthe(currentlysingle)managernode:

$dockerswarmjoin-tokenmanager

Toaddamanagertothisswarm,runthefollowingcommand:

dockerswarmjoin\

--tokenSWMTKN-1-5blnptt38eh9d3s8lk8po3069vbjmz7k7r3falkm20y9v9hefx-a4v5olovq9mnvy7v8ppp63r23\

192.168.0.143:2377

Theoutputshowsthetokenandtheentirecommandthatneedstobeexecutedonthemachinethatissupposedtobecomethemanager.Afterexecutingit,weshouldseethatanewmanagerwasadded.

Anotheroptiontoaddamanageristopromoteitfromtheworkerroleusingthedockernodepromote<node>command.Inordertogetitbacktotheworkerrole,wecanusethedockernodedemote<node>command.

Supposewehaveaddedtwoadditionalmanagers;weshouldseethefollowingoutput:

$dockernodels

IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS

4mrrmibdrpa3yethhmy13mwzqubuntu-manager2ReadyActive

kzgm7erw73tu2rjjninxdb4wp*ubuntu-managerReadyActiveLeader

pkt4sjjsbxx4ly1lwetieuj2nubuntu-manager1ReadyActiveReachable

Notethatthenewmanagershavethemanagerstatussettoreachable(orleftempty),whiletheoldmanageristheleader.ThereasonforthisisthefactthatthereisalwaysoneprimarynoderesponsibleforallSwarmmanagementandorchestrationdecisions.TheleaderiselectedfromthemanagersusingtheRaftconsensusalgorithm,andwhenitisdown,anewleaderiselected.

Raftisaconsensusalgorithmthatisusedtomakedecisionsindistributedsystems.Youcanreadmoreabouthowitworks(andseeavisualization)athttps://raft.github.io/.AverypopularalternativealgorithmforthesamegoaliscalledPaxos.

Supposeweshutdowntheubuntu-managermachine;let'shavealookathowthenewleaderwaselected:

$dockernodels

IDHOSTNAMESTATUSAVAILABILITYMANAGERSTATUS

4mrrmibdrpa3yethhmy13mwzqubuntu-manager2ReadyActiveReachable

kzgm7erw73tu2rjjninxdb4wpubuntu-managerReadyActiveUnreachable

pkt4sjjsbxx4ly1lwetieuj2n*ubuntu-manager1ReadyActiveLeader

Notethatevenwhenoneofthemanagersisdown,theswarmcanworkcorrectly.

Thereisnolimitforthenumberofmanagers,soitmaysoundthatthemoremanagersthebetterfaulttolerance.It'strue,however,havingalotofmanagershasanimpactontheperformancebecauseallSwarm-state-relateddecisions(forexample,addinganewnodeorleaderelection)havetobeagreedbetweenallmanagersusingtheRaftalgorithm.Asaresult,thenumberofmanagersisalwaysatradeoffbetweenthefaulttoleranceandtheperformance.

TheRaftalgorithmitselfhasaconstraintonthenumberofmanagers.Distributeddecisionshavetobeapprovedbythemajorityofnodes,calledaquorum.Thisfactimpliesthatanoddnumberofmanagersisrecommended.

Tounderstandwhy,let'sseewhatwouldhappenifwehadtwomanagers.Inthatcase,themajorityistwo,soifanyofthemanagersisdown,thenit'snotpossibletoreachthequorumandthereforeelecttheleader.Asaresult,losingonemachinemakesthewholeclusteroutoforder.Weaddedamanager,butthewholeclusterbecamelessfaulttolerant.Thesituationwouldbedifferentinthecaseofthreemanagers.Then,themajorityisstilltwo,solosingonemanagerdoesnotstopthewholecluster.Thisisthefactthateventhoughit'snottechnicallyforbidden,onlyoddnumberofmanagersmakessense.

Themorethemanagersinthecluster,themoretheRaft-relatedoperationsareinvolved.Then,themanagernodesshouldbeputintothedrainavailabilityinordertosavetheirresources.

SchedulingstrategySofar,wehavelearnedthatthemanagerautomaticallyassignsaworkernodetoatask.Inthissection,wedivedeeperintowhatautomaticallymeans.WepresenttheDockerSwarmschedulingstrategyandawaytoconfigureitaccordingtoourneeds.

DockerSwarmusestwocriteriaforchoosingtherightworkernode:

Resourceavailability:Schedulerisawareoftheresourcesavailableonnodes.Itusestheso-calledspreadstrategythatattemptstoschedulethetaskontheleastloadednode,provideditmeetsthecriteriaspecifiedbylabelsandconstraints.Labelsandconstraints:

Labelisanattributeofanode.Somelabelsareassignedautomatically,forexample,node.idornode.hostname;otherscanbedefinedbytheclusteradmin,forexample,node.labels.segmentConstraintisarestrictionappliedbythecreatoroftheservice,forexample,choosingonlynodeswiththegivenlabel

Labelsaredividedintotwocategories,node.labelsandengine.labels.Thefirstoneisaddedbytheoperationalteam;thesecondoneiscollectedbyDockerEngine,forexample,operatingsystemorhardwarespecifics.

Asanexample,ifwewouldliketoruntheTomcatserviceontheconcretenode,ubuntu-worker1,thenweneedtousethefollowingcommand:

$dockerservicecreate--constraint'node.hostname==ubuntu-worker1'tomcat

Wecanalsoaddacustomlabeltothenode:

$dockernodeupdate--label-addsegment=AAubuntu-worker1

Theprecedingcommandaddedalabel,node.labels.segment,withthevalueAA.Then,wecanuseitwhilerunningtheservice:

$dockerservicecreate--constraint'node.labels.segment==AA'tomcat

Thiscommandrunsthetomcatreplicasonlyonthenodesthatarelabeledwiththegivensegment,AA.

Labelsandconstraintsgiveustheflexibilitytoconfigurethenodesonwhichservicereplicaswouldberun.Thisapproach,eventhoughvalidinmanycases,shouldnotbeoverused,sinceit'sbesttokeepthereplicasdistributedonmultiplenodesandletDockerSwarmtakecareoftherightschedulingprocess.

DockerComposewithDockerSwarmWehavedescribedhowtouseDockerSwarminordertodeployaservice,whichinturnrunsmultiplecontainersfromthegivenDockerimage.Ontheotherhand,thereisDockerCompose,whichprovidesamethodtodefinethedependenciesbetweencontainersandenablesscalingcontainers,buteverythingisdonewithinoneDockerhost.Howdowemergebothtechnologiessothatwecanspecifythedocker-compose.ymlfileandautomaticallydistributethecontainersonacluster?Luckily,thereisDockerStack.

IntroducingDockerStackDockerStackisamethodtorunmultiple-linkedcontainersonaSwarmcluster.TounderstandbetterhowitlinksDockerComposewithDockerSwarm,let'stakealookatthefollowingfigure:

DockerSwarmorchestrateswhichcontainerisrunonwhichphysicalmachine.Thecontainers,however,don'thaveanydependenciesbetweenthemselves,soinorderforthemtocommunicate,wewouldneedtolinkthemmanually.DockerCompose,incontrast,provideslinkingbetweenthecontainers.Intheexamplefromtheprecedingfigure,oneDockerimage(deployedinthreereplicatedcontainers)dependsonanotherDockerimage(deployedasonecontainer).Allcontainers,however,runonthesameDockerhost,sothehorizontalscalingislimitedtotheresourcesofonemachine.DockerStackconnectsbothtechnologiesandallowsusingthedocker-compose.ymlfiletorunthecompleteenvironmentoflinkedcontainersdeployedonaclusterofDockerhosts.

UsingDockerStackAsanexample,let'susethecalculatorimagethatdependsontheredisimage.Let'ssplittheprocessintofoursteps:

1. Specifyingdocker-compose.yml.2. RunningtheDockerStackcommand.3. Verifyingtheservicesandcontainers.4. Removingthestack.

Specifyingdocker-compose.ymlWealreadydefinedthedocker-compose.ymlfileinthepreviouschaptersanditlookedsimilartothefollowingone:

version:"3"

services:

calculator:

deploy:

replicas:3

image:leszko/calculator:latest

ports:

-"8881:8080"

redis:

deploy:

replicas:1

image:redis:latest

Notethatallimagesmustbepushedtotheregistrybeforerunningthedockerstackcommandsothattheywouldbeaccessiblefromallnodes.Itisthereforenotpossibletobuildimagesinsidedocker-compose.yml.

Withthepresenteddocker-compose.ymlconfiguration,wewillrunthreecalculatorcontainersandonerediscontainer.Theendpointofthecalculatorservicewillbeexposedonport8881.

<strong>$dockerstackdeploy--compose-filedocker-compose.ymlapp</strong><br/><strong>Creatingnetworkapp_default</strong><br/><strong>Creatingserviceapp_redis</strong><br/><strong>Creatingserviceapp_calculator</strong>

Dockerplanstosimplifythesyntaxsothatthestackwordwouldnotbeneeded,forexample,dockerdeploy--compose-filedocker-compose.ymlapp.Atthetimeofwriting,it'sonlyavailableintheexperimentalversion.

VerifyingtheservicesandcontainersTheserviceshavestarted.Wecancheckthattheyarerunningwiththedockerservicelscommand:

$dockerservicels

IDNAMEMODEREPLICASIMAGE

5jbdzt9wolorapp_calculatorreplicated3/3leszko/calculator:latest

zrr4pkh3n13fapp_redisreplicated1/1redis:latest

WecanlookevencloserattheservicesandcheckonwhichDockerhoststheyhavebeendeployed:

$dockerservicepsapp_calculator

IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE

jx0ipdxwdilmapp_calculator.1leszko/calculator:latestubuntu-managerRunningRunning57secondsago

psweuemtb2wfapp_calculator.2leszko/calculator:latestubuntu-worker1RunningRunningaboutaminuteago

iuas0dmi7abnapp_calculator.3leszko/calculator:latestubuntu-worker2RunningRunning57secondsago

$dockerservicepsapp_redis

IDNAMEIMAGENODEDESIREDSTATECURRENTSTATE

8sg1ybbggx3lapp_redis.1redis:latestubuntu-managerRunningRunningaboutaminuteago

Wecanseethatoneofthecalculatorcontainersandtherediscontainersarestartedontheubuntu-managermachine.Twoothercalculatorcontainersrunontheubuntu-worker1andubuntu-worker2machines.

Notethatweexplicitlyspecifiedtheportnumberunderwhichthecalculatorwebserviceshouldbepublished.Therefore,weareabletoaccesstheendpointviathemanager'sIPaddresshttp://192.168.0.143:8881/sum?a=1&b=2.Theoperationreturns3asaresultandcachesitintheRediscontainer.

RemovingthestackWhenwe'redonewiththestack,wecanremoveeverythingusingtheconvenientdockerstackrmcommand:

$dockerstackrmapp

Removingserviceapp_calculator

Removingserviceapp_redis

Removingnetworkapp_default

UsingDockerStackallowsrunningtheDockerComposespecificationontheDockerSwarmcluster.Notethatweusedtheexactdocker-compose.ymlformat,whichisagreatbenefit,becausethereisnoneedtospecifyanythingextrafortheSwarm.

ThemergerofbothtechnologiesgivesustherealpowerofdeployingapplicationsonDockerbecausewedon'tneedtothinkaboutindividualmachines.Allweneedtospecifyishowour(micro)servicesaredependentoneachother,expressitinthedocker-compose.ymlformat,andletDockertakecareofeverythingelse.Thephysicalmachinescanbethentreatedsimplyasasetofresources.

AlternativeclustermanagementsystemsDockerSwarmisnottheonlysystemforclusteringDockercontainers.Eventhoughit'stheoneavailableoutofthebox,theremaybesomevalidreasonstoinstallathird-partyclustermanager.Let'swalkthroughthemostpopularalternatives.

KubernetesKubernetesisanopensourceclustermanagementsystemoriginallydesignedbyGoogle.Eventhoughit'snotDocker-native,theintegrationissmooth,andtherearemanyadditionaltoolsthathelpwiththisprocess;forexample,komposecantranslatethedocker-compose.ymlfilesintoKubernetesconfigurationfiles.

Let'stakealookatthesimplifiedarchitectureofKubernetes:

KubernetesissimilartoDockerSwarminawaythatitalsohasmasterandworkernodes.Additionally,itintroducestheconceptofapodthatrepresentsagroupofcontainersdeployedandscheduledtogether.Mostpodshaveafewcontainersthatmakeupaservice.Podsaredynamicallybuiltandremoveddependingonthechangingrequirements.

Kubernetesisrelativelyyoung.Itsdevelopmentstartedin2014;however,it'sbasedonGoogle'sexperience,andthisisoneofthereasonswhyit'soneofthemostpopularclustermanagementsystemsavailableonthemarket.TherearemoreandmoreorganizationsthatmigratedtoKubernetes,suchaseBay,Wikipedia,andPearson.

ApacheMesosApacheMesosisanopensourceschedulingandclusteringsystemstartedattheUniversityofCalifornia,Berkeley,in2009,longbeforeDockeremerged.ItprovidesanabstractionlayeroverCPU,diskspace,andRAM.OneofthegreatadvantagesofMesosisthatitsupportsanyLinuxapplication,notnecessarily(Docker)containers.Thisiswhyit'spossibletocreateaclusteroutofthousandsofmachinesanduseitforbothDockercontainersandotherprograms,forexample,Hadoop-basedcalculations.

Let'slookatthefigurepresentingtheMesosarchitecture:

ApacheMesos,similartootherclusteringsystems,hasthemaster-slavearchitecture.Itusesnodeagentsinstalledoneverynodeforcommunication,anditprovidestwotypesofschedulers,Chronos-forcron-stylerepeatingtasksandMarathon-providingaRESTAPItoorchestrateservicesandcontainers.

ApacheMesosisverymaturecomparedtootherclusteringsystems,andithasbeenadoptedinalargenumberoforganizations,suchasTwitter,Uber,andCERN.

ComparingfeaturesKubernetes,DockerSwarm,andMesosareallgoodchoicesfortheclustermanagementsystem.Allofthemarefreeandopensource,andallofthemprovideimportantclustermanagementfeaturessuchasloadbalancing,servicediscovery,distributedstorage,failurerecovery,monitoring,secretmanagement,androllingupdates.AllofthemcanalsobeusedintheContinuousDeliveryprocesswithouthugedifferences.Thisisbecause,intheDockerizedinfrastructure,theyalladdressthesameissue,theclusteringofDockercontainers.Nevertheless,obviously,thesystemsarenotexactlythesame.Let'shavealookatatablepresentingthedifferences:

DockerSwarm Kubernetes ApacheMesos

Dockersupport Native

SupportsDockerasoneofthecontainertypesinthepod

Mesosagents(slaves)canbeconfiguredtohostDockercontainers

Applicationtypes

Dockerimages

Containerizedapplications(Docker,rkt,andhyper)

AnyapplicationthatcanberunonLinux(alsocontainers)

Applicationdefinition

DockerComposeconfiguration

Podsconfiguration,replicasets,replicationcontrollers,services,anddeployments

Applicationgroupsformedinthetreestructure

Setupprocess Verysimple

Dependingontheinfrastructure,itmayrequirerunningonecommandormanycomplexoperations

Fairlyinvolved,itrequiresconfiguringMesos,Marathon,Chronos,Zookeeper,andDockersupport

API DockerRESTAPI RESTAPI Chronos/Marathon

RESTAPI

UserInterface

Dockerconsoleclient,third-partywebapplications,suchasShipyard

Consoletools,nativeWebUI(KubernetesDashboard)

OfficialwebinterfacesforMesos,Marathon,andChronos

Cloudintegration

Manualinstallationrequired

Cloud-nativesupportfrommostproviders(Azure,AWS,GoogleCloud,andothers)

Supportfrommostcloudproviders

Maximumclustersize 1,000nodes 1,000nodes 50,000nodes

Provideshorizontalpod

Marathonprovidesautoscalingbasedonresource(CPU/Memory)

Autoscaling Notavailable autoscalingbasedontheobservedCPUusage

consumption,numberofrequestspersecond,andqueuelength

Obviously,apartfromDockerSwarm,Kubernetes,andApacheMesos,thereareotherclusteringsystemsavailableinthemarket.Theyare,however,notthatpopularandtheirusagedecreasesovertime.

Nomatterwhichsystemyouchoose,youcanuseitnotonlyforthestaging/productionenvironmentsbutalsotoscaleJenkinsagents.Let'shavealookathowtodoit.

ScalingJenkinsTheobvioususecasesforserverclusteringarethestagingandproductionenvironments.Whenused,it'senoughtoattachaphysicalmachineinordertoincreasethecapacityoftheenvironment.Nevertheless,inthecontextofContinuousDelivery,wemayalsowanttoimprovetheJenkinsinfrastructurebyrunningJenkinsagent(slave)nodesonacluster.Inthissection,wetakealookattwodifferentmethodstoachievethisgoal.

DynamicslaveprovisioningWesawdynamicslaveprovisioninginChapter3,ConfiguringJenkins.WithDockerSwarm,theideastaysexactlythesame.Whenthebuildisstarted,theJenkinsmasterrunsacontainerfromtheJenkinsslaveDockerimage,andtheJenkinsfilescriptisexecutedinsidethecontainer.DockerSwarm,however,makesthesolutionmorepowerfulsincewearenotlimitedtoasingleDockerhostmachinebutcanproviderealhorizontalscaling.AddinganewDockerhosttotheclustereffectivelyscalesupthecapacityoftheJenkinsinfrastructure.

Atthetimeofwriting,theJenkinsDockerplugindoesnotsupportDockerSwarm.OneofthesolutionsistouseKubernetesorMesosastheclustermanagementsystem.EachofthemhasadedicatedJenkinsplugin:KubernetesPlugin(https://wiki.jenkins.io/display/JENKINS/Kubernetes+Plugin)andMesosPlugin(https://wiki.jenkins.io/display/JENKINS/Mesos+Plugin).

Nomatterhowtheslavesareprovisioned,wealwaysconfigurethembyinstallingtheappropriatepluginandaddingtheentrytotheCloudsectioninManageJenkins|ConfigureSystem.

JenkinsSwarmIfwedon'twanttousethedynamicslaveprovisioning,thenanothersolutionforclusteringJenkinsslavesistouseJenkinsSwarm.WedescribedhowtouseitinChapter3,ConfiguringJenkins.Here,weaddthedescriptionforDockerSwarm.

First,let'shavealookathowtoruntheJenkinsSwarmslaveusingtheDockerimagebuiltfromtheswarm-client.jartool.ThereareafewofthemavailableonDockerHub;wecanusethecsanchez/jenkins-swarm-slaveimage:

$dockerruncsanchez/jenkins-swarm-slave:1.16-master-username-password-namejenkins-swarm-slave-2

ThiscommandexecutionshouldhaveexactlythesameeffectastheonpresentedinChapter3,ConfiguringJenkins;itdynamicallyaddsaslavetotheJenkinsmaster.

Then,togetthemostofJenkinsSwarm,wecanruntheslavecontainersontheDockerSwarmcluster:

$dockerservicecreate--replicas5--namejenkins-swarm-slavecsanchez/jenkins-swarm-slave-master-disableSslVerification-username-password-namejenkins-swarm-slave

TheprecedingcommandstartsfiveslavesontheclusterandattachesthemtotheJenkinsmaster.PleasenotethatitisverysimpletoscaleJenkinshorizontallybyexecutingthedockerservicescalecommand.

ComparisonofdynamicslaveprovisioningandJenkinsSwarmDynamicslaveprovisioningandJenkinsSwarmcanbebothrunonaclusterthatresultsinthearchitecturepresentedinthefollowingdiagram:

Jenkinsslavesarerunontheclusterandthereforeareveryeasilyscaledupanddown.IfweneedmoreJenkinsresources,wescaleupJenkinsslaves.Ifweneedmoreclusterresources,weaddmorephysicalmachinestothecluster.

ThedifferencebetweenthetwosolutionsisthatthedynamicslaveprovisioningautomaticallyaddsaJenkinsslavetotheclusterbeforeeachbuild.Thebenefitofsuchapproachisthatwedon'tevenhavetothinkabouthowmanyJenkinsslavesshouldberunningatthemomentsincethenumberautomaticallyadaptstothenumberofpipelinebuilds.Thisiswhy,inmostcases,thedynamicslaveprovisioningisthefirstchoice.Nevertheless,JenkinsSwarmalsocarriesafewsignificantbenefits:

Controlofthenumberofslaves:UsingJenkinsSwarm,weexplicitlydecidehowmanyJenkinsslavesshouldberunningatthemoment.Statefulslaves:ManybuildssharethesameJenkinsslave,whichmaysoundlikeadrawback;however,itbecomesanadvantagewhenabuildrequiresdownloadingalotofdependentlibrariesfromtheinternet.Inthe

caseofdynamicslaveprovisioning,tocachethedependencies,wewouldneedtosetupasharedvolume.Controlofwheretheslavesarerunning:UsingJenkinsSwarm,wecandecidenottorunslavesontheclusterbuttochoosethehostmachinedynamically;forexample,formanystartups,whentheclusterinfrastructureiscostly,slavescanbedynamicallyrunonthelaptopofadeveloperwhoisstartingthebuild.

ClusteringJenkinsslavesbringalotofbenefitsanditiswhatthemodernJenkinsarchitectureshouldlooklike.Thisway,wecanprovidethedynamichorizontalscalingoftheinfrastructurefortheContinuousDeliveryprocess.

ExercisesInthischapter,wehavecoveredDockerSwarmandtheclusteringprocessindetail.Inordertoenhancethisknowledge,werecommendthefollowingexercises:

1. Setupaswarmclusterthatconsistsofthreenodes:

UseonemachineasthemanagernodeandtwomachinesasworkernodesYoucanusephysicalmachinesconnectedtoonenetwork,machinesfromthecloudprovider,orVirtualBoxmachineswiththesharednetworkCheckthattheclusterisconfiguredproperlyusingthedockernodecommand

2. Run/scaleahelloworldserviceonthecluster:

TheservicecanlookexactlythesameasdescribedintheexercisesforChapter2,IntroducingDockerPublishtheportsothatitwillbeaccessiblefromoutsideoftheclusterScaletheservicetofivereplicasMakearequesttothe"helloworld"serviceandcheckwhichofthecontainersisservingtherequest

3. ScaleJenkinsusingslavesdeployedontheSwarmcluster:

UseJenkinsSwarmordynamicslaveprovisioningRunapipelinebuildandcheckthatitisexecutedononeoftheclusteredslaves

SummaryInthischapter,wetookalookattheclusteringmethodsforDockerenvironmentsthatenablesettingupthecompletestaging/production/Jenkinsenvironment.Herearethekeytakeawaysfromthechapter:

Clusteringisamethodofconfiguringasetofmachinesinawaythat,inmanyrespects,canbeviewedasasinglesystemDockerSwarmisthenativeclusteringsystemforDockerDockerSwarmclusterscanbedynamicallyconfiguredusingbuilt-inDockercommandsDockerimagescanberunandscaledontheclusterusingthedockerservicecommandDockerStackisamethodtoruntheDockerComposeconfigurationonaSwarmclusterThemostpopularclusteringsystemsthatsupportDockerareDockerSwarm,Kubernetes,andApacheMesosJenkinsagentscanberunonaclusterusingthedynamicslaveprovisioningortheJenkinsSwarmplugin

Inthenextchapter,wewilldescribethemoreadvancedaspectsoftheContinuousDeliveryprocessandpresentthebestpracticesforbuildingpipelines

AdvancedContinuousDelivery

Inthelastchapter,wecoveredhowserverclusteringworksandhowwecanuseittogetherwithDockerandJenkins.Inthischapter,wewillseeamixtureofdifferentaspectsthatareveryimportantintheContinuousDeliveryprocessbuthavenotbeendescribedyet.

Thischaptercoversthefollowingpoints:

ExplaininghowtoapproachdatabasechangesinthecontextofContinuousDeliveryIntroducingtheideaofdatabasemigrationandrelatedtoolsExploringdifferentapproachestobackwards-compatibleandbackwards-incompatibledatabaseupdatesUsingparallelstepsintheJenkinspipelineCreatingaJenkinssharedlibraryPresentingawaytorollbackproductionchangesIntroducingContinuousDeliveryforlegacysystemsExploringhowtopreparezero-downtimedeploymentsPresentingContinuousDeliverybestpractices

ManagingdatabasechangesSofar,wehavefocusedontheContinuousDeliveryprocess,whichwasappliedtoawebservice.Asimplepartofthiswasthatwebservicesareinherentlystateless.Thisfactmeansthattheycanbeeasilyupdated,restarted,clonedinmanyinstances,andrecreatedfromthegivensourcecode.Awebservice,however,isusuallylinkedtoitsstatefulpart,adatabasethatposesnewchallengestothedeliveryprocess.Thesechallengescanbegroupedintothefollowingcategories:

Compatibility:ThedatabaseschemaandthedataitselfmustbecompatiblewiththewebserviceallthetimeZero-downtimedeployment:Inordertoachievezero-downtimedeployment,weuserollingupdates,whichmeansthatadatabasemustbecompatiblewithtwodifferentwebserviceversionsatthesametimeRollback:Arollbackofadatabasecanbedifficult,limited,orsometimesevenimpossiblebecausenotalloperationsarereversible(forexample,removingacolumnthatcontainsdata)Testdata:Database-relatedchangesaredifficulttotestbecauseweneedtestdatathatisverysimilartoproduction

Inthissection,IwillexplainhowtoaddressthesechallengessothattheContinuousDeliveryprocesswillbeassafeaspossible.

UnderstandingschemaupdatesIfyouthinkaboutthedeliveryprocess,it'snotreallythedataitselfthatcausedifficultiesbecausewedon'tusuallychangethedatawhenwedeployanapplication.Thedataissomethingthatiscollectedwhilethesystemisliveintheproduction;whereas,duringdeployment,weonlychangethewaywestoreandinterpretthisdata.Inotherwords,inthecontextoftheContinuousDeliveryprocess,weareinterestedinthestructureofthedatabase,notexactlyinitscontent.Thisiswhythissectionconcernsmainlyrelationaldatabases(andtheirschemas)andfocuseslessonothertypesofstoragesuchasNoSQLdatabases,wherethereisnostructuredefinition.

Tounderstandthisbetter,wethinkofRedis,whichwehavealreadyusedinthisbook.Itstoredthecacheddata,soeffectivelyitwasadatabase.Nevertheless,itrequiredzeroeffortfromtheContinuousDeliveryperspectivesinceitdidn'thaveanydatastructure.Allitstoredwasthekey-valueentries,whichdoesnotevolveovertime.

NoSQLdatabasesusuallydon'thaveanyrestrictingschemaandthereforesimplifytheContinuousDeliveryprocessbecausethereisnoadditionalschemaupdatesteprequired.Thisisahugebenefit;however,itdoesn'tnecessarilymeanthatwritingapplicationswithNoSQLdatabasesissimplerbecausewehaveputmoreeffortintodatavalidationinthesourcecode.

Relationaldatabaseshavestaticschemas.Ifwewouldliketochangeit,forexample,toaddanewcolumntothetable,weneedtowriteandexecuteaSQLDDL(datadefinitionlanguage)script.Doingthismanuallyforeverychangerequiresalotofworkandleadstoerror-pronesolutions,inwhichtheoperationsteamhastokeepinsyncthecodeandthedatabasestructure.Amuchbettersolutionistoautomaticallyupdatetheschemainanincrementalmanner.Suchasolutioniscalleddatabasemigration.

IntroducingdatabasemigrationsDatabaseschemamigrationisaprocessofincrementalchangestotherelationaldatabasestructure.Let'shavealookatthefollowingdiagramtounderstanditbetter:

Thedatabaseintheversionv1hastheschemadefinedbytheV1_init.sqlfile.Additionally,itstoresthemetadatarelatedtothemigrationprocess,forexample,itscurrentschemaversionandthemigrationchangelog.Whenwewanttoupdatetheschema,weprovidethechangesintheformofaSQLfile,suchasV2_add_table.sql.Then,weneedtorunthemigrationtoolthatexecutesthegivenSQLfileonthedatabase(italsoupdatesthemetatables).Ineffect,thedatabaseschemaisaresultofallsubsequentlyexecutedSQLmigrationscripts.Next,wewillseeanexampleofamigration.

Migrationscriptsshouldbestoredintheversioncontrolsystem,usuallyinthesamerepositoryasthesourcecode.

Migrationtoolsandthestrategiestheyusecanbedividedintotwocategories:

Upgradeanddowngrade:Thisapproach,forexample,usedbytheRuby

onRailsframework,meansthatwecanmigrateup(fromv1tov2)anddown(fromv2tov1).Itallowsthedatabaseschematorollback,whichmaysometimesendupindataloss(ifthemigrationislogicallyirreversible).Upgradeonly:Thisapproach,forexample,usedbytheFlywaytool,onlyallowsustomigrateup(fromv1tov2).Inmanycases,thedatabaseupdatesarenotreversible,forexample,removingatablefromthedatabase.Suchachangecannotberolledbackbecauseevenifwerecreatethetable,wehavealreadylostallthedata.

Therearemanydatabasemigrationtoolsavailableonthemarket,outofwhichthemostpopularareFlyway,Liquibase,andRailMigrations(fromtheRubyonRailsframework).Asanextsteptounderstandhowsuchtoolswork,wewillseeanexamplebasedontheFlywaytool.

Therearealsocommercialsolutionsprovidedfortheparticulardatabases,forexample,Redgate(forSQLServer)andOptimDatabaseAdministrator(forDB2).

UsingFlyway

Let'suseFlywaytocreateadatabaseschemaforthecalculatorwebservice.Thedatabasewillstorethehistoryofalloperationsthatwereexecutedontheservice:thefirstparameter,thesecondparameter,andtheresult.

WeshowhowtousetheSQLdatabaseandFlywayinthreesteps:

1. ConfiguringtheFlywaytooltoworktogetherwithGradle.2. DefiningtheSQLmigrationscripttocreatethecalculationhistorytable.3. UsingtheSQLdatabaseinsidetheSpringBootapplicationcode.

ConfiguringFlywayInordertouseFlywaytogetherwithGradle,weneedtoaddthefollowingcontenttothebuild.gradlefile:buildscript{dependencies{classpath('com.h2database:h2:1.4.191')}}…plugins{id"org.flywaydb.flyway"version"4.2.0"}…flyway{url='jdbc:h2:file:/tmp/calculator'user='sa'}

Here'saquickcommentontheconfiguration:

WeusedtheH2database,whichisanin-memory(andfile-based)databaseWestorethedatabaseinthe/tmp/calculatorfileThedefaultdatabaseuseriscalledsa(systemadministrator)

InthecaseofotherSQLdatabases(forexample,MySQL),theconfigurationwouldbeverysimilar.TheonlydifferenceisintheGradledependenciesandtheJDBCconnection.

Afterthisconfigurationisapplied,weshouldbeabletoruntheFlywaytoolbyexecutingthefollowingcommand:$./gradlewflywayMigrate-i

Thecommandcreatedthedatabaseinthefile/tmp/calculator.mv.db.Obviously,ithasnoschemasincewehaven'tdefinedanythingyet.

Flywaycanbeusedasacommand-linetool,viaJavaAPI,orasapluginforthepopularbuildingtoolsGradle,Maven,andAnt.

DefiningtheSQLmigrationscriptThenextstepistodefinetheSQLfilethataddsthecalculationtableintothedatabaseschema.Let'screatethesrc/main/resources/db/migration/V1__Create_calculation_table.sqlfilewiththefollowingcontent:

createtableCALCULATION(

IDintnotnullauto_increment,

Avarchar(100),

Bvarchar(100),

RESULTvarchar(100),

primarykey(ID)

);

Notethemigrationfilenamingconvention,<version>__<change_description>.sql.TheSQLfilecreatesatablewithfourcolumns,ID,A,B,RESULT.TheIDcolumnisanautomaticallyincrementedprimarykeyofthetable.Now,wearereadytoruntheFlywaycommandtoapplythemigration:

$./gradlewflywayMigrate-i

…

Successfullyapplied1migrationtoschema"PUBLIC"(executiontime00:00.028s).

:flywayMigrate(Thread[DaemonworkerThread2,5,main])completed.Took1.114secs.

Thecommandautomaticallydetectedthemigrationfileandexecuteditonthedatabase.

Themigrationfilesshouldbealwayskeptintheversioncontrolsystem,usually,togetherwiththesourcecode.

AccessingdatabaseWeexecutedourfirstmigration,sothedatabaseisprepared.Toseethecompleteexample,weshouldalsoadaptourprojectsothatitwouldaccessthedatabase.

Let'sfirstconfiguretheGradledependenciestousetheH2databasefromtheSpringBootproject.Wecandothisbyaddingthefollowinglinestothebuild.gradlefile:

dependencies{

compile("org.springframework.boot:spring-boot-starter-data-jpa")

compile("com.h2database:h2")

}

Thenextstepistosetupthedatabaselocationandthestartupbehaviorinthesrc/main/resources/application.propertiesfile:

spring.datasource.url=jdbc:h2:file:/tmp/calculator;DB_CLOSE_ON_EXIT=FALSE

spring.jpa.hibernate.ddl-auto=validate

ThesecondlinemeansthatSpringBootwillnottrytoautomaticallygeneratethedatabaseschemafromthesourcecodemodel.Onthecontrary,itwillonlyvalidateifthedatabaseschemaisconsistentwiththeJavamodel.

Now,let'screatetheJavaORMentitymodelforthecalculationinthenewsrc/main/java/com/leszko/calculator/Calculation.javafile:

packagecom.leszko.calculator;

importjavax.persistence.Entity;

importjavax.persistence.GeneratedValue;

importjavax.persistence.GenerationType;

importjavax.persistence.Id;

@Entity

publicclassCalculation{

@Id

@GeneratedValue(strategy=GenerationType.AUTO)

privateIntegerid;

privateStringa;

privateStringb;

privateStringresult;

protectedCalculation(){}

publicCalculation(Stringa,Stringb,Stringresult){

this.a=a;

this.b=b;

this.result=result;

}

}

TheentityclassrepresentsthedatabasemappingintheJavacode.Atableisexpressedasaclassandeachcolumnasafield.ThenextstepistocreatetherepositoryforloadingandstoringtheCalculationentities.

Let'screatethesrc/main/java/com/leszko/calculator/CalculationRepository.javafile:

packagecom.leszko.calculator;

importorg.springframework.data.repository.CrudRepository;

publicinterfaceCalculationRepositoryextendsCrudRepository<Calculation,Integer>{}

Finally,wecanusetheCalculationandCalculationRepositoryclassestostorethecalculationhistory.Let'saddthefollowingcodetothesrc/main/java/com/leszko/calculator/CalculatorController.javafile:

...

classCalculatorController{

...

@Autowired

privateCalculationRepositorycalculationRepository;

@RequestMapping("/sum")

Stringsum(@RequestParam("a")Integera,@RequestParam("b")Integerb){

Stringresult=String.valueOf(calculator.sum(a,b));

calculationRepository.save(newCalculation(a.toString(),b.toString(),result));

returnresult;

}

}

Now,whenwestarttheserviceandexecutethe/sumendpoint,eachsummingoperationisloggedintothedatabase.

Ifyouwouldliketobrowsethedatabasecontent,thenyoucanaddspring.h2.console.enabled=truetotheapplication.propertiesfile,andthenbrowsethedatabaseviathe/h2-consoleendpoint.

WeexplainedhowthedatabaseschemamigrationworksandhowtouseitinsideaSpringproject,builtwithGradle.Now,let'shavealookathowitintegrateswithintheContinuousDeliveryprocess.

ChangingdatabaseinContinuousDeliveryThefirstapproachtousedatabaseupdatesinsidetheContinuousDeliverypipelinecouldbetoaddastagewithinthemigrationcommandexecution.Thissimplesolutionwouldworkcorrectlyformanycases;however,ithastwosignificantdrawbacks:

Rollback:Asmentionedbefore,it'snotalwayspossibletorollbackthedatabasechange(Flywaydoesn'tsupportdowngradesatall).Therefore,inthecaseofservicerollback,thedatabasebecomesincompatible.Downtime:Theserviceupdateandthedatabaseupdatearenotexecutedexactlyatthesametime,whichcausesdowntime.

Thisleadsustotwoconstraintsthatwewillneedtoaddress:

ThedatabaseversionneedstobecompatiblewiththeserviceversionallthetimeThedatabaseschemamigrationisnotreversible

Wewilladdresstheseconstraintsfortwodifferentcases:backwards-compatibleupdatesandnon-backwards-compatibleupdates.

Backwards-compatiblechangesBackwards-compatiblechangesaresimpler.Let'slookatthefollowingfiguretoseehowtheywork:

SupposetheschemamigrationDatabasev10isbackwards-compatible.IfweneedtorollbacktheServicev1.2.8release,thenwedeployServicev1.2.7,andthereisnoneedtodoanythingwiththedatabase(databasemigrationsarenotreversible,sowekeepDatabasev11).Sincetheschemaupdateisbackwards-compatible,Servicev.1.2.7worksperfectlyfinewithDatabasev11.ThesameappliesifweneedtorollbacktoServicev1.2.6,andsoon.Now,supposeDatabasev10andallothermigrationsarebackwards-compatible,thenwecouldrollbacktoanyserviceversionandeverythingwouldworkcorrectly.

Thereisalsonoproblemwiththedowntime.Ifthedatabasemigrationiszero-downtimeitself,thenwecanexecuteitfirstandthenusetherollingupdatesfortheservice.

Let'slookatanexampleofabackwards-compatiblechange.Wewillcreateaschemaupdatethataddsacreated_atcolumntothecalculationtable.Themigrationfilesrc/main/resources/db/migration/V2__Add_created_at_column.sqllooksasfollows:

altertableCALCULATION

addCREATED_ATtimestamp;

Besidesthemigrationscript,thecalculatorservicerequiresanewfieldintheCalculationclass:

...

privateTimestampcreatedAt;

...

WealsoneedtoadjustitsconstructorandthenitsusageintheCalculatorControllerclass:

calculationRepository.save(newCalculation(a.toString(),b.toString(),result,Timestamp.from(Instant.now())));

Afterrunningtheservice,thecalculationhistoryisstoredtogetherwiththecreated_atcolumn.Notethatthechangeisbackwards-compatiblebecauseevenifwereverttheJavacodeandleavethecreated_atcolumninthedatabase,everythingwouldworkperfectlyfine(therevertedcodedoesnotaddressthenewcolumnatall).

Non-backwards-compatiblechangesNon-backwards-compatiblechangesarewaymoredifficult.Lookingatthepreviousfigure,ifdatabasechangev11wasbackwards-incompatible,itwouldbeimpossibletorollbacktheserviceto1.2.7.Inthiscase,howcanweapproachnon-backwards-compatibledatabasemigrationssothatrollbacksandzero-downtimedeploymentswouldbepossible?

Tomakealongstoryshort,wecanaddressthisissuebyconvertinganon-backwards-compatiblechangeintoachangethatisbackwards-compatibleforacertainperiodoftime.Inotherwords,weneedtoputintheextraeffortandsplittheschemamigrationintotwoparts:

Backwards-compatibleupdateexecutednow,whichusuallymeanskeepingsomeredundantdataNon-backwards-compatibleupdateexecutedaftertherollbackperiodtimethatdefineshowfarbackwecanrevertourcode

Toillustratethisbetter,let'slookatthefollowingimage:

Let'sthinkaboutanexampleofdroppingacolumn.Aproposedmethodwouldincludetwosteps:

1. Stopusingthecolumninthesourcecode(v1.2.5,backwards-compatibleupdate,executedfirst).

2. Dropthecolumnfromthedatabase(v11,non-backwards-compatibleupdate,executedaftertherollbackperiod).

AllserviceversionsuntilDatabasev11canberolledbacktoanypreviousversion,theservicesstartingfromServicev1.2.8canberolledbackonlywithintherollbackperiod.Suchapproachmaysoundtrivialbecauseallwedidwasdelaythecolumnremovalfromthedatabase.However,itaddressesboththerollbackissueandthezero-downtimedeploymentissue.Asaresult,itreducestheriskassociatedwiththerelease.Ifweadjusttherollbackperiodtoareasonableamountoftime,forexample,inthecaseofmultiplereleasesperdaytotwoweeks,thentheriskisnegligible.Wedon'tusuallyrollmanyversionsback.

Droppingacolumnwasaverysimpleexample.Let'shavealookatamoredifficultscenarioandrenametheresultcolumninourcalculatorservice.Wepresenthowtodothisinafewsteps:

1. Addinganewcolumntothedatabase.2. Changingthecodetousebothcolumns.3. Mergingthedatainbothcolumns.4. Removingtheoldcolumnfromthecode.5. Droppingtheoldcolumnfromthedatabase.

altertableCALCULATION<br/>addSUMvarchar(100);

Asaresult,afterexecutingthemigration,wehavetwocolumns:resultandsum.

ChangingthecodetousebothcolumnsThenextstepistorenamethecolumninthesourcecodemodelandtousebothdatabasecolumnsforthesetandgetoperations.WecanchangeitintheCalculationclass:publicclassCalculation{...privateStringsum;...publicCalculation(Stringa,Stringb,Stringsum,TimestampcreatedAt){this.a=a;this.b=b;this.sum=sum;this.result=sum;this.createdAt=createdAt;}

publicStringgetSum(){returnsum!=null?sum:result;}}

Tobe100%accurate,inthegetSum()method,weshouldcomparesomethinglikethelastmodificationcolumndate(notexactlynecessarytoalwaystakethenewcolumnfirst).

Fromnowon,everytimeweaddarowintothedatabase,thesamevalueiswrittentoboththeresultandsumcolumns.Whilereadingsum,wefirstcheckifitexistsinthenewcolumn,andifnot,wereaditfromtheoldcolumn.

Thesameresultcanbeachievedwiththeuseofdatabasetriggersthatwouldautomaticallywritethesamevaluesintobothcolumns.

Allthechangeswemadesofarwerebackwards-compatible,sowecanrollbacktheserviceanytimewewant,toanyversionwewant.

updateCALCULATION<br/>setCALCULATION.sum=CALCULATION.result<br/>whereCALCULATION.sumisnull;

Westillhavenolimitsfortherollback;however,ifweneedtodeploytheversionbeforethechangeinstep2,thenthisdatabasemigrationneedstoberepeated.

publicclassCalculation{<br/>...<br/>privateStringsum;<br/>...<br/>publicCalculation(Stringa,Stringb,Stringsum,TimestampcreatedAt){<br/>this.a=a;<br/>this.b=b;<br/>this.sum=sum;<br/>this.createdAt=createdAt;<br/>}<br/><br/>publicStringgetSum(){<br/>returnsum;<br/>}<br/>}

Afterthisoperation,wenolongerusetheresultcolumninthecode.Notethatthisoperationisonlybackwards-compatibleuptostep2.Ifweneedtorollbacktostep1,thenwecouldlosethedatastoredafterthisstep.

DroppingtheoldcolumnfromthedatabaseThelaststepistodroptheoldcolumnfromthedatabase.Thismigrationshouldbeperformedaftertherollbackperiodwhenwearesurewewon'tneedtorollbackbeforestep4.

Therollbackperiodcanbeverylongsincewearen'tusingthecolumnfromthedatabaseanymore.Thistaskcanbetreatedasacleanuptask,soeventhoughit'snon-backwards-compatible,thereisnoassociatedrisk.

Let'saddthefinalmigration,V5__Drop_result_column.sql:

altertableCALCULATION

dropcolumnRESULT;

Afterthisstep,wefinallycompletedthecolumnrenamingprocedure.Notethatallwedidwascomplicatetheoperationalittlebit,inordertostretchitintime.Thisreducedtheriskofbackwards-incompatibledatabasechangesandallowedzero-downtimedeployments.

SeparatingdatabaseupdatesfromcodechangesSofar,inallfigures,wepresentedthatdatabasemigrationsareruntogetherwithservicereleases.Inotherwords,eachcommit(whichimplieseachrelease)tookbothdatabasechangesandcodechanges.However,therecommendedapproachistomakeaclearseparationthatacommittotherepositoryiseitheradatabaseupdateoracodechange.Thismethodispresentedinthefollowingimage:

Thebenefitofdatabase-servicechangeseparationisthatwegetthebackwards-compatibilitycheckforfree.Imaginethatthechangesv11andv1.2.7concernonelogicalchange,forexample,addinganewcolumntothedatabase.Then,wefirstcommitdatabasev11,sothetestsintheContinuousDeliverypipelinecheckifdatabasev11workscorrectlywithservicev.1.2.6.Inotherwords,theycheckifdatabaseupdatev11isbackwards-compatible.Then,wecommitthev1.2.7change,sothepipelinechecksifdatabasev11worksfinewithservicev1.2.7.

Thedatabase-codeseparationdoesnotmeanthatwemusthavetwoseparateJenkinspipelines.Thepipelinecanalwaysexecuteboth,butweshouldkeepitasagoodpracticethatacommitiseitheradatabaseupdateoracodechange.

Tosumup,thedatabaseschemachangesshouldbeneverdonemanually.Instead,weshouldalwaysautomatethemusingamigrationtool,executedasapartoftheContinuousDeliverypipeline.Weshouldalsoavoidnon-backwards-compatibledatabaseupdatesandthebestwaytoassurethisistocommit

separatelythedatabaseandcodechangesintotherepository.

AvoidingshareddatabaseInmanysystems,wecanspotthatthedatabasebecomesthecentralpointthatissharedbetweenmultipleservices.Insuchacase,anyupdatetothedatabasebecomesmuchmorechallengingbecauseweneedtocoordinateitbetweenallservices.

Forexample,imaginewedevelopanonlineshopandwehaveaCustomerstablethatcontainsthefollowingcolumns:firstname,lastname,username,password,email,anddiscount.Therearethreeservicesthatareinterestedinthecustomer'sdata:

Profilemanager:Thisenableseditinguser'sdataCheckoutprocessor:Thisprocessesthecheckout(readsusernameandemail)Discountmanager:Thisanalyzesthecustomer'sordersandsetsthesuitablediscount

Let'slookatthefollowingimagethatpresentsthissituation:

Theyaredependentonthesamedatabaseschema.Thereareatleasttwoissueswithsuchanapproach:

Whenwewanttoupdatetheschema,itmustbecompatiblewithallthreeservices.Whileallbackwards-compatiblechangesarefine,anynon-backwards-compatibleupdatebecomeswaymoredifficultoreven

impossible.EachservicehasaseparatedeliverycycleandaseparateContinuousDeliverypipeline.Then,whichpipelineshouldweuseforthedatabaseschemamigrations?Unfortunately,thereisnogoodanswertothisquestion.

Forthereasonsmentionedpreviously,eachserviceshouldhaveitsowndatabaseandtheservicesshouldcommunicateviatheirAPIs.Followingourexample,wecouldapplythefollowingrefactoring:

Thecheckoutprocessorshouldcommunicatewiththeprofilemanager'sAPItofetchthecustomer'sdataThediscountcolumnshouldbeextractedtoaseparatedatabase(orschema),andthediscountmanagershouldtaketheownership

Therefactoredversionispresentedinthefollowingimage:

Suchanapproachisconsistentwiththeprinciplesofthemicroservicearchitectureandshouldalwaysbeapplied.ThecommunicationoverAPIsiswaymoreflexiblethanthedirectdatabaseaccess.

Inthecaseofmonolithicsystems,adatabaseisusuallytheintegrationpoint.Sincesuchanapproachcausesalotofissues,it'sconsideredasananti-pattern.

PreparingtestdataWehavealreadypresenteddatabasemigrationsthatkeepthedatabaseschemaconsistentbetweentheenvironmentsasasideeffect.Thisisduetothefactthatifwerunthesamemigrationscriptsonthedevelopmentmachine,inthestagingenvironment,orintheproduction,thenwewouldalwaysgettheresultinthesameschema.However,thedatavaluesinsidethetablesdiffer.Howcanwepreparethetestdatasothatitwouldeffectivelytestoursystem?Thisisthetopicofthissection.

Theanswertothisquestiondependsonthetypeofthetest,anditisdifferentforunittesting,integration/acceptancetesting,andperformancetesting.Let'sexamineeachcase.

UnittestingInthecaseofunittesting,wedon'tusetherealdatabase.Weeithermockthetestdataonthelevelofthepersistencemechanism(repositories,dataaccessobjects)orwefaketherealdatabasewithanin-memorydatabase(forexample,H2database).Sinceunittestsarecreatedbydevelopers,theexactdatavaluesareusuallyinventedbydevelopersandtheydon'tmattermuch.

Integration/acceptancetestingIntegrationandacceptancetestsusuallyusethetest/stagingdatabase,whichshouldbeassimilaraspossibletotheproduction.Oneapproach,takenbymanycompanies,istosnapshottheproductiondataintostagingthatguaranteesthatitisexactlythesame.Thisapproach,however,istreatedasananti-patternforthefollowingreasons:

Testisolation:Eachtestoperatesonthesamedatabase,sotheresultofonetestmayinfluencetheinputoftheothersDatasecurity:ProductioninstancesusuallystoresensitiveinformationandarethereforebettersecuredReproducibility:Aftereverysnapshot,thetestdataisdifferent,whichmayresultinflakytests

Fortheprecedingreasons,thepreferredapproachistomanuallypreparethetestdatabyselectingasubsetoftheproductiondata,togetherwiththecustomerorthebusinessanalyst.Whentheproductiondatabasegrows,it'sworthrevisitingitscontenttoseeifthereareanyreasonablecasesthatshouldbeadded.

ThebestwaytoadddatatothestagingdatabaseistousethepublicAPIofaservice.Thisapproachisconsistentwithacceptancetests,whichareusuallyblack-box.What'smore,usingtheAPIguaranteesthatthedataitselfisconsistentandsimplifiesdatabaserefactoringbylimitingdirectdatabaseoperations.

PerformancetestingTestdatafortheperformancetestingisusuallysimilartoacceptancetesting.Onesignificantdifferenceistheamountofdata.Inordertotesttheperformancecorrectly,weneedtoprovidesufficientvolumeofinputdata,atleastaslargeasavailableontheproduction(duringthepeaktime).Forthispurpose,wecancreatedatagenerators,whichareusuallysharedbetweenacceptanceandperformancetests.

PipelinepatternsWealreadyknoweverythingthatisnecessarytostartaprojectandsetuptheContinuousDeliverypipelinewithJenkinsandDocker.ThissectionisintendedtoextendthisknowledgewithafewoftherecommendedgeneralJenkinspipelinepractices.

ParallelizingpipelinesThroughoutthisbook,wehavealwaysexecutedthepipelinesequentially,stagebystage,stepbystep.Thisapproachmakesiteasytoreasonthestateandtheresultofthebuild.Ifthereisfirsttheacceptanceteststageandthenthereleasestage,itmeansthatthereleasewon'teverhappenuntiltheacceptancetestsaresuccessful.Sequentialpipelinesaresimpletounderstandandusuallydonotcauseanysurprises.Thisiswhythefirstmethodtosolveanyproblemistodoitsequentially.

However,insomecases,thestagesaretime-consumingandit'sworthrunningtheminparallel.Averygoodexampleisperformancetests.Theyusuallytakealotoftime,soassumingtheyareindependentandisolated,itmakessensetoruntheminparallel.InJenkins,wecanparallelizethepipelineontwodifferentlevels:

Parallelsteps:Withinonestage,parallelprocessesrunonthesameagent.ThismethodissimplebecauseallJenkinsworkspace-relatedfilesarelocatedononephysicalmachine,however,asalwayswiththeverticalscaling,theresourcesarelimitedtothatsinglemachine.Parallelstages:Eachstagecanberuninparallelonaseparateagentmachinethatprovideshorizontalscalingofresources.Weneedtotakecareofthefiletransferbetweentheenvironments(usingthestashJenkinsfilekeyword)ifafilecreatedinthepreviousstageisneededontheotherphysicalmachine.

Bythetimeofwritingthisbook,parallelstagesarenotavailableinthedeclarativepipeline.ThefeatureissupposedtobeaddedinJenkinsBlueOceanv1.3.Inthemeantime,theonlypossibilityistousethedeprecatedfeatureintheGroovy-basedscriptingpipeline,asdescribedhereathttps://jenkins.io/doc/book/pipeline/jenkinsfile/#executing-in-parallel.

Let'slookathowitlooksinpractice.Ifwewouldliketoruntwostepsinparallel,theJenkinsfilescriptshouldlookasfollows:

pipeline{

agentany

stages{

stage('Stage1'){

steps{

parallel(

one:{echo"parallelstep1"},

two:{echo"parallelstep2"}

)

}

}

stage('Stage2'){

steps{

echo"runafterbothparallelstepsarecompleted"

}

}

}

}

InStage1,withtheuseoftheparallelkeyword,weexecutetwoparallelsteps,oneandtwo.NotethatStage2isexecutedonlyafterbothparallelstepsarecompleted.Thisiswhysuchsolutionsareperfectlysafetoruntestsinparallel;wecanalwaysbesurethatthedeploymentstageisrunonlyafterallparallelizedtestshavealreadypassed.

ThereisaveryusefulplugincalledParallelTestExecutorthathelpstoautomaticallysplittestsandruntheminparallel.Readmoreathttps://jenkins.io/doc/pipeline/steps/parallel-test-executor/.

Theprecedingdescriptionconcernedtheparallelstepslevel.Theothersolutionwouldbetouseparallelstagesandthereforeruneachstageonaseparateagentmachine.Thedecisiononwhichtypeofparallelismtouseusuallydependsontwofactors:

HowpowerfultheagentmachinesareHowmuchtimethegivenstagetakes

Asageneralrecommendation,unittestsarefinetoruninparallelsteps,butperformancetestsareusuallybetteroffonseparatemachines.

ReusingpipelinecomponentsWhentheJenkinsfilescriptgrowsinsizeandbecomesmorecomplex,wemaywanttoreuseitspartsbetweensimilarpipelines.

Forexample,wemaywanttohaveseparate,butsimilar,pipelinesfordifferentenvironments(dev,QA,prod).AnothercommonexampleinthemicroserviceworldisthateachservicehasaverysimilarJenkinsfile.Then,howdowewriteJenkinsfilescriptssothatwedon'trepeatthesamecodealloveragain?Therearetwogoodpatternsforthispurpose,parameterizedbuildandsharedlibraries.Let'sdescribethemonebyone.

BuildparametersWealreadymentionedinChapter4,ContinuousIntegrationPipeline,thatapipelinecanhaveinputparameters.Wecanusethemtoprovidedifferentusecaseswiththesamepipelinecode.Asanexample,let'screateapipelineparametrizedwiththeenvironmenttype:

pipeline{

agentany

parameters{

string(name:'Environment',defaultValue:'dev',description:'Which

environment(dev,qa,prod)?')

}

stages{

stage('Environmentcheck'){

steps{

echo"Currentenvironment:${params.Environment}"

}

}

}

}

Thebuildtakesoneinputparameter,Environment.Then,allwedointhisstepisprinttheparameter.Wecanalsoaddaconditiontoexecutedifferentcodefordifferentenvironments.

Withthisconfiguration,whenwestartthebuild,wewillseeapromptfortheinputparameter,asfollows:

Parametrizedbuildcanhelpreusethepipelinecodeforscenarioswhenitdiffersjustalittlebit.Thisfeature,however,shouldnotbeoverusedbecausetoomanyconditionscanmaketheJenkinsfiledifficulttounderstand.

Sharedlibraries

Theothersolutiontoreusethepipelineistoextractitspartsintoasharedlibrary.

AsharedlibraryisaGroovycodethatisstoredasaseparatesource-controlledproject.ThiscodecanbelaterusedinmanyJenkinsfilescriptsaspipelinesteps.Tomakeitclear,let'shavealookatanexample.Asharedlibrarytechniquealwaysrequiresthreesteps:

1. Createasharedlibraryproject.2. ConfigurethesharedlibraryinJenkins.3. UsethesharedlibraryinJenkinsfile.

CreatingasharedlibraryprojectWestartbycreatinganewGitproject,inwhichweputthesharedlibrarycode.EachJenkinsstepisexpressedasaGroovyfilelocatedinthevarsdirectory.

Let'screateasayHellostepthattakesthenameparameterandechoesasimplemessage.Thisshouldbestoredinthevars/sayHello.groovyfile:/***Helloworldstep.*/defcall(Stringname){echo"Hello$name!"}

Human-readabledescriptionsforsharedlibrarystepscanbestoredinthe*.txtfiles.Inourexample,wecouldaddthevars/sayHello.txtfilewiththestepdocumentation.

Whenthelibrarycodeisdone,weneedtopushittotherepository,forexample,asanewGitHubproject.

ConfigurethesharedlibraryinJenkinsThenextstepistoregisterthesharedlibraryinJenkins.WeopenManageJenkins|ConfigureSystem,andfindtheGlobalPipelineLibrariessection.There,wecanaddthelibrarygivingitanamechosen,asfollows:

Wespecifiedthenameunderwhichthelibraryisregisteredandthelibraryrepositoryaddress.Notethatthelatestversionofthelibrarywillbeautomaticallydownloadedduringthepipelinebuild.

WepresentedimportingtheGroovycodeasGlobalSharedLibrary,buttherearealsootheralternativesolutions.Readmoreathttps://jenkins.io/doc/book/pipeline/shared-libraries/.

UsesharedlibraryinJenkinsfileFinally,wecanusethesharedlibraryintheJenkinsfilescript.

Let'shavealookattheexample:

pipeline{

agentany

stages{

stage("Hellostage"){

steps{

sayHello'Rafal'

}

}

}

}

If"Loadimplicitly"hadn'tbeencheckedintheJenkinsconfiguration,thenwewouldneedtoadd"@Library('example')_"atthebeginningoftheJenkinsfilescript.

Asyoucansee,wecanusetheGroovycodeasapipelinestepsayHello.Obviously,afterthepipelinebuildcompletes,intheconsoleoutput,weshouldseeHelloRafal!.

Sharedlibrariesarenotlimitedtoonestep.Actually,withthepoweroftheGroovylanguage,theycanevenactastemplatesforentireJenkinspipelines.

RollingbackdeploymentsIrememberthewordsofmycolleague,aseniorarchitect,"Youdon'tneedmoreQAs,youneedafasterrollback".WhilethisstatementisoversimplifiedandtheQAteamisoftenofgreatvalue,thereisalotoftruthinthissentence.Thinkaboutit;ifyouintroduceabugintheproductionbutrollitbacksoonafterthefirstuserreportsanerror,thenusuallynothingbadhappens.Ontheotherhand,ifproductionerrorsarerarebutnorollbackisapplied,thentheprocesstodebugtheproductionusuallyendsupinlongsleeplessnightsandanumberofdissatisfiedusers.ThisiswhyweneedtothinkabouttherollbackstrategyupfrontwhilecreatingtheJenkinspipeline.

InthecontextofContinuousDelivery,therearetwomomentswhenthefailurecanhappen:

Duringthereleaseprocess,inthepipelineexecutionAfterthepipelinebuildiscompleted,inproduction

Thefirstscenarioisprettysimpleandharmless.Itconcernsacasewhentheapplicationisalreadydeployedtoproductionbutthenextstagefails,forexample,thesmoketest.Then,allweneedtodoisexecuteascriptinthepostpipelinesectionforthefailurecase,whichdowngradestheproductionservicetotheolderDockerimageversion.Ifweuseblue-greendeployment(asdescribedlaterinthischapter),theriskofanydowntimeisminimalsinceweusuallyexecutetheload-balancerswitchasthelastpipelinestage,afterthesmoketest.

Thesecondscenario,whenwenoticeaproductionbugafterthepipelineissuccessfullycompleted,ismoredifficultandrequiresafewcomments.Here,theruleisthatweshouldalwaysreleasetherolledbackserviceusingexactlythesameprocessasthestandardrelease.Otherwise,ifwetrytodosomethingmanually,inafasterway,weareaskingfortrouble.Anynonrepetitivetaskisrisky,especiallyunderstress,whentheproductionisoutoforder.

Asasidenote,ifthepipelinecompletessuccessfullybutthereisaproductionbug,thenitmeansthatourtestsarenotgoodenough.So,thefirstthingafterrollbackistoextendtheunit/acceptancetest

suiteswiththecorrespondingscenarios.

ThemostcommonContinuousDeliveryprocessisonefullyautomatedpipelinethatstartsbycheckingoutthecodeandendsupwithreleasetotheproduction.

Thefollowingfigurepresentshowthisworks:

WealreadypresentedtheclassicContinuousDeliverypipelinethroughoutthisbook.Iftherollbackshoulduseexactlythesameprocess,thenallweneedtodoisrevertthelatestcodechangefromtherepository.Asaresult,thepipelineautomaticallybuilds,tests,andfinally,releasestherightversion.

Repositoryrevertsandemergencyfixesshouldneverskipthetestingstagesinthepipeline.Otherwise,wemayendupwithareleasethatisstillnotworkingcorrectlybecauseofanotherissuethatmakesdebuggingevenharder.

Thesolutionisverysimpleandelegant.Theonlydrawbackisthedowntimethatweneedtospendonthecompletepipelinebuild.Thisdowntimecanbeavoidedifweuseblue-greendeploymentorcanaryreleases,inwhichcases,weonlychangetheloadbalancersettingtoaddressthehealthyenvironment.

Therollbackoperationbecomeswaymorecomplexinthecaseoforchestratedreleases,duringwhichmanyservicesaredeployedatthesametime.Thisisoneofthereasonswhyorchestratedreleasesaretreatedasananti-pattern,especiallyinthemicroserviceworld.Thecorrectapproachistoalwaysmaintainbackwardscompatibility,atleastforsometime(likewepresentedforthedatabaseatthebeginningofthischapter).Then,it'spossibletoreleaseeachserviceindependently.

AddingmanualstepsIngeneral,theContinuousDeliverypipelinesshouldbefullyautomated,triggeredbyacommittotherepository,andendupaftertherelease.Sometimes,however,wecan'tavoidhavingmanualsteps.Themostcommonexampleisthereleaseapproval,whichmeansthattheprocessisfullyautomated,butthereisamanualsteptoapprovethenewrelease.Anothercommonexampleismanualtests.Someofthemmayexistbecauseweoperateonthelegacysystem;someothersmayoccurwhenatestsimplycannotbeautomated.Nomatterwhatthereasonis,sometimesthereisnochoicebuttoaddamanualstep.

Jenkinssyntaxoffersakeywordinputformanualsteps:

stage("Releaseapproval"){

steps{

input"Doyouapprovetherelease?"

}

}

Thepipelinewillstopexecutionontheinputstepandwaituntilit'smanuallyapproved.

Rememberthatmanualstepsquicklybecomeabottleneckinthedeliveryprocess,andthisiswhytheyshouldalwaysbetreatedasasolutionthat'sinferiortocompleteautomation.

Itissometimesusefultosetatimeoutfortheinputinordertoavoidwaitingforeverforthemanualinteraction.Aftertheconfiguredtimeiselapsed,thewholepipelineisaborted.

ReleasepatternsInthelastsection,wediscussedtheJenkinspipelinepatternsusedtospeedupthebuildexecution(parallelsteps),helpwiththecodereuse(sharedlibraries),limittheriskofproductionbugs(rollback),anddealwithmanualapprovals(manualsteps).Thissectionpresentsthenextgroupofpatterns,thistimerelatedtothereleaseprocess.Theyaredesignedtoreducetheriskofupdatingtheproductiontoanewsoftwareversion.

Wealreadydescribedoneofthereleasepatterns,rollingupdates,inChapter8,ClusteringwithDockerSwarm.Here,wepresenttwomore:blue-greendeploymentandcanaryreleases.

Blue-greendeploymentBlue-greendeploymentisatechniquetoreducethedowntimeassociatedwiththerelease.Itconcernshavingtwoidenticalproductionenvironments,onecalledgreen,theothercalledblue,aspresentedinthefollowingfigure:

Inthefigure,thecurrentlyaccessibleenvironmentisblue.Ifwewanttomakeanewrelease,thenwedeployeverythingtothegreenenvironmentand,attheendofthereleaseprocess,changetheloadbalancertothegreenenvironment.Asaresult,auser,allofasudden,startsusingthenewversion.Thenexttimewewanttomakearelease,wemakechangestotheblueenvironmentand,attheend,wechangetheloadbalancertoblue.Weproceedthesameeverytime,switchingfromoneenvironmenttoanother.

Theblue-greendeploymenttechniqueworkscorrectlywithtwoassumptions:environmentisolationandnoorchestratedreleases.

Thissolutiongivestwosignificantbenefits:

Zerodowntime:Allthedowntimefromtheuserperspectiveisamomentofchangingtheloadbalanceswitch,whichisnegligibleRollback:Inordertorollbackoneversion,it'senoughtochangebackthe

loadbalanceswitchblue-greendeploymentinclude:Database:Schemamigrationscanbetrickyincaseofarollback,soit'sworthusingthepatternspresentedatthebeginningofthischapterTransactions:RunningdatabasetransactionsmustbehandedovertothenewdatabaseRedundantinfrastructure/resources:Weneedtohavedoubletheresources

Therearetechniquesandtoolstoovercomethesechallenges,sotheblue-greendeploymentpatternishighlyrecommendedandwidelyusedintheITindustry.

Youcanreadmoreabouttheblue-greendeploymenttechniqueontheexcellentMartinFowler'sbloghttps://martinfowler.com/bliki/BlueGreenDeployment.html.

CanaryreleaseCanaryreleasingisatechniquetoreducetheriskassociatedwithintroducinganewversionofthesoftware.Similartoblue-greendeployment,itusestwoidenticalenvironments,aspresentedinthefollowingfigure:

Also,similartotheblue-greendeploymenttechnique,thereleaseprocessstartsbydeployinganewversionintheenvironmentthatiscurrentlyunused.Here,however,thesimilaritiesend.Theloadbalancer,insteadofswitchingtothenewenvironment,issettolinkonlyaselectedgroupofuserstothenewenvironment.Allthereststillusetheoldversion.Thisway,anewversioncanbetestedbysomeusersandincaseofabug,onlyasmallgroupisaffected.Afterthetestingperiod,allusersareswitchedtothenewversion.

Thisapproachhassomegreatbenefits:

Acceptanceandperformancetesting:Iftheacceptanceandperformancetestingisdifficulttoruninthestagingenvironment,thenit'spossibletotestitinproduction,minimizingtheimpactonasmallgroupofusers.Simplerollback:Ifanewchangecausesafailure,thenrollingbackisdonebyswitchingbackalluserstotheoldversion.A/Btesting:IfwearenotsurewhetherthenewversionisbetterthantheUXortheperformanceperspective,thenit'spossibletocompareitwiththe

oldversion.

Canaryreleasingsharesthesamedrawbacksasblue-greendeployment.Theadditionalchallengeisthatwehavetwoproductionsystemsrunningatthesametime.Nevertheless,canaryreleasingisanexcellenttechniqueusedinmostcompaniestohelpwiththereleaseandtesting.

YoucanreadmoreaboutthecanaryreleasingtechniqueontheexcellentMartinFowler'sbloghttps://martinfowler.com/bliki/CanaryRelease.html.

WorkingwithlegacysystemsAllwehavedescribedsofarappliessmoothlytogreenfieldprojects,forwhichsettingupaContinuousDeliverypipelineisrelativelysimple.

Legacysystemsare,however,waymorechallengingbecausetheyusuallydependonmanualtestsandmanualdeploymentsteps.Inthissection,wewillwalkthroughtherecommendedscenariotoincrementallyapplyContinuousDeliverytoalegacysystem.

Asastepzero,IrecommendreadinganexcellentbookbyMichaelFeathers,WorkingEffectivelywithLegacyCode.Hisideasonhowtodealwithtesting,refactoring,andaddingnewfeaturesclearmostoftheconcernsabouthowtoautomatethedeliveryprocessforlegacysystems.

Formanydevelopers,itmaybetemptingtocompletelyrewritealegacysystem,ratherthanrefactorit.Whiletheideaisinterestingfromadeveloper'sperspective,itisusuallyabadbusinessdecisionthatresultsinproductfailure.YoucanreadmoreaboutthehistoryofrewritingtheNetscapebrowserinanexcellentblogpostbyJoelSpolsky,ThingsYouShouldNeverDo,athttps://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i.

ThewaytoapplytheContinuousDeliveryprocessdependsalotonthecurrentproject'sautomation,thetechnologiesused,thehardwareinfrastructure,andthecurrentreleaseprocess.Usually,itcanbesplitintothreesteps:

1. Automatingbuildanddeployment.2. Automatingtests.3. Refactoringandintroducingnewfeatures.

Let'slookatthemindetail.

AutomatingbuildanddeploymentThefirststepincludesautomatingthedeploymentprocess.ThegoodnewsisthatinmostlegacysystemsIhaveworkedwith,therewasalreadysomeautomationinplace,forexample,intheformofshellscripts.

Inanycase,theactivitiesforautomateddeploymentincludesthefollowing:

Buildandpackage:SomeautomationusuallyalreadyexistsintheformofMakefile,Ant,Maven,anyotherbuildtoolconfiguration,oracustomscript.Databasemigration:Weneedtostartmanagingthedatabaseschemainanincrementalmanner.ItrequiresputtingthecurrentschemaasaninitialmigrationandmakingallthefurtherchangeswithtoolssuchasFlywayorLiquibase,asalreadydescribedinthischapter.Deployment:Evenifthedeploymentprocessisfullymanual,thenthereisusuallyatext/wikipagedescriptionthatneedstobeconvertedintoanautomatedscript.Repeatableconfiguration:Inlegacysystems,configurationfilesareusuallychangedmanually.Weneedtoextracttheconfigurationanduseaconfigurationmanagementtool,asdescribedinChapter6,ConfigurationManagementwithAnsible.

Aftertheprecedingsteps,wecanputeverythingintoadeploymentpipelineanduseitasanautomatedphaseafteramanualUAT(useracceptancetesting)cycle.

Fromtheprocessperspective,it'sworthalreadystartingreleasingmoreoften.Forexample,ifthereleaseisyearly,trytodoitquarterly,thenmonthly.Thepushforthatfactorwilllaterresultinfaster-automateddeliveryadoption.

AutomatingtestsThenextstep,usuallymuchmoredifficult,istopreparetheautomatedtestsforthesystem.ItrequirescommunicatingwiththeQAteaminordertounderstandhowtheycurrentlytestthesoftwaresothatwecanmoveeverythingintoanautomatedacceptancetestsuite.Thisphaserequirestwosteps:

Acceptance/sanitytestsuite:WeneedtoaddautomatedteststhatreplacesomeoftheregressionactivitiesoftheQAteam.Dependingonthesystem,theycanbeprovidedasablack-boxSeleniumtestorCucumbertest.(Virtual)testenvironments:Atthispoint,weshouldbealreadythinkingoftheenvironmentsinwhichourtestswouldberun.Usually,thebestsolutiontosaveresourcesandlimitthenumberofmachinesrequiredistovirtualizethetestingenvironmentusingVagrantorDocker.

TheultimategoalistohaveanautomatedacceptancetestsuitethatwillreplacethewholeUATphasefromthedevelopmentcycle.Nevertheless,wecanstartwithasanitytestthatwillshortlycheckifthesystemiscorrectfromtheregressionperspective.

Whileaddingtestscenarios,rememberthatthetestsuiteshouldexecuteinreasonabletime.Forsanitytests,itisusuallylessthan10minutes.

RefactoringandintroducingnewfeaturesWhenwehaveatleastthefundamentalregressiontestingsuite,wearereadytoaddnewfeaturesandrefactortheoldcode.It'salwaysbettertodoitinsmallpieces,stepbystepbecauserefactoringeverythingatonceusuallyendsupinachaosthatleadstoproductionfailures(notclearlyrelatedtoanyparticularchange).

Thisphaseusuallyincludesthefollowingactivities:

Refactoring:Thebestplacetostartrefactoringtheoldcodeiswherethenewfeaturesareexpected.Startingthisway,wearepreparedforthenewfeaturerequeststocome.Rewrite:Ifweplantorewritepartsoftheoldcode,weshouldstartfromthecodethatisthemostdifficulttotest.Thisway,weconstantlyincreasethecodecoverageinourproject.Introducingnewfeatures:Duringthenewfeatureimplementation,it'sworthusingthefeaturetogglepattern.Then,incaseanythingbadhappens,wecanquicklyturnoffthenewfeature.Actually,thesamepatternshouldbeusedduringrefactoring.

Forthisphase,it'sworthreadinganexcellentbookbyMartinFowler,Refactoring:ImprovingtheDesignofExistingCode.

Whiletouchingtheoldcode,it'sgoodtofollowtheruletoalwaysaddapassingunittestfirst,andonlythen,changethecode.Withthisapproach,wecandependonautomationtocheckthatwedon'tchangethebusinesslogicbyaccident.

UnderstandingthehumanelementWhileintroducingtheautomateddeliveryprocesstoalegacysystem,it'spossibleyouwillfeel,morethananywhereelse,thehumanfactor.Inordertoautomatethebuildprocess,weneedtocommunicatewellwiththeoperationsteam,andtheymustbewillingtosharetheirknowledge.ThesamestoryappliestothemanualQAteam;theyneedtobeinvolvedinwritingautomatedtestsbecauseonlytheyknowhowtotestthesoftware.Ifyouthinkaboutit,boththeoperationsandQAteamsneedtocontributetotheprojectthatwilllaterautomatetheirwork.Atsomepoint,theymayrealizethattheirfutureinthecompanyisnotstableandbecomelesshelpful.ManycompaniesstrugglewithintroducingtheContinuousDeliveryprocessbecauseteamsdonotwanttogetinvolvedenough.

Inthissection,wediscussedhowtoapproachlegacysystemsandthechallengestheypose.IfyouareinprogressofconvertingyourprojectandorganizationintotheContinuousDeliveryapproach,thenyoumaywanttohavealookattheContinuousDeliveryMaturityModel,whichaimstogivestructuretotheprocessofadoptingtheautomateddelivery.

AgooddescriptionoftheContinuousDeliveryMaturityModelcanbefoundathttps://developer.ibm.com/urbancode/docs/continuous-delivery-maturity-model/.

Exercises

Inthischapter,wehavecoveredvariousaspectsoftheContinuousDeliveryprocess.Sincepracticemakesperfect,werecommendthefollowingexercises:

1. UseFlywaytocreateanon-backwards-compatiblechangeintheMySQLdatabase:

UsetheofficialDockerimage,mysql,tostartthedatabaseConfigureFlywaywithproperdatabaseaddress,username,andpasswordCreateaninitialmigrationthatcreatesauserstablewiththreecolumns:id,email,andpasswordAddasampledatatothetableChangethepasswordcolumntohashed_password,whichwillstorethehashedpasswordsSplitthenon-backwards-compatiblechangeintothreemigrationsasdescribedinthischapterYoucanuseMD5orSHAforhashingCheckthat,asaresult,thedatabasestoresnopasswordsinplaintext

2. CreateaJenkinssharedlibrarywithstepstobuildandunittestGradleprojects:

CreateaseparaterepositoryforthelibraryCreatetwofilesinthelibrary:gradleBuild.groovyandgradleTest.groovyWritetheappropriatecallmethodsAddthelibrarytoJenkinsUsethestepsfromthelibraryinapipeline

SummaryThischapterwasamixtureofvariousContinuousDeliveryaspectsthatwerenotcoveredbefore.Thekeytakeawaysfromthechapterareasfollows:

Databasesareanessentialpartofmostapplicationsandshould,therefore,beincludedintheContinuousDeliveryprocess.Databaseschemachangesarestoredintheversioncontrolsystemandmanagedbydatabasemigrationtools.Therearetwotypesofdatabaseschemachange:backwards-compatibleandbackwards-incompatible.Whilethefirsttypeissimple,thesecondrequiresabitofoverhead(splittomultiplemigrationsspreadovertime).Adatabaseshouldnotbethecentralpointofthewholesystem.Thepreferredsolutionistoprovideeachservicewithitsowndatabase.Thedeliveryprocessshouldalwaysbepreparedfortherollbackscenario.Threereleasepatternsshouldalwaysbeconsidered:rollingupdates,blue-greendeployment,andcanaryreleasingLegacysystemscanbeconvertedintotheContinuousDeliveryprocessinsmallstepsratherthanallatonce.

BestpracticesThankyouforreadingthisbook.IhopeyouarereadytointroducetheContinuousDeliveryapproachtoyourITprojects.Asthelastsectionofthisbook,Iproposealistofthetop10ContinuousDeliverypractices.Enjoy!

Practice1–ownprocesswithintheteam!Owntheentireprocesswithintheteam,fromreceivingrequirementstomonitoringtheproduction.Asoncesaid:"Aprogramrunningonthedeveloper'smachinemakesnomoney."Thisiswhyit'simportanttohaveasmallDevOpsteamthattakescompleteownershipofaproduct.Actually,thatisthetruemeaningofDevOps-DevelopmentandOperationsfromthebeginningtotheend:

OwneverystageoftheContinuousDeliverypipeline:howtobuildthesoftware,whattherequirementsareinacceptancetests,andhowtoreleasetheproduct.Avoidhavingapipelineexpert!Everymemberoftheteamshouldbeinvolvedincreatingthepipeline.Findagoodwaytosharethecurrentpipelinestate(andtheproductionmonitoring)amongteammembers.Themosteffectivesolutionisbigscreensintheteamspace.Ifadeveloper,QA,andITOperationsengineerareseparateexperts,thenmakesuretheyworktogetherinoneagileteam.Separateteamsbasedonexpertiseresultintakingnoresponsibilityfortheproduct.Rememberthatautonomygiventotheteamresultsinhighjobsatisfactionandexceptionalengagement.Thisleadstogreatproducts!

Practice2–automateeverything!Automateeverythingfrombusinessrequirements(intheformofacceptancetests)tothedeploymentprocess.Manualdescriptions,wikipageswithinstructionsteps,theyallquicklybecomeoutofdateandleadtotribalknowledgethatmakestheprocessslow,tedious,andunreliable.This,inturn,leadstoaneedforreleaserehearsalsandmakeseverydeploymentunique.Don'tgodownthispath!Asarule,ifyoudoanythingforthesecondtime,automateit:

Eliminateallmanualsteps;theyareasourceoferrors!Thewholeprocessmustberepeatableandreliable.Don'tevermakeanychangesdirectlyinproduction!Useconfigurationmanagementtoolsinstead.Usepreciselythesamemechanismtodeploytoeveryenvironment.Alwaysincludeanautomatedsmoketesttocheckifthereleasecompletedsuccessfully.Usedatabaseschemamigrationstoautomatedatabasechanges.Useautomaticmaintenancescriptsforbackupandcleanup.Don'tforgettoremoveunusedDockerimages!

Practice3–versioneverything!Versioneverything:softwaresourcecode,buildscripts,automatedtests,configurationmanagementfiles,ContinuousDeliverypipelines,monitoringscripts,binaries,anddocumentation.Simplyeverything.Makeyourworktask-based,whereeachtaskresultsinacommittotherepository,nomatterwhetherit'srelatedtorequirementgathering,architecturedesign,configuration,orthesoftwaredevelopment.Ataskstartsontheagileboardandendsupintherepository.Thisway,youmaintainasinglepointoftruthwiththehistoryandreasonsforthechanges:

Bestrictabouttheversioncontrol.Everythingmeanseverything!Keepsourcecodeandconfigurationinthecoderepository,binariesintheartifactrepository,andtasksintheagileissuetrackingtool.DeveloptheContinuousDeliverypipelineasacode.Usedatabasemigrationsandstoretheminarepository.Storedocumentationintheformofmarkdownfilesthatcanbeversion-controlled.

Practice4–usebusinesslanguageforacceptancetests!Usebusiness-facinglanguageforacceptanceteststoimprovethemutualcommunicationandthecommonunderstandingoftherequirements.WorkcloselywiththeproductownertocreatewhatEricEvancalledtheubiquitouslanguage,acommondialectbetweenthebusinessandtechnology.Misunderstandingsaretherootcauseofmostprojectfailures:

Createacommonlanguageanduseitinsidetheproject.UseanacceptancetestingframeworksuchasCucumberorFitNessetohelpthebusinessteamunderstandandgettheminvolved.Expressbusinessvaluesinsideacceptancetestsanddon'tforgetaboutthemduringdevelopment.It'seasytospendtoomuchtimeonunrelatedtopics!Improveandmaintainacceptancetestssothattheyalwaysactasregressiontests.Makesureeveryoneisawarethatapassingacceptancetestsuitemeansagreenlightfromthebusinesstoreleasethesoftware.

Practice5–bereadytorollback!Bereadytorollback;soonerorlateryouwillneedtodoit.Remember,Youdon'tneedmoreQAs,youneedafasterrollback.Ifanythinggoeswronginproduction,thefirstthingyouwanttodoistoplaysafeandcomebacktothelastworkingversion:

DeveloparollbackstrategyandtheprocessofwhattodowhenthesystemisdownSplitnon-backwards-compatibledatabasechangesintocompatibleonesAlwaysusethesameprocessofdeliveryforrollbacksandforstandardreleasesConsiderintroducingblue-greendeploymentsorcanaryreleasesDon'tbeafraidofbugs,theuserwon'tleaveyouifyoureactquickly!

Practice6–don'tunderestimatetheimpactofpeopleDon'tunderestimatetheimpactofpeople.Theyareusuallywaymoreimportantthantools.Youwon'tautomatethedeliveryiftheITOperationsteamwon'thelpyou.Afterall,theyhavetheknowledgeaboutthecurrentprocess.ThesameappliestoQAs,business,andeveryoneinvolved.Makethemimportantandinvolved:

LetQAsandIToperationsbeapartoftheDevOpsteam.Youneedtheirknowledgeandskills!Providetrainingtomembersthatarecurrentlydoingmanualactivitiessothattheycanmovetoautomation.Favorinformalcommunicationandaflatstructureoforganizationoverhierarchyandorders.Youwon'tdoanythingwithoutgoodwill!

Practice7–buildintraceability!Buildintraceabilityforthedeliveryprocessandworkingsystem.Thereisnothingworsethanafailurewithoutanylogmessages.Monitorthenumberofrequests,thelatency,theloadofproductionservers,thestateoftheContinuousDeliverypipeline,andeverythingyoucanthinkofthatcouldhelpyoutoanalyzeyourcurrentsoftware.Beproactive!Atsomepoint,youwillneedtocheckthestatsandlogs:

Logpipelineactivities!Inthecaseoffailure,notifytheteamwithaninformativemessage.Implementproperloggingandmonitoringoftherunningsystem.UsespecializedtoolsforsystemmonitoringsuchasKibana,Grafana,orLogmatic.io.Integrateproductionmonitoringintoyourdevelopmentecosystem.Considerhavingbigscreenswiththecurrentproductionstatsinthecommonteamspace.

Practice8–integrateoften!Integrateoften,actually,allthetime!Assomeonesaid:Continuousismoreoftenthanyouthink.Thereisnothingmorefrustratingthanresolvingmergeconflicts.ContinuousIntegrationislessaboutthetoolandmoreabouttheteampractice.Integratethecodeintoonecodebaseatleastafewtimesaday.Forgetaboutlong-lastingfeaturebranchesandahugenumberoflocalchanges.Trunk-basedevelopmentandfeaturetogglesforthewin!

Usetrunk-baseddevelopmentandfeaturetogglesinsteadoffeaturebranches.Ifyouneedabranchorlocalchanges,makesurethatyouintegratewiththerestoftheteamatleastonceaday.Alwayskeepthetrunkhealthy;makesureyouruntestsbeforeyoumergeintothebaseline.Runthepipelineaftereverycommittotherepositoryforafastfeedbackcycle.

Practice9–buildbinariesonlyonce!Buildbinariesonlyonceandrunthesameoneoneachoftheenvironments.NomatteriftheyareinaformofDockerimagesorJARpackages,buildingonlyonceeliminatestheriskofdifferencesintroducedbyvariousenvironments.Italsosavestimeandresources:

Buildonceandpassthesamebinarybetweenenvironments.Useartifactrepositorytostoreandversionbinaries.Don'teverusethesourcecoderepositoryforthatpurpose.Externalizeconfigurationsanduseaconfigurationmanagementtooltointroducedifferencesbetweenenvironments.

Practice10–releaseoften!Releaseoften,preferablyaftereachcommittotherepository.Asthesayinggoes,Ifithurts,doitmoreoften.Releasingasadailyroutinemakestheprocesspredictableandcalm.Stayawayfrombeingtrappedintherarereleasehabit.Thatwillonlygetworseandyouwillendupwithreleasingonceayearhavingathreemonths'preparationperiod!

RephraseyourdefinitionofdonetoDonemeansreleased.Takeownershipofthewholeprocess!Usefeaturetogglestohide(fromusers)featuresthatarestillinprogress.Usecanaryreleasesandquickrollbacktoreducetheriskofbugsintheproduction.Adoptazero-downtimedeploymentstrategytoenablefrequentreleases.