continuous delivery with containers: the good ... - jax london · mechanical sympathy: docker and...
TRANSCRIPT
ContinuousDeliverywithContainers:TheGood,theBad,andtheUgly
DanielBryant@danielbryantuk
Containers:Expectationsversusreality
10/10/2017 @danielbryantuk
“DevOps”
Settingthescene…
• Continuousdeliveryisalargetopic• Nobusinessfocustoday(valuestreametc)• PaaSandServerless aresuperinteresting…• ButI’massumingyou’reall-inoncontainers
• Focusingtodayontheprocessandtooling• Nolivecodingtoday• Mini-bookcontainsmoredetails(thanksnginx!)
10/10/2017 @danielbryantuk
bit.ly/2jWDSF7
TL;DR– ContainersandCD
• Containerimagebecomesthebuildpipeline‘singlebinary’
• Addingmetadatatocontainersimagesisvital,butchallenging
• Mustvalidatecontainerconstraints(NFRs)• Cultivatecontainer‘mechanicalsympathy’
10/10/2017 @danielbryantuk
@danielbryantuk
• IndependentTechnicalConsultant,CTOatSpectoLabs• Architecture,DevOps,Java,microservices,cloud,containers
• ContinuousDelivery(CI/CD)advocate
• Leadingchangethroughtechnologyandteams
10/10/2017 @danielbryantuk
ContinuousDelivery
10/10/2017 @danielbryantuk
ContinuousDelivery
• Producevaluableandrobustsoftwareinshortcycles
• Optimising forfeedbackandlearning
• Not (necessarily)ContinuousDeployment
10/10/2017 @danielbryantuk
Creationofabuildpipelineismandatoryforcontinuousdelivery
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
TheImpactofcontainersonCD
10/10/2017 @danielbryantuk
Containertechnology(andCD)
• OS-levelvirtualisation• cgroups,namespaces,rootfs
• Packageandexecutesoftware
• Containerimage==‘singlebinary’
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
Creatingapipelineforcontainers
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
Makeyourdevenvironmentlikeproduction
• Developlocallyorcopy/codeincontainer
• Mustbuild/testcontainerslocally• Perform(atleast)happypathtests
10/10/2017 @danielbryantuk
QuickAside:Running*entire*systemlocally
10/10/2017 @danielbryantuk
https://news.ycombinator.com/item?id=13960107https://opencredo.com/working-locally-with-microservices/https://www.datawire.io/telepresence/ |https://hoverfly.io/
Makeyourdevenvironmentlikeproduction
• Developlocallyorcopy/codeincontainer
• Mustbuild/testcontainerslocally• Perform(atleast)happypathtests
• Useidenticalbaseimagesfromproduction• Withsameconfiguration
10/10/2017 @danielbryantuk
Lessonlearned:Dockerfile contentissuper important
• OSchoice
• Configuration
• Buildartifacts
• Exposingports
• Java• JDKvsJREandOraclevsOpenJDK?
• Golang• Staticallycompiledbinaryinscratch?
• Python• Virtualenv?
10/10/2017 @danielbryantuk
Pleasetalktothesysadminpeople:Theiroperationalknowledgeisinvaluable
10/10/2017 @danielbryantuk
Differenttestandprodcontainers?
• Create“test”versionofcontainer• FullOS(e.g.Ubuntu)• Testtoolsanddata
• Easytoseeapp/configurationdrift
• Usetestsidecarcontainersinstead
• ONTESTproposalbyAlexiLedenev
10/10/2017 @danielbryantuk
http://blog.terranillius.com/post/docker_testing/
Dockermulti-stagebuilds
10/10/2017 @danielbryantuk
http://blog.alexellis.io/mutli-stage-docker-builds/https://github.com/moby/moby/pull/31257https://github.com/moby/moby/pull/32063
Javaspecificstuff…
10/10/2017 @danielbryantuk
github.com/oracle/docker-images/tree/master/OracleJava jdk.java.net/9/ea
Hotoffthepress:Modularity
• Createminimalruntimeimages
• “jlink deliversaself-containeddistributionofyourapplicationandtheJVM,readytobeshipped.”
• Benefits:• Reducedfootprint• Performance• Security
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
BuildingimageswithJenkins
• Myreportcoversthis
• Buildasusual…
• BuildDockerImage• CloudbeesDockerBuildandPublishPlugin
• Pushimagetoregistry
10/10/2017 @danielbryantuk
Storinginanimageregistry(DockerHub)
10/10/2017 @danielbryantuk
Metadata– Bewareof“latest”DockerTag
• Bewareofthe‘latest’Dockertag
• “Latest”simplymeans• thelastbuild/tagthatranwithoutaspecifictag/versionspecified
• Ignore“latest”tag• Versionyourtags,everytime• danielbryantuk/test:2.4.1
10/10/2017 @danielbryantuk
Lessonlearned:Metadataisvaluable
• Applicationmetadata• Version/GITSHA
• Buildmetadata• Builddate• Imagename• Vendor
• Qualitymetadata• QAcontrol,signedbinaries,ephemeralsupport• Securityprofiles(AppArmor),Securityauditedetc
10/10/2017 @danielbryantuk
Metadata- AddingLabelsatbuildtime
• DockerLabels
• Addkey/valuedatatoimage
10/10/2017 @danielbryantuk
Metadata- AddingLabelsatbuildtime
• Microscaling Systems’Makefile
• LabellingautomatedbuildsonDockerHub (h/tRossFairbanks)• Createfile‘/hooks/build’
• label-schema.org• microbadger.com
10/10/2017 @danielbryantuk
Metadata- AddingLabelsatruntime
10/10/2017 @danielbryantuk
$ docker run -d --labeluk.co.danielbryant.lbname=frontdoor nginx
• Can’docker commit’,butcreatesnewimage
• Notpossibletoupdaterunningcontainer
• DockerProposal:Updatelabels #21721
LizRice(andAqua)totherescue!
10/10/2017 @danielbryantuk
github.com/aquasecurity/manifesto
Externalregistrywithmetadatasupport
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
Componenttesting
10/10/2017 @danielbryantuk
Testing:JenkinsPipeline(ascode)
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
Testingindividualcontainers
10/10/2017 @danielbryantuk
Integrationtesting
10/10/2017 @danielbryantuk
IntroducingDockerCompose
10/10/2017 @danielbryantuk
DockerCompose&JenkinsPipeline
10/10/2017 @danielbryantuk
EphemeralKubernetesClusters
• Kubernaut (WIP)
• Managesapoolofclusters
• ”Claim”afreshcluster
• UseHelmtoinstalldependencies
10/10/2017 @danielbryantuk
TestingNFRsinthebuildpipeline
• PerformanceandLoadtesting• Gatling/jmeter• Flood.io
• Securitytesting• Findsecbugs /OWASPDependencycheck• Bdd-security(OWASPZAP)/Arachni• Gauntlt /Serverspec• DockerBenchforSecurity/CoreOSClair
10/10/2017 @danielbryantuk
DelayingNFRstothe‘LastResponsibleMoment’
• Newsflash!• Sometimesthelastresponsiblemomentisup-front!
• Containers/microservices don’tmakethiseasier• Sometimesmoredifficult…
10/10/2017 @danielbryantuk
Mechanicalsympathy:DockerandJava
• WatchforJVMcgroup/taskset awareness• getAvailableProcessors()mayincorrectlyreportthenumberofcpus inDocker(JDK-8140793)• Runtime.availableProcessors()ignoresLinuxtaskset command(JDK-6515172)• Default fork/jointhreadpoolsizes(andothers)isbasedfromhostCPUcount
• Setcontainermemoryappropriately• JVMrequirements=Heapsize(Xmx)+Metaspace +JVMoverhead• Accountfornativethreadrequirementse.g.threadstacksize(Xss)
• Entropy• Hostentropycansoonbeexhaustedbycryptooperations
10/10/2017 @danielbryantuk 46
Deployment
10/10/2017 @danielbryantuk
skillsmatter.com/skillscasts/10668-looking-forward-to-daniel-bryant-talk
docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.deploy-existing-version.html
Observabilityiscoretocontinuousdelivery
10/10/2017 @danielbryantuk
www.infoq.com/articles/monitoring-containers-at-scale
Containersarenotasilverbullet
10/10/2017 @danielbryantuk
Movingtocontainers:Goingall-in?
10/10/2017 @danielbryantuk
OR
ShouldIbuildmyowncontainerplatform?
Probablynot(UnlessyouareGoogle,AWSorIBM)
Whateveryoudecide…pushitthroughapipelineASAP!
10/10/2017 @danielbryantuk
Usingcontainersdoesnotobviatetheneedforgoodarchitecturalpractices
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
https://speakerdeck.com/caseywest/containercon-north-america-cloud-anti-patterns
Summary
10/10/2017 @danielbryantuk
Insummary
• Continuousdeliveryisvitallyimportantinmodernarchitectures/ops
• Containerimagesmustbethe(single)sourceoftruthwithinpipeline• Andmetadataaddedasappropriate…
• Mechanicalsympathyisimportant(assertpropertiesinthepipeline)• Notalldevelopersareoperationallyaware
• Thetoolingisnowbecomingstable/mature• Weneedtore-applyexistingCDpracticeswithnewtechnologies/tooling
10/10/2017 @danielbryantuk
Bedtimereading
10/10/2017 @danielbryantuk
Thanksforlistening
• Anyquestions?
• Feelfreetocontactme• @danielbryantuk• [email protected]
10/10/2017 @danielbryantuk
bit.ly/2jWDSF7
Comingsoon!
Bonusslides(forextracontext)
10/10/2017 @danielbryantuk
Containerise anexisting(monolithic)app?
• For
• Weknowthemonolithwell
• Allowshomogenizationofthepipelineanddeploymentplatform
• Canbeademonstrablewinfortechandthebusiness
• Against
• Canbedifficult(100+linescripts)
• Oftennotdesignedforoperationwithincontainers,norcloudnative
• Puttinglipstickonapig?
10/10/2017 @danielbryantuk
Keylessonslearned
• Conductanarchitecturalreview• ArchitectureforDevelopers,bySimonBrown• ArchitectureInterview,bySusanFowler
• Lookfordataingress/egress• Filesystemaccess
• Supportresourceconstraints/transience• Optimise forquickstartupandshutdown• Evaluateapproachtoconcurrency• Storeconfiguration(secrets)remotely
10/10/2017 @danielbryantuk
Newdesignpatterns
10/10/2017 @danielbryantuk
bit.ly/2efe0TP
Microservices…
Containersandmicroservices arecomplementary
Testinganddeploymentchange
10/10/2017 @danielbryantuk
https://specto.io/blog/recipe-for-designing-building-testing-microservices.html
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk
10/10/2017 @danielbryantuk