Docker on OpenStack - CORE this report, we describe about docker, its basics and importance of docker containers in comparison ... Docker on OpenStack ... Containers on Google Cloud

Download Docker on OpenStack - CORE  this report, we describe about docker, its basics and importance of docker containers in comparison ... Docker on OpenStack ... Containers on Google Cloud

Post on 27-May-2018

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

<ul><li><p>Docker on OpenStack </p><p>August 2014</p><p>Author :Nitin Agarwal nitinagarwal3006@gmail.com</p><p>Supervisor(s) :Belmiro Moreira</p><p> CERN openlab Summer Student Report 2014</p></li><li><p> CERN openlab Summer Student Report 2014 </p><p>ProjectSpecificationCERNisestablishingalargescaleprivatecloudbasedonOpenStackaspartoftheexpansionofthecomputinginfrastructureforstoringthedatacomingoutoftheLargeHadronCollider(LHC)experiments.Asthedatacomingoutofthedetectorsisincreasingcontinuouslythatneedstobestoredinthedatacenter,weneedmorephysicalresources(moremoney)andsinceVirtualmachinestakeslotofCPUandmemoryoverheadandminutesforcreatingtheimages,bootingupandforsnapshottingaswell.SoherecomesthesolutiontouseDockercontainers.Dockerisanopenplatformtobuild,shipandrundistributedapplications.DockerbeingacontainerbasedvirtualisationframeworkmakesuseofLXC.DockercontainersarelightweightandfastanddockermakesuseofUnionFileSystemwhichmakesitunique.DockercomeswiththeDockerIndex/Hubwhereyoucanstoreandsharethedockerimages.ThisprojectinvolvestheunderstandingofDockeranddockercontainersindetail,deploymentofprivateDockerRegistryaswellastheintegrationofdockerwithOpenstacktoenabletheNovacomputeservicetousethedockerAPIascomputedriverinsteadofthelibvirtAPI.</p><p> 1|Page </p></li><li><p> CERN openlab Summer Student Report 2014 </p><p>AbstractAtCERN,withtheeverincreasingamountofdatacomingoutofthedetectorsthatneedstobestoredinthedatacenter,newwaysaresoughttohelpanalyzeandstorethisdataaswellashelpresearchersperformtheirownexperiments.Tohelpoffersolutionstosuchproblems,CERNhasemployedtheuseofcloudcomputingandinparticularOpenStackanopensourceandscalableplatformforbuildingpublicandprivateclouds.OpenStackisusedtoview,create,andmanageresourcesinacloudandautomatethetasks.ComputenodesformtheresourcecoreoftheOpenStackComputecloud,providingtheprocessing,memory,networkandstorageresourcestoruninstances.Asthedataisincreasingcontinuouslyaround50PB/secandabout5PB/dayofdatathatneedstobestored,CERNislookingfornewwaystoutilisethehardwareresourcesofthedatacentermoreefficiently.InthisprojectweoutlineanddocumenttheintegrationofDockerwiththeNovacomputeserviceofOpenStack(Devstack,Packstack),deploymentofprivateDockerRegistryatCERNforpushingandpullingthedockerimages.ToallowtheNovacomputeservicetousetotheDockerAPIascomputedriverinsteadoftheLibvirtdriverandtoallownovatobootthedockerimages,weneedtostorethedockerimagesinglancethatactsasanindependentdockerregistryafterconfiguration.Inthisreport,wedescribeaboutdocker,itsbasicsandimportanceofdockercontainersincomparisontovirtualmachines,stepsfordeployingandconfiguringtheprivateDockerRegistryatCERNandstepsforconfiguringtheNovatousedockerdriverinDevstackonUbuntucloudimageandPackstackonRHEL7.</p><p> 2|Page </p></li><li><p> CERN openlab Summer Student Report 2014 </p><p>TableofContents1. Introduction...5</p><p>2. OpenStack.7</p><p>2.1. Overview72.2. InstallingandRunningOpenStack..82.3. WorkingwithNovaCLI.82.4. WorkingwithGlanceCLI..9</p><p>3. Docker9</p><p>3.1. Overview....93.2. BasicsoftheDockerSystem.103.3. AdvantagesofusingDocker.123.4. HowareDockerContainerslightweight...133.5. DockerContainersvsVMs143.6. InstallingDockeronRHEL7..14</p><p>4. DockerContainersandImages.15</p><p>4.1. RunningDockerContainers..154.2. WorkingwithDockerImages.174.3. UniqueAdvantagesofDockeroverothercontainertechnologies...18</p><p>5. DockerRegistry...19</p><p>5.1. Overview......195.2. DeployingyourownprivateDockerRegistry..195.3. PushingandPullingImagesfromRegistry..23</p><p>6. DockeronOpenStack....23</p><p>6.1. Overview......236.2. NovaDockerArchitecture..246.3. ConfiguringOpenStacktoenableDocker...25</p><p>6.3.1. InstallingDockerforOpenStack...256.3.2. NovaConfiguration266.3.3. GlanceConfiguration.26</p><p>6.4. DeploymentwithDevstack266.5. UploadingDockerImagestoGlance276.6. BootingInstancesusingNova..27</p><p>7. DockeronPackstack.28</p><p>7.1. Overview.28</p><p> 3|Page </p></li><li><p> CERN openlab Summer Student Report 2014 </p><p>7.2. InstallingPackstackonRHEL7...287.3. ConfiguringPackstacktoenableDocker29</p><p>7.3.1. InstallingDockerforPackstack297.3.2. NovaConfiguration....297.3.3. GlanceConfiguration.31</p><p>7.4. UploadingDockerImagestoGlance....317.5. BootingInstancesusingNova...32</p><p>8. ContainersonGoogleCloudPlatform.33</p><p>8.1. Overview.338.2. ContainerVMs....348.3. Kubernetes......34</p><p>9. Conclusion...35</p><p>10. Bibliography.36</p><p> 4|Page </p></li><li><p> CERN openlab Summer Student Report 2014 </p><p>1IntroductionLet'sconsiderthedeploymentofarelativelysimpleapplicationWordpress.AtypicalWordpressinstallationrequiresApache2,PHP5,MySQL,Wordpresssourcecode,MySQLdatabasewithWordpressconfiguredtousethisdatabase,apacheconfigurationtoloadthePHPmodule,enablethesupportforURLrewritingand.htaccessfiles,DocumentRootpointingtotheWordpresssources.Whiledeployingandrunningasystemlikethisonourserver,wemayrunintosomeproblemsandchallengesnamelyIsolation,Security,Upgrades,downgrades,Snapshotting,backingup,Reproducibility,Constrainresources,EaseofinstallationandEaseofremoval.AtCERN,wehavearound50PB/secofdatacomingoutofthedetectorsandabout5PB/daytobestoredintheserversdeployedatthedatacenter,wemakeuseofOpenStacktoview,create,andmanageresourcesinacloudandautomatethetasks.ComputenodesformtheresourcecoreoftheOpenStackComputecloud,providingtheprocessing,memory,networkandstorageresourcestoruninstances.Whenwedecidetoruneachindividualapplicationonaseparatevirtualmachine,mostofourproblemsgoawaybutwecomeacrossotherissues:</p><p> Money:canweactuallyreallyaffordbootingupaninstanceforeveryapplicationweneed?Alsocanwepredicttheinstancesizewewillneed,becauseifweneedmoreresourceslater,weneedtostoptheVMtoupgradeitoroverpayforresourceswedon'tendupusing.</p><p> Time:manyoperationsrelatedtovirtualmachinesaretypicallyslow.Bootingtakes</p><p>minutes,snapshottingtootakesminutes,creatinganimagetakesminutes.Theworldkeepsturningandwedon'thavesomuchoftime!</p><p>SousingDocker,Containerbasedvirtualisationframeworkandanopenplatformtobuild,shipandrundistributedapplicationsisthesolution.Dockercontainersarelightweightandfast.BootingupaVMisabigdealasittakesupfewminutestogetstartedandasignificantamountofmemorywhereasbootingupaDockercontainerisfastandusesverylittleCPUandmemoryoverhead.Almostcomparabletostartingaregularprocess.Notonlyrunningacontainerisfast,buildinganimageandsnapshottingthefilesystemisfastaswell.DockercontainersareportabletoanyoperatingsystemthatrunsDockerwhetherit'sUbuntuorCentOS.</p><p> Isolation:Dockerisolatesapplicationsatthefilesystemandnetworkinglevel.Itfeelsalotlikerunning"real"virtualmachinesinthatsense.</p><p> 5|Page </p></li><li><p> CERN openlab Summer Student Report 2014 </p><p> Reproducibility:Wecanbuildthesystemjustthewaywelike(eitherbylogginginand</p><p>aptgetinallsoftware,orusingaDockerfile),thencommitthechangestoanimage.Wecannowinstantiateasmanyinstancesofitaswewantortransfertoanimagetoanothermachinetoreproduceexactlythesamesetup.</p><p> Security:Dockercontainersaremoresecurethanregularprocessisolation.Link</p><p> Constrainresources:DockercurrentlysupportslimitingCPUusagetoacertainshare</p><p>ofCPUcycles,memoryusagecanalsobelimited.Restrictingdiskusageisnotdirectlysupportedasofyet.</p><p> Easeofinstallation:DockerhasDockerHub/Registry,arepositorywithofftheshelf</p><p>dockerimageswecaninstantiatewithasinglecommand.</p><p> Easeofremoval:Ifwedon'tneedanapplicationanymore,justdestroythecontainer.</p><p> Upgrades,downgrades:Bootupthenewversionofanapplicationfirst,thenswitchovertheloadbalancerfromtheoldporttothenewone.</p><p> Snapshotting,backingup:Dockersupportscommittingandtaggingofimages,which</p><p>incidentally,unlikesnapshottingaVMisinstant.</p><p> 6|Page </p><p>http://www.google.com/url?q=http%3A%2F%2Fblog.docker.com%2F2013%2F08%2Fcontainers-docker-how-secure-are-they%2F&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNGbMiKKW9t5E8U3eMvWMyah6Vw3nQhttp://www.google.com/url?q=http%3A%2F%2Fblog.docker.com%2F2013%2F08%2Fcontainers-docker-how-secure-are-they%2F&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNGbMiKKW9t5E8U3eMvWMyah6Vw3nQhttp://www.google.com/url?q=http%3A%2F%2Fblog.docker.com%2F2013%2F08%2Fcontainers-docker-how-secure-are-they%2F&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNGbMiKKW9t5E8U3eMvWMyah6Vw3nQ</p></li><li><p> CERN openlab Summer Student Report 2014 </p><p>2OpenStackOpenstackisacloudoperatingsystemthatcontrolslargepoolsofcompute,storage,andnetworkingresourcesthroughoutadatacenter.2.1OverviewTheOpenStackprojectcontainsvariouscomponentsthatindividuallyprovidecompute,storage,networkingandthedashboardbuttogethercreateafunctioningcloudoperatingsystem(OS).</p><p>ThecomponentsofOpenStackare:</p><p> Compute(Nova)theInfrastructureasaService(IaaS)systemprovidingvirtualmachinestohostswithnovacomputeinstalled.</p><p> IdentityService(Keystone)providestheauthenticationandauthorizationforall</p><p>OpenStackcomponents.</p><p> ImageService(Glance)animagerepositoryforallvirtualdiskimages.Glancecanalsobeconfiguredtostoretheseimagesonaremotecluster,suchasCeph.</p><p> Dashboard(Horizon)theuserinterfacetoeasilycontrolmostaspectsofthe</p><p>OpenStackcomponents.Asanalternative,theOpenStackAPIcanbeused.</p><p> 7|Page </p></li><li><p> CERN openlab Summer Student Report 2014 </p><p> Networking(Neutron)providesnetworkingasaservicebyallowinguserstocreate</p><p>theirownnetworksandinterfacesaswellasmanageIPs.</p><p> ObjectStorage(Swift)isahighlyavailableanddistributedobject/blobstore.</p><p> BlockStorage(Cinder)providesblockstorageasaservice2.2InstallingandRunningOpenStackWehaveinstalledDevstackontheUbuntu14.04imageinadedicatedVM.DevStackisasetofscriptsandutilitiestoquicklydeployanOpenStackcloud.Clonethegithubrepositoryofthedevstackbyexecutingthecommand:&gt;&gt;&gt;gitclonehttps://github.com/openstackdev/devstackTostartadevcloudrunthefollowingNOTASROOT(seeDevStackExecutionEnvironmentbelowformoreonuseraccounts):&gt;&gt;&gt;./stack.shWhenthescriptfinishesexecuting,youshouldbeabletoaccessOpenStackendpoints,likeso:</p><p> Horizon:http://myhost/ Keystone:http://myhost:5000/v2.0/</p><p>WealsoprovideanenvironmentfilethatyoucanusetointeractwithyourcloudviaCLI:#sourceopenrcfiletoloadyourenvironmentwithOpenStackCLIcreds.openrcORsourceopenrc#listinstancesnovalist2.3WorkingwithNovaCLINovaisacomputingprojectforOpenStack.Thelistofallthecommandsthatcanbeexecutedwithnovacanbeseenhere.SomeofthemostcommonnovaclientcommandstogetfamiliarisedwithandworkonDockerwithOpenStackarementionedbelow:</p><p> novabootBootanewserver. novadeleteImmediatelyshutdownanddeletespecifiedserver(s). novaflavorlistPrintalistofavailable'flavors'(sizesofservers).</p><p> 8|Page </p><p>http://www.google.com/url?q=http%3A%2F%2Fdocs.openstack.org%2Fuser-guide%2Fcontent%2Fnovaclient_commands.html&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNG3K96xHxyqAPF_zNXV6cthZLzVMQ</p></li><li><p> CERN openlab Summer Student Report 2014 </p><p> novaimagecreateCreateanewimagebytakingasnapshotofarunningserver. novaimagedeleteDeletespecifiedimage(s). novaimagelistPrintalistofavailableimagestobootfrom. novalistListactiveservers. novaservicelistShowalistofallrunningservices.Filterbyhost&amp;binary. novashowShowdetailsaboutthegivenserver.</p><p>2.4WorkingwithGlanceCLI GlanceistheimageserviceforOpenStack.Itservesasanimagerepositoryforallvirtualdiskimages.Thelistofallthecommandsthatcanbeexecutedwithglancecanbeseenhere.SomeofthemostcommonglanceclientcommandstogetfamiliarisedwithandworkonDockerwithOpenStackarementionedbelow:</p><p> glanceimagelistListimagesyoucanaccess. glanceimageshowDescribeaspecificimage. glanceimagecreateCreateanewimage. glanceimagedeleteDeletespecifiedimage(s).</p><p>3Docker3.1OverviewDockerisanopenplatformtobuild,shipandrundistributedapplications.Dockerconsistsof:</p><p> DockerEngine,alightweightandpowerfulopensourcecontainervirtualizationtechnologycombinedwithaworkflowforbuildingandcontainerizingyourapplications.</p><p> DockerHub/Registry,aSaaSserviceforsharingandmanagingyourapplication</p><p>stacksandautomatingworkflows.Dockerletsyouquicklyassembleapplicationsfromcomponentsandeliminatesthefrictionthatcancomewhenshippingcode.Asaresult,ITcanshipfasterandrunthesameapp,unchanged,onlaptops,datacenterVMsandonanycloudinfrastructure.Dockerletsyougetyourcodetestedanddeployedintoproductionasfastaspossible.Thenextwaveofvirtualization,andonethathasthepotentialtodisplacehypervisorbasedvirtualizationonLinuxplatforms,isuponusnowthatDocker,thesoftwarecontainerandapplicationpackagingsystem.</p><p> 9|Page </p><p>http://www.google.com/url?q=http%3A%2F%2Fdocs.openstack.org%2Fuser-guide%2Fcontent%2Fglanceclient_commands.html&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNGGCW-uAcK8yPsYY7evQsDlhK6Oow</p></li><li><p> CERN openlab Summer Student Report 2014 </p><p>3.2BasicsoftheDockerSystem</p><p> Dockermakespackagingallofthepartsofanapplicationthetools,configurationfiles,libraries,andmoreintoamuchsimplertask.It'sabitlikeavirtualmachine,conceptually,allowingforthesegmentationofasinglepowerfulmachinetobesharedwithmanyapplicationswiththeirownindividualspecificconfigurationrequirements,andwithoutallowingthoseapplicationstointerferewithoneanother.Exceptunlikeavirtualmachine,applicationsrunnativelyontheunderlyingLinuxkernel,andeachisjustverycarefullysegmentedfromoneanotherandfromtheunderlyingoperatingsystem.DockercurrentlyusesLinuxContainers,whichrunsinthesameoperatingsystemasitshostandprovidessystemlevelvirtualisation.Thisallowsittosharealotofthehostoperatingsystemresources.Itusesaufsasthefilesystem.DockerusesLinuxcgroupsandnamespacestoisolateprocessesfromeachothersotheyappeartorunontheirownsystem.Dockerconsistsofthreeparts:DockerDaemon,DockerImages,theDockerRepositorieswhichtogethermakeLinuxContainereasyandfuntouse.DockerDaemonrunsasrootandorchestratesallrunningcontainers.Justasvirtualmachinesarebasedonimages,DockerContainersarebasedonDockerimages.Theseimagesaretinycomparedtovirtualmachineimagesandarestackable.</p><p> 10|Page </p></li><li><p> CERN openlab Summer Student Report 2014 </p><p>AuFSisalayeredfilesystem,soyoucanhaveareadonlypart,andawritepart,andmergethosetogether.Soyoucouldhavethecommonpartsoftheoperatingsystemasreadonly,whicharesharedamongstallofyourcontainers,andthengiveeachcontaineritsownmountforwriting.Namespaces:Dockerusesnamespacestoprovidetheisolatedworkspaceforcontainers.Whenwerunacontainer,Dockercreatesasetofnamespacesforthatcontainer.Thisprovidesalayerofisolation:eachaspectofacontainerrunsinitsownnamespaceanddoesnothaveaccessoutsideit.SomeofthenamespacesthatDockerusesare:</p><p> Thepidnamespace:Usedforprocessisolation(PID:ProcessID). Thenetnamespace:Usedformanagingnetworkinterfaces(NET:Networking). Theipcnamespace:UsedformanagingaccesstoIPCresources(IPC:InterProcess</p><p>Communication). Themntnamespace:Usedformanagingmountpoints(MNT:Mount). Theutsnamespace:Usedforisolatingkernelandversionidentifiers.(UTS:Unix</p><p>TimesharingSystem).Controlgroups:Dockeralsomakesuseofanothertechnologycalledcgroupsorcontrolgroups.Akeytorunningapplicationsinisolationistohavethemonlyusetheresourcesyouwant.Thisensurescontainersaregoodmultitenantcitizensonahost.ControlgroupsallowDockertoshareavailablehardwareresourcestocontainersand,ifrequired,setuplimitsandconstraints.Forexample,limitingthememoryavailabletoaspecificcontainer.Unionfilesystems,orUnionFS,arefilesystemsthatoperatebycreatinglayers,makingthemverylightweightandfast.Dockerusesunionfilesystemstoprovidethebuildingblocksforcontainers.Dockercanmakeuseofseveralunionfilesystemvariantsincluding:AUFS,btrfs,vfs,andDeviceMapper.ContainerFormat:Dockercombinesthesecomponentsintoawrapper...</p></li></ul>