docker on openstack - core this report, we describe about docker, its basics and importance of...

Download Docker on OpenStack - CORE this report, we describe about docker, its basics and importance of docker containers in comparison ... Docker on OpenStack ... Containers on Google Cloud

Post on 27-May-2018

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Docker on OpenStack

    August 2014

    Author :Nitin Agarwal nitinagarwal3006@gmail.com

    Supervisor(s) :Belmiro Moreira

    CERN openlab Summer Student Report 2014

  • CERN openlab Summer Student Report 2014

    ProjectSpecificationCERNisestablishingalargescaleprivatecloudbasedonOpenStackaspartoftheexpansionofthecomputinginfrastructureforstoringthedatacomingoutoftheLargeHadronCollider(LHC)experiments.Asthedatacomingoutofthedetectorsisincreasingcontinuouslythatneedstobestoredinthedatacenter,weneedmorephysicalresources(moremoney)andsinceVirtualmachinestakeslotofCPUandmemoryoverheadandminutesforcreatingtheimages,bootingupandforsnapshottingaswell.SoherecomesthesolutiontouseDockercontainers.Dockerisanopenplatformtobuild,shipandrundistributedapplications.DockerbeingacontainerbasedvirtualisationframeworkmakesuseofLXC.DockercontainersarelightweightandfastanddockermakesuseofUnionFileSystemwhichmakesitunique.DockercomeswiththeDockerIndex/Hubwhereyoucanstoreandsharethedockerimages.ThisprojectinvolvestheunderstandingofDockeranddockercontainersindetail,deploymentofprivateDockerRegistryaswellastheintegrationofdockerwithOpenstacktoenabletheNovacomputeservicetousethedockerAPIascomputedriverinsteadofthelibvirtAPI.

    1|Page

  • CERN openlab Summer Student Report 2014

    AbstractAtCERN,withtheeverincreasingamountofdatacomingoutofthedetectorsthatneedstobestoredinthedatacenter,newwaysaresoughttohelpanalyzeandstorethisdataaswellashelpresearchersperformtheirownexperiments.Tohelpoffersolutionstosuchproblems,CERNhasemployedtheuseofcloudcomputingandinparticularOpenStackanopensourceandscalableplatformforbuildingpublicandprivateclouds.OpenStackisusedtoview,create,andmanageresourcesinacloudandautomatethetasks.ComputenodesformtheresourcecoreoftheOpenStackComputecloud,providingtheprocessing,memory,networkandstorageresourcestoruninstances.Asthedataisincreasingcontinuouslyaround50PB/secandabout5PB/dayofdatathatneedstobestored,CERNislookingfornewwaystoutilisethehardwareresourcesofthedatacentermoreefficiently.InthisprojectweoutlineanddocumenttheintegrationofDockerwiththeNovacomputeserviceofOpenStack(Devstack,Packstack),deploymentofprivateDockerRegistryatCERNforpushingandpullingthedockerimages.ToallowtheNovacomputeservicetousetotheDockerAPIascomputedriverinsteadoftheLibvirtdriverandtoallownovatobootthedockerimages,weneedtostorethedockerimagesinglancethatactsasanindependentdockerregistryafterconfiguration.Inthisreport,wedescribeaboutdocker,itsbasicsandimportanceofdockercontainersincomparisontovirtualmachines,stepsfordeployingandconfiguringtheprivateDockerRegistryatCERNandstepsforconfiguringtheNovatousedockerdriverinDevstackonUbuntucloudimageandPackstackonRHEL7.

    2|Page

  • CERN openlab Summer Student Report 2014

    TableofContents1. Introduction...5

    2. OpenStack.7

    2.1. Overview72.2. InstallingandRunningOpenStack..82.3. WorkingwithNovaCLI.82.4. WorkingwithGlanceCLI..9

    3. Docker9

    3.1. Overview....93.2. BasicsoftheDockerSystem.103.3. AdvantagesofusingDocker.123.4. HowareDockerContainerslightweight...133.5. DockerContainersvsVMs143.6. InstallingDockeronRHEL7..14

    4. DockerContainersandImages.15

    4.1. RunningDockerContainers..154.2. WorkingwithDockerImages.174.3. UniqueAdvantagesofDockeroverothercontainertechnologies...18

    5. DockerRegistry...19

    5.1. Overview......195.2. DeployingyourownprivateDockerRegistry..195.3. PushingandPullingImagesfromRegistry..23

    6. DockeronOpenStack....23

    6.1. Overview......236.2. NovaDockerArchitecture..246.3. ConfiguringOpenStacktoenableDocker...25

    6.3.1. InstallingDockerforOpenStack...256.3.2. NovaConfiguration266.3.3. GlanceConfiguration.26

    6.4. DeploymentwithDevstack266.5. UploadingDockerImagestoGlance276.6. BootingInstancesusingNova..27

    7. DockeronPackstack.28

    7.1. Overview.28

    3|Page

  • CERN openlab Summer Student Report 2014

    7.2. InstallingPackstackonRHEL7...287.3. ConfiguringPackstacktoenableDocker29

    7.3.1. InstallingDockerforPackstack297.3.2. NovaConfiguration....297.3.3. GlanceConfiguration.31

    7.4. UploadingDockerImagestoGlance....317.5. BootingInstancesusingNova...32

    8. ContainersonGoogleCloudPlatform.33

    8.1. Overview.338.2. ContainerVMs....348.3. Kubernetes......34

    9. Conclusion...35

    10. Bibliography.36

    4|Page

  • CERN openlab Summer Student Report 2014

    1IntroductionLet'sconsiderthedeploymentofarelativelysimpleapplicationWordpress.AtypicalWordpressinstallationrequiresApache2,PHP5,MySQL,Wordpresssourcecode,MySQLdatabasewithWordpressconfiguredtousethisdatabase,apacheconfigurationtoloadthePHPmodule,enablethesupportforURLrewritingand.htaccessfiles,DocumentRootpointingtotheWordpresssources.Whiledeployingandrunningasystemlikethisonourserver,wemayrunintosomeproblemsandchallengesnamelyIsolation,Security,Upgrades,downgrades,Snapshotting,backingup,Reproducibility,Constrainresources,EaseofinstallationandEaseofremoval.AtCERN,wehavearound50PB/secofdatacomingoutofthedetectorsandabout5PB/daytobestoredintheserversdeployedatthedatacenter,wemakeuseofOpenStacktoview,create,andmanageresourcesinacloudandautomatethetasks.ComputenodesformtheresourcecoreoftheOpenStackComputecloud,providingtheprocessing,memory,networkandstorageresourcestoruninstances.Whenwedecidetoruneachindividualapplicationonaseparatevirtualmachine,mostofourproblemsgoawaybutwecomeacrossotherissues:

    Money:canweactuallyreallyaffordbootingupaninstanceforeveryapplicationweneed?Alsocanwepredicttheinstancesizewewillneed,becauseifweneedmoreresourceslater,weneedtostoptheVMtoupgradeitoroverpayforresourceswedon'tendupusing.

    Time:manyoperationsrelatedtovirtualmachinesaretypicallyslow.Bootingtakes

    minutes,snapshottingtootakesminutes,creatinganimagetakesminutes.Theworldkeepsturningandwedon'thavesomuchoftime!

    SousingDocker,Containerbasedvirtualisationframeworkandanopenplatformtobuild,shipandrundistributedapplicationsisthesolution.Dockercontainersarelightweightandfast.BootingupaVMisabigdealasittakesupfewminutestogetstartedandasignificantamountofmemorywhereasbootingupaDockercontainerisfastandusesverylittleCPUandmemoryoverhead.Almostcomparabletostartingaregularprocess.Notonlyrunningacontainerisfast,buildinganimageandsnapshottingthefilesystemisfastaswell.DockercontainersareportabletoanyoperatingsystemthatrunsDockerwhetherit'sUbuntuorCentOS.

    Isolation:Dockerisolatesapplicationsatthefilesystemandnetworkinglevel.Itfeelsalotlikerunning"real"virtualmachinesinthatsense.

    5|Page

  • CERN openlab Summer Student Report 2014

    Reproducibility:Wecanbuildthesystemjustthewaywelike(eitherbylogginginand

    aptgetinallsoftware,orusingaDockerfile),thencommitthechangestoanimage.Wecannowinstantiateasmanyinstancesofitaswewantortransfertoanimagetoanothermachinetoreproduceexactlythesamesetup.

    Security:Dockercontainersaremoresecurethanregularprocessisolation.Link

    Constrainresources:DockercurrentlysupportslimitingCPUusagetoacertainshare

    ofCPUcycles,memoryusagecanalsobelimited.Restrictingdiskusageisnotdirectlysupportedasofyet.

    Easeofinstallation:DockerhasDockerHub/Registry,arepositorywithofftheshelf

    dockerimageswecaninstantiatewithasinglecommand.

    Easeofremoval:Ifwedon'tneedanapplicationanymore,justdestroythecontainer.

    Upgrades,downgrades:Bootupthenewversionofanapplicationfirst,thenswitchovertheloadbalancerfromtheoldporttothenewone.

    Snapshotting,backingup:Dockersupportscommittingandtaggingofimages,which

    incidentally,unlikesnapshottingaVMisinstant.

    6|Page

    http://www.google.com/url?q=http%3A%2F%2Fblog.docker.com%2F2013%2F08%2Fcontainers-docker-how-secure-are-they%2F&sa=D&sntz=1&usg=AFQjCNGbMiKKW9t5E8U3eMvWMyah6Vw3nQhttp://www.google.com/url?q=http%3A%2F%2Fblog.docker.com%2F2013%2F08%2Fcontainers-docker-how-secure-are-they%2F&sa=D&sntz=1&usg=AFQjCNGbMiKKW9t5E8U3eMvWMyah6Vw3nQhttp://www.google.com/url?q=http%3A%2F%2Fblog.docker.com%2F2013%2F08%2Fcontainers-docker-how-secure-are-they%2F&sa=D&sntz=1&usg=AFQjCNGbMiKKW9t5E8U3eMvWMyah6Vw3nQ

  • CERN openlab Summer Student Report 2014

    2OpenStackOpenstackisacloudoperatingsystemthatcontrolslargepoolsofcompute,storage,andnetworkingresourcesthroughoutadatacenter.2.1OverviewTheOpenStackprojectcontainsvariouscomponentsthatindividuallyprovidecompute,storage,networkingandthedashboardbuttogethercreateafunctioningcloudoperatingsystem(OS).

    ThecomponentsofOpenStackare:

    Compute(Nova)theInfrastructureasaService(IaaS)systemprovidingvirtualmachinestohostswithnovacomputeinstalled.

    IdentityService(Keystone)providestheauthenticationandauthorizationforall

    OpenStackcomponents.

    ImageService(Glance)animagerepositoryforallvirtualdiskimages.Glancecanalsobeconfiguredtostoretheseimagesonaremotecluster,suchasCeph.

    Dashboard(Horizon)theuserinterfacetoeasilycontrolmostaspectsofthe

    OpenStackcomponents.Asanalternative,theOpenStackAPIcanbeused.

    7|Page

  • CERN openlab Summer Student Report 2014

    Networking(Neutron)providesnetworkingasaservicebyallowinguserstocreate

    theirownnetworksandinterfacesaswellasmanageIPs.

    ObjectStorage(Swift)isahighlyavailableanddistributedobject/blobstore.

    BlockStorage(Cinder)providesblockstorageasaservice2.2InstallingandRunningOpenStackWehaveinstalledDevstackontheUbuntu14.04imageinadedicatedVM.DevStackisasetofscriptsandutilitiestoquicklydeployanOpenStackcloud.Clonethegithubrepositoryofthedevstackbyexecutingthecommand:>>>gitclonehttps://github.com/openstackdev/devstackTostartadevcloudrunthefollowingNOTAS