git + docker tutorial - nersc-docker... · git history • git is a ‘version control system’,...

Tony Wildish!

Git + Docker tutorial

Preamble

•  Thispresenta,on,thetutorialmaterial–  h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip–  h"ps://www.nersc.gov/users/computa>onal-systems/genepool/genepool-training-and-tutorials/

•  Pre-requisites–  Seeh"ps://bitbucket.org/TWildish/git-docker-tutorial/overview

–  PleasetellmeyoudidthatalreadyJ•  Today:–  3:00–4:00:gitoverview+hands-onexercises–  4:00–5:00:dockeroverview+hands-onexercises–  Familiaritywithwhat’spossible,ratherthanadeep-dive– Workedexamplesofhowtodothings

-2-

This tutorial

•  Git–  Basicsofrepositories,localandremote–  Howtorecoverfrommistakes–  Workingwithbranches–  Workingwithteams

•  Docker–  Variouswaystorun&managedockercontainers–  Arealbioinforma>csapplica>onexample

•  ThankstoMichaelBarton–  Howtogetdatainto/outofadockercontainer–  Howtobuildasimpledockercontainer–  ShiTer–dockeronCori,Edison,and(eventually)Genepool

-3-h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip

Git history

•  Gitisa‘VersionControlSystem’,(VCS)•  Gitmanagescollec,onsoffiles(text,smallbinaries)–  Trackstheirhistory,versions–  Tracksmul>pledevelopmentpaths–  Letsyourecoverpreviousversions

•  GitistheVCS,don’tbotherwithanythingelse–  CVS:ConcurrentVersionSystem->completelyobsolete–  SVN:SubVersioN->mostlyobsolete(shouldbe!)

•  DesignedbyLinusTorvalds(hewhogaveusLinux!)•  Q:Whatdoes‘git’standfor?


Why use git?

•  Security–  Neverloseyourcodeagain–  Codeissafeagainstdiskfailure/earthquakes/meteors

•  Convenience–  Easilydeployyourcodeinseveralplaces–  Easilymanageseveralversions(prod,dev,…)

•  Community–  Shareyourcodewithothers–  Acceptbug-fixes&contribu>onsincontrolledmanner

•  DidImen,on…–  Neverloseyourcodeagain


Git components

•  Command-lineinterface,the‘git’command•  Server‘hos,ng’plaRorms,web-interface,API–  Github.com:theoriginalgithos>ngservice–  Bitbucket.com:usedbyLBNL/JGI–  Gitlab.com:recentplacormwithcon>nuousintegra>on

•  Hos,ngplaRormsbringaddedvalue–  Issuetracking:bugreports,coupledtogithistory– Wiki:managingdocumenta>on–  Teammgmt:differentroles(admin,developer,user)–  Accessmgmt:read/write,read-only,private,public–  ‘web-hooks’:performcustomac>onsbasedontriggers


Git concepts

•  Repository–  Localorremote,aplacewheregitkeepsyourfiles

•  Onyourlocaldisk,oronaremoteserver

•  Workingarea–  Partofyourlocalrepository,youedityourcodethere

•  Stagingarea–  Partofthelocalrepositorywheregittrackschangestoyourworkingarea

•  Branches,tags– Waystomanagesub-groupsoffilesinarepository


Git workflow

•  Changefilesinyourworkingarea•  Tellgitaboutthechanges–  Thisaddsthefilestothe‘stagingarea’–  Atthispoint,s>llpossibletoundo,leavingnotrace

•  Committhosechanges– Makethempermanent,addthemtotherepository–  Nowthosechangescanberecovered,any>melater

•  Pushthechangestoaremoterepository–  Copyyourlocalrepositorytoaremoteserver–  Nowyouhavearemotebackup


More git concepts

•  Clone–  Alocalcopyofaremoterepository–  Youcanchangetheclone–youownit–  Accesstoremoterepositorycontrolledbyitsowner

•  Fork–  Aremotecopyofanotherremoterepository–  Youownthefork,whichyoucannowcloneandchange

•  Anon-concept:‘TheCentralRepository’–  Gitiscompletelydecentralized–  Canworkwithmul>pleremoterepositories,simultaneously

•  Confused?–  Let’sgetstuckintotheexercises...


Git exercises

•  Cookbookapproach:–  Cancut-&-paste,butbe"ertotypeincommandsyourself

•  Today:doexercises1,3,and4ifyouhave,me–  1)BasicCommitandTag–  2)UndoingMistakes–  3)UsingARemoteRepository–  4)UsingBranches–  5)WorkinginTeams

•  Feelfreetoworkthroughtherestatyourownpace


Docker

-11-

Docker overview

•  Dockerisa‘containertechnology’–  Linux-specific

•  can’trunMacOSX,Windowsindockercontainers•  ButcanrundockercontainersonMacOSX&Windows

•  Similartovirtualmachines,butmorelightweight–  Smaller,fastertostart,easiertomaintainandmanage–  Lighteronsystemresources=>vastlymorescalable

•  Notavirtualmachine–  Sharestheunderlyinghostopera>ngsystem–  Lessfullyisolatedfromthehost=>securityconcerns– Moreofanapplica>on-wrapperonsteroids


Docker components

•  The‘docker’command-linetool–  Abitofakitchen-sink,yourone-stopshopforeverythingdocker

•  Thedocker-daemon–  Worksbehindthescenestocarryoutac>ons–  Managescontainerimages,processes–  Buildscontainerswhenrequested–  Runsasroot,notauser-spacedaemon

•  Docker.com–  Allthingsdocker:installa>on,documenta>on,tutorials

•  Dockerhub.com–  Repositoryofdockercontainers.Manyotherrepositoriesexist


Docker concepts •  Image

–  Ashrink-wrappedchunkofs/w+itsexecu>onenvironment•  Imagetags

–  Iden>fydifferentversionsofanimage–  Anamespaceforsepara>ngyourimagesfromotherpeoples

•  Imageregistry–  Aplaceforsharingimageswithawidercommunity–  Dockerhub.com,plussomedomain-specificregistries

•  Container–  Aprocessinstan>atedfromanimage

•  Dockerfile–  Arecipeforbuildinganimage:download,compile,configure…–  CanshareeithertheDockerfile,ortheimage,orboth


Docker images: layers

•  Imagesusethe‘overlayfilesystem’concept–  Imageisbuiltbyaddinglayerstoabase–  EachcommandintheDockerfileaddsanewlayer–  Eachlayeriscachedindependently–  Layerscanbesharedbetweenmul>pleimages–  Changeinonelayerinvalidatesallfollowinglayers

•  Forcesrebuild(similarto‘make’dependencies…)

•  Performanceconsidera,ons–  Toomanylayerscanimpedeperformance–  Toofewcancauseexcessiverebuilding–  Buildingproduc>on-qualityimagestakescare,prac>ce


Docker exercises

•  Again,acookbookapproach•  Today:1,3and4aremostinteres,ng–  1)RunningImages–  2)Cleaningup–  3)RunningaBioboxContainer–  4)Crea,ngaDockerImage–  5)RunningonCoriwithShiTer

-16-

h"ps://bitbucket.org/TWildish/git-docker-tutorial/get/master.zip

git + docker tutorial - nersc-docker... · git history • git is a ‘version control system’,...

Documents