kubernetes - pywaw.orgpywaw.org/media/slides/pywaw-70-kubernetes-taking... · kubernetes kubernetes...

Kubernetes

Introduction

WOJCIECHBARCZYŃSKI(hiring)SeniorSoftwareEngineerLeadofWarsawTeam-SMACCSystemEngineerbackgroundInterests:workingsoftwareHobby:teachingsoftwareengineering

BACKGROUNDAtopAIFinTech➡microservicesandk8sBeforerenewtechstackofatopIndonesianmobileecommerce3.5ywithOpenstack,1000+nodes,21datacentersIdonotlikeINFRA:D

KUBERNETES

Kubernetes-greekforhelmsmanRunandManagescontainersInspiredbyGoogle'sBorgIntegratedwithAWS,GCP,AzureBecominganintegrationplatformforlargeecosystem

ManagesApplicationsnotMachines!

GOALS

Utilizedresourcesnearly100%Gettoapplications/servicesmindsetEnforcelooselycouplesoftware-12factorapps,Amazon-APIapproachBestpractisesincluded,e.g.,nameservice,metadatadiscovery,...

CURRENTWINNER

«AmazonjoinedKuberneteson10.08.2017»

WHYKUBERNETES?DataCenterasaBlackBoxBatteriesforyour(12factor)apps

WHYKUBERNETES?GiveyoucompletecontroloveryourapplicationwithsimpleyamlconfigfilesUselabelstoauto-wireyourapptomonitoring,logging,andalarmingLetyouto,almostforget,abouttheinfrastructure

BatteriesLoadBalancingNameServiceDiscoveryMetadataandAnnotationsupportDecoupledinterfaceandimplementationLabeledbasedmatching

DATACENTERASABLACKBOX

KUBERNETES

Kubernetes

IngressController

Node

Node

Node

Node

App

DockerImage

ServiceDeployment

k8sconfig:

makedocker_push;kubectlcreate-fapp-srv-dpl.yaml

SCALEUP!SCALEDOWN!

Kubernetes

IngressController

Node

Node

Node

NodeApp

App

App

scale3x

kubectl--replicas=3-fapp-srv-dpl.yaml

INGRESSCONTROLLERapi.smacc.io/v1/users➡service:users-v1api.smacc.io/v2/users➡service:users-v2api.smacc.io/accounts➡service:accountssmacc.io➡service:website

INGRESSCONTROLLER

API

BACKOFFICE1

DATA

WEB

ADMIN

BACKOFFICE2

BACKOFFICE3

API.DOMAIN.COM

DOMAIN.COM/WEB

BACKOFFICE.DOMA

IN.COM

ORCHESTRATORPRIVATENETWORKINTERNET

API

LISTEN

(DOCKER,SWARM,MESOS...)

ROLLINGUPDATES!

Kubernetes

IngressController

Node

Node

Node

NodeApp

App

App

DockerImagev2

kubectlsetimagedeployment/appapp=app:v2.0.0

ROLLINGUPDATES!

Kubernetes

IngressController

Node

Node

Node

Node

App

App

DockerImagev2

ROLLINGUPDATES!

Kubernetes

IngressController

Node

Node

Node

Node

App

App

DockerImagev2

App

ROLLINGUPDATES!

Kubernetes

IngressController

Node

Node

Node

NodeApp

App

App

DockerImagev2

LOADBALANCING

KubernetesWorker

KubernetesWorker

KubernetesWorker

Node

Port30000

Node Node

App

App

KubernetesWorker

Node

App

<<Requests>>

B

Service

Port30000

Port30000

Port30000

LoadBalancer

RESISTANCE!

Kubernetes

IngressController

Node

Node

Node

NodeApp

App

App

RESISTANCE!

Kubernetes

IngressController

Node

Node

NodeApp

App

App

RESISTANCE!WhenthenodediesinflamesWhenotherapps(withhigherguaranteedquotas)eatsallmemoryWhenyouneedtodrainnodesbeforeupgradeYoucaneasilyscaleup,createmachineandjoinittocluster(easierwithkopsoronGCE)

FEDERATION

GlobalLoadBalancer

App App App

On-premisePoland

Amazoneu-west-1

Googleasia-southeast1

MUCHMOREPlug-and-playintegrations:

integrationwithAWS,GoogleCloudPlatform,andAzuremultipledriversfornetwork,storage,...youcanrunonminikube

MUCHMOREKubernetesadministratedwithkubernetes:

everythingruninpodse.g.,youdeployyourlogcollectorsfork8saspods:http://wbarczynski.pl/centralized-logging-for-kubernetes-with-fluentd-and-elasticsearch/

BASICCONCEPTS

Name Purpose

Service Interface ServiceName,port,labels,annotations

Deployment Factory Howmanypodswithwhichdockerimages,labels

Pod Implementation 1+dockerimagesrunningin1pod

BASICCONCEPTSconfig/secret➡configandfilesingress-controller➡urlpattern➡service

SERVICEservice.yaml:

apiVersion:v1kind:Servicemetadata:name:api-statusspec:ports:-port:80protocol:TCPselector:app:api-status

SERVICE#createtheserviceanddeploymentkubectlcreate-fapi-status-srv.yamlkubectlcreate-fapi-status-dpl.yaml

#gettoarunningdocker(inapod)kubectl-itexecapp-999-8zh1p/bin/bash

#checkwhethernameserviceworkscurlhttp://api-status/healthOK

BASICCONCEPTS

Service

Pods

Labels

Deployment Deployment

<<Creates>><<Creates>>

ServiceNameServicePort

deployment.yamlapiVersion:apps/v1beta1kind:Deploymentmetadata:name:api-status-nginxapp:api-statusspec:replicas:1template:metadata:labels:name:api-status-nginxapp:api-statusspec:containers:-name:nginximage:nginx

CONFIGenvvariablesindeployment:

env:-name:SEARCH_ENGINE_USERvalue:mighty_mouse

CONFIGfeedenvsfromconfigmaps:

env:-name:SEARCH_ENGINE_USERvalueFrom:configMapKeyRef:name:my-configkey:search.user

CONFIGyoucanshipfilesusingconfigmaps/secrets

kubectlcreateconfigmapmy-config-file--from-file=config.json

CONFIGYoucanalsorunyourown:

HashiCorpConsuloretcdHashiCorpVault

METADATAANDANNOTATIONSAuto-wiringPrecisediscoveryReportingLabelingtargetsforsecurityscansLabelingcriticalservicesforoncall(seealertmanager)

MONITORINGWITHKUBERNETESYoudeployamemcachedExposeditsprometheusmetricsonmetrics/Howtoshipmetrics?

ANNOTATIONS!memcached-0-deployment.yaml

https://github.com/skarab7/kubernetes-memcached

---apiVersion:v1kind:Servicemetadata:name:memcached-0labels:app:memcachedkubernetes.io/name:"memcached"role:shard-0tier:backendannotations:prometheus.io/scrape:"true"prometheus.io/scheme:"http"prometheus.io/path:"metrics"prometheus.io/port:"9150"

INGRESSCONTROLLERWITHTRAEFIK?

ANNOTATIONS!Usetraefikinsteadofbuilt-inreverseproxy

apiVersion:extensions/v1beta1kind:Ingressmetadata:name:api-statusnamespace:productionannotations:kubernetes.io/ingress.class:traefikspec:rules:-host:api.example.comhttp:paths:-path:/statusbackend:serviceName:api-status

LABELS!Monitoringrulethatuseslabels:

AlertManager

ALERTProductionAppServiceInstanceDownIFup{environment="production",app=~".+"}==0FOR4mANNOTATIONS{summary="Instanceof{{$labels.app}}isdown",description="Instance{{$labels.instance}}ofapp{{$labels.app}}hasbeendownformorethan4minutes"}

LABELS!Callsbifthelabelisseverity=page:

AlertManager

group_by:[cluster]#Ifanalertisn'tcaughtbyaroute,sendittothepager.receiver:team-pagerroutes:-match:severity:pagereceiver:team-pager

receivers:-name:team-pageropsgenie_configs:-api_key:$API_KEYteams:example_team

THEREISSOMUCHMOREresourcequotaseventsinKubernetesreadinessprobeslivenessprobesvolumesstatefulnamespaces...

KUBERENTESAwesomecommand-lineResilientplatformsimpleYAMLfilestosetupyourservice,servicediscoveryincludedannotationsandmetadatadiscoveryincluded

0.1➡1.0Yourcomponentneedstogetmuchmoresmarter.

SERVICESELF-CONSCIOUSNESSYourendpoint:

metrics/alertrules/-[WIP]health/orhealthz/info/

DEEPLOOKINSIDEwhenIamreadytoserverequestswhenIneedtorestartmyselfwhattodowhendependentservicesaredown...

DEEPLOOKINSIDEAmIreallystateless?Caching?fail-fast,startfast

RELATIONSWITHOTHERSmaster-workerrelationshipswaitingforotherresources/services

12FACTORAPPSfindservicesbynameorURImovetheimportantconfigtoenvironmentvariables

LOGGINGlogstashjsonformatmakeconfigurablewithENVvariable

EFKorELK

WHATWITHYOURDATABASESKeepitinaseparated(k8s)clusterThebest,gowithDaaSWithStateful,youcanrunyourdbink8s

Longdiscussion...

MIGRATIONOFENVStaging,production,canary,green/blue...:

Ifyouhave$$$,haveaseparatedk8sclusterIfnot,useNamespaces

APPSINNEWWORLD12factorapps(Heroku,2012)muchmuchsmartermuchfastermuchmorepredictablemuchhardertodevelop:DForgingexperienceintocode[WIP]:https://github.com/microdevs

THANKYOU

(hiring)WojciechBarczyński(wojciech.barczynski@smacc.io)

Backupslides

6+1STEPSThebig1-makingyourappsmarter

1.CLEANUPSinglescriptforrepo-Makefile[1]ResurrecttheREADME

[1]Withzshorbashauto-completionplugininyourterminal.

2.GETBACKALLTHEKNOWLEDGEPuppet,Chef,...➡DockerfileChecktheinstances➡Dockerfile,README.rstNagios,...➡README.rst,checks/

3.INTRODUCERUN_LOCALmakerun_localAnicesectiononhowtoruninREADME.rstUse:docker-compose

Themostcrucialpoint.

4.GETTOKUBERNETESmakekube_create_configmakekube_applyGeneratetheyamlfilesifyourenvsdiffer

5.CONTINUOUSDEPLOYMENTSimplecomponents:

testcode,builddocker,pushtodockerreporuntherollingupdate:kubectlsetimagedeployment/api-statusnginx=nginx:1.9.1IuseTravisCI

5.CONTINUOUSDEPLOYMENTComplexcomponents:

withlabel-basedmatching,theskyisthelimit

6.KEEPITRUNNINGBrigdethenewwithold:

Youcanaddyourexternalserviestothek8sNameServiceYoucanbridgeKubernetesservicestoyourServiceDiscovery[1]

[1]YoucansubscribetoK8Seventstokeep,e.g.,yourconsulinsync