package your java application using docker and kubernetes · service discovery & lb: docker...

PackageyourJavaApplicationusingDockerandKubernetes

Arun Gupta, @arungupta

Docker Captain Java Champion

JavaOne Rock Star (4 years) NetBeans Dream Team

Silicon Valley JUG Leader Author Runner

Lifelong learner

2

DockerWorkflow

3

OrchestrationFrameworks▪Developer

– Core concepts – Cluster – Single container – Multi-container

– Service discovery & LB – Persistent Volumes – Local development

▪Ops – Multiple master – Scheduler – Rules and constraints – Monitoring – Rolling Update – Cloud/commercial support

4

CoreConcepts:DockerManager

5

docker swarm init --listen-addr <ip>:2377

CoreConcepts:DockerWorker

6

docker swarm join --token <worker_token> <manager>:2377

CoreConcepts:AddMoreWorkers

7

docker swarm join --token <worker_token> <manager>:2377

CoreConcepts:Primary/SecondaryMaster

8

docker swarm join --manager --token <manager_token> --listen-addr <master2>:2377 <master1>:2377

Swarm Manager

Swarm Worker Swarm WorkerSwarm Worker Swarm WorkerSwarm Worker

Swarm Manager Swarm Manager

Raft Consensus Group

Gossip Network

primary secondary secondary Container

CoreConcepts:Cluster

Strongly consistent Replicated (Raft based) Extremely fast (in-memory reads)

CoreConcepts:ReplicatedService

10

docker service create --replicas 3 --name web jboss/wildfly

CoreConcepts:NodeFailure

11

X

CoreConcepts:Desired!=Actual

12

CoreConcepts:Reconcile

13

CoreConcepts:ContainerFailure

14

X

CoreConcepts:Desired!=Actual

15

CoreConcepts:Reconcile

16

CoreConcepts:Scale

17

docker service scale web=6

CoreConcepts:GlobalService

18

docker service create --mode=global --name=prom prom/prometheus

©2016CouchbaseInc.

CoreConcepts:Kubernetes▪Pods: colocated group of containers that share an IP, namespace, storage volume ▪Replica Set: manages the lifecycle of pods and ensures specified number are running (next gen Replication Controller) ▪Service: Single, stable name for a set of pods, also acts as LB ▪Label: used to organize and select group of objects

19

“db”

port 8091 port 8091

NodeDocker

Pod

Containers

©2016CouchbaseInc.

CoreConcepts:Kubernetes▪Node: Machine or VM in the cluster ▪Master: Central control plane, provides unified view of the cluster

– etcd: distributed key-value store used to persist Kubernetes system state

▪Worker: Docker host running kubelet (node agent) and proxy services

– Runs pods and containers – Monitored by systemd (CentOS) or

monit (Debian)

20

Master

API Server(pods, services, …)

Controller Manager

etcdetcdetcd

Scheduler

Worker

Docker

Kubelet Proxy

©2016CouchbaseInc.

Master

Worker

Docker

21

Kubelet

API Server(pods, services, …)

Controller ManagerScheduler

etcdetcdetcd

Proxy

Worker

Docker

Kubelet Proxy

kubectl

Internet

Load Balancer

Kubernetes Cluster

ServiceDiscovery&LB:Docker

▪Docker Compose – Define and run multi-container applications – Configuration defined in one or more files

– docker-compose.yml (default)– docker-compose.override.yml (default)– Multiple files specified using -f

– Deployed as Docker Stack– Great for dev, staging, and CI

22

ServiceDiscoverywithDocker

23

docker stack deploy --compose-file=docker-compose.yml webapp

24

StackService

Task

25

docker service create --replicas 3 --name web -p 8080:8080 jboss/wildfly

LoadBalancing:Docker

Load Balancer

©2016CouchbaseInc.

ServiceDiscovery:Kubernetes

▪Service: Abstract a set of pods as a single IP and port – Simple TCP/UDP load balancing

▪Creates environment variables in other pods or DNS resolution ▪Stable endpoint for pods to reference

– Allows list of pods to change dynamically

26

©2016CouchbaseInc.

ServiceDiscoverywithKubernetes

27

©2016CouchbaseInc.

Node

Couchbase Service

CouchbaseService:Kubernetes

28

©2016CouchbaseInc.

“backend”

“frontend”

ServiceandReplicaSet:Kubernetes

29

Couchbase Service

DockerVolumes:PersistentContainers

30

Implicit Per-Container

Explicit Per-Container Per-Host Multi-Host

What? Default sandbox Explicit volume Directory on hostStorage on

distributed file systems

Location /var/lib/docker/volumes on the host

/var/lib/docker/volumes on the host

Mounted within container

Ceph, GlusterFS, NFS, …

Container crash Directory unavailable Directory unavailable Yes Yes

Host crash Directory unavailable Directory unavailable No Yes

Shared No Yes Yes (host only) Yes (cluster wide)

DockerVolumePlugin

▪“Batteries included, but replaceable” ▪Includes default driver for host-based volumes ▪Plugins enables containers to be integrated with external storage systems

– For example, Amazon EBS, Azure Storage and GCE Persistent Disks

31

DockerVolumePluginArchitecture

32

Docker Client

Docker Host

Plugin Client

PluginDaemon

StorageBackend

Storage Backend

StorageBackend

StorageBackend

local

DockerVolumePluginwithPortworx

33

Docker Client

Docker Host

Portworx Client PX-Dev Amazon

EBSportworx

©2016CouchbaseInc.

KubernetesVolume

▪Directory accessible to the containers in a pod ▪Volume outlives any containers in a pod ▪Common types

– hostPath– nfs– awsElasticBlockStore– gcePersistentDisk

34

©2016CouchbaseInc.

KubernetesPersistentVolume

35

Provision Network Storage Request Storage Use Claim

PersistentVolume PersistentVolumeClaim Claims are mounted as

volumes

1 2 3

http://blog.couchbase.com/2016/july/stateful-containers-kubernetes-amazon-ebs

©2016CouchbaseInc.

Amazon Web Services

“worker”“master”

KubernetesVolume:PersistentContainer

36

“worker”

Kubernetes Cluster

Pod

Couchbase Docker

Container

Replication Controller Publicly

accessible Service

Storage

©2016CouchbaseInc.

Amazon Web Services


KubernetesVolume:PersistentContainer

37

“worker” EBS

http://blog.couchbase.com/2016/july/stateful-containers-kubernetes-amazon-ebs

©2016CouchbaseInc.

Amazon Web Services


Kubernetes:PersistentContainerswithPortworx

38

“worker”

EBS

S3SAN

SSD

HDD

https://github.com/arun-gupta/couchbase-kubernetes/tree/master/cluster-petset-portworx

Development:Docker▪Docker Community Edition

– Docker for Mac/Windows/Linux – Monthly edge and quarterly stable releases – Native desktop or cloud provider experience

39

Development:Kubernetes

▪Single node cluster – minikube

▪Multi-node cluster – kops – kube-aws (CoreOS + AWS) – kube-up (deprecated) – Google Cloud, Azure, Tectonic, …

40

RollingUpdate:Docker

41

docker service update web --image wildfly:2 --update-parallelism 2 --update-delay 10s

2 2

22

2

2

1 1

11

1

1

©2016CouchbaseInc.

RollingUpdate:Kubernetes

42

Application Service

1 1 1

2 2 2 etcd

webapp-rc

webapp-rc-xxxxwebapp-rc

https://github.com/arun-gupta/kubernetes-java-sample/tree/master/rolling-update

Monitoring:Docker

43

▪docker container stats CLI ▪Docker Remote API ▪docker system events CLI ▪In-built Prometheus endpoint ▪cAdvisor

Monitoring:Docker

44

©2016CouchbaseInc.

Worker

45

Kubelet cAdvisor

Monitoring:Kubernetes

WorkerKubelet cAdvisor

WorkerKubelet cAdvisor

InfluxDB + Grafana Heapster

Master

References

▪Docker: docker.io ▪Kubernetes: kubernetes.io ▪Slides: https://github.com/javaee-samples/docker-java

48

http://docker.io

http://kubernetes.io

https://github.com/javaee-samples/docker-java

www.modsummit.com

www.developersummit.com

http://www.modsummit.com/

http://www.developersummit.com/

package your java application using docker and kubernetes · service discovery & lb: docker...

Documents