kubernetes: where we are, where we’re going and why

KubernetesWhere we are, where we’re going and why

Brendan BurnsSenior Staff Software Engineer

Where are we?

Where are you?

Where are we going?

Where are we going?

We’re making reliable, scalable, agile distributed systems a CS101 exercise

Scaling Kubernetes to 1M QPS

Building a demo to 1M QPS

nginx-service10.0.0.3

Loadbot Replication Controller

Building a demo to 1M QPS

nginx-service10.0.0.3

Kubernetes API Server

Data Aggregator

pods whereapp=loadboat

data-service10.0.0.4

Demo GUI

The road ahead

1

2

3

Kubernetes 1.1

Looking ahead to Kubernetes 1.2

Flags in the distance

Kubernetes 1.1

Autoscaling BatchJobs

HTTP Load Balancing

Resource Overcommit

IP Tables Kube Proxy

Newkubectl tools

1M QPS, 1000+ nodes* and much more!

Kubernetes 1.1

Autoscaling 1M QPS1k+ Nodes

HTTP Load Balancing

Resource Overcommit


Newkubectl tools

And much more!

Releasing today!

Rolling out to Google Container Engine this week[new clusters]

Rolling out to existing Container Engine clusters in ~2 weeks[send us a note if you want it earlier]

Ingress for HTTP Load Balancing [Beta]

Service-foo: 24.1.2.3 Service-bar: 24.4.5.6

Ingress for HTTP Load Balancing

Service-foo: 10.0.0.1 Service-bar 10.0.0.2

api.company.com24.7.8.9

http://api.company.com/foo http://api.company.com/bar

Ingress for HTTP Load Balancing

Service-foo: 10.0.0.1 Service-bar 10.0.0.2


http://api.company.com/foo http://api.company.com/bar

Ingress API

Ingress API

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: testspec: rules: - host: k8s.io http: paths: - path: /foo backend: serviceName: fooSvc servicePort: 80 - path: /bar backend: serviceName: barSvc servicePort: 80

fooSvc barSvc

http://k8s.io/foo http://k8s.io/bar

Ingress API

apiVersion: extensions/v1beta1kind: Ingressmetadata: name: testspec: rules: - host: k8s.io http: paths: - backend: serviceName: k8sSvc servicePort: 80 - host: j7a.io http: paths: - backend: serviceName: j7aSvc servicePort: 80

k8sSvc j7aSvc

http://k8s.io/* http://j7a.io/*

Implementing Ingress

Ingress Object Ingress Controller● GCE● HAProxy*● ...

Horizontal Pod Autoscaling [Beta]

Service-foo: 10.0.0.1


http://api.company.com/foo





https://www.flickr.com/photos/davedehetre/4440211085


apiVersion: extensions/v1beta1

kind: HorizontalPodAutoscaler

metadata:

name: php-apache

spec:

scaleRef:

kind: ReplicationController

name: php-apache

namespace: default

minReplicas: 1

maxReplicas: 10

cpuUtilization:

targetPercentage: 50

https://www.flickr.com/photos/davedehetre/4440211085

But why is it called “Horizontal”?

Jobs [Beta]




Jobs



http://api.company.com/foo Start Finish

Jobs

Start Finish

apiVersion: extensions/v1beta1kind: Jobmetadata: name: ffmpegspec: selector: matchLabels: app: ffmpeg template: metadata: labels: app: ffmpeg spec: containers: - name: ffmpeg image: ffmpeg restartPolicy: OnFailure

Jobs

Start Finish

apiVersion: extensions/v1beta1kind: Jobmetadata: name: ffmpegspec: selector: matchLabels: app: ffmpeg # run 5 times before done completions: 5

...

Jobs

Start Finish

apiVersion: extensions/v1beta1kind: Jobmetadata: name: ffmpegspec: selector: matchLabels: app: ffmpeg # run 5 times before done completions: 5 parallelism: 2...

Jobs are a foundation

https://www.flickr.com/photos/dgoomany/4976873174 https://www.flickr.com/photos/kubina/278696130

Shard numbers, Scheduled Jobs, Workflow and more...

New APIs: HTTP Load Balancing with Ingress

fooSvc barSvc

http://k8s.io/foo http://k8s.io/bar

New APIs: Horizontal Pod AutoScaling

New APIs: Jobs

Start Finish

But that’s not all...

Autoscaling BatchJobs

HTTP Load Balancing

Resource Overcommit


Newkubectl tools

1M QPS, 1000+ nodes* and much more!

Memory Overcommit

Guaranteed

Burstable

Best Effort

Memory Overcommit

Resource Class Request Limit

Best Guaranteed 10G 10G

Better Burstable 5G 10G

Good Best Effort - 10G

IPTables Proxy

kubectl improvements

kubectl run -i --tty shell --image=busybox -- sh

kubectl attach -i --tty my-pod

kubectl edit pods my-pod

kubectl apply ...

Rolling update improvements

The road ahead

1

2

3

An overview of Kubernetes 1.1



Why are we all here?

It’s not to run N nodes or M containersFrankly, it’s not even to run containersIt’s to build applicationsIt’s to operate applicationsIt’s to update applications

And to do it all easily.

Envisioning distributed systems as applications

type LeaderFn func(isLeader bool, leaderName string)

type LeaderElector interface {InstallLeaderFunc(leaderFn LeaderFn)Run()

}


func myLeaderFn(leader bool, leaderName string) {if leader {

fmt.Println("I'm the leader")} else {

fmt.Printf("%s is the leader\n", leaderName)}

}


package main

func leaderFn(leader bool, leaderName string) { … }

func main() {elector := impl.NewLeaderElector()elector.InstallLeaderFunc(leaderFn)elector.Run()

}


myLeaderElector

./myLeaderElector --replicas=3 --daemon


myLeaderElector

./myLeaderElector --replicas=3 --daemon

myLeaderElector

etcd

myLeaderElector

etcd

myLeaderElector

etcd


myLeaderElector

./myLeaderElector --stop

myLeaderElector

etcd

myLeaderElector

etcd

myLeaderElector

etcd


myLeaderElector

./myLeaderElector --stop

The road ahead

1

2

3

An overview of Kubernetes 1.1



Looking forward to Kubernetes 1.2 : Pre-built applications

Google Deployment Manager is Open Sourced!https://github.com/kubernetes/deployment-manager

Deis announces Helmhttps://github.com/deis/helm

And more to come...

backend10.0.0.3

frontend24.1.2.3

https://github.com/kubernetes/deployment-manager

https://github.com/kubernetes/deployment-manager

https://github.com/deis/helm

https://github.com/deis/helm

Cluster-1

Looking forward to Kubernetes 1.2 : Cross cluster management

New Open Source UXhttps://github.com/kubernetes/dashboard

Cross cluster service import/export

TBD...

Cluster-2

Ubernetes

Looking forward to Kubernetes 1.2 : Simplified Config

!! generator.kubernetes.io/java/v1

name: my-java-app

java: 7

jar: some/path/to/my.jar

replicas: 2

resources:

cpu: 1.0

memory: 10G

Thank you!

kubernetes: where we are, where we’re going and why

Technology