tectonic summit 2016: networking for kubernetes
TRANSCRIPT
![Page 1: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/1.jpg)
Networking for KubernetesA Tale from the Trenches
Cloud Engineering, eBaySreekanth Pothanis
![Page 2: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/2.jpg)
Networking is inherently hard!
Complexities of running on openstack
Scale
Multitenancy
Interoperability with Legacy
![Page 3: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/3.jpg)
Private Network model with Openstack SDN
Dedicated kube router provisioned in neutron
Private Networks
Subnet per node
![Page 4: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/4.jpg)
L3 Routed Model
NIPAP as IPAM
Subnet per node
Fully routable pods
![Page 5: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/5.jpg)
Network 2.0
Abstract out network boundaries from nodes to arbitrary network scopes
IP blocks are allocated to these network scopes
Scopes can represent a host or a higher level aggregation
Supports legacy and other complex network zoning
![Page 6: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/6.jpg)
Network 2.0
node
pod
pod Network Scope
IPAM
node
pod
pod
Allocation Pools
Network Scope
Allocation Pool
1 uuid1
2 uuid2
![Page 7: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/7.jpg)
IPAM controllerCluster admin creates network scopes + allocation pools
Kubernetes Nodes are associated with Scopes
IPAM Controller assigns IP based on scope of the node selected by Kube scheduler
Pods are annotated with IPs
Tessnet plugin configures the pods with annotated IP
Kube Scheduler
IPAM controller
TessNetPlugin
Pod: myPod
Host: A
Pod:myPod
Node A
notMyPod
myPod 10.10.11.4
Tessnet Pluginkubelet
Network Scope1
Allocation Pools10.10.12.0/2210.11.1.0/24
Node: A
Node: B
Node: C Network Scope2
API Server
Host: AIP: 10.10.1.4
Pod: myPod
Host: AIP: 10.10.1.4
Pod: myPod
“network_scope”: “netscope1”
![Page 8: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/8.jpg)
Networking 2.0 -- host
OVS
ARP Proxy
![Page 9: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/9.jpg)
Service to POD
Kube’s default implementation creates LBs on Nodes
Load balance on pods directly
Neutron LBaaS Pool
Neutron LBaaS VIP
POD POD POD POD
![Page 10: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/10.jpg)
eBay Ingress
Application Topology
POOL
Application VIP
VIP
GTM Load Balanced Pool
POOL
VIP
POOL
VIP
Region 1 Region 2 Region 3
Global Name (omg.g.ebay.com)
MONITOR MONITOR MONITOR
Application VIPApplication VIP
![Page 11: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/11.jpg)
Ingress controller
Ingress: myIngressStatus: VIP-1 IP GTM name
Ingress controller
API Server
LBMS
DNS
GTM
Ingress: myIngress
Ingress: myIngressStatus: VIP-1 IP
Ingress: myIngressStatus: VIP-1 IP GTM name
![Page 12: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/12.jpg)
DNS
apiVersion: v1kind: Servicemetadata: annotations: network.tess.io/kube2udns: "nginx.spothanis.svc.32.tess.io.\t3600\tIN\tA\t10.149.4.27"
Kube2DNS controller
DNS
APIServer
![Page 13: Tectonic Summit 2016: Networking for Kubernetes](https://reader033.vdocuments.site/reader033/viewer/2022042706/5871ae051a28abda6a8b60bd/html5/thumbnails/13.jpg)
Future work
Network Policy Enforcement
Globally federated Ingress -- SLB based