container orchestration integration: openstack kuryr
TRANSCRIPT
![Page 1: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/1.jpg)
Container Orchestration Integration:
OpenStack Kuryr and
Apache Mesos@takufukushima
![Page 2: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/2.jpg)
@takufukushima• MidoNet contributor sometimes
• OpenStack Kuryr contributor these days
![Page 3: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/3.jpg)
Agenda1. Introduction to Docker and Apache
Mesos
2. The history of Docker and Apache Mesos Networking
3. OpenStack Kuryr as the building block
4. Summary
![Page 4: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/4.jpg)
1. Introduction to Docker and Apache Mesos
![Page 5: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/5.jpg)
We need the cluster manager• We distribute workloads to containers on
hosts or VM instances
• Docker and other containers are building blocks
• We want to manage them from the bird’s-eye view
![Page 6: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/6.jpg)
Container cluster managers• Docker native
• Docker Swarm
• Apache Mesos
• Docker based
• Google Kubernetes
![Page 7: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/7.jpg)
rocks
• Blazing fast (VM? Huh?)
• Great ecosystem
• e.g., DockerHub, Meetups
• Golang dev hipstersAnd nice art works
(seriously)
![Page 8: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/8.jpg)
The dark side of
• “fundamentally flawed”
• “It’s The Future”
• “So I just need to split my simple CRUD app into 12 microservices, each with their own APIs which call each others’ APIs but handle failure resiliently, put them into Docker containers, launch a fleet of 8 machines which are Docker hosts running CoreOS, “orchestrate” them using a small Kubernetes cluster running etcd, figure out the “open questions” of networking and storage, and then I continuously deliver multiple redundant copies of each microservice to my fleet. Is that it?”
![Page 9: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/9.jpg)
rocks• The core of Mesosphere DCOS
• Originally research project of UCB RAD (AMP) lab
• Great ecosystem and use cases
• Twitter, Apple, Airbnb, eBay and so on
• Pluggable frameworks
• Apache Aurora, Chronos, Marathon
![Page 10: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/10.jpg)
architecture
Retrieved from http://radar.oreilly.com/2015/10/swarm-v-fleet-v-kubernetes-v-mesos.html
![Page 11: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/11.jpg)
and
• Mesos has few containerizers
• cgroups and namespace based containerizer
• Docker containerizer
• External containerizer
![Page 12: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/12.jpg)
2. The history of Docker and Apache Mesos Networking
![Page 13: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/13.jpg)
Docker networking• docker0 bridge
• veth pairs and netns
• --icc and --link
• --net
• bridge, container, host, none
• NAT by iptables
![Page 14: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/14.jpg)
Extended Docker networking• CoreOS flannel
• For Kubernetes
• Weave
• SocketPlane
• pipeworks
![Page 15: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/15.jpg)
![Page 16: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/16.jpg)
![Page 17: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/17.jpg)
![Page 18: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/18.jpg)
libnetwork• Networking component as a plugin
• docker network command
• Drivers separated from Docker core
• bridge
• overlay
• none
• Remote driver opened up for everyone
![Page 19: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/19.jpg)
overlay driver• SocketPlane
• Container communication over the hosts
• VXLAN
• libkv for storing the network state in the distributed datastore
• --cluster-store and --cluster-advertise
• etcd, Consul and ZooKeeper
![Page 20: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/20.jpg)
network.CreateEndpoint()
controller.NewNetwork()
InterfaceInfo
endpoint.Join() endpoint.Leave()
endpoint.Delete()
network.Delete()
sandbox.Info
e.g., netns
Network Controller
Driver
Built-in Remote
Endpoint
BridgeHostNone
OverlayCalicoKuryrWeave
Network Network
Endpoint Endpoint
IP addressesMAC addressesRoutesDNS entries
Container Container
OptionLabel
CLI
Sandbox Sandbox
Container Container
libnetwork CNM
![Page 21: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/21.jpg)
networking
• Almost the same as Docker
• especially if you’re using Docker as the containerizer
• Containers share the IP of the slaves
• NAT and netns
![Page 22: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/22.jpg)
integration point• External Containerizer Program (ECP)
• Slaves delegate the containerising to ECP
• It’s just building the Docker command
• Protobuf data is passed through stdin and stdout
• ENV vars can be used for additional data
![Page 23: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/23.jpg)
new networking
• IPAM server
• IPAM client on masters and slaves
• Network Isolator Module (NIM) on slaves
• Cleanup Module on masters
![Page 24: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/24.jpg)
new networking
Retrieved from https://github.com/apache/mesos/blob/master/docs/networking-for-mesos-managed-containers.md
![Page 25: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/25.jpg)
3. OpenStack Kuryr as a building block
![Page 26: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/26.jpg)
OpenStack? Why is it
related with Docker?
![Page 27: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/27.jpg)
OpenStack and Docker• OpenStack and Docker are exclusive for each other at
this point
• Multi tenancy
• Strict resource isolation
• OpenStack Magnum
• Docker managed by OpenStack
• Docker containers on VM instances
• OpenStack Kolla
![Page 28: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/28.jpg)
Revisiting OpenStack Neutron• Neutron is a networking component of OpenStack
• Networking resource allocation through the API
• Vendor agnostic APIs
• Many network controllers supporting these APIs
• The model of libnetwork is getting close to Neutron’s one
![Page 29: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/29.jpg)
Maximizing the developers effects: Investment for the most effective way
![Page 30: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/30.jpg)
OpenStack Kuryr• A new component in “Neutron Stadium”
• A translator between Neutron and libnetwork
• Map the API calls on the remote driver into Neutron’s API calls
![Page 31: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/31.jpg)
OpenStack Kuryr• A new component in “Neutron Stadium”
• A translator between Neutron and libnetwork
• Map the API calls on the remote driver into Neutron’s API calls
![Page 32: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/32.jpg)
OpenStack Kuryr• A new component in “Neutron Stadium”
• A translator between Neutron and libnetwork
• Map the API calls on the remote driver into Neutron’s API calls
![Page 33: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/33.jpg)
Kuryr architcture
Neutron
Daemon
Host
DistributedDatastore
DistributedDatastoreDistributed
Datastore
ContainerContainerContainer
Daemon
Host
ContainerContainerContainer
Daemon
Host
ContainerContainerContainer
Kuryr Kuryr Kuryr
Keystone
![Page 34: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/34.jpg)
![Page 35: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/35.jpg)
Kuryr as a translator
![Page 36: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/36.jpg)
Kuryr as a translator
![Page 37: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/37.jpg)
4. Summary
![Page 38: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/38.jpg)
Container networking made easy• Container networking had some issues
• The new networking models and APIs are emerging
• OpenStack Kuryr can be the common building block
![Page 39: Container Orchestration Integration: OpenStack Kuryr](https://reader031.vdocuments.site/reader031/viewer/2022021507/5876d5801a28ab1d238b5745/html5/thumbnails/39.jpg)
Kuryr as a translator
The end of slides. Any questions?