container networking and openstack

Container Networking and Openstack Fernando Sanchez - @fernandosanchez Fawad Khaliq - @fawadkhaliq March, 2016

Copyright © PLUMgrid, Inc. 2011-2015

Agenda

Containers, Orchestration and Networking

•  Containers: runtimes

•  Container Orchestration tools

•  Container Networking models

•  Early (host, bridge, container modes)

•  Modern (CNM, CNI)

Containers and Openstack

•  Magnum

•  Kuryr PLUMgrid's involvement in the container ecosystem

Containers, Orchestration and Networking


Containers: runtimes

•  LXC - Linux standard •  Docker - Docker

•  Rocket - CoreOS •  Drawbridge – Microsoft •  LXD - Canonical (uses LXC) •  And also an open container project: runC


Containers: Orchestration

•  Swarm (plus Docker Datacenter) - Docker

•  Kubernetes - Google

•  Mesos - Berkeley, Airbnb, Twitter, Apple…. Mesosphere

•  Fleet, Tectonic - CoreOS

•  Rancher – Rancher Labs •  others…


Early: •  Bridge mode •  Host mode •  Container mode

Evolution: •  Container Networking Model (CNM) •  Container Networking Interface (CNI)

Container Networking strategies


Bridge-mode Networking

•  A private “docker0” bridge is provisioned on the host

•  A private namespace for each container is provisioned inside that bridge

•  Containers’ ethX are mapped to private bridge interfaces

•  NAT (w/Iptables) is used to map between each private container and the host’s public interface

Container networking(I)


Bridge-mode Networking

•  Each container needs a port mapped from eth0 into their private interface

•  IPtables keeps a NAT session

in the conntrack per each container mapping public/private

•  Different ports required on public per container

•  Default mode of operation

•  Use of NAT à Performance? Scalability? Operations? Orchestration?

Container networking(I)


Host-mode Networking

•  Docker reuses the networking namespace of the main host (it’s virtually “the same interface” for the host and the container)

•  The only way to share the interface is to use different ports per container.

•  Recommended for production in Docker

•  Faster… but secure?

Container networking (II)


Container-mode Networking

•  Docker reuses the networking namespace of another container.

•  Typically, create a container with bridge mode then create a set of containers sharing the interface with the first container

•  The only way to share the interface is to use different ports per container.

•  Kubernetes used work like this, where a set of containers sharing the bridge is a POD

•  Recently moved to CNI (see later)

•  On the way out?

Container networking (III)

eth0 iptables

vethxxx

docker0

vethxxx

LO

eth0

Docker host 1.2.3.4:80

172.17.0.17:49154

LO

eth0

172.17.0.17:49155

Nginx container 1 Nginx container 2


libnetwork Docker’s interface between the docker daemon and the network

Docker

Remote drivers (Weave, PLUMgrid, Kuryr, etc.)

libnetwork

Native drivers (Null, Bridge, Overlay)


•  Network Sandbox = Isolated environment for the container’s network config…. (similar to a “network namespace”)

•  Endpoint = Interface •  Network = Group of endpoints

Container networking (IV) CNM : Container Networking Model (Docker)


docker network

create Create a network

connect Connect container to a network

disconnect Disconnect container from a network

inspect Display network information

ls List all networks

rm Remove a network

Docker network CLI


$ docker network create \

—driver overlay \

—subnet=192.168.1.0/24 \

test-net

Docker: create a network


$ docker network ls

NETWORK ID NAME DRIVER

a0ee12ceb7ed test-net overlay

Docker: list networks


$ docker network connect \

test-net test-cont

$ docker run \

—itd —name=test-cont \

—net=test-net test-image

Docker: connect a network


$ docker network disconnect \

test-net test-cont

Docker: disconnect a network


$ docker network delete test-net

Docker: delete a network


Treats container / pod of containers synonymous to Linux network namespaces

Networks described on JSON based format for network and IPAM config in /etc/cni/net.d {

"name": “test-net", "type": ”bridge", #type of network/plugin bridge,macvlan,ipvlan,

COMMERCIAL "bridge": "cni0", "isGateway": true, "ipMasq": true, "ipam": { "type": "host-local", #can be used for pluggable ipams "subnet": "10.22.0.0/16", "routes": [ { "dst": "0.0.0.0/0" } ] } }

Plugin supports two commands: add/remove container to/from network

Kubernetes has already adopted CNI and code went in for initial draft recently.

Container networking modes (IV) CNI : Container Networking Interface (CoreOS, Kubernetes)


•  Magnum: provides orchestration using Swarm, Mesos and Kubernetes

•  Kuryr: provides networking for containers using Neutron as a backend.

Containers and container networking in OpenStack


•  Provides container orchestration from an Openstack API, abstracting the underlying Container Orchestration Engines (Swarm, Kubernetes, Mesos).

•  Uses Heat, Nova, Neutron, user-data, cloud-init.

•  Uses Nova instances on Neutron networks to launch container orchestration engines (COEs)

•  Containers are launched inside Nova instances. Support for bare metal container is not there yet.

•  Use non-Neutron networking options for container right now.

Magnum: Container-as-a-service


•  Works as a proxy for libnetwork to provide networking for Docker using Neutron as a backend.

•  Plans to support both CNM and CNI. •  Kubernetes integration in discussion •  Is not integrated with Magnum yet. •  Kuryr is currently a remote driver for libnetwork.

Kuryr: Networking for containers with OpenStack


Kuryr: Networking for containers with OpenStack

PLUMgrid’s involvement in the container ecosystem


•  Remote driver for libnetwork •  Available at:

•  https://github.com/plumgrid/libnetwork-plugin •  Uses PLUMgrid’s Open Networking suite

PLUMgrid libnetwork plugin


$ docker network create



$ docker network connect


Demo of PLUMgrid libnetwork plugin and integration with Swarm


•  PLUMgrid is an active member of the Kuryr development community.

•  Working on the nested container networking design in the OpenStack community. This will bring Neutron networking to Magnum containers.

•  Design specification can be found here

•  https://review.openstack.org/#/c/269039/

Kuryr, Magnum, Neutron Integration: Nested container


VM1

c1 c2

VM2

c1 c2

router1

vm-network2 vm-network1

cnt-network2 cnt-network1

What’s the nested container problem?


•  https://docs.docker.com/engine/userguide/networking/work-with-networks/

• https://github.com/kubernetes/kubernetes/tree/master/pkg/kubelet/network/cni

• https://docs.docker.com/engine/userguide/networking/get-started-overlay/

• http://blog.kubernetes.io/2016/01/why-Kubernetes-doesnt-use-libnetwork.html

• http://mail-archives.apache.org/mod_mbox/mesos-issues/201601.mbox/%3CJIRA.12910446.1446664866000.159860.1453306119973@Atlassian.JIRA%3E

• https://github.com/kubernetes/kubernetes/search?utf8=%E2%9C%93&q=cni

References

container networking and openstack

Documents