Agenda1. What is container networking? 2. Weave Net – what it is and how it works 3. Encryption & performance, Weave Net case
study 4. What is Kubernetes Network Policy & why do I
need it? 5. Kubernetes Network Policy example & demo
What is networking?
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 7 – e.g. HTTP
…
Layer 4 – e.g. TCP, UDP
What is networking?
VM10.0.0.3
VM10.0.0.4
VM10.0.0.1
VM10.0.0.2
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 7 – e.g. HTTP
…
Layer 4 – e.g. TCP, UDPL2 connectivity
Adding containers
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 7 – e.g. HTTP
…
VM10.0.0.3
VM10.0.0.4
VM10.0.0.1
VM10.0.0.2
C1
C2
C3
L2 connectivityLayer 4 – e.g. TCP, UDP
Original Docker model
VM10.0.0.1
C1
C2
docker run -p 80:8080 nginx
:80
:8080
eth0 docker0vethX
NAT
VM10.0.0.2
C3
Original Docker model
VM10.0.0.1
VM10.0.0.2
C2
docker run -p 3306:3306 mysql
:80C1:8080
eth0 docker0vethX
NATC3:3306
eth0 docker0vethX:3306
New model used by CNI (Kubernetes, etc) and
Docker Overlay (libnetwork)
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 7 – e.g. HTTP
…
Layer 4 – e.g. TCP, UDP
elevates the baseline for platforms from L2 to L3
VM
C110.0.0.1
C210.0.0.2
C310.0.0.3
New model IP addresses for “services” as well,
aggregation of pods
service VIP10.1.0.1
“api”
VM
C110.0.0.1
C210.0.0.2
C310.0.0.3
Problem
service VIP10.1.0.1
how do you provide connectivity
between hosts when each container
needs its own IP? in any environment?
Options for IP-per-pod connectivity• Underlay, or “programmable fabric”
• IaaS provides the routing • Kubernetes supports this with
GCE, AWS • “Send all traffic to 10.1.1.0/24 to this
VM” • Limit on the number of routes per host • Network typically limited to a region • Hard to do cross or hybrid-cloud
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 7 – e.g. HTTP
…
Layer 4 – e.g. TCP, UDP
Options for IP-per-pod connectivity• Overlay, like Weave Net
• Encapsulates L2 Ethernet frames on top of L4 to provide a “flat virtual L2 network”
• Expose virtual L2 network back to Linux which then runs normally (ARP, etc)
• It can run anywhere, only requirement is UDP/TCP between hosts
• Can create virtual networks which span networks, even across NAT
• Automatically allocates IP addresses for containers with IPAM
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 7 – e.g. HTTP…
Layer 4 – e.g. TCP, UDP
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 4 – e.g. TCP, UDP
Options for IP-per-pod connectivity• Overlay, like Weave Net
• Encapsulates L2 Ethernet frames on top of L4 to provide a “flat virtual L2 network”
• Expose virtual L2 network back to Linux which then runs normally (ARP, etc)
• It can run anywhere, only requirement is UDP/TCP between hosts
• Can create virtual networks which span networks, even across NAT
• Automatically allocates IP addresses for containers with IPAM
Layer 7 – e.g. HTTP…
Layer 2
Layer 3
Layer 4
Layer 2
Layer 3
Layer 4
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 4 – e.g. TCP, UDP
AWS DC
Doesn’t that sound a bit slow?• Actually, no • Graph shows
fastdp overlay within 89% of native on two AWS c3.8xlarge instances
• We’ll come on to encryption later…
Weave Net – what it is and how it works• Control plane, gossip &
CRDTs• Forms a dynamic mesh of
peers with least cost routing • Favours availability &
partition tolerance, like the Internet
• Only needs consensus initially when allocating IPAM ranges
• Data path, a few options• Fastdp, uses VXLAN to form
tunnels which are handled entirely in kernel, fast
• Sleeve, fallback user-space mode which does packet sniffing like wireshark (libpcap)
• AWS VPC mode, directly “programs the fabric”
Encryption & performance, Weave Net case study
• Prior to Weave Net 1.9, you had to choose between • fast • secure
• Now you can have both! • We added IPsec support to our fastdp data path
mode
VXLAN and IPsec• VXLAN
• Encapsulates Ethernet traffic over UDP, forming a “tunnel”
• Implemented in Weave Net using OVS, Open vSwitch kernel module
• IPsec • Secures the VXLAN UDP packets which
contain the overlay’s L2 packets • Weave Net’s control plane sets this all up for
youLayer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 7 – e.g. HTTP…
Layer 4 – e.g. TCP, UDP
Layer 2 – e.g. Ethernet
Layer 3 – e.g. IP
Layer 4 – e.g. TCP, UDP
VXLAN / OVS
IPsec / ESP
Weave Network Policy ControllerBlocking unwanted network traffic in Kubernetes
What is Kubernetes Network Policy and why do I need it?
with thanks to @bboreham for the
weave-npc slides!
Kubernetes NetworkPolicykind: NetworkPolicy metadata: name: presentation-policy spec: podSelector: tier: presentation ingress: - ports: - protocol: tcp port: 80
Presentation Tier
Middle Tier
Data Tier
:80
Kubernetes NetworkPolicykind: NetworkPolicy metadata: name: middle-tier-policy spec: podSelector: tier: middle ingress: - from: - podSelector: matchLabels: tier: presentation
Presentation Tier
Middle Tier
Data Tier
Controller
Kubernetes Master
host1weave-npc
host2weave-npc
iptables iptables
watch on policies, pods
Top-level iptables rulesFORWARD chain:-o weave -j WEAVE-NPC -o weave -j DROP
WEAVE_NPC chain:-m state --state RELATED,ESTABLISHED -j ACCEPT -m state --state NEW -j WEAVE-NPC-DEFAULT -m state --state NEW -j WEAVE-NPC-INGRESS
Per-policy iptables rulesWEAVE-NPC-DEFAULT chain: -m set --match-set weave-v/q_G.;Q?uK]BuDs2 dst -j ACCEPT -m set --match-set weave-k?Z;25^M}|1s7P3|H dst -j ACCEPT ... WEAVE-NPC-INGRESS chain: -m set --match-set weave-LuMDZrBg:KsT9Xll[ src -m set --match-set weave-hR9K[Olp~d>@1wQu/ dst -j ACCEPT -m set --match-set weave-hR9K[Olp~d>@1wQu/ src -m set --match-set weave-hR9K[Olp~d>@1wQu/ dst -j ACCEPT ...
What could possibly go wrong?Back in the FORWARD chain: -o weave -m state --state NEW -j NFLOG --nflog-group 86
We subscribe to this via ulogd so we can print:TCP connection from 10.32.0.7:56648 to 10.32.0.11:80 blocked by Weave NPC.
Also exported as a Prometheus metric
Demo in 2 parts
Part 2 - Network Policy: https://www.weave.works/guides/cloud-testdrive-part-4-secure-container-firewalls/
Other topics
• Kubernetes 101 • Continuous delivery: hooking up my CI/CD
pipeline to Kubernetes • How do I monitor this stuff?
We have talks on all these topics in the Weave user group!
Thanks! Questions?
We are hiring!DX in San Francisco
Engineers in London & SF
weave.works/weave-company/hiring
See also:weave.works/docs/net/latest/how-it-works/github.com/weaveworks/go-odptools.ietf.org/html/rfc2406
CRDTs• Merge operation ⊕ for data structure such that:
• Associative — A ⊕ (B ⊕ C) = (A ⊕ B) ⊕ C • Commutative — A ⊕ B = B ⊕ A • Idempotent — A ⊕ A = A
Examples of CRDTs• Add-only set • Latest-value-wins singleton (using timestamp) • Set with deletion: give every element an id,
maintain a set of “deleted elements” using add-only set
• Many more… http://arxiv.org/abs/1608.03960
Using Weave Mesh• General form of update: A ⊕ B ⇒ C
A = local state, B = incoming update, C = resulting state
• Golang interface: • mergeReceived = B • mergeDelta = difference from A to C • mergeComplete = C