understanding network and service virtualization
DESCRIPTION
Talk at ONUG Fall 2014TRANSCRIPT
• Status Quo
• SDN and other buzzwords explained
• Deploying Network Virtualization ‒ Show and Tell
• Deploying Service Virtualization
• Vendor solution survey and landscape
• Docker Networking ‒ Show and Tell
Agenda
2
Technology Trends
Application Rollout Today
• Poor automation for VLAN, Service contexts, and VRFs
• Poor legacy application design?
Web Tier
Application
Tier
Database Tier
Typical Data Center Design
5
Rack
Core
Aggregation
Edge
Application group A
Application group B
Problem: Network not ready for
Over 70% of today’s servers are Virtual Machines, but VMs are not treated as first class citizens by the network
‒ East-west traffic poorly managed
‒ Lack of prioritization and rate-limiting at VM level
‒ Traffic between VMs on same server often unsupervised
‒ IP/MAC overlap not allowed, and addressing limited by VLANs
6
VM VM
VM VM
VM VM
VM VM
VMs Containers
Symptoms of a broader problem with lack of proper network abstractions and policy layering
Trend #1: Network Virtualization
Dynamic, Programmable, Automated
7
Computing Infrastructure
SDN-based Virtualized Network Platform
Storage Infrastructure
Network Virtualization Requirements
8
Integration with legacy network
End-to-end visibility of VM traffic
Traffic isolation across virtual networks
• Support bare metal
servers, appliances and gateways
• VLAN, VxLAN, GRE support, allowing IP overlap across tenants
• Edge-based control of VM traffic and scalable host tracking
Troubleshooting support
Application policy Orchestrating virtual L4-L7 services
• End-to-end visibility that maps Virtual to Physical scalably
• Provisioning, and chaining of virtual services
• Application level policy across and within virtual networks
Trend #2: Service Virtualization
9
Internet Internet
NFV
Step 1. Virtualizing network functions Step 2. Chaining/Stitching them
NFV in Data Centers
1. Virtualizing the L4-L7 network service appliance (e.g., Load-balancer)
2. Chaining services to ensure that the traffic is routed through virtual appliances
3. Optimizing service delivery for applications
• Increasing number of virtual appliances
• Increasing CPU or memory of each appliance
• Placement of virtual appliances
• Offloading certain tasks to NIC or switch
10
Compute Orchestration
SDN control
Open-source?
Trend #3: New Infrastructure Tools
11
Business Potential of SDN and NFV
12
Business How?
Reduced time to revenue Speed up of service provisioning
OpEx saving Automated operations and easier management of resources
New revenue Through new business models centered around on-demand usage
Feature velocity Introduce changes quickly according to business logic needs
Improved policy compliance Ensure that cloud workload is compliant with enterprise policies (e.g., access control)
Reduced OpEx during upgrades Introduce new functions and service by replacing just software stack
Software-defined Networking (SDN):
Technology behind network virtualization
“Software-defined Network”
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware
Simple Packet Forwarding Hardware Simple Packet
Forwarding Hardware
Network Operating System
OpenFlow or
other API
North-bound
interface API
Unchanged mgmt API
Future Mode of Operation:
Lower complexity and cost,
Granular traffic management,
Dynamic and Automated
LB service
FW service
IP routing service
14
Legacy
Router
Design choice: Insertion
1. In-network: Existing/green-field network fabrics upgraded to support SDN
2. Overlay: WITHOUT changing fabric, the intelligence is added to edge-devices,
‒ as an additional appliance (e.g., bump-in-wire managed by controller)
‒ as enhanced server kernel bridge (e.g., OpenVSwitch in x86 hypervisors)
15
Control Path OpenFlow Hardware switch
Data path (Hardware)
Figure courtesy of
Martin Casada @ VMware
Design choice: Purist vs Hybrid
Hybrid approaches 1.Exclusively through
embedded control plane: e.g., Yang modeled NetConf, OpFlex
2.Embedded control plane exists, but FIB reprogrammable directly: e.g., Hybrid switches with rule overridden by OpenFlow
3.Programming both embedded control plane and FIB: e.g., Open vSwitch
Data plane
Control plane
Mgmt plane Orchestration
Purist SDN architecture,
where flow-based abstraction
programs all hardware
Extnl. Control plane
Mgmt plane Orchestration
Hybrid control plane where the
hardware contains a more open platform for adding logic
Intl. Control plane
Data plane
Centralized Control
17
OpenFlow Switch
OpenFlow Switch
OpenFlow Switch
Controller
Distributed Control Controller
Controller
Controller
Design choice: Distributed operation
OpenFlow Switch
OpenFlow Switch
OpenFlow Switch
Buzzwords: Open vSwitch, OVSDB,
OpenDayLight, OpenStack
• Embraced by industry (including OpenStack, and Intel ) as de facto server networking software
24
Open vSwitch
Physical switch
OVSDB + Optionally OpenFlow
Open vSwitch
Controller
Open vSwitch
VM VM VM VM
Open vSwitch: Most popular S/w switch
Tunnels
• Vendor-driven consortium (with Cisco, Brocade, and others) for developing open-source SDN controller platform
OpenDayLight Controller
25
Orchestration
North-bound API
Application
Controller
South-bound API
Dataplane elements
OpenStack Network Mgmt
26
Typical workflow 1. Create a network 2. Associate a subnet
with the network 3. Boot a VM and
attach it to the network
4. Delete the VM 5. Delete any ports 6. Delete the
network
Network Virtualization App
SDN Controller
pSwitch
pSwitch
vSwitch
vSwitch
OVSDB OpenFlow
Neutron API
ODL Mech driver
ML2 Plugin
OpenStack Networking in OpenDaylight
► Overlay-based OpenStack Networking supported today with L2 forwarding and flooding
VLAN, GRE, VxLAN based segmentation
NAT and Distributed L3 Virtual Routing
Distributed ARP responder
ACL/Security policies for ingress and egress
Stateless load-balancing service
<#>
Deploying Network Virtualization
Goal
Computing Infrastructure
SDN-based Virtualized Network Platform
Deployment mode #1: Underlay
VPN termination, L3 routing
VM VM VM VM VM VM IP 192.168.1.2, MAC 0x1
VM VM VM VM VM VM VM VM VM VM VM VM
VM VM VM VM VM VM
Controller
cluster
CLI, REST, GUI
IP 192.168.1.2, MAC 0x2 IP 192.168.2.2, MAC 0x1
IP 192.168.1.2, MAC 0x3
IP 192.168.1.2, MAC 0x2 IP 192.168.1.2, MAC 0x1
IP 192.168.2.1, MAC 0x2 IP 192.168.1.3, MAC 0x4
Tenant membership
decided based on
{switch-port, MAC, IP}
tuple in each flow
30
VNet identified using VLANs, VxLANs or GRE
Internet
Custom routing
by controller
• Problem: OpenFlow switches have resource limitations ‒ Weak CPU incapable of doing traffic summarization, frequent
statistics reporting, and packet marking
‒ Flow-table limitation in switches (e.g., 1500 exact match entries)
‒ Switch-controller communication limits (e.g., 200 packet_in/sec)
‒ Firmware does not always expose the full capabilities of the chipset
• Solution: ‒ Next generation of hardware customized for OpenFlow
‒ New TCAMs with larger capacity
‒ Intelligent traffic aggregation
‒ Minimal offloading to vSwitches
Performance Limitations
31
Legacy L3 routing
Legacy L2 switching
VM VM VM VM VM VM
10.1.1.0/24 10.1.2.0/24 10.2.1.0/24
10.1.1.1 10.1.1.2 10.1.2.1 10.1.2.2 10.2.1.1 10.2.1.2
VM VM VM VM VM VM VM VM VM VM VM VM
VM VM VM VM VM VM
vDP vDP vDP vDP vDP vDP
Controller
cluster
Internet Logical link
v/p-Gateway CLI, REST, GUI
Deployment mode #2: Overlay
vDP: Virtual Data Plane
VM addressing
masked from fabric
Tunnels
Tenant membership
decided by virtual
interface on the vSwitch
vDP
VxLAN Tunneling
33
• Between VxLAN Tunnel End Points (VTEP) in each host server
• UDP port numbers allows better ECMP hashing
• In absence of SDN control plane, IP multicast is used for layer-2 flooding (broadcasts, multicasts and unknown unicasts)
VTEP outer MAC header
Outer IP header
Outer UDP header
VxLAN header
Original L2 packet
VxLAN flags Reserved
24bit VN ID Reserved
Source port VxLAN port UDP Length Checksum
• Solution: ‒ Offload it to the top-of-
rack leaf switch
‒ Use hardware gateway
• Problem: ‒ Overlay mode is CPU
hungry at high line rates and has anecdotally fared poorly in real world
Performance Limitations
34
Throughput Recv side cpu
Send side cpu
Linux Bridge: 9.3 Gbps 85% 75%
OVS Bridge: 9.4 Gbps 82% 70%
OVS-STT: 9.5 Gbps 70% 70%
OVS-GRE: 2.3 Gbps 75% 97%
Source: http://networkheresy.com/2012/06/08/the-overhead-of-software-tunneling/
• Combined overlay and underlay (fabric) to achieve: ‒ end-to-end visibility
‒ complete control
‒ best mix of both worlds
• Also called P+V or Overlay-Underlay ‒ Vendors are converging towards this architecture
• The integration may need 1) link-local VLANs or 2) integration with VM manager to detect VM profile
Deployment mode #3: Hybrid
35
Deploying Network Service Virtualization
36
VM VM VM VM VM VM VM VM VM VM VM VM
vNF vNF vNF vNF vNF vNF
CLI, REST, GUI
Typical Deployment Mode is Overlay
vNF: Virtualized Network Function
Services can be single-tenanted and multi-tenanted
vNF vNF vNF vNF
Traffic to vFirewall
Traffic to
dst VM Traffic to
VIP
Network
Controller
Service
Controller
Compute
Controller
Service Type: Stateful and Stateless
38
OVS
VM 1 VM 2 VM 1
Host
OVS
VM 2
Host
Dst = VIP 1 Dst = VIP 2
Stateless service: No additional appliance needed
Stateful service: Virtual function deployed in VM or container
VM 3
Change header and Fwd to specific VM
Traffic proxied to specific VM
Stateless load-balancing and access policies can be done in this manner
Service Scaling: Scale-out and Scale-up
• Scale-out: ‒ Deploy more network function instances
‒ Scale-out of workload is also necessary
• Scale-up: ‒ Give more resources to each network function instance
‒ Offloading simple tasks to vSwitch, pSwitch or pAppliance
39
Combined Solution
OVS
VM 1 VM 2 VM 1
Host 1
OVS
VM 2
Host 2
OpenStack
Dst = VIP 1 Dst = VIP 2
Controller
Orchestration
Network Plumbing
VM 3
Service rollout and chaining
L2-L7 Service orchestration
DC Network Virtualization
Policy/ QoS
Tro
ub
le-
sho
oti
ng
UI/
An
alyt
ics
Compute
L3 Spine
VTEP Leaf
Vendor Landscape:
Taxonomy is Important
Rack
Four types of SDN solutions
1. SDN-Dataplane
‒ Traffic handling devices
Physical
Virtual
2. SDN-Control
‒ Decoupled control plane
OpenFlow++
Overlay
3. SDN-Fabric
‒ Combined data and control plane
4. SDN-Mgmt
‒ Extensible mgmt software and API
Core
Aggregation
Edge
Controller cluster
Management/
Orchestration
Virtual switches
Server manager
42
Vendor Ecosystem
Data plane
(Elements used for traffic
handling)
Controller solutions
(Decoupled control
plane)
Fabric
(Combined data
and control plane)
Management
(Extensible mgmt
software and API)
L2-L4
routing
SDN-D-PSwitch
SDN-D-VSwitch
SDN-C-OpenFlow
SDN-C-Overlay
SDN-D-Fabric SDN-N-Mgmt
43
(*Not necessarily complete)
Vendor Ecosystem
Data plane
(Elements used for traffic
handling)
Controller solutions
(Decoupled control
plane)
Fabric
(Combined data
and control plane)
Management
(Extensible mgmt
software and API)
L4-L7
services
SDN-S-Dataplane SDN-S-Control SDN-S-Fabric SDN-S-Orchestrator
44
(*Not necessarily complete)
Docker Networking
45
• Over the past few years, LXC came up as an alternative to VM for running workload on hosts
• Each container is a clone of the host OS
• Docker brought Linux containers to prominence
‒ Tracks application configuration and possibly archives to DockerHub
Linux Containers
46
Container 1
App X
Container 2
Container 3
Host OS
Guest root
App Y
Guest root
App Z
Guest root
Docker
• Excellent way to track application dependencies and configuration in a portable format.
• For instance the Dockerfile on the right can be used to spawn a container with nginx LB and accessed at a host port
$ docker build XYZ
$ docker images
$ docker run -i --name=nginx1 -d –i nginx
$ docker ps
$ docker inspect nginx1
47
# Pull base image.
FROM dockerfile/ubuntu
# Install Nginx.
RUN \
add-apt-repository -y ppa:nginx/stable && \
apt-get update && \
apt-get install -y nginx && \
rm -rf /var/lib/apt/lists/* && \
echo "\ndaemon off;" >> /etc/nginx/nginx.conf && \
chown -R www-data:www-data /var/lib/nginx
# Define mountable directories.
VOLUME ["/etc/nginx/sites-enabled", "/etc/nginx/certs", "/etc/nginx/conf.d", "/var/log/nginx"]
# Define working directory.
WORKDIR /etc/nginx
# Define default command.
CMD ["nginx"]
# Expose ports.
EXPOSE 80
EXPOSE 443
Networking Still in Early Stages
Today Docker usage is predominantly within a single laptop or host. The default network on right is allocated to the nginx container we spawned.
But, folks are exploring connecting containers across hosts.
48
"NetworkSettings": { "Bridge": "docker0", "Gateway": "172.17.42.1", "IPAddress": "172.17.0.15", "IPPrefixLen": 16, "MacAddress": "02:42:ac:11:00:0f", "PortMapping": null, "Ports": { "443/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "49157" } ], "80/tcp": [ { "HostIp": "0.0.0.0", "HostPort": "49158" } ] }
Nginx
container Container X
Host 1
Bash shell
container Container Y
docker0
Open vSwitch
Host 2
Internet
Open vSwitch
docker0
vxlan vxlan vxlan vxlan Other
cluster
hosts
Networking borrowed from VM world
Container and VM networking unified
• Edge-based overlays are even more important in container world.
• Open vSwitch already supports network namespaces
• VxLAN provides:
‒ isolation,
‒ improves L2/L3 scalability,
‒ allows overlapping MAC/IP address
Docker Engine
OVS OVS OVS
Conta
iner Conta
iner Conta
iner
Conta
iner Conta
iner Conta
iner
VM V VM
Kubernetes OpenStack
VxLAN Tunneled network
Neutron
OVS agent
Networking Redefined
Going forward, all Networking is SDN, with varying architectures and networking policy being compiled down.
All operational goodness from the computing world is brought into networking world to make it unified.
Summary
• Looking at service virtualization separately is not wise. Recommend a joint evaluation
• VM and container networking work with similar network abstractions ‒ But at different scale and velocity
• Edge-based intelligence using Open vSwitch, and VxLAN overlay is powerful.
• Open Daylight is catching up.
Thank you.
Slidehare.net/sdnhub