neutron behind the scenes
TRANSCRIPT
Outline Neutron Architecture
Neutron and Linux Networking
Neutron and OpenvSwitch
Network Packet Flows
Distributed Virtual Router
Neutron Overview OpenStack’s networking component, networking-as-a-service
Create, configure, manage software-defined networks
First introduced in Havana release, renamed from ‘Quantum’
Pluggable architecture enabling additional servicese.g. load-balancing-as-a-service (LBaaS), VPN-as-a-service (VPNaaS), DNS-as-a-service (DNSaaS)
interface with L2 & L3 hardware, SDN controllers etc.
Neutron in OpenStack Neutron uses Keystone (Identity Service) for authenticating / authorizing all API requests
Integration with Horizon enabling tenants to create networks / subnets and cloud users to connect instances to tenant / provider networks through virtual NICs
Interaction with Nova for the required ‘plugging’ of the virtual NICs to the VM instances
Neutron Terminology Provider Network
Networks defined in OpenStack that map to the existing data center’s physical networkSupport for either flat or VLAN network typesCan be shared among the tenants
Tenant NetworkNetworks defined by OpenStack tenants’ usersSupport for VLAN and GRERoute between tenant networks using the L3 agent and the Neutron routers
Image taken from https://developer.rackspace.com/blog/neutron-networking-the-building-blocks-of-an-openstack-cloud/
A Basic Setup
Image taken from http://docs.openstack.org/havana/install-guide/install/yum/content/figures/3/a/common/figures/Neutron-PhysNet-Diagram.png
• Demo Setup in VMWare Workstation• Management Network (10.0.0.0/24)
• 10.0.0.51 controller• 10.0.0.61 network• 10.0.0.71 computeA• 10.0.0.91 compute
• Data Network (10.0.1.0/24)• 10.0.1.161 network• 10.0.1.171 computeA• 10.0.1.191 compute
• External Network (192.168.254.0/24)• 192.168.254.129 controller• 192.168.254.131 computeA• 192.168.254.133 computeC
Tenant Network Features• Support for Overlapping IPs
• Multiple Tenants having the same internal subnets (e.g. 192.168.1.0/24) but still remain isolated
• Configuration• DHCP / Host routes / DNS name servers• Metadata (e.g. SSH-key injection)
• Security Groups• Ingress / Egress Rule Configuration
• Floating IPs (1:1 NAT)
• Support for IPv6
Neutron Component Communication
L2 Agent(s) L3 / DHCP / Other Agents
Message Queue
neutron-server Database
Neutron Components neutron-server
Implements the REST API (v2) callsInterfaces with other OpenStack services via RPC callsDatabase persistenceTranslates between API and plugin calls
The plugin implements the core networking (network ports, networks and subnets, IP addressing etc.)Either implements directly or proxies to dedicated 3rd party hardware/software solution
ML2 PluginA modular framework replacing previous monolithic plugins (openvswitch, linuxbridge)
Network Type DriversFlat, GRE , VLAN, VXLAN TypeDrivers
Network Mechanism DriversOpenvSwitch , Linuxbridge, Cisco Nexus etc.
Allows multiple mechanisms to be used simultaneously
Open vSwitch An open source, software-based virtual switch
Supports distribution across multiple physical servers
Supports many features e.g. VLAN,GRE, VXLAN, NetFlow, sFlow, LACP
The OpenvSwitch plugin is loaded by the Neutron serviceReceives API calls and stores the corresponding network data in a database backend
The OpenvSwitch agent that runs on each compute nodeReads the database and implements the network model / configures the network flows on the local vSwitch instance.
GRE (Generic Routing Encapsulation) Used to create virtual point-to-point links over IP (L2-in-L3)
The GRE Header
It has some benefits over VLANs such as being a routable protocol and not requiring e.g. 802.1Q on physical switches / physical network interfaces
Linux Network Namespaces & Neutron The ability to create logical and separate network stacks assigned to group of processes
Each stack has its own routing table, firewall rules, network interfaces
L3 agent creates a network namespace per virtual router and dnsmasq process
Create and attach veth (virtual Ethernet interfaces) to a network namespace
Connect the network namespaces to physical network interfaces via a bridge (e.g. linuxbridge or Openvswitch bridge)
A Network Packet Flow through Neutron
Image taken from http://pinrojas.com/2014/07/29/theres-real-magic-behind-openstack-neutron/
Packet exits the VM’s ethX interface through a tap virtual Ethernet interface that connects to a Linux Bridge where iptables rules are applied.
ipset optimization
Connect the Linux Bridge with the OpenvSwitch br-int using a veth pair (qvb & qvo) – VLAN tagging at br-int
Pass traffic to br-tun through patch interfaces
Reach network node through the GRE tunnel
Pass traffic to br-int (network node) through patch interfaces
Deliver to virtual router and then exit though br-ex
Neutron @ Juno Distributed Virtual Router – DVR
“router_distributed” configuration under neutron.conf
agent_mode= dvr_snat for network node
agent_mode = dvr for compute node(s)
Need for l2population mechanism
Compute node(s) connects to provider network
SNAT still centralized for VMs without floating IPs
Neutron @ Juno North-South (floating IP)
DNAT per compute node when floatingIP is in useDefault route via FIP-NSPoint-to-point link between the router and the FIP namespacesProxy-ARP for floating IPs on br-ex
East-West traffic (tenant subnet-to-subnet) DVR router with the same IP and MAC address at each compute host Local DVR router maintains ARP cache of all the subnet hosts Avoid communication through the central L3 agent or ARP broadcasts
References http://docs.openstack.org/juno/install-guide/install/apt/content/section_neutron-networking.html
http://pinrojas.com/2014/07/29/theres-real-magic-behind-openstack-neutron/
http://www.slideshare.net/carlbaldwin/dvr-slides
http://superuser.openstack.org/articles/bridges-and-tunnels-a-drive-through-openstack-networking
http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html
https://www.openstack.org/summit/openstack-summit-hong-kong-2013/session-videos/presentation/openstack-neutron-modular-layer-2-plugin-deep-dive
https://openstack.redhat.com/Networking_in_too_much_detail
http://www.slideshare.net/carlbaldwin/dvr-slides
http://kimizhang.wordpress.com/2014/11/25/building-redundant-and-distributed-l3-network-in-juno
Thank You!
Appendix – Useful Commandsps aux | grep ovs - Show running OpenvSwitch processes
virsh list -- Show running KVM instances
virsh dumpxml instance-XXXX | less -- Show KVM instance configuration
brctl show -- Show linux bridges and their connected interfaces
ipset list -- View the ipset defined
ethtool -S qvbXXXX - Print the interface index of the veth's peer
ip link list | grep ^XX - Print the veth's peer interface
ovs-vsctl list-br - Lists existing OVS bridges
ovs-vsctl list-ports br-int - Lists ports per OVS bridge
Appendix – Useful Commandsovs-vsctl show - Brief overview of the database contents
ovs-ofctl show br-tun -- Switch information such as flow tables and ports
ovs-ofctl dump-flows br-tun -- Flows
ip netns ls - List network namespaces
ip netns exec $ROUTER1 ip addr - Network interfaces for a specific network namespaceip netns exec $ROUTER1 ip route - Routing table for a specific network namespace
ip netns exec $ROUTER1 iptables -t nat - IP tables NAT rules for a specific network namespace