Inside the Architecture of Neutron

Mark McClainmmcclain (at) yahoo-inc.com

Why Create Neutron?

• Rich Topologies

• Technology Agnostic

• Extensible

• Advance Services Support

• Load Balancing, VPN, Firewall

The Basics

What does the user see?

Compute API

Network API

Storage APIGUI, CLI, API Libs

KVM

ML2 Plugin

Ceph

Abstractions

Net110.0.0.0/24

Nova

Neutron

L2 virtual network

virtual port

virtual server

virtual interface (VIF)

virtual subnet

VM110.0.0.2

VM210.0.0.2

Architecture

Design Goals

• Unified API

• Small Core

• Pluggable Open Architecture

• Extensible

OpenStack The Operator View

Basic Deployment

neutron-server

L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent

L3 AgentL3 Agent

L3 AgentL3 Agent

Database

L3 Agent

DHCP Agent

L2 Agent

Message Queue

Adv Services

neutron-server

REST API SERVICE RPC SERVICE

PLUGIN

REST API SERVICE

• REST API

• HTTP(S) Python WSGI Application

• Customary TCP port is 9696

• Exposes logical resources

• networks, subnets, ports, etc

• Request/Response Serialization

neutron-server

REST API SERVICE RPC SERVICE

PLUGIN

RPC SERVICE

• RPC Service

• AMQP via Oslo messaging modules

• Enables bidirectional agent communication

• Optional

neutron-server

REST API SERVICE RPC SERVICE

PLUGIN

• PLUGIN

• Written in Python

• Only one active

• Must implement V2 API calls

• Optional database access

• Optional extension support

The Plugin

ML2Plugin

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

The Plugin

NeutronPluginBaseV2

NeutronDbPluginV2

ML2Plugin

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

Plugin Extensions

• Add logical resources to the REST API

• Discovered by server at startup

• REST: /v2.0/extensions

• Common Extensions

• Binding, DHCP, L3, Provider, Quota, Security Group

• Other Extensions

• Allowed Addresses, Extra Routes, Metering

Monolithic Plugin

• Full implementation of core resources

• Two types:

• Proxy

• Direct control PLUGIN

ML2: Modular Layer 2 Plugin

• Full V2 Plugin Implementation

• Delegates calls to proper L2 drivers

• Two kinds of drivers

• Type Driver

• Mechanism Driver Mech Mgr

PLUGIN

Type Mgr

L2 Agent

L2 Agent

• Runs on hypervisor

• Communicates with server via RPC

• Watch and notify when devices added/removed

• Wires new devices

• Proper network segment

• Security Group Rules

Dive Into the OVS Agent

• OVS

• What does it actually do?

• How do we get isolation?

• VLAN, Overlays: GRE, VXLAN

• Processing loop

Linux Network Namespace

• Isolated copy of network stack

• private loopback

• scope limited to namespace

• can reuse addresses

• Explicit configuration needed to connect

• Processes can spawn within namespace

lo

eth1

eth0

lo

eth1

eth0

lo

eth1

eth0

Host A B

br-int

L3 Agents

Network Node

L3 Agent

• Run on Network Node

• Uses Namespaces

• Metadata Agent (if enabled)

Network Node

Core

Hypervisor Hypervisor Hypervisor

L3 Agent How it’s implemented

• Manages Collection of Network Namespaces

• Isolated IP Stacks

• Forwarding Enabled

• net.ipv4.ip_forward=1

• Static Routing

• Metadata Proxy

lo

eth1

eth0

lo

qg-2

qr-1

lo

qg-b

qr-e

Host A B

br-ex

Configuration Agents

Configuration Agents: DHCP

• RPC based notifications

• dnsmasq

• Isolation Support via Network Namespaces

• Multiple copies for HA

Configuration Agents: Metadata Proxy

• Proxies Metadata requests to Nova

• Routed Networks

• process embedded in router

• Non-routed Networks

• static route redirects traffic running in DHCP namespace

Configuration Agents: Metadata Proxy

curl http://168.254.169.254/openstack/latest/meta_data.json

Tenant

VM

Nova Metadata Service

Meta NS Proxy

Metadata Agent

Unix Domain Socket

Management Network

X-Router-Id: 2bc7c882-d612-438c-a334-0047f2b5c2d7 X-Forwarded-For: 10.0.0.1 X-Instance-ID: aaaaaaaa-aaaa-aaaa-aaaaaaaaaaaa

Booting a VM• nova boot

Booting a VM• nova boot

• create port

• notify DHCP of new port

Booting a VM• nova boot

• create port

• notify DHCP of new port

• create device

• new in Icehouse wait

Booting a VM• nova boot

• create port

• notify DHCP of new port

• libvirt create device

• new in Icehouse wait

• wire port

Booting a VM• nova boot

• create port

• notify DHCP of new port

• libvirt create device

• new in Icehouse wait

• wire port

• boot

Load Balancer as a Service

• Service Plugin

• Driver based

• Agent w/Driver

• Agent communicates over RPC

• Open Source requires namespaces

• Others interact with other systems

LB Agent

HAProxy

VPN as a Service

• Service Plugin

• Driver based

• Agent w/Driver

• Communicates over RPC

• Openswan

L3 Agent

Router

Metadata Proxy

VPN Driver

Firewall as a Service

• Edgewall

• Service Plugin

• Driver based

• Agent w/Driver

• Communicates over RPC

• Experimental

L3 Agent

Router

Metadata Proxy

Firewall Driver

Differences

• Different Design Decisions

• Sync with backend system

• L2 Agent Optional

• Not all implement same extensions

Summary

Open vSwitch / Linux Bridge

Ryu OpenFlow Controller

• Unified API

• Small Core

• Pluggable Open Architecture

• Multiple Vendor Support

• Extensible

More Information

• Cloud Administrator Guide

• http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html

• Network v2.0 API

• http://developer.openstack.org/api-ref-networking-v2.html

Questions?