simplifying openstack networks with routing on the host: gerard chami + scott laffer, cumulus...

v

Simplifying OpenStack Networks with Routing on the Host

Scott Laffer and Gerard Chami

5th of May, 2016

cumulusnetworks.com 2

• Overview

• Who are we?

• Why care about the plumbing?

• What options are there?

• Demo

Agenda

Who are we?


Scott Laffer Technical Support Engineer

[email protected] @slaffah

Gerard Chami Technical Support Engineer

[email protected] @gerardchami

Transformation: First Servers, Now Networking


First: Compute

Transformed

LOCKED

Now: Networking Transforms

OPEN

Open Networking Enables Platform Choice and Affordable Capacity

cumulusnetworks.com

Applications, OS and Hardware

Open Ecosystem Agile, open, scalable with unprecedented cost savings

4

“NetDevOps” – using existing DevOps tools for networking

Operational efficiencies, increased deployment speed

OpenStack + Cumulus – Own the Rack with Linux


.

Why OpenStack? .

Why Cumulus Linux?

Open source and associated ability to innovate

No vendor lock-in

Affordable

Commoditized hardware

Bridge the gap between your sysadmins and network engineers

Treat your switch like a server

OpEx and CapEx savings

Disaggregated HW and SW

Linux throughout your entire rack!

Why care about the plumbing?


ML2 Type Driver Choices – “The What”

Flat Type Driver All subnets assigned are placed in the same Layer-2 broadcast domain. Commonly used for defining a single provider network (single pool of external IP addresses).


VLAN Type Driver Each OpenStack subnet is assigned to a different VLAN. Discussed in detail in the Cumulus OpenStack Validated Design Guide. VxLAN Type Driver Each OpenStack subnet is assigned to a different VxLAN.

Looks similar to a typical Cumulus VxLAN design except VTEP can be in the host.

Other Linux Bridge OvS Vendor Other GRE VLAN VxLAN

Core Plugin (ML2)

Type Manager

Type Driver

Mechanism Manager

Mechanism Driver

Other Linux Bridge OvS Vendor Other GRE VLAN VxLAN

ML2 Mechanism Driver Choices – “The How”

Linux Bridge Provides Layer-2 and Layer-3 connectivity on a compute node using traditional bridging constructs.


OpenVswitch (OVS) Provides Layer-2 and Layer-3 connectivity on a compute node using networking stack that sits on top of the Linux Kernel. It does not use the Linux Kernel API.

Cumulus Linux Instantiates/Destroys VLANs on a Cumulus Switch after a tenant network is created/deleted on the OpenStack Compute Nodes

Core Plugin (ML2)

Type Manager

Type Driver

Mechanism Manager

Mechanism Driver

ML2 Framework providing a way to configure L2/L3 connectivity on any networking platform such as the linux kernel (linuxbridge) or OpenVSwitch

Design 1: ML2 + VLAN: MLAG Between Host/Leaf and Leaf/Spine

§  Overall: A well known and common design using MLAG at the spine layer, MLAG at leaf layer, but least scalable and least flexible. An “old school” but proven network design.


§  Considerations: §  VLANs statically assigned but doesn’t scale well §  STP heavy between Leaf/Spine and Leaf/Host §  MLAG difficult to manage at scale. §  Using Cumulus ML2 Mechanism driver to

dynamically add/remove VLANs doesn’t make sense. How do you add/remove VLANs from spines consistently?

§  Cumulus “Stickiness”: §  Better automation story §  Better operational story

•  Common tools for operation switch and server

§  Validated Design Guide certified

L2

ML2 Pair

Design 2: MLAG at Top-of-rack, IP Fabric Between Leaf/Spine

Overall: Uses less MLAG, more Layer-3, VxLAN, and is therefore more scalable. Caveat: Utilizes third-party SDN overlays, which could add to overall complexity.


§  Considerations: §  Scales better than L2 + MLAG §  SDN Overlays dynamically provision VxLAN on the switch

•  SDN overlay – Midokura, Nuage, PLUMgrid •  Hierarchical Port Binding with Cumulus Mechanism Driver (alpha)

§  Future “Upsides”: §  Scales better than L2 + MLAG §  SDN Overlays dynamically provision VxLAN on

the switch •  SDN overlay – Midokura, Nuage, PLUMgrid •  Hierarchical Port Binding with Cumulus Mechanism Driver (alpha)

§  Cumulus “Stickiness”: §  Simple Layer-3 config for IP fabric §  BGP/OSPF unnumbered §  HPB + Cumulus ML2 in production

ML2 Pair

L2

L3

ECMP

Design 3– Layer-3 to the Host: Single Attach

Overall: “Good Enough” for single links from hosts to switches, and recommended by Openstack.org Caveat: Not unique/novel - other networking vendors can accomplish this


§  Considerations: §  Application need to be distributed §  Not recommended for those who believe in dual

attaching host §  VTEP on the host §  VXLAN offload NICs recommended

§  Cumulus “Stickiness”: §  Eliminated STP (Spanning Tree)

L3

ECMP

L3

Design 4: Layer-3 to the Host: Multiple Attach (Quagga on the Host)

§  Overall: The best overall networking solution with OpenStack and Cumulus Networks in large configurations. 100% simple and flexible architecture with Layer-3 networking using Linux quagga package extendable to other software solutions.


§  Considerations: §  VXLAN offload network interfaces recommended §  Succeeds in docker container environments §  Supports more than two links from hosts to

switches for load balancing

§  Cumulus “Stickiness”: §  Simplified infrastructure config §  Server/switch/rack mobility §  Major reduction in IPv4 addressing §  Requires Cumulus Quagga package

ECMP

ECMP

OpenStack Network Design Decision Tree


Tenant Separation method?

VLAN Type Driver on host

VxLAN Type Driver on host

IP Fabric Between Leaf/Spine

Number of Host to Switch Links?

1 host to switch link

2 or more host to switch links

All L2/MLAG - Leaf/Spine MLAG, Host/Leaf MLAG, applies Cumulus Validated Design Guide

All L3 - Assign L3 address on host interfaces. Unnumbered for IP fabric switch interfaces.

All L3 - L3 BGP/OSPF unnumbered config all the way to the host. Install Linux Quagga package from Cumulus on each host.

“past”

“present”

“future”

Cumulus Networks

Demo Time

14

Not just a party trick…


© 2016 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.

§ Thank You!


Unleashing the Power of Open Networking

v

Backup Slides


Getting Started with the Validated Design Guide


§  Detailed 54-page HOWTO deployment guide - two spine and four leaf switches

§  Install and configure switches and compute nodes

§  Closely mimics production architecture

VMware® vSphere and Cumulus Networks® Validated Solution GuideDeploying VMware® vSphere with Cumulus® Linux® Switches

Big Data Hadoop and Cumulus Networks®

Validated Solution Guide

Deploying Apache Hadoop with Cumulus®

Linux®

Switches

0

0

00

0

00

000

0

01

1

0

1

11

1 1 111

1

111

1

Demo: Cumulus VX "Rack-on-a-Laptop" Part I (L2 + MLAG)


VirtualBox Appliance Contains:

§  Two Cumulus VX leaf nodes + Two RDO compute nodes

§  Custom tenant creation and tear-down script

§  Command line input via any local Web browser

§  Cumulus ML2 mechanism driver enabled – create 1 or 2 tenants

http://tinyurl.com/RackOnALaptop

OpenStack ControllerCompute Node (Nova)

Network Node (Neutron)Dashboatd Node (Horizon)

Compute Node

192.168.100.4/24192.168.100.3/24

192.168.100.2/24192.168.100.1/24Mgmt Bridge

Leaf 1 Leaf 2802.1q bond

Virtual ExperienceCumulus VX

Virtual ExperienceCumulus VX

swp18

swp17

swp18

swp17

swp32s0

ens0p9

swp32s0

ens0p9

host1 host2

Demo: Cumulus VX "Rack-on-a-Laptop" Part II (L3 to the Host)


VirtualBox Appliance Contains:

§  One Cumulus VX spine node + Two Cumulus VX leaf nodes + Two RDO compute nodes + One Debian external router

§  Custom tenant creation and tear-down script

§  Command line input via any local Web browser

§  Quagga packages on each compute node for Layer-3 to the host with BGP unnumbered http://tinyurl.com/RackOnALaptop-2

OpenStack Network Design: Layer 2 vs. Layer 3


VMVM

bridge - <>bridge - <>

subinterface

taptap

subinterface

802.1q trunk 802.1q trunk

802.1q bond

VMVM

bridge - <>bridge - <>

subinterface

taptap

tap tap

taptap

subinterface

vRouter

L3 Agent

DHCP AgentDHCP Agent

3 4

VXLAN –> Tunnel IP

Server1 Network Node

172.16.1.1

172.16.1.2

192.168.40.2192.168.40.3/24

VM

br-<random> br-<random> br-external

TAP

VXLAN-2061eth0

eth0 eth0 eth0

swp1 swp8

swp47

vRouter

VXLAN-2061

Mgmt Network

1

2 5

6

203.0.113.1/24

203.0.113.2/24

Layer 2 + VLAN Layer 3 + VXLAN

simplifying openstack networks with routing on the host: gerard chami + scott laffer, cumulus...

Technology