Trellis Introduction

Saurav Das, Charles Chan & Jono Hart

with contributions from many more …

CORD Build 2017, San JoseJanuary 14, 2018

• WhatisTrellis?

• WhyTrellis?

• TrellisRoadmap

Outline

TrellisMulti-purposeleaf-spinefabricdesignedforNFV

WhiteBoxHardware

OpenSourceSoftware

SDNBased

L2bridged

L3routed

Trellis– Multi-purposeLeaf-SpineFabric

Access & Trunk VLANsIPv4 & IPv6 & MPLS SRIPv4 & IPv6 Multicast DHCP L3 relay (IPv4/v6)vRouter BGPv4/v6(ext.)

IPmulticast

ONOS Cluster

WhiteBoxSwitchAccton6712

LeafSwitch

24x40Gportsdownlinktoservers

8x40GportsuplinktodifferentspineswitchesECMPacrossalluplinkports

GEmgmt.

WhiteBoxSwitchAccton6712

SpineSwitch

32x40Gportsdownlinktoleafswitches

GEmgmt.

BRCMASIC

OF-DPA

IndigoOFAgent

OpenFlow 1.3

OCP:OpenComputeProjectONL:OpenNetworkLinuxONIE:OpenNetworkInstallEnvironmentBRCM:BroadcomMerchantSiliconASICsOF-DPA:OpenFlow Datapath Abstraction

Leaf/SpineSwitchSoftwareStack

tocontroller

OCPSoftware

(ONL,ONIE)

OCP Bare Metal Hardware

Disaggregation– Bare-metal+Open-Source

IngressPort

Table

Phy Por

t

Vlan Table

Termin-ation MACTable

Multi-cast

Routing Table

UnicastRoutingTable

MPLSTable

BridgingTable

ACLPolicyTable

L2 FloodGroup

L3 ECMPGroup

Phy Port

Phy Port

Phy Port

Phy Port

Phy Port

MPLSLabelGroup

MPLSLabelGroup

L3McastGroup

L2 InterfaceGroup

L2 InterfaceGroup

FabricASICPipeline* (BRCM’sOF-DPA)

Vlan 1 Table

MPLSL2

PortTable

* Simplified view

Allowsprogrammingofallflow-tables&port-groupsviaOpenFlow1.3

WhyOF-DPA?

OF 1.0

L2 InterfaceGroup

Phy Port

L2 InterfaceGroup

OF 1.3

AchievesDataplane Scale

6

TrellisControlPlaneRedundancy

ONOS Instance 1

ONOS Instance 2

ONOS Instance 3

ONOS Instance 4

ONOS Instance 5

M B B

M B B

M

M

B B

B BM

M B B

B BB B

M

M

BBSwitches simultaneously connect to several controller instances.only 1 controller instance is master, several other instances are backups

Mastership is decided by controllersSwitches have no say

Controller instances simultaneously connect to several switches.Any controller instance can be master or backup for any switch

Spreading mastership over controller instances contributes to scale

MB

= Master= Backup

TrellisControlPlaneRedundancy

ONOS Instance 1

ONOS Instance 2

ONOS Instance 3

ONOS Instance 4

ONOS Instance 5

M B

M B

M B

B

M

B

M BSwitches simultaneously connect to several controller instances.only 1 controller instance is master, several other instances are backups

Mastership is decided by controllersSwitches have no say

Controller instances simultaneously connect to several switches.Any controller instance can be master or backup for any switch

Spreading mastership over controller instances contributes to scale

MB

= Master= Backup= Retry

R

R

R R

M M

R

R

M

Losing controller instances redistributes switch mastership

Switches continue to retry lost connections

Management watchdog can

reboot lostcontroller instances

TrellisDataPlaneRedundancy

Spine Switch Spine Switch

ECMP group ECMP group

Leaf Switch Leaf Switch

Leaf SwitchLeaf SwitchAccess

Equipment

Datacenter Leaf-Spine Fabric Underlay

Virtual Network Overlay

Unified SDN ControlOf Underlay & Overlay

ONOSControllerCluster&

Apps

Trellis is the enabling Network Infrastructure for CORD

Trellis Provides Common control over underlay & overlay networks, including• Service Composition for Tenant Networks• Distributed Virtual Routing • Optimized Delivery of Multicast Traffic Streams

Trellis– CORDNetworkInfrastructure

R,E,M-Access

AccessLinks

ONOS

vRouterOther Apps

OverlayControl

Underlay Control

OtherApps

XOS

vSG

vSG

vSG

VNF

VNF

VNFVNF

VNF VNFVNF

VNF VNF VNFVNF

VNF

OVS

OVS

OVS

OVS

OVS

3

WhiteBox WhiteBoxWhiteBoxWhiteBox

WhiteBox WhiteBox WhiteBox WhiteBox

WhiteBox WhiteBox WhiteBoxWhiteBoxWhiteBox

WhiteBoxWhiteBoxWhiteBox

CORDArchitectureMetro

Router

• WhatisTrellis?

• WhyTrellis?

• TrellisRoadmap

Outline

• TrellisisdesignedforServiceProviders&NFV

WhyTrellis?

ToR ToR ToR

Spine Spine Spine

ToR ToR ToR ToR

SDNController(ONOSCluster)

FabricControl

vOLTControl

AAAControl

DHCPRelay

OpenFlow1.3

vRouterControl

…

Bare-metalOpen-source

Leaf-SpineFabric

ToR

NETCONF …

CORDController(XOS)

Maple QumranAX

XGS-PONWhitebox OLT(EdgeCore)

Multi-vendorONT/ONUs&RG

VOLTHA

VOLTHA

CDN

vSG

CDN

vSG

DataplaneVNFsinEdgeCompute

MetroRouters

ControlPlaneVNFsasSDNapps

ServiceCreationandOrchestration

3rdPartyServiceVNFs

Maple QumranAX

Trellis in CORD

• TrellisisdesignedforServiceProviders&NFV

• SDNallowssimpler/easier/optimizedfeatures

WhyTrellis?

vRouterasaVNF?

Dataplane

Control Plane(OSPF, BGP ..)

Management (CLI, SNMP, NetCONF)

Control Plane(OSPF, BGP ..)

Dataplane

Management (CLI, SNMP, NetCONF)

VNF = vRouter VM(vCPE, vBNG,

vPGW, vBRAS)

CPDP

vRouterVM

Underlay Network

VNF

VNF

VNF

Issues: HairpinningEmbedded control plane complicates scale-out 16

VNFM(VNF Manager)

DP

DP

DP

DP

DP

CP

Issue: Still hairpinning through a load-balancer

vRouter inSoftware?

ONOSControllerCluster

ONOSControllerCluster

VMVM

Trellis vRouter

Underlay Control

OSPFI-BGP

vRouter app

Quagga(or other)

RouterRouter

External

ONOSOverlay Control

OVS

OVS

VNF VNF

OVS

OVS

17

vRouter Operation

ONOSControllerCluster

ONOSControllerCluster

VMVM

Trellis vRouter

Underlay Control

OSPFI-BGP

vRouter

Quagga(or other)

RouterRouter

ONOSOverlay Control

OVS

OVS

VNF VNF

OVS

OVS

18

External

vRouter Operation

ONOSControllerCluster

ONOSControllerCluster

VMVM

Underlay Control

OSPFI-BGP

vRouter

Quagga(or other)

RouterRouter

ONOSOverlay Control

OVS

OVS

VNF VNF

OVS

OVS

19

External

vRouter OperationTrellis vRouter à Implemented as a big distributed router

à Presents entire n/w infrastructure as a single router to outside world

• Trellis is designed for Service Providers & NFV

• SDN allows simpler/easier/optimized features

• SDN + Programmable pipelines == New features

Why Trellis?

SDN + Programmable Pipelines

21

ONOS

configurationcontrol

control configP4Runtime gNMI

P4

OpenConfig

• Trellis is designed for Service Providers & NFV

• SDN allows simpler/easier/optimized features

• SDN + Programmable pipelines == New features

• Open-source == SP ownership & customizability

Why Trellis?

• What is Trellis?

• Why Trellis?

• Trellis Roadmap

Outline

TrellisFeaturesUsedinCORD• Bridging&VLANs(Access/Trunk/Native)• IPv4Unicast&MulticastRouting• vRouter– BGPv4&StaticRouting

TrellisFeaturesTo-Be-UsedinCORD• Dual-Homing• IPv6• DHCPL3relay• Pseudowires

TrellisRoadmapvs.Trellis-CORDRoadmap

UpcomingONOS1.12release(Dec2017)• Dual-homing(released1.11.1)fixesforknownissues• Pseudowiresupport• InitialQoSsupport• IPv6additionalfeatures

• DHCPv6relayadditionalfeatures(contributedbyNokia)• IPv6Multicast(contributedbyNokia)• IPv6RouterAdvertisementapp(contributedbyInfosys)

• SupportforNewASICs&Bare-metalswitches• BroadcomQumran(QMX)• CaviumXpliant• Quantaswitches(QCTLY8)• BarefootTofinousingP4(notin1.12release)

TrellisRoadmap

RoadmapSee here for complete roadmap

https://docs.google.com/spreadsheets/d/1SmKxsZ9iHtLHoEtvsntsJ98tLf_cYT_qXuuO7eAUtt4/edit?usp=sharing

• What is Trellis?• Multi-purpose leaf-spine fabric • Built with à bare-metal hardware + open-source software + SDN• Overlay + Underlay when used in CORD

• Why Trellis?• Designed for SP use cases & NFV• SDN allows simpler, easier & more-optimized features• SDN + P4 = new features in hardware• Open-source gives SP’s ownership & full customizabilty

• Trellis Roadmap• Different roadmaps for Trellis, and the use of Trellis in CORD

Summary

Thankyou!andmore…