nexus 7000 series innovations: m3 module, dci, scale

Data Center Switching Product Management TeamDecember 2016

Cisco Nexus 7000 Series SwitchesDesigning Data Center Interconnect

2© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Confidential

Agenda

Nexus 7000 Product Update

Nexus 7000 DCI Technologies


Switching Infrastructure for Today’s Data Centers

Investment Protection

Architectural Flexibility

Operational Simplicity

Open and Programmable

Secure, Scalable, and Resilient

One Operating System Cisco NX-OS

Cisco Nexus 9000 Series






Cisco Data Center Fabric Solutions

APPLICATION CENTRIC INFRASTRUCTURE PROGRAMMABLE FABRIC

Disaggregated approach based on Open standards

Support 3rd party SDN Controller

SDN across the entire Nexus Product line

Rich inter-domain support

Turnkey integrated solution with security, centralized management, compliance and scale

Automated application centric-policy model with embedded security

Broad and deep ecosystem

CONTROLLER

OPEN APIs

VXLAN with BGP EVPN

OPEN APIs


Cisco Nexus 7700 Series Switch Family26

RU

14 R

U

9 R

U

Cisco Nexus® 770010-Slot



Environmental EFFICIENCY True front-to-back airflow

Smaller FOOTPRINTMore compact

Fabric BANDWIDTH1.32 Tbps

100G Density

40G Density

1G / 10G Density

192

384

768

96

192

384

48

96

192

12

24

48

3 R

U



FabricPath

FCoEM1(2008)

80G/slot

Cisco Nexus 7000 Series Module Evolution

M2(2012)240G/slot

F1(2010)230G/slot

F2(2011)480G/slot

F3(2013)1.2T/slot

M3(2016)1.2T/slot

OTV

MPLS

LISP

EoMPLS/VPLS

Layer 3Layer 2

SampledNetFlow

FEX

Large Tables

Large Buffers

FullNetFlow

VXLANFSAOffload

L2-L2 GW

10G FSA 256-bitMACsec

40G / 100G


FabricPath

FCoE

Cisco Nexus 7000 Series Module Evolution

M2(2012)240G/slotF3(2013)

1.2T/slot

M3(2016)1.2T/slot

OTV

MPLS

LISP

EoMPLS/VPLS

Layer 3Layer 2

SampledNetFlow

FEX

Large Tables

Large Buffers

FullNetFlow

VXLANFSAOffload

L2-L2 GW

10G FSA 256-bitMACsec

40G / 100G


DC CORE | DC INTERCONNECT

24-Port 40GE

48-Port 1/10GE

Enhanced Scale | Enhanced Security | Deployment Flexibility | Investment Protection

Cisco Nexus 7000 M3 Series Modules

12-Port 100GE

48-Port 1/10GE

24-Port 40GE


256-bit AES MACsec#

48 1/10 GE Ports (SFP+) 24 40 GE Ports (QSFP) 12 100 GE Ports (QSFP28)

On all ports/speeds

Multi-Core Fabric Services Accelerator (FSA)Enhanced Performance for BFD, Netflow, and Other Distributed Fabric Services

New Cisco M3 ASIC VXLAN, OTV, LISP*, MPLS FabricPath*, Classic L2/L3 Cisco TrustSec – SGT, SXP, SGACLs

Larger Tables 2M* FIB Entries 384K* MAC Entries 128K ACL/QOS Entries

Advanced Parser Layer 2 to Layer 2 Gateway* GTP Hashing

Deeper Buffers 31.25MB per 10GE Port 125MB per 40GE Port 350MB per 100GE Port

* Hardware Capability. Software support may be available in the future. See NX-OS Software Release Notes.

Cisco Nexus 7000 M3 Series Modules

# MKA support may be available in a later release.

Compatible with Supervisor 2/2E and Fabric 2 Modules | VDC Interoperability with F3 or M2 I/O Modules


QSFP28 Optics for 100G connectivity

QSFP Optics for 40G connectivity

Approximately 6µsec cross-fabric latency

Nexus 7700 M3 12-Port 100G I/O Module


Introducing the M3-Series into New or Existing Chassis

M3 and F3 or M2 modules interoperate at the Lowest Common Feature Set

(F3 + M3) VDC

Full Layer2 and Layer3 Interoperability

No L3 Proxy

M3

F3

(M3 only) VDC

Full Layer2 and Layer3 Interoperability

M3 – 48p 10G

M3 – 24p 40G


VDC Interface Allocation – M3-Series Modules

M3 40G6-port

port-group

VDC 1 VDC 2 VDC 3 VDC 4

M3 10G24-port

port-group

Port-group size varies depending on I/O module typeVDC Allocation on port-group boundaries – Aligns ASIC resources to VDCs

M3 100G2-port

port-group


4x10G Port Breakout Capability

Seamless 10G aggregation into dense 40G/100G portsBreakout per port not per line card | No need to reload

• Direct-attach active/passive copper breakout cables• Direct-attach active optical breakout cables• Fiber breakout cables (not included with optics transceivers)

Nexus 7000 24-Port 40GE

M3 Series I/O Module


F3 Series I/O Module


M3 Series I/O Module


F3 Series I/O Module


Nexus 7000 Series – Designed for DC & Campus Core

VPC BASED DESIGN

Classic STP Limitation 50% of all Links not utilized Complex to Harden

No STP Blocked Ports Full Links Utilization Faster Convergence

Simple to Configure Higher Fabric Bandwidth Consistent Latency

SpineScales to provide fabric bandwidth

LeafScales to provide access port density

Spine

Leaf

Horizontal Scale Out

FABRIC BASED DESIGNSTP BASED DESIGN

Workload Mobility | Application Communication | Port Density | Bandwidth


DCNM 10

DCNM 10 is a comprehensive toolbox for automated cloud-scale deployments

TOPOLOGY DISPLAYS OVERLAY, UNDERLAY and STORAGE NETWORKS – VXLAN ENABLED

OVERVIEWAUTOMATED LAN, SAN, and PROGRAMMABLE FABRIC MANAGER

POAP and AUTO-PROVISIONING FUNCTIONS FOR OVERLAY and UNDERLAY NETWORKS

MULTI-SITE, MULTI-TENANT, MULTI-FABRIC TURNKEY INFRASTRUCTURE

SUPPORTS ALL NEXUS and MDS SWITCHES

ENTERPRISE SCALE 500-1000 DEVICES

IMAGE, CONFIGURATION, and PATCH MANAGEMENT

INTEGRATES WITH VMWARE, OPENSTACK, REST


Data Center Interconnect Technologies


Nexus 7000 Series – Designed for DCI

WAN

L3 INTERCONNECT: IP VRF-LITE, MPLS, LISP

HITLESS ISSU, STATEFUL PROCESS RESTART, GRACEFUL INSERT & REMOVAL

L2 INTERCONNECT: FABRICPATH, VPC, OTV, LISP, VPLS, VXLAN

SOLUTION

BENEFITSLEVERAGE PROVEN & MATURE DCI TECHNOLOGIES AND IMPLEMENTATIONS

CONTROLLER

VXLAN with BGP EVPN

PROGRAMMABLE FABRIC

STP [OR] VPC BASED

STP, vPC, FabricPath

ACI


Challenges in Traditional Layer 2 VPNsFlooding Behavior

- Unknown Unicast for MAC propagation- Unicast Flooding reaches all sites

Pseudo-wire Maintenance

- Full mesh of Pseudo-wire is complex- Head-End replication is a common problem

Multi-Homing

- Requires additional Protocols & extends STP- Malfunctions impacts multiple sites


EoMPLS

VPLSDark Fiber

Data Center Interconnect – Traditional Layer 2 Extensions


EoMPLS

VPLSDark Fiber

Data Center Interconnect – Traditional Layer 2 Extensions

• vPC or FabricPath• Applies easily for dual site interconnection• Over dark fiber or protected D-WDM• Easy crypto using end-to-end 802.1AE

• OTV – Overlay Transport Virtualization• MAC in IP

• EoMPLS & VPLS & A-VPLS & H-VPLS• PE style• Multi-tenants• Most deployed today

Ethernet

MPLS

IP


Use Case: MACsec for Secure DCIsSingle Access dark Fiber Connectivity

Datacenter 1 Datacenter 2

Nexus 7000 Nexus 7000

Dual Access with dark Fiber ConnectivityDatacenter 1 Datacenter 2



VPC

VPC

Datacenter 1 Datacenter 2Nexus 7000 Nexus 7000


MPLS Core

Nexus 7000s as Bulk Encrypters for Self managed MPLS DCI Cores


OTV Enhancements

Loopback Address as Join Interface

50% more MAC Addresses per Site

50% more MAC Addresses across all Sites

* Hardware Capability. Software support may be available in the future. See NX-OS Software Release Notes.


Interconnecting Fabrics with Nexus 7000 Switches and F3/M3 Series Cards

FabricPath VXLAN EVPN

EthernetFabricPath to VLAN

HandoffVLAN to VXLAN

Handoff

Interconnecting fabrics using Nexus 7000 Switches and F3/M3 cards requires two VDCs


Loopback Address as OTV Join Interface - Benefits

Enables the use of multiple uplinks & ECMP in the core for better resiliency and traffic depolarization

DC EastDC West

Core


Challenges in Traditional Layer 2 VPNsSolved by OTV

Flooding Behavior

- Unknown Unicast for MAC propagation- Unicast Flooding reaches all sites

Pseudo-wire Maintenance

- Full mesh of Pseudo-wire is complex- Head-End replication is a common problem

Multi-Homing

- Requires additional Protocols & extends STP- Malfunctions impacts multiple sites

✔ ✔ ✔Control-Plane Based

Learning Dynamic Encapsulation Native AutomatedMulti-Homing


Yet Another Layer 2 Extension

Control-Plane

Multi-Homing

LoopPrevention

FaultContainment

TransportAgnostic

MulticastOptimization

Path Diversity

Multi-Site

GoodFabricPath ✖ ✔1 ✔✔ ✖ ✖ ✖ ✔ ✖VXLAN (Flood&Learn) ✖ ✔1 ✔2 ✖ ✔ ✔ ✔✔ ✖

BetterVXLAN BGP EVPN ✔ ✔1 ✔2 ✔✔ ✔✔ ✔ ✔✔ ✖

VPLS ✖ ✔1 ✔✔ ✖ ✖ ✖ ✔ ✔Best OTV ✔✔ ✔✔ ✔✔ ✔✔ ✔✔ ✔✔ ✔✔ ✔✔

1) Only with Multi-Chassis Link Aggregation (MC-LAG / VPC)

2) Limited Overlay Loop Prevention


MPLS


F3/M3 With MPLS L3 VPN• Highest density of 40G MPLS

• Available on M3 or M3 + F3 VDC

• VRF-Lite supported at FCS

L2L3 MPLS PE

MPLS P

MPLS Core

F3/M3 F3/M3


LISP*

* M3 Hardware Capability. Software support may be available in the future. See NX-OS Software Release Notes.


WAN/Campus

• Similar problem scale to DNS• Leverage demand based protocols

• A directory of hosts• Location as well as policy• Location != Routing

• Keep routing lean • Move all host state to LISP directory

• Minimize state on the routers and switches (cache on demand)

Handling host state at large scale with LISP

Branch/Closet

LISP XTR

DC 1 DC 2

LISP Host directory


WAN/Campus

• The Fabric can be based on any technology:• ACI, EVPN (PF), NSX

• LISP routers will take host routes received from the fabric and register them with the LISP directory

LISP Host Directory Services for any fabricBranch/Closet

LISP XTR

DC 1 DC 2

Local host routes

Local host routes


ACI WAN/DCI Handoff*



Nexus 7000

AUTO-PROVISION

OpFlex

TENANT SEGMENTATION

APIC

WAN/DCI OR

DC CORE

SOLUTION

SECURITY POLICY ENFORCEMENT AT ACI LEAF

PER-TENANT REACHABILITY WITH MP-BGP

GROUP POLICY AUTOMATION WITH OPFLEX

BENEFITSMULTI-DC WORKLOAD MOBILITY

LEVERAGE PROVEN/MATURE DCI TECHNOLOGIES AND IMPLEMENTATIONS

Nexus 7000 Series – ACI WAN/DCI Handoff


DCI Summary



Nexus 7000 Series – Designed for Interconnecting Fabrics

WAN

L3 INTERCONNECT: IP VRF-LITE, MPLS, LISP

HITLESS ISSU, STATEFUL PROCESS RESTART, GRACEFUL INSERT & REMOVAL

L2 INTERCONNECT: FABRICPATH, VPC, OTV, LISP, VPLS, VXLAN

SOLUTION

BENEFITSLEVERAGE PROVEN & MATURE DCI TECHNOLOGIES AND IMPLEMENTATIONS

CONTROLLER

VXLAN with BGP EVPN

PROGRAMMABLE FABRIC

STP [OR] VPC BASED

STP, vPC, FabricPath

ACI

nexus 7000 series innovations: m3 module, dci, scale

Technology