Download - Cisco Nexus Back 2 Basic
B k 2B iDataCenter TechnicalOverviewSeriesBack 2 BasicsData Center Technical Overview Series
Nexus 7000October 14Nexus 7000
IMPORTANT: Audio is being broadcast directly to your computer speakers, so make sure they are functional. No need to dial in separately.
Back 2 BasicsToday’sPresenterToday s Presenter
CurrentCisco Data Center Consulting Systems E i i li i i N 7000Engineer specializing in Nexus 7000 partner enablement. Located in Rosemont, Chicago
PastPastCisco Security Systems Engineer, Sr. Network Engineer at IPG (Inter Public Group), Network Engineer at 3com/USRobotics and Motorola
Raj Chacko
Engineer at 3com/USRobotics and Motorola
Raj Chacko
Cisco Nexus 7000 Series Switch & NX-OS Roadmap
Cisco Nexus 7000Back to Basics
Raj Chacko CCIE R&S SecurityRaj Chacko, CCIE R&S, Security
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 32
The Evolving Data Center and New ChallengesThe Evolving Data Center and New Challenges
ImpactEmerging Challenges Impact
Higher I/O requirementsNetwork/ Storage
Emerging Challenges
Sophisticated Greater east-west
bandwidth Rapid provisioning/
SophisticatedVirtualization
10G ready wiring Server/cabling density
Physical InfrastructureApplication Complexity
WAN optimization
Application PerformanceCloud Computing and XaaS
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 4
WAN optimization Application Acceleration
Transforming the DC with New Technologies
Consolidation
A t ti
Utility Market
Virtualization
Automation
MultiMulti--SP CloudSP Cloud
Unified Computing SP CloudUnified Computing SP Cloud
Private CloudPrivate Cloud
Data Center NetworkingData Center Networking
Unified Fabric ArchitectureUnified Fabric Architecture
2008 HA with ISSUVPC VDC
Today Unified Fabric Fabric Extender OTVFabricPath
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 5
FabricPath
2011+Cloud-centric Networking
ServicesLISP
The Cisco Nexus Switching Family
Complete switching portfolio
Consistent operating system across all platforms Consistent operating system across all platforms
Infrastructure scalability, transport flexibility and operational manageability
Nexus 7010 Nexus 7018Nexus 1000V Virtual Switch
1K1KCisco Nexus 1000V
2008
x86
Nexus 2000 Fabric Extender
Nexus 5000Nexus 4000
NX-OS Operating System
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 6
Data Center Network Manager
Introducing the Cisco Nexus 7000
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
Nexus 7000 PlatformIndustry’s First Data Center Class Platform
Nexus 7000 and NX-OS 10 & 18 Sl t i• 10 & 18 Slot versions
• 15+ Terabit System• Unified Fabric Ready• Modern Modular OS• Modern, Modular OS• Device Virtualization • Cisco TrustSec • Continuous Operations p
Nexus 7010 8 I/O Slots + 2 Supervisor Slots
Nexus 7018 16 I/O Slots + 2 Supervisor Slots8 I/O Slots + 2 Supervisor Slots
Front to Back Airflow256 10GbE (4:1) / 64 Ports line rate384 10/100/1000 Ports
16 I/O Slots + 2 Supervisor SlotsSide to Side Airflow512 10GbE (4:1) / 128 Ports line rate768 10/100/1000 Ports
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 8
Cisco NX-OS Multi-protocol Operating SystemData Center Network Manager (DCNM)
384 10/100/1000 Ports 768 10/100/1000 Ports
Nexus 7010 Chassis System statusLEDs
ID LEDs on all FRUs
Front-to-back airflow
LEDs
Integrated cablemanagement Air exhaust
Optional locking front
d
managementwith cover
System fan traysdoors System fan trays
Fabric fan trays
21RU
Locking ejector levers Two chassis
Supervisor slots (5-6)
Crossbar fabric modules
e e s Two chassis per 7’ rack
Payload slots(1-4, 7-10)
modules
Power supplies
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 9Front Rear
Air intake with optional filter Common equipment
removes from rearN7K-C7010 9
Nexus 7018 ChassisSystem status
Systemfan trays
LEDs
Integrated cablemanagement Optional front
door
Side-to-side airflow
Supervisor slots (9-10)
Crossbar fabric25RUslots (9-10) fabric
modules
Common equipment Payload slotsremoves from rear(1-8, 11-18)
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 10Front Rear
Power supply air intake
Power supplies
N7K-C7018 10
Supervisor EngineSupervisor Engine
Performs control plane and management functions
D l 1 66GH I l X i h 4GB DRAM Dual-core 1.66GHz Intel Xeon processor with 4GB DRAM
2MB NVRAM, 2GB internal bootdisk, compact flash slots
Out-of-band 10/100/1000 management interface Out-of-band 10/100/1000 management interface
Always-on Connectivity Management Processor (CMP) for lights-out management
Console and auxiliary serial ports
USB ports for file transfer N7K-SUP1
ID LED AUX Port USB Ports CMP Ethernet
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
Console Port
AUX PortManagement
Ethernet
USB Ports CMP Ethernet
Reset ButtonStatusLEDs
Compact FlashSlots
Management InterfacesManagement Interfaces
Management Ethernet 10/100/1000 interface used exclusively for
system management Belongs to dedicated “management” VRF
Prevents data plane traffic from entering/exiting fromPrevents data plane traffic from entering/exiting from mgmt0 interface
Cannot move mgmt0 interface to another VRFCannot assign other system ports to management VRF
Connectivity Management Processor(CMP) Ethernet Connects to standalone, always-on
i i imicroprocessor on supervisor engineRuns lightweight software with network stackCompletely independent of NX-OS on main CPU
Provides ‘lights out’ remote management and
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
Provides lights out remote management and disaster recovery via 10/100/1000 interface
Removes need for terminal servers
Nexus 7000 Line Module Portfolio(1 Gig and under)( g )M1 Series 1GbE
$15K $27K $27KTBD
N7K-M148GT-11
48 x 10/100/1000 48 x 1GigE 48 x 10/100/1000
N7K-M148GS-11 N7K-M148GS-11LN7K-M148GT-11L
48 x 1GigE
(Shipping) (Shipping) (Target Cairo 2HCY10) (FCS’d May 25th)
48 x 10/100/1000
46 Gbps Fabric
Copper
48 x 1GigE
46 Gbps Fabric
SFP
48 x 10/100/1000
46 Gbps Fabric
Copper
48 x 1GigE
46 Gbps Fabric
SFPCopper SFP Copper SFP
“XL” Capable
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 13
L2 / L3
60 Mpps Forwarding Capacity
Nexus 7000 Line Module Portfolio – 10 GigM1 Series 10 Gigabit EthernetM1 Series 10 Gigabit Ethernet
$70K$44K$70K
N7K-M132XP-12
32 x 10GigE 32 x 10GigE
N7K-M132XP-12L N7K-M108X2-12L
8 x 10GigE(Shipping) (Cairo Target Oct 2010) (FCS’d May 25)
4:1 Oversubscribed
SFP+
4:1 Oversubscribed
SFP+
1:1 Line rate
X2
60 Mpps
80 Gbps Fabric
60 Mpps
80 Gbps Fabric
120 Mpps
80 Gbps Fabric
“XL” CapableFEX Support
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 14L2 / L3
F1 Series: High-Performance Layer 2 10GbE ModulesHigh Performance Layer 2 10GbE Modules
32-port 10G SFP+ F1 moduleOct 2010
SFP+ and RJ 45 10G DCB I/O Hi h f SFP+ and RJ-45 10G DCB I/O modules
1G/10G dual-speed switch-on-chip design
High performance230 Gbps fabric connectivity
320 Gbps local switching
480 Mpps forwarding per module
Layer 2 with L3/L4 servicesL3 Routing provided by M1 Modules
Multi-protocol – Classic Ethernet,
480 Mpps forwarding per module
7.68 Billion pps per 7018
SKU N7K-F132XP-15=vPC, L2MP, DCB, FCoE
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 15
Integrated Forwarding EngineIntegrated Forwarding Engine
Advanced hardware forwarding engineengine
Up to 60Mpps IPv4 unicast, 30Mpps IPv6 unicast throughput
M1 Series Forwarding Engine M1 Series Forwarding EngineEqual to Cat 6K EARL 8
Integrated on every I/O module(NOT a FRU)(NOT a FRU)
Non-XL XLFIB TCAM 128K Up to 1MIPv4 Routes 128K Up to 1M
Table sizes
IPv6 Routes 64K Up to 500KClassification TCAM (ACL and QoS) 64K 128KNetFlow TCAM (Ingress and Egress) 512K 512KMAC table 128K 128K
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
MAC table 128K 128KBridge Domains (VDC + VLAN) 16K 16K
Crossbar Switch Fabric Module Each fabric module provides 46Gbps per I/O module slot
Up to 230Gbps per slot with 5 fabric modules
Initially shipping I/O modules do not leverage full fabric bandwidth
Maximum 80G per slot with 10G moduleFuture modules leverage additional available fabric bandwidth
Access to fabric controlled using QoS aware central Access to fabric controlled using QoS-aware central arbitration with VOQ
N7K-C7010-FAB-1
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
N7K-C7018-FAB-1
Current I/O Module CapacityFabric Modules
1CrossbarFabricASICs
1Gbps I/O modules Requires 1 fabric for full
b d idth
46Gbps/slot
2CrossbarFabricASICs
Requires 2 fabrics for N+1 redundancy
bandwidth
46Gbps/slot
3CrossbarFabricASIC
46Gbps92Gbps138Gbps184Gbps230Gbpsper slot bandwidth 46Gbps/slot
ASICs
4CrossbarF b i
4th and 5th fabric modules provide additional redundancy for current M1 cards, and full bandwidth for F1
per slot bandwidth
46Gb / l t FabricASICs
5Crossbar
and full bandwidth for F1
10Gbps I/O modules Requires 2 fabrics for full
46Gbps/slot
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 18
CrossbarFabricASICs
Requires 3 fabrics for N+1 redundancy
Requires 2 fabrics for full bandwidth 46Gbps/slot
18
Access to Fabric BandwidthAccess to Fabric Bandwidth
Access to fabric controlled using central arbitration
Arbiter ASIC on supervisor engine provides fabric arbitration
Bandwidth capacity on egress modules represented by Virtual Output Queues (VOQ ) t i t f b i(VOQs) at ingress to fabric
I/O modules interface with arbiter to gain access to VOQs
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
NX OS: Purpose Built for the Data CenterNX-OS: Purpose Built for the Data Center
IOS
Catalyst
NX-OS
SAN OS
Nexus
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 20
SAN-OS Release 4.1
MDS
NX-OS Modular ArchitectureNX OS Modular Architecture
Storage ProtocolsLayer 3 Protocols Layer 2 ProtocolsM
anag
er OSPF
BGP
EIGRP
GLBP
HSRP
IGMP
PIM SNMP
VLAN
PVLAN
UDLD
CDP
802.1XSTP
LACP CTS
VSANs
FCIP
Zoning
FSPF
IVR …
System Infrastructure
HA
M
… … Future
Kernel (Linux)
Based on MDS-9000 Series SAN-OS Every process runs in protected memory for fault containment Automatic stateful process restart
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
Automatic stateful process restart Modular code only runs in DRAM when invoked
Cisco Confidential
Data Center Class AvailabilityIn Service Software Upgrade (ISSU)
Upgrade and reboot
In Service Software Upgrade (ISSU)
Initiate stateful failoverUpgrade and reboot
Upgrade and reboot I/O CPUIn Service Software UpgradesIn Service Software UpgradesIn Service Software UpgradesIn Service Software Upgrades
Release 4.1
Release4.2
PF P M c.
Active
PF P M etc.
StandbyReleas
e 4.1Release
4.2
Minimize Planned Downtime Upgrades are possible Upgrades are possible
between minor and majorbetween minor and major e 4.1
Kernel
OSP BG
P
PIM etc
HA Manager
N7K Data PlaneKernel
HA Manager
OSP
BG
PPI
M ebetween minor and major between minor and major software releasessoftware releases
Critical components for LAN Critical components for LAN + SAN vision+ SAN vision
N7K Data Plane
R lRelease
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 22
I/O Module Images
Release 4.1
Release4.2
Data Center Class AvailabilityStateful Process RestartStateful Process Restart
P
Restart process!Avoid Network Re-convergence
Stateful Process RestartStateful Process RestartStateful Process RestartStateful Process Restart
BG
P
OSP
F
PIM
TCP/
UD
P
IPv6
STP
HSR
P
LAC
P
etc
HA Manager PSS
Processes can restart in Processes can restart in milliseconds and maintain state milliseconds and maintain state from state database (PSS)from state database (PSS)
Net effect is zero impact toNet effect is zero impact to Kernel
N7K Data Plane
Net effect is zero impact to Net effect is zero impact to neighbor relationshipsneighbor relationships
Supported for all L2Supported for all L2 protocols as well as OSPFv2
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 23
Unified FabricIncreased Efficiency Simplified OperationsIncreased Efficiency, Simplified Operations
Mgmt Network
Mgmt Network
Front-End Front-End NetworkNetwork
Backup NetworkBackup Network
UnifiedFabricUnifiedFabric
Storage NetworkStorage Network
Back-End Network
Back-End Network
Unified FabricUnified Fabric
NetworkNetwork NetworkNetwork
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 24
Unified FabricUnified Fabric
Delivering Unified I/OFibre Channel over EthernetFibre Channel over Ethernet
Data Center Ethernet StandardsData Center Ethernet StandardsData Center Ethernet StandardsData Center Ethernet Standards
Unified I/O TransportUnified I/O TransportUnified I/O TransportUnified I/O Transport
Mapping FC frames over Ethernet Transport Eth tEthernet Transport
Enables Fibre Channel to run over a lossless Ethernet medium
Fibre Channel
Ethernet
medium
Single Adapter, less device proliferation, lower power consumption
Channel Traffic
consumption
NO gateways required
net
er E er er C F S
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 25
Ethe
rnH
ead
FCoE
Hea
d
FCH
ead FC Payload C
RC
EOF
FCS
Network Stack Comparison
SCSI
iSCSI
FCP
Less Overheadthan
Less OverheadthaniSCSI
FCIPFC
than FCIP or iSCSI
than FCIP or iSCSI
IP
TCP
FCoE
Ethernet
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 26
Physical WireSCSI iSCSI FCIP FCoE FC
Virtualization with VDC – 1 to Many
VDC 1VDC 1
Layer 3 Protocols
VDC 2VDC 2
Layer 2 Protocols
VDC 1
VDC 2
Layer 2 Protocols Layer 3 Protocols
VLAN
PVLAN
OSPF
BGP
EIGRP
GLBP
HSRP
IGMP
UDLD
CDP
802.1XSTP
LACP PIMCTS SNMP
OSPF
BGP
EIGRP
GLBP
HSRP
IGMP
PIM SNMP
VLAN
PVLAN
UDLD
CDP
802.1XSTP
LACP CTS
Infrastructure
VDC 3LACP PIMCTS SNMP
……
VDC 4
……
VDC – Virtual Device Context (Up to 4)
Fl ibl i /di ib i f h d d f
Infrastructure
Kernel
Flexible separation/distribution of hardware resources and software components
Complete data plane and control plane separation, Physical ports allocated to VDCs
Complete software fault isolation
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 27
Securely delineated administrative contexts
Forwarding engine scalability with appropriate interface allocation
VDC - Enabling Network Consolidation
=VDC
VDC 2
VDC 4
Device Partitioning into Multiple
VDCExtranet
VDCProd
VDCDMZ
Multiple Contexts
Lower Capital Expenditure
Consolidate multiple devices
Remove interconnect links
Reduce Fewer number of devices to manage
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 28
Operational Costs Lowers overall data center power draw
Virtual Device Contexts VDC Resource Utilization (Layer 2)VDC Resource Utilization (Layer 2)
Layer 2 learning with multiple active VDC’s also has an impact on resource utilization - MAC addresses learnt in a VDC are only propagated to other linecards when that linecard has a port in that VDCin that VDC…
Switch FabricX
Linecard 1 Linecard 2 Linecard 3MAC Table MAC Table MAC Table
MAC “A” MAC “A”
1/1 1/2 1/3 1/4 2/1 2/2 2/3 2/4 3/1 3/2 3/3 3/4
MAC A MAC A
VDC
30
VDC
20
VDC
20
VDC
20
VDC
10
VDC
10
VDC
30
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 29
MAC Address A MAC “A” is propagated to linecard 2 and 3 but only linecard 2 installs MAC due to local port being in VDC 10
Virtual Device Contexts VDC Resource Utilization (Layer 3)VDC Resource Utilization (Layer 3)
VDC 10 VDC 20 VDC 30FIB and ACL TCAM resources are more effectively utilized
Linecard 1 Linecard 2 Linecard 3 Linecard 4 Linecard 5 Linecard 6 Linecard 7 Linecard 8
FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM
effectively utilized…
128K 128K 128K 128K 128K 128K 128K 128K
ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 30
64K 64K 64K 64K 64K 64K 64K 64K
Virtual Device Contexts VDC AdministrationVDC Administration
Super User can access all global configuration commands, can create/delete VDC’s and perform , p
resource allocation across VDC’s…
VDC Administrator can change any configuration for resources allocated to that VDC and can also
create user roles specific to that VDC with a b t f fi ti dsubset of configuration commands…
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 31
VDC User Role is a restricted role based access for a given VDC and can perform configuration as
defined by the VDC Administrator…
Enhancing Layer2 Scalability – Multi-Chassis SolutionVirtual Port Channel (vPC)
L2
Virtual Port Channel (vPC)
SiSiSiSi
Virtual Port Channel
Physical Topology Logical Topology
Bi sectional BW with vPC
Non-vPC vPC
Virtual Port ChannelBi-sectional BW with vPC
vPC is a Port-channeling concept extending link aggregation to two
h i l i h
Uses all available uplink bandwidth Enable seamless VM Mobility,
separate physical switches Allows the creation of resilient L2
topologies based on Link Aggregation
y,Server HA Clusters
Scale Available Layer 2 Bandwidth Grow the size of the layer 2
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 32
Aggregation. Eliminates the need for STP in
the access-distribution Layer
Grow the size of the layer 2 network
Simplify Network Design
Feature Overview & Terminology vPC Terminology
vPC peer – a vPC switch, one of a pair
vPC member port – one of a set of ports (port channels) that form a vPC
vPC peer-keepalive vPC peer-link
vPC Terminology
vPC – the combined port channel between the vPC peers and the downstream device
vPC peer-link – Link used to synchronize state between vPC peer devices, must be 10GbE
link
CFS protocol
between vPC peer devices, must be 10GbE
vPC peer-keepalive link – the keepalive link between vPC peer devices, i.e., backup to the vPC peer-link
vPC peer
vPC b
vPCvPC
vPC VLAN – one of the VLANs carried over the peer-link and used to communicate via vPC with a peer device.
non vPC VLAN One of the STP VLANs not
member port
member port
non-vPC VLAN – One of the STP VLANs not carried over the peer-link
CFS – Cisco Fabric Services protocol, used for state synchronization and configuration validation
vPC
non-vPC device
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
between vPC peer devices
Building a vPC DomainSteps to setup vPC
1. Configure globally a vPC domain on both vPC devices2. Configure a Peer-keepalive link on both vPC peer switches (make sure is operational)
NOTE: When a vPC domain is configured the keepalive must be operational to allow a PC d i t f ll fvPC domain to successfully form.
3. Configure (or reuse) an interconnecting port-channel between the vPC peer switches4. Configure the inter-switch channel as Peer-link on both vPC devices (make sure is
operational)operational)5. Configure (or reuse) Port-channels to dual-attached devices6. Configure a unique logical vPC and join port-channels across different vPC peers
vPC peer-keepalive link
vPC peer-link
vPC peer
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 34
vPC vPC member port
Standalone Port-channel
Attaching to a vPC DomainIEEE 802 3ad and LACP
Definition:Port-channel for devices for devices dual-attached to
IEEE 802.3ad and LACP
the vPC pair.Provides local load balancing for port-channel
membersSTANDARD 802.3ad port channel
Access Device RequirementsSTANDARD 802.3ad capabilitySTANDARD 802.3ad capabilityLACP Optional
Recommendations: vPC
Use LACP when available for better failover and mis-configuration protection
vPC member
port
RegularPort-
channel port
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
Overlay Transport VirtualizationOverlay Transport Virtualization
OTV is a “MAC in IP” technique to
Technology PillarsOTV is a MAC in IP technique to
extend Layer 2 domains OVER ANY TRANSPORT
Protocol LearningDynamic Encapsulation
Preserve Failure Boundary
No Pseudo-Wire State Maintenance
Optimal Multicast
First platform to support OTV starting with 5.0(3) release!
Nexus 7000
Built-in Loop Prevention
Automated Multi-homing
Optimal Multicast Replication
Multipoint Connectivity
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 36
Site IndependencePoint-to-Cloud Model
OTV Data Plane: Inter-Site Packet FlowOTV Data Plane: Inter Site Packet Flow1. Layer 2 lookup on the destination MAC.
MAC 3 is reachable through IP B. 2. The Edge Device encapsulates the frame.3 The Core deli ers the packet to the Edge
4. The Edge Device on site East receives and decapsulates the packet.
5. Layer 2 lookup on the original frame. MAC 3 is a local MAC
Core3. The Core delivers the packet to the Edge
Device on site East.MAC 3 is a local MAC.
6. The frame is delivered to the destination.
MAC TABLE MAC TABLE3
OTVOTV OTVOTV OTVOTV OTVOTV
MAC TABLE
VLAN MAC IF100 MAC 1 Eth 2
100 MAC 2 Eth 1
MAC TABLE
VLAN MAC IF100 MAC 1 IP A
100 MAC 2 IP AL 25
IP A IP BMAC 1 MAC 3MAC 1 MAC 3L 21 Encap
2Decap
4IP A IP B
100 MAC 3 IP B
100 MAC 4 IP B
IP A IP BMAC 1 MAC 3 100 MAC 3 Eth 3
100 MAC 4 Eth 4
Layer 2Lookup
IP A IP BMAC 1 MAC 3MAC 1 MAC 3Layer 2Lookup
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 37
MAC 1 MAC 3MAC 1 MAC 3
West Site
MAC 1 MAC 3EastSite
6
OTV Data Plane EncapsulationOTV Data Plane Encapsulation OTV adds a 42 Byte IP encapsulation.
The outer IP header is followed by an OTV shim header, which contains yinformation about the overlay (vlan, overlay number, etc).
The 802.1Q header is extracted from the original frame and the VLANfield copied over into the OTV shim header.
The OTV Edge Device can also map the 802.1p CoS bits to the outer IP header’s DSCP field as well as to the OTV Shim header.
802.1QDMAC SMAC Eth Payload
802.1Q
Co
6B 6B 2B 20B 8B
DMAC SMACEther Type IP Header
Original Frame 4B
CRC
VLA
N
OTV Shim
ToSToS
oS
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 38
42 Byte encapsulation (same as VPLSoGRE)
OTV Control PlaneNeighbor Discovery in a Multicast-Enabled CoreNeighbor Discovery in a Multicast Enabled Core
OTV Adjacencies are establishedover the mcast group in the core
OTV OTV Control PlaneOTVOTV OTVOTV
East
Control Plane Control PlaneOTVOTV
Multicast-enabled Core
WestEast
The end resultEmulation of a multi-access link-
The end resultEmulation of a multi-access link-
The mechanismEdge Devices (EDs) join an ASM
The mechanismEdge Devices (EDs) join an ASM
Core
OTVOTVOTV
Control Pl Emulation of a multi-access link-
layer multicast environment Link-local Neighbor DiscoveryAdjacencies are maintained over
the multicast group
Emulation of a multi-access link-layer multicast environment Link-local Neighbor DiscoveryAdjacencies are maintained over
the multicast group
Edge Devices (EDs) join an ASMmulticast group in the core. They join as hosts (no PIM on EDs)OTV hellos and updates are
encapsulated in IP and sent to the lti t
Edge Devices (EDs) join an ASMmulticast group in the core. They join as hosts (no PIM on EDs)OTV hellos and updates are
encapsulated in IP and sent to the lti t
South
Plane
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 39
g pA single update reaches all
neighbors
g pA single update reaches all
neighbors
multicast groupEDs are both sources and
receivers
multicast groupEDs are both sources and
receivers
OTV Control PlaneNeighbor Discovery in a Multicast-Enabled Core (1)Neighbor Discovery in a Multicast Enabled Core (1)
Multicast-enabled CoreOTV
Control PlaneOTV
Control PlaneOTVOTV OTVOTVEncap
2
OTV Hello 3 CoreReplication
1
4
OTV Hello 5
ASM Group
WestEastIP A
IGMP Report IGMP ReportIP B
Encap
IP A Mcast GOTV Hello IP A Mcast GOTV Hello
Decap
IP A Mcast GOTV Hello
S G oup
West
IGM
P R
eporIP C
OTVOTV
rtIP C
Decap4
IP A Mcast GOTV Hello
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 40
South
OTV Control Plane
OTV Hello 5
OTV Control PlaneNeighbor Discovery in a Multicast-Enabled Core (2)Neighbor Discovery in a Multicast Enabled Core (2)
The West Site sees that the hello contains its ID.
10
Multicast-enabled Core
OTV Control Plane
OTV Control PlaneOTVOTV OTVOTV
8 CoreReplication
Decap9
Decap9
The OTV Adjacency is Established
OTV Hello OTV Hello
ASM Group
WestEastIP A
IGMP Report IGMP ReportIP B
p
IP C Mcast GOTV Hello
DecapASM Group
West p p
IGM
P R
ep
IP C
OTVOTV
portIP C
Encap7
IP C Mcast GOTV Hello
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 41
South
OTV Control Plane
OTV Hello The South Site sends its hello with West’s address
in the TLV
6From Bottom to Top
OTV Control PlaneMAC Address Advertisements – Multicast CoreMAC Address Advertisements Multicast Core
Every time an Edge Device learns a new MAC address, the OTV control plane will advertise it together with its associated VLAN IDs and IP next hop.
The IP next hops are the addresses of the Edge Devices through which these MACs addresses are reachable in the core.
A single OTV update can contain multiple MAC addresses for different VLANs.
A single update reaches all neighbors, as it is encapsulated in the same ASMmulticast group used for the neighbor discovery.
VLAN MAC IF
100 MAC A IP A
4
Core
3 New MACs are learned on VLAN 100
Vlan 100 MAC A
Vlan 100 MAC B
OTV update is replicated by the core
3
100 MAC A IP A
100 MAC B IP A
100 MAC C IP A
3 New MACs are learned on VLAN 100
1
Core
IP AWest
EastVlan 100 MAC C
VLAN MAC IF
100 MAC A IP A
4
3
2
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 42South-East
100 MAC B IP A
100 MAC C IP A
ConfigurationConfigurationOTV CLI Configuration
Connects to the core Used to join the Overlay network
interface Overlay0
Connects to the core. Used to join the Overlay network. Its IP address is used as source IP for the OTV encap
ASM/Bidir group in the core used for the OTV Control Plane.
SSM group range used to carry the site’sinterface Overlay0otv join-interface Ethernet1/1otv control-group 239.1.1.1otv data group 232 192 1 0/24
SSM group range used to carry the site s mcast traffic data.
otv data-group 232.192.1.0/24otv extend-vlan 100-150
otv site-vlan 99 Site VLANs being extended by OTV
VLAN used within the Site for communication between the site’s Edge Devices
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 43
Cisco NXOS SoftwareLayer 2 Multipathing - Feature Sets
16-Way Equal Cost Multipathing (ECMP) t L 2
Layer 2 Multipathing Feature Sets
at Layer 2
Mac-in-Mac – Hierarchical Addressing with built in protocol checks (RPF,TTL) MacMac--inin--MacMac
Up to 16-Way L2 ECMP
Optimized MAC Learning – learn based on conversations.
ISIS Control Plane – leveraging an MacMac--inin--MacMacUp to 16Way
L2 ECMP
established routing protocol for Layer 2 ‘routing’
Interoperability with existing classic ethernet networksethernet networks• VPC + – allows VPC into a L2MP cloud
• STP Boundary Termination
M lti T l idi t ffi
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 44
Multi-Topology – providing traffic engineering capabilities
Cisco FabricPath OverviewCisco FabricPath Overview
CiscoCisco FabricPathFabricPath
Data Plane Innovation Control Plane Innovation
Cisco Cisco FabricPathFabricPath
No MAC learning via flooding
Plug-n-Play Layer 2 IS-IS Support unicast and
Routing, not bridging Built-in loop-mitigation
Time-to-Live (TTL)
ppmulticast Fast, efficient, and scalable Equal Cost MultipathingTime to Live (TTL)
RPF CheckEqual Cost Multipathing
(ECMP) VLAN and Multicast Pruning
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 45Cisco Nexus PlatformCisco Nexus Platform
Cisco NXCisco NX--OSOS
Data Plane OperationData Plane Operation
FabricPath header is imposed by ingress switchEncapsulation to creates hierarchical address scheme
Ingress and egress switch addresses are used to make “Routing” decision
No MAC learning required inside the L2 Fabric
C
A
FabricPath Header S11
S42
S11 S42FabricPath Routing
STP DomainSTP DomainSTP DomainSTP DomainFabricPathFabricPathFabricPathFabricPathS11 S42
A
DATA
Ingress Switch Egress Switch
S11 S42
A C
A C
C
A
DATAL2 Bridging
A C
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 46
STP Domain 1STP Domain 1STP Domain 1STP Domain 1 STP Domain 2STP Domain 2STP Domain 2STP Domain 2
A CDATA
A C A C
Control Plane OperationControl Plane Operation
Assigned switch addresses to all FabricPath enabled switches Plug-N-Play L2 IS-IS is used to manage forwarding topology
automatically (no user configuration required) Compute shortest, pair-wise paths Support equal-cost paths between any FabricPath switch
pairspairs
S1 S2 S3 S4
FabricPathRouting Table
FabricPathRouting Table
Switch IF
S1 L1
L1L2
S11 S12 S42L2 FabricL2 Fabric
L3
L4
S2 L2
S3 L3
S4 L4
S12 L1, L2, L3, L4
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 47
L2 FabricL2 Fabric… …
S42 L1, L2, L3, L4
Unicast with FabricPathUnicast with FabricPath
Support more than 2 active paths (up to 16) across the FabricForwarding decision based on ‘FabricPath Routing Table’
Increase bi-sectional bandwidth beyond port-channel High availability with N+1 path redundancy
S1 S2 S3 S4
L1L2
L3
Switch
IF
… …
S42 L1, L2, L3, L4
S11 S12 S42L2 FabricL2 Fabric
L4S42 L1, L2, L3, L4
MAC IF
A 1/1
… …
C S42/
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 48
A C
C S1/1
Multicast with FabricPathMulticast with FabricPath
Several ‘Trees’ are rooted in key location inside the fabric
Forwarding through distinct ‘Trees’
All Switches in L2 Fabric share the same view for each ‘Tree’ Multicast traffic load-balanced across these ‘Trees’
Root for Tree #1
Root for Tree #2
Root for Tree #3
Root for Tree #4Tree #1 Tree #2 Tree #3 Tree #4
Ingress switch for FabricPath decides which “tree” to be used and add tree number in the header
L2 FabricL2 Fabrictree number in the header
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 49A C
FabricPath ConfigurationFabricPath Configuration
No L2 IS-IS configuration required
New ‘feature-set’ keyword introduced to allow multiple conditional services required by FabricPath to be enabled in one shot
Simplified operational model – only 3 CLIs to get FabricPath up and running
N7K(config)# feature-set fabricpathN7K(config)# vlan 10-19N7K(config-vlan)# mode fabricpathN7K(config)# interface port-channel 1N7K(config-if)# switchport mode
N7K(config)# feature-set fabricpathN7K(config)# vlan 10-19N7K(config-vlan)# mode fabricpathN7K(config)# interface port-channel 1N7K(config-if)# switchport mode
L2 FabricL2 Fabric
N7K(config if)# switchport mode fabricpathN7K(config if)# switchport mode fabricpath
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 50
FabricPath PortCE Port
N7000 + FEX Single Access LayerN 7000Nexus 7000
Fabric Extender (2248 FEX)
• Nexus 7000 + FEX is single management – FEX/Nexus 7000• Nexus 2000 FEX is like a Line Card to the Nexus 7000• No Spanning Tree between FEX and Nexus 7000
• Nexus 7000 + FEX is single management – FEX/Nexus 7000• Nexus 2000 FEX is like a Line Card to the Nexus 7000• No Spanning Tree between FEX and Nexus 7000
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 51
No Spanning Tree between FEX and Nexus 7000• Nexus 7000 maintains all management and configuration
No Spanning Tree between FEX and Nexus 7000• Nexus 7000 maintains all management and configuration
Nexus 2248 Fabric Extender – Shipping
48x1GE/100Mb 4x10GE
Beacon &Beacon & Status LEDs
Beacon & Status LEDs
Power Supplies, Redundant & Hot Swappable
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 52Redundant, Hot Swappable Fans
Nexus 2000 with Nexus 7000Benefits of the N7K + N2KBenefits of the N7K + N2K
Cor
e • Combines benefits of Top of Rack (ToR) and End of Row (EoR) network
egat
ion
architectures• Reduces cable runs• Cross Nexus architecture provides
Agg
re
pInvestment protection
• Reduce management points in the network
cces
s
• Solution for higher density 1G (i.e. 96 port 1G module)
• Ensures feature consistency across
Ac hundreds or thousands of server
portsVM VM
VMVM VMVM
VM VM
N2K / N7K
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID
N2K / N7K 1 GEVM VM
VMVM VMVM
VM VM
N2K / N7K 1 GE
Major Competitors in this spaceMajor Competitors in this space
Juniper EX8208 is shipping
Juniper EX8216 is getting positioned in the DC
HP with a bag of switches from various vendors.
H3C“me too” features of
Brocade / Foundry gaining market share in modular switching
FCoE on DCX
Arista with its latest modular switch – 7500
Big Claims like..
… 5x Performance
Touting Stratus architecture
IBM teaming up with Juniper in Switching
Nexus 7000 Series
Leading with lower price
Aggressive marketing against N7K with
Aggressively taking Foundry boxes to channels
… 5x Performance
… 1/10th the Power Consumption
… ½ the footprint
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 54
Juniper in Switching against N7K with incorrect claims
© 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID © 2009 Cisco Systems, Inc. All rights reserved. Cisco ConfidentialPresentation_ID 55