brocade

BCEFP in a Nutshell Study Guide for Exam 150-810

Brocade University

Revision 0513

Corporate Headquarters - San Jose, CA USAT: (408) [email protected]

European Headquarters - Geneva, SwitzerlandT: +41 22 799 56 [email protected]

Asia Pacific Headquarters - SingaporeT: [email protected]

2013 Brocade Communications Systems, Inc. All Rights Reserved.

ADX, AnyIO, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, ICX, MLX, MyBrocade, OpenScript, VCS, VDX, and Vyatta are registered trademarks, and HyperEdge, The Effortless Network, and The On-Demand Data Center are trademarks of Brocade Communications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may be trademarks of their respective owners.

Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning any equipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this document at any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not be currently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained in this document may require an export license from the United States government

Revision 0513

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionIntroduction to BCEFP in a Nutshell First Edition

Objective: The BCEFP Nutshell guide is designed to help you prepare for the BCEFP Certification, exam number 150-810.

Audience: The BCEFP Nutshell self-study guide is intended for those who have successfully completed the CEF 200 Certified Ethernet Fabric Training and the CEFP 300-WBT Advanced Ethernet Fabric Administration course, and who wish to undertake self-study or review activities before taking the actual BCEFP exam. The BCEFP guide is not intended as a substitute for classroom training or hands-on time with Brocade products.

How to make the most of the BCEFP guide: The BCEFP guide summarizes the key topics on the BCEFP exam for you in an easy to use format. It is organized closely around the exam objectives. We suggest this guide be used in conjunction with our free online knowledge assessment test. To benefit from the BCEFP guide, we strongly recommend you have successfully completed the CEF 200 Certified Ethernet Fabric Training and the CEFP 300-WBT Advanced Ethernet Fabric Administration course.

We hope you find this useful in your journey towards BCEFP Certification, and we welcome your feedback by sending an email to [email protected].

Joe CannataCertification Manager 2013 Brocade Communications Systems, Inc. All Rights Reserved. iii

Brocade Certified Ethernet Fabric Professional in a Nutshell Second EditionTable of ContentsBrocade VCS Fabric Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Brocade VCS Distributed Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Distributed Fabric Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Masterless control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Shared port profile information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Automatic distribution of zoning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1VCS Fabric: Use Case Aggregation Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1VCS Fabric: Use Case Combined Access/Aggregation Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Hardware Products and Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Brocade VDX Product Comparisons and Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Brocade VDX Switch Protocol and Fabric Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Brocade VDX 6720 Data Center Switches Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Brocade VDX 6720-60 Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Brocade VDX 6710-54 Data Center Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Brocade VDX 6730-32 Data Center Switch Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Brocade VDX 6730-32 Data Flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Brocade VDX 6730-76 Data Center Switch Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Brocade VDX 8770-8 and VDX 8770-4 Data Center Switch Overview . . . . . . . . . . . . . . . . . 12

VDX 8770 Port Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14VDX 8770 Interface Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15VDX 8770 Switch Fabric Module (SFM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15VDX 8770 Line Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Slot Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16VDX8770 CID Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16VDX8770 Management Module (MM) See Figure14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Long Distance ISL Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Brocade VDX Licenses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

VCS Fabric License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18FCoE License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Ports on Demand (POD) License . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18NOS v3.0.0 Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20TRILL Defined. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

TRILL Frames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Hop Count. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

VCS Fabric Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Automatic Layer 2 Adjacency Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22RBridge ID Conflicts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23VCS Edge Port Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

L2 Forwarding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2013 Brocade Communications Systems, Inc. All Rights Reserved. v

Brocade Certified Ethernet Fabric Professional in a Nutshell Second EditionFSPF Protocol for ECMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24VCS Layer 2 ECMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25VCS Fabric L2 Multi-Destination Tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Multi-cast Tree Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Traffic Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26VCS Known Unicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26VCS Multicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31IGMP Snooping in a VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32VCS Fabric Edge Loop Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33How Edge Loop Detection Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34BUM Storm Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

L3: Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36VCS Fabric - Layer 3 Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

VCS Fabric L3 Routing Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36How OSPF works in VCS Fabric. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Assigning OSPF areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Configuring a Summary-Address for the NSSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Using Designated Routers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Assigning virtual links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40Using track ports and track priority with VRRP and VRRP-E. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41VRRP-E differences for basic configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Short Path Forwarding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42Enabling preemption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

VCS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Verifying RBridge ID and VCS ID Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44VCS ISL Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Fabric ISL Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Verifying VCS Fabric ISL Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Verifying VCS Fabric ISL Formation (cont.) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47NOS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

vLAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50vLAG Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50vLAG Example Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51vLAG Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52LAG Type Brocade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54Configuring the vLAG Ignore Split. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54The minimum-links Command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55Support for Static MAC Pre-Provisioning on vLAG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55LACP troubleshooting tips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56xSTP reconvergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

QOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58vi 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell Second EditionRandom Early Discard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Brocade VCS Fabric QoS Restrictions for Layer 3 features in VCS mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Configuring QoS profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Configuring Brocade VCS Fabric QoS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59CEE Map CoS Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59FCoE Queuing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60Deficit weighted round robin scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60Port-Based Policer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

Data Center Bridging (DCB) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61DCB Enhancements to Ethernet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62DCBX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62ETS Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63ETS and Class of Service. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Priority Flow Control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Strict Priority Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Data Center Bridging Map Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Defining a priority group table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Defining a Priority-Table Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

FCoE and iSCSI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67FCoE Co-Existence with LAG/vLAG. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Configuring FCoE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68Configuring FCoE and LLDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69Enable and Disable FCoE Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69FCoE Show Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70iSCSI TLV Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71Assigning an FCoE map onto a LAG member . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72Zone Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

Zone merging scenarios: Default access mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72The show name-server detail command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

FC Bridging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74LSAN naming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Brocade VCS Fabric and Brocade Fibre Channel SAN Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Principal Routing Bridge Availability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76FC-FC Routing to Bridge FCoE and FC Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76Managing domain IDs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76FC to FC Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Brocade VDX Switch TRILL Termination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77Edge to BB Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

AMPP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79VM Mobility Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Solution Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79Port Profile Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

vCenter Integration Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81vCenter Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 2013 Brocade Communications Systems, Inc. All Rights Reserved. vii

Brocade Certified Ethernet Fabric Professional in a Nutshell Second EditionNOS vCenter Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82Life of a Port-Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Configuring FCoE Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Management and Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85sFlow Global Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85SNMP community strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Collecting Support Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Login Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86TACACS+ Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86RBAC Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86DH-CHAP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87Configuring Security Profiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87ACL Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87ACL Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88Firmware Download. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88Automatic firmware synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88Firmware Upgrade Rules for vLAGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89Firmware Download Error Handling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

Troubleshooting Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89FRU Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89Display the SFP information for a Specific Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Verifying LACPDUs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90Ping Failure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91BNA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91Brocade Network Advisor v11.3 Enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

VDX Additional Functionality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93Virtual IP Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

Selecting the MTU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94Registered State Change Notification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95Fabric Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95show firmwaredownloadhistory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95show port-channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95oscmd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

Taking the Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98viii 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition

List of FiguresVCS Fabric Use Case: Aggregation Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2VCS Fabric Use Case: Combined Access/Aggregation Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Brocades VDX 6720 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Brocade VDX 6720-24 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Brocade VDX 6720-60 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Brocade VDX 6710-54 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Brocade VDX 6710-54 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Brocade VDX 6730-32 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10Brocade VDX 6730-32 Ports Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11Brocade VDX 6730-76 Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12VDX 8770-8 Switch Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13VDX 8770-4 Switch Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14VDX 8770 Port Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15VDX 8770 Management Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17TRILL Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21Layer 2 Adjacency Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22FSPF for ECMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24ECMP Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25VCS Fabric L2 Multi-Destination Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26Unicast Ethernet Frame Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27Unicast Ethernet Frame with TRILL Header Added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28TRILL Ethernet Frame: Link Transport Header Added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29TRILL Ethernet Frame: Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30Ethernet Frame: End of Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31VCS Multicast Data Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32IGMP Snooping in a VCS Fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33VCS Fabric Edge Loop Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34VCS Fabric L3 Routing Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38The virtual servers are dynamically moved between Host Server 1 and Host Server 2. . . . . . . . . . . . . . . .43show VCS command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44show vcs detail command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44show fabric all command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47show fabric islports command output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48vLAGs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50vLAGs with MCTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .52vLAG Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53MAC Pre-Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56Wrr schedule - two queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60ETS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64ETS and Class of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64FCoE Co-Existence with LAG/vLAG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68 show fcoe interface brief command example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70Default Access mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73Brocade VCS Fabric to Brocade FC SAN Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75FC to FC Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77Brocade VDX Switch Trill Termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78Edge to BB Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78VM Migration Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79 2013 Brocade Communications Systems, Inc. All Rights Reserved. ix

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition

Port Profiles: Distributed Network Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80Port Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81NOS and vCenter Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83x 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionList of TablesBrocade VDX Product Comparisons and Positioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Brocade VDX Switch Protocol and Fabric Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5FSPF Key to Figure17 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 2013 Brocade Communications Systems, Inc. All Rights Reserved. xi

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionBrocade VCS Fabric Technology

Brocade VCS Distributed Intelligence

Distributed Fabric Services Self-forming fabric (with minimum configuration) Information shared across all fabric members Fabric aware of all connected devices

Masterless control Switch or link failure does not require full fabric reconvergence

Shared port profile information Automatic Migration of Port Profiles (AMPP) Enables seamless virtual server migration

Automatic distribution of zoning Zoning changes are automatically distributed through out the fabricBrocade VCS Fabric technology supports a fully distributed control plane. That means every switch is aware of the entire network topology. You can configure it in a ring, a mesh, or a tree. You can configure enough links to make it entirely non-blocking or configure the topology to be over subscribed at whatever level you choose. Every switch knows about every attached device and Virtual Machine (VM).

When a VM is attached to the network, the network configuration is automatically bound to the VM and then distributed throughout the Brocade VCS Fabric cloud. If the VM moves, the destination port already knows what to do and the VM is automatically reconnected to all of its old resources. The network configuration automatically migrates with the VM.

VCS Fabric: Use Case Aggregation Layer Low cost, highly flexible logical chassis at aggregation layer. See Figure1. Building block scalability Per port price of a ToR switch Availability, reliability, manageability of a chassis Flexible subscription ratios Ideal aggregator for 1 Gbps ToR switches 2013 Brocade Communications Systems, Inc. All Rights Reserved. 1

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition Optimized multi-path network No single point failure No passive connections STP not necessary

FIGURE 1 VCS Fabric Use Case: Aggregation Layer

In this use case, Brocade VCS Fabric technology is used as an aggregation technology for 1 GbE server connections. This design gives the architect a building-block approach to network scalability, leveraging lower-cost switches at the aggregation layer. This design approach provides the ability to scale by adding another Brocade VDX switch into the fabric. Stackable 1 GbE switches are used at ToR, and those switches are connected into an Ethernet fabric. The Ethernet fabric is very flexible and can be scaled like building blocks.2 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionSubscription ratios can be anything, including 1:1 through the fabric. This use case example shows a three-switch stack configuration at the ToR and two 10 GbE links per switch for a total of 6 links to the Brocade VCS Fabric technology aggregation layer through a single LAG.

This building block design enables you to pay as you grow. To increase port count, simply add a Brocade VDX switch non-disruptively into the fabric. Since the Brocade VCS fabric looks and acts like a single logical entity, minimal management is required moving forward. This is an advantage of using Brocade VCS Fabric in the Access/Aggregation Layers.

VCS Fabric: Use Case Combined Access/Aggregation Layers Flatter, simpler network design. See Figure2.

- Logical two-tier architecture- VCS fabrics at the edge

Greater Layer 2 scalability/flexibility- Increased sphere of VM mobility- Seamless network expansion

Optimized multi-path network- All paths are active- No single point failure- STP not necessary

Convergence ready- End-to-end enhanced Ethernet (DCB)- Multi-hop FCoE support- iSCSI DCB support

2013 Brocade Communications Systems, Inc. All Rights Reserved. 3

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 2 VCS Fabric Use Case: Combined Access/Aggregation Layers

Storage is now introduced to the Ethernet fabric. This use case illustrates end-to-end native multi-hop FCoE and iSCSI DCB connectivity.

This use case illustrates two ways the fabric can be configured. In this diagram, a ToR Mesh architecture is used. The benefit is that a true flat network edge, where the switches are connected to its peers. The challenge of this design is to maintain common subscription ratios between each server (between racks). This topology is a single stage design. 4 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionHardware Products and Features

Brocade VDX Product Comparisons and Positioning

Brocade VDX Switch Protocol and Fabric Support

The Brocade VDX 6710, 6730, and 8770 switches support convergence through iSCSI, FCoE, and FC protocol

TABLE 1 Brocade VDX Product Comparisons and Positioning

Brocade VDX 6710 Brocade VDX 6720 Brocade VDX 6730Brocade VDX 8770-4Brocade VDX 8770-8

Physical Description 1U 6 x 10 GbE SFP+

and 48 x 1 GbE GE copper

2U 24 x 10 GbE SFP+

ports 60 x 10 GbE SFP+

ports

2U 24 x 10 GbE

SFP+ and 8 x 8 Gbps FC ports

60 x 10 GbE SFP+ ports and 16 x 8 Gbps FC ports

8770-4: 8U8770-8: 15U

48 x 1 GbE line card provides up to 48 SFP/SFP-cop-per ports

48 x 10 GbE line card provides up to 48 SFP+ ports

12 x 40 GbE line card provides up to 12 40 GbE QSFP ports

Licenses Brocade VCS Fabric License

POD License Brocade VCS Fabric

License FCoE License

POD License Brocade VCS

Fabric License FCoE License

Brocade VCS Fabric License

FCoE License Layer 3 Advanced Services

Positioning High Performance, low latency, enabled 1 Gbps switch provid-ing cost- effective connectivity to 1G servers

High performance, low latency, enabled 10 GbE switch targeted at virtual data centers. Enables end- to-end LAN/SAN convergence solutions with VCS fabric

High Performance, low latency enabled convergence switch with FC connectivity

High Performance, low latency enabled modular convergence switch with FC connec-tivity

TABLE 2 Brocade VDX Switch Protocol and Fabric Support

Product iSCSI Ethernet FCoE FC VCS Fabric

Brocade VDX 6710 Yes 1 Yes No 2 No Yes 3

Brocade VDX 6720 Yes Yes Yes No Yes

Brocade VDX 6730 Yes Yes Yes Yes 4 Yes

Brocade VDX 8770 Yes Yes Yes No Yes 2013 Brocade Communications Systems, Inc. All Rights Reserved. 5

Brocade Certified Ethernet Fabric Professional in a Nutshell First Editionsupport and also are part of a VCS fabric. The VDX 6710 is a Ethernet switch that can be part of a VCS fabric.

Note 1: The 1GbE ports do not support DCB so the iSCSI TLV is not supported however iSCSI traffic can still go across these ports. iSCSI TLV is supported on the 10 GbE ports.

Note 2: Cannot attach an FCoE device to a port on this switch however FCoE traffic can flow through the 10 G (Trill) ports as long as the destination is not on that switch. As the switch would look at the destination RB and modify the outer header with next hop information so the switch would never look at the type of traffic in that frame.

Note 3: The 1 GbE ports can not be used as ISL connections to other switches in the fabric.

Note 4: The FC ports can only be used as E_Port (Brocade VDX switch) to EX_Port (SAN router) connection. Cannot directly attach a FC end device to any of the FC ports on the Brocade VDX 6730.

Brocade VDX 6720 Data Center Switches Overview Built for the Virtual Data Center. See Figure3.

- Uses Brocade fabric switching eAnvil2 ASIC- Supports Brocade Network Operating System (NOS) including VCS technology

Performance and Density- 24- and 60-port models (VDX 6720-24 and VDX 6720-60)- Ports on Demand (POD) enables 24 to 60 port configurations- 600ns latency for 24 port and 1.8usecs for 60 port (when spanning port groups)

Configuration Flexibility- 1 Gbps or 10 Gbps supported on every port- Front-to-back or back-to-front airflow

Enables Network Convergence- Complete FCoE support including multi-hop (license required)- iSCSI Data Center Bridging support (DCB)6 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 3 Brocades VDX 6720

VDX 6720-24 platform contains one eAnvil2 ASIC. Each eAnvil2 ASIC supports a Routing Engine (RTE) and each RTE supports 24 ports, 1 or 10 GbE. Every packet coming in through the external SFP+ interfaces destined to external ports are switched in a single stage within the eAnvil2 ASIC.

Port groups are shown in Figure4.

FIGURE 4 Brocade VDX 6720-24 Port Groups 2013 Brocade Communications Systems, Inc. All Rights Reserved. 7

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionBrocade VDX 6720-60 Data FlowThe Brocade VDX 6720-60 platform contains nine eAnvil2 ASICs:

Six front end eAnvil2 ASICs Each provides 10 external SFP+ interfaces Switching bandwidth from switch backplane (data rate at full duplex): 6720-60: 1200 Gbps Forwarding bandwidth from ports (data rate at full duplex):6720-60: 896 Mpps Port groups are shown in Figure5. .

FIGURE 5 Brocade VDX 6720-60 Port Groups

Brocade VDX 6710-54 Data Center SwitchThe Brocade VDX 6710-54 is a low-cost Ethernet switch that is VCS capable. Uses Brocade fabric switching eAnvil2 ASIC and supports Brocade Network Operating System (NOS) including Brocade VCS Fabric technology. See Figure6.

54 total ports 6 x 10 GbE DCB capable optical ports. The 10 GbE ports are designed to be used as uplink ports to the VCS fabric however do support the direct attachment of 10 GbE devices. FCoE direct attached devices are not supported on this switch.

48 x 1 GbE copper ports 8 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 6 Brocade VDX 6710-54


Over subscription: There is no over subscription. From the Main CPU there is a connection to each of the three ASICs. Shown is only the connection to one of the eAnvil2 ASICs.

FIGURE 7 Brocade VDX 6710-54 Port Groups

Brocade VDX 6730-32 Data Center Switch OverviewUses Brocade fabric switching eAnvil2 ASIC. See Figure8.

Supports Brocade Network Operating System (NOS) including Brocade VCS Fabric technology

Configuration flexibility: Ports on Demand (POD) enables ports to be added. Default is 16 ports enabled with a single 8 POD license available to support a total of 32 ports

32 total ports 2013 Brocade Communications Systems, Inc. All Rights Reserved. 9

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition 8 x 8 Gbps FC ports 24 x 10 GbE DCB capable ports. Twinax and SFP optical connectivity options

Front-to-back or back-to-front airflow

Supports Network Convergence: Complete FCoE support including multi-hop (license required) and iSCSI Data Center Bridging support (DCB).

FIGURE 8 Brocade VDX 6730-32

Brocade VDX 6730-32 Data FlowAs shown above, the Brocade VDX 6730-32 platform contains one eAnvil2 and one Condor2 ASIC. The eAnvil2 ASIC support 1 RTE (Routing Engine) and each RTE supports 24, 10/1 Gbps Ethernet Ports. So every packet coming in through the external SFP+ interfaces destined to external ports are switched in a single stage within the eAnvil2 ASIC.


There is no over subscription for network traffic. FC Traffic over subscription worst case would be 240:64 (15:4)

240: 24 x 10 GbE FCoE devices 64: 8 x 8 Gbps FC traffic between the eAnvil2 ASIC and the Condor 210 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 9 Brocade VDX 6730-32 Ports Groups

Brocade VDX 6730-76 Data Center Switch OverviewUses Brocade fabric switching eAnvil2 ASIC.

Supports Brocade Network Operating System (NOS) including Brocade VCS Fabric technology

Configuration flexibility: Ports on Demand (POD) enables ports to be added. Default is 40 ports enabled with two 10 POD licenses available to support a total of 60 GE ports

76 total ports 16 x 8 Gbps FC ports 60 x 10 GbE DCB capable ports: Twinax and SFP optical connectivity optionsSupports Network Convergence: Complete FCoE support including multi-hop (license required) and iSCSI Data Center Bridging support (DCB)

Port groups are shown in Figure10. 2013 Brocade Communications Systems, Inc. All Rights Reserved. 11

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 10 Brocade VDX 6730-76 Port Groups

Brocade VDX 8770-8 and VDX 8770-4 Data Center Switch Overview Two models are available: 4 and 8 line card (LC) switch models LCs are available in 1, 10 and 40 Gbps models In 8-slot switch, up to: See Figure11

- 384 x 10 GbE or 1 GbE user ports- 96 x 40 GbE user ports12 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 11 VDX 8770-8 Switch Design

In 4-slot switch, up to: See Figure12

92 x 10 GbE or 1 GbE user ports 48 x 40 GbE user ports 2013 Brocade Communications Systems, Inc. All Rights Reserved. 13

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 12 VDX 8770-4 Switch Design

In 4-slot switch, up to:

192 x 10 GbE or 1 GbE user ports 48 x 40 GbE user ports

- Fully redundant Management Modules (MM), Switch Fabric Modules (SFM) and PSUs- 4 microsecond (s) latency from any port to any port- 100 GbE/SDN-ready and 4 Tbps-capable backplane to support future releases- VCS mode support only (no Standalone mode)

VDX 8770 Port Numbering

Brocade VDX 8770-8Ports are numbered from 1 through 48, from top to bottom, with the odd-numbered ports on the right row and the even-numbered ports on the left row when installed in the switch.

Brocade VDX 8770-4:Ports are numbered from 1 through 48, from left to right, with the odd-numbered ports on the upper row and the even-numbered ports on the lower row when installed in the switch. See Figure1314 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 13 VDX 8770 Port Numbering

For both the VDX 8770-4 and VDX 8770-8:

ISL Trunking is supported only on the 48x10GbE line card. The port groups can consist of up to 8 ports. The octet groups are ports 1-8, 9-16, 17-24, 25-32, 33-40, and 41-48.

VDX 8770 Interface NomenclatureThe 10GE SFP+ interfaces in LC48x10G line card are named TenGigabitEthernet or in short TE

The 40GE QSFP interfaces in the LC12x40G line card are named as FortyGigabitEthernet or in short FO

The 1GE SFP+ interfaces in LC48x1G line card are named GigabitEthernet or in short GE

VDX 8770 Switch Fabric Module (SFM)SFMs contain 3 x Condor3 ASICs for switching packets between LCs

Hot swappable SFM modules provides resiliency - VDX 8770-8 is N+1 with loss of one SFM - VDX 8770-4 is not N+1; if one SFM is lost

There are message link ports between the line card Hawks and the SFMs: - For VDX 8770-8, S3 and S4 have to be populated with at least one SFM - For VDX 8770-4, S1 and S2 have to be populated with at least one SFM

The SFM blade has an early-removal indicator that enables the software to re-route the data traffic flowing through the SFM to the other SFMs in the system thereby providing a zero-frame-drop performance during SFM hot-plug-out. Based on the available bandwidth, the user may or may not experience a small percentage loss. 2013 Brocade Communications Systems, Inc. All Rights Reserved. 15

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionVDX 8770 Line Cards SFP+ and QSFP media interfaces 3 Hawk ASICs provide the traffic manager functionality 6 Wolverine ASICs provide packet processor functionality Local CPU to offload MM and provide distributed processing No local switching, all traffic is sent to SFMs for switching by Condor3 ASICs Brocade VDX 8770-8

Slot ConfigurationInterface modules are registered with the system by type, and the slot must be configured with the correct type before you can install an interface module in that slot. When you install a new interface module, the system checks whether or not a previous configuration is associated with the slot. The following rules apply when you install or replace an interface module: When you install an interface module and boot it up to an online state in a slot that was never occupied or configured, the module type information is automatically detected and saved to the database. No special configuration is required. If you install an interface module in a slot that was previously occupied by an interface module of the same type and the slot is configured for that same type, you can hot-swap the modules without powering off the interface modules. No slot configuration changes are required.

If the slot was previously configured for a different type of interface module, the installation fails and the module is faulted with a "Type mismatch" error. A RASlog error message is generated. You must power off the interface module and clear the slot configuration with the no linecard command before you can configure the slot for a new interface module.

VDX8770 CID Card 2 Cards - (reads are from CID 1 unless problems occur, writes are to both)

Located behind fan 2 on VDX8770-4, fan 1 on VDX8770-8

Critical data sets (cannot be written by NOS)

A header with the CID part number, serial number, and other data about the CID card Chassis PN/SN License ID/WWN Non-critical data sets (can be written by NOS)

FRU history log (not verified) Table with management IPs, switch name Power off list for line cards DCE MAC data containing switch MAC addresses16 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionVDX8770 Management Module (MM) See Figure14Handles management and control plane functions for switch fabric modules, line cards and all other FRUs

MMs are redundant, with one active and one standby MM

NOS 3.0 requires cold reboot of standby MM during failover and firmware upgrades

Connections:

10 Gbps Ethernet connection between the two MMs 1 Gbps Ethernet connection between MM and LCs PCIe connection between MM, proprietary buses and switch fabric modules

FIGURE 14 VDX 8770 Management Module

Long Distance ISL SupportWith NOS v2.1.1 or higher, the maximum ISL distance is 10 km

Supported on the Brocade VDX 6720-60 and 6730-76 switches

All other Brocade switches support a maximum distance of 200 m

Use the long-distance-isl command to specify one of these distances: 200 m (default)

2000 m

5000 m

10000 m

Here is an example:

VDX11(conf-if-te-11/0/14)# long-distance-isl ?Possible completions:

2000 2000 meter distance link (Warning: It May disable other ISLs in the port group)5000 5000 meter distance link (Warning: It May disable other ISLs in the port group)10000 10,000 meter distance link (Warning: It May disable other ISLs in the port group)

Additional notes: 2013 Brocade Communications Systems, Inc. All Rights Reserved. 17

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition For a 10 km ISL link, no other ISL links are allowed on the same eAnvil2 ASIC. For 2 km and 5 km ISL links, another short distance ISL link can be configured. A maximum of 3 PFCs (per priority flow control) can be supported on a long distance ISL link. Use the no long-distance-isl command to revert to the default value which is 200 m. For more information refer to the Brocade Network OS Administrators Guide Supporting Network OS v2.1.1, 53-1002491.

Brocade VDX Licenses

VCS Fabric License If the Brocade VCS Fabric does not exceed two Brocade VDX switches, no VCS Fabric license is required.

(There is one exception if the other node has an Advanced Service bundle, see below for more details)

A two switch Brocade VCS Fabric license is built into NOS If there are more than two nodes in the fabric, each switch requires a VCS Fabric license

FCoE License Required to support FCoE on Brocade VDX 6720, 6730 and 8770 switches. Required to activate FC ports on Brocade VDX 6730-32 and 6730-76 switches. An FCoE license enables FCoE on the Brocade VDX platforms. Since FCoE requires a VCS fabric, a VCS Fabric license is a prerequisite for enabling FCoE in a fabric with more than two switches. In addition, a separate FCoE license is required to enable FCoE on a VCS edge port. Without an FCoE license, FCoE logins are not permitted, and all FCoE commands with the exception of a few commands return an error of No FCoE license present when executed. See release notes for more information on FC port activation and the FCoE license.

Ports on Demand (POD) License Brocade VDX 6720-24 and 6730-32: One 8-port POD license Base switch has 16 ports enabled Brocade VDX 6720-60 and 6730-76: Two 10-port POD licenses Base switch has 40 ports enabled

NOS v3.0.0 Licensing Supports all NOS 2.0/2.1/2.1.1-supported VDX 67XX platforms using same license rules for legacy

features

Supports new chassis-based platforms (VDX 8770)18 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition Legacy licensable features:- FCoE- Ports on Demand (POD1 and POD2) - not on VDX 6710 or VDX 8770- VCS Fabric

New licensable features in NOS v3.0.0- Layer 3 - only supported in VCS mode, no license required on VDX 67XX platforms- Advanced Services - (FCoE + VCS Fabric + Layer 3 bundle) - only on VDX 8770

Adding the Advanced Services bundle, which includes the VCS Fabric license, requires the other node in a 2-node fabric to have a VCS license 2013 Brocade Communications Systems, Inc. All Rights Reserved. 19

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionVCS FabricBrocade VCS Fabric technology leverages proven Fibre Channel fabric protocols to build a TRILL-based fabric.

TRILL DefinedTransparent Interconnection of Lots of Links (TRILL) provides a solution for shortest path frame routing for multihop Layer 2 Ethernet that:

Supports arbitrary topologies (ring, mesh, star) Uses link-state routing protocols

- The link-state protocol is performed by every switching node in the network. The basic concept of link-state routing is that every node constructs a map of the connectivity to the network, in the form of a graph, showing which nodes are connected to which other nodes. Each node then independently calculates the next best logical path from it to every possible destination in the network. The collection of best paths will then form the node's routing table.

- Brocade uses Fabric Shortest Path First (FSPF) as the routing protocol- TRILL is currently a proposed Internet Engineering Task Force (IETF) standard and is an Equal-Cost

Multi-Path (ECMP)-capable protocol. Equal-cost multi-path routing (ECMP) is a routing strategy where next-hop packet forwarding to a single destination can occur over multiple "best paths" which tie for top place in routing metric calculations. Multipath routing can be used in conjunction with most routing protocols, since it is a per-hop decision that is limited to a single router. It potentially offers substantial increases in bandwidth by load-balancing traffic over multiple paths

TRILL uses a link state-based control plane to form loop-free optimized paths between a source and destination. TRILL provides the following features:

Minimal configuration required Load balancing among multiple paths Forwarding loop mitigation without the need for STP Support of multiple points of attachment to the TRILL network. In STP, a single node with multiple

attachments to a single spanning tree segment always receives and sends traffic over only one of the those attachment points. TRILL must manage all traffic, including multicast and broadcast traffic, so as not to create traffic loops involving Ethernet segments with multiple TRILL attachment points. This includes multiple attachments to a single TRILL node and attachments to multiple TRILL nodes. Support for multiple attachments can improve support for forms of mobility that induce topology changes

Support for broadcast and multicast Behaves as a normal bridge to devices outside the TRILL network

TRILL FramesTRILL encapsulates an Ethernet frame to specify information to route the frame through the network (Standard frames as well as 2.5 k Mini Jumbo frames are also supported). See Figure15.20 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition Link transport header: A header that is added and removed at each hop and is used to specify the next hop RBridge and the transmitting RBridge This is also known as the outer MAC header

TRILL header: Specifies nickname values of the egress and ingress RBridges, unless the frame is multi-destined, in which case the egress nick.name specifies the root of the distribution tree on which the frame is being sent. The egress nickname is Equivalent to the RBridge ID.

FIGURE 15 TRILL Frame

Hop CountThe Hop Count field is a 6-bit unsigned integer. An RBridge drops frames received with a hop count of zero, otherwise it decrements the hop count. (This behavior is different from IPv4 and IPv6 in order to support the later addition of a trace route-like facility that would be able to get a hop count exceeded from an egress RBridge.)

For known unicast frames, the ingress RBridge should set the hop count in excess of the number of RBridge hops it expects to the egress RBridge to allow for alternate routing later in the path. For multi-destination frames, the hop count should be set by the ingress RBridge to at least the expected number of hops to the most distant RBridge.

To accomplish this, RBridge RBn calculates, for each branch from RBn of the specified distribution tree rooted at RBi, the maximum number of hops in that branch. Multi-destination frames are of particular danger because a loop involving one or more distribution tree forks could result in the rapid generation of multiple copies of the frame, even with the normal TTL mechanism. It is for this reason that multi-destination frames are subject to a stringent Reverse Path Forwarding Check and other checks. As an optional additional traffic control measure, when forwarding a multi-destination frame onto a distribution tree branch, transit RBridge RBn may decrease the hop count by more than 1, unless decreasing the hop count by more than 1 would result in a hop count insufficient to reach all destinations in that branch of the tree rooted at RBi. Using a hop count close or equal to the minimum needed on multi-destination frames provides additional protection against problems with temporary loops when forwarding. Although the RBridge MAY decrease the hop count of multi-destination frames by more than 1, under the circumstances described above, the RBridge forwarding a frame must decrease the hop count by at least 1, and discards the frame if it cannot do so because the hop count is 0. The option to decrease the hop count by more than 1 under the circumstances described above applies only to multi-destination frames, not to known unicast frames. 2013 Brocade Communications Systems, Inc. All Rights Reserved. 21

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionVCS Fabric FormationBrocade VCS Fabric technology leverages proven Fibre Channel fabric protocols to build a TRILL-based fabric. The main functions of the fabric formation protocols are:

Confirming that each switch in the VCS fabric is assigned the same VCS ID Confirming that each switch in the VCS fabric is assigned a unique RBridge ID Create a network topology database using a link state routing protocol (FSPF) Compute a broadcast tree to distribute fabric broadcast and multicast trafficBefore connecting VDX switches, the following should be configured:

Set the VCS Fabric ID Set the RBridge ID Enable VCS Fabric mode

Automatic Layer 2 Adjacency FormationBrocade VCS Fabric forms adjacencies with its directly connected Brocade VCS Fabric-enabled neighboring switches. See Figure16.

These adjacencies are called Inter Switch Links (ISLs) The ISLs can be separate links or Brocade proprietary VCS Fabric trunks

FIGURE 16 Layer 2 Adjacency Formation22 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionRBridge ID ConflictsIf there is a conflict for an RBridge ID, one of the offending switches must have their RBridge ID changed

Valid RBridge IDs are 1-239 An RBridge ID change requires a reboot of the switch.The switch needs to be rebooted because an

RBridge ID change requires a reset to all configuration databases to accommodate the new RBridge ID

VCS Edge Port ConfigurationVCS fabric edge ports are switch ports used to connect to external devices including end-stations or non-VCS Fabric mode switches or routers.

Standard Layer 2 configuration options are available (i.e. VLANs and LAGs) STP is not configurable on edge ports. STP is not configurable on edge ports, but VCS will pass BPDU

traffic through the fabric. STP sees the fabric as a wire.

When an edge port converts to an ISL port, the edge port configuration is ignored. Similarly, when an ISL port converts to an edge port, all the user provisioned edge port configuration for that port is activated.

LACP can be configured on edge ports only with either Brocade or Standard types. If Brocade is chosen, so that Link Reset (LR) primitives are exchanged properly, make sure the edge peering device is a Brocade Converged Network Adapter (CNA), a standalone Brocade VDX switch, or a Brocade 8000 switch. 2013 Brocade Communications Systems, Inc. All Rights Reserved. 23

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionL2 Forwarding

FSPF Protocol for ECMPBrocade VCS Fabrics uses the FSPF routing protocol to distribute link-state information of all ISLs. FSPF is a Link State Path Selection protocol, similar to OSPF, which is an Interior Gateway Protocol (IGP) widely used in IP networks. FSPF keeps track of the state of the links on all switches in the Fabric. It also associates a cost with each link. The protocol computes paths from a switch to all the other switches in the fabric, by adding the cost of all the links traversed by the path, and choosing the path that minimizes the cost. See Figure17.

FSPF is similar to Layer 3 routing protocols like OSPF. Although it has roots from OSPF, FSPF only defines and implements point-to-point links. In other words, there is no concept of a designated router (DR) and a backup designated router (BDR), areas or summarization, or anything similar like that being managed in FSPF.

FSPF forms a single adjacency per fabric trunk.

FIGURE 17 FSPF for ECMP

TABLE 3 FSPF Key to Figure17From RB3

Destination RB Outgoing Interface

RB1 te 3/0/1, te 3/0/3

RB2 te 3/0/1, te 3/0/3

RB4 te 3/0/1, te 3/0/3

RB5 te 3/0/1

RB6 te 3/0/324 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionVCS Layer 2 ECMPWith NOS v2.0.0a, up to 8 ECMP paths are supported per switch. When multiple ECMP paths exist, the traffic is load balanced across all available equal cost paths based on a hash from fields in the frames. The hash algorithm can be changed using the fabric ecmp load-balance command. For more information see the.Network OS Administrators Guide Supporting Network OS v2.1.1 manual. See Figure18.

IP: MAC DA, MAC SA, VLAN, IP DA, IP SA, TCP/UDP ports FCoE: Input port, MAC DA, MAC SA, VLAN, D_ID, S_ID, OX_ID Other: MAC DA, MAC SA, VLAN

FIGURE 18 ECMP Paths

VCS Fabric L2 Multi-Destination TreeVCS Fabric technology uses FSPF to calculate a loop-free multi-destination tree root switch. The multi-destination tree is calculated after the unicast routes are computed. The following rules determine the multi-destination root RBridge:

RBridge ID with the highest priority. The default multicast RBridge priority is zero. The default can be changed using the fabric route mcast RBridge ID priority command. The priority range is 0 through 255.

Lowest RBridge IDWhen multiple links are available between two nodes in the multicast tree The one with the highest BW at that instant is selected. Like if there is a ISL with 40G and a ISL with 20G then the 40G is chosen. If there are more than one equal bandwidth ISLs, then the port on the lowest switch WWN and lowest port will be chosen. The following rules determine which down stream links will be selected.

Highest bandwidth Lowest WWN - lowest port on that switchIn Figure19, RB5 is not the lowest RBridge number but it has been assigned the highest priority 2013 Brocade Communications Systems, Inc. All Rights Reserved. 25

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 19 VCS Fabric L2 Multi-Destination Tree

Multi-cast Tree ChangesThe following scenarios cause changes to multi-destination tree:

Root failure or removal from the VCS fabric All possible primary links to the neighbor RB closest to the multi-destination root are down Increasing the priority of an existing RBridge to be the highest priority will cause the tree to change/

rebuild

Data Path

Traffic TypesThe data path through the fabric changes depending on the type of traffic. Traffic is classified into two distinct types:

Known unicast (destination MAC is known) Broadcast, unknown unicast, or multicast

VCS Known Unicast Data PathFor a known unicast frame, no flooding occurs when the one of the following occurs:

MAC has been previously learned RBridges know the RB that is hosting the MAC FSPF knows the shortest path to get to the destination RBridge because the routing topology has already

been created26 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionLearning of a source MAC to an ingress RBridge is done through the data plane. The ingress RBridge encapsulates the native Ethernet frame with TRILL headers until the frame reaches the destination RBridge. The outer MAC header is changed at every hop. The frame enters the fabric through the ingress RB (RB1). The source MAC (ES_A) is learned by RB1. See Figure20.

FIGURE 20 Unicast Ethernet Frame Example

The ES_B is a known MAC and RB1 knows that MAC ES_B is hosted by RB2 RB1 adds a TRILL header with a egress RB nickname of RB2. See Figure21. 2013 Brocade Communications Systems, Inc. All Rights Reserved. 27

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 21 Unicast Ethernet Frame with TRILL Header Added

RB1 uses the FSPF routing table to determine the path to RB2. See Figure22. RB1 adds the link transport header which specifies:

- The next hop RB for the path (RB3)- Itself as the Outer MAC SA 28 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 22 TRILL Ethernet Frame: Link Transport Header Added

As a transit switch, RB3 removes the previous link transport header and replaces it with one that specifies the next hop RB. See Figure23. 2013 Brocade Communications Systems, Inc. All Rights Reserved. 29

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 23 TRILL Ethernet Frame: Data Path

The frame arrives at the egress RB, RB2 RB2 removes the TRILL header and delivers the frame to the end station. See Figure23. 30 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 24 Ethernet Frame: End of Data Path

VCS Multicast Data Path Unknown MAC, broadcast, or multicast traffic enters the fabric Traffic is flooded and sent to multicast root. See Figure25. 2013 Brocade Communications Systems, Inc. All Rights Reserved. 31

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 25 VCS Multicast Data Path

IGMP Snooping in a VCS FabricIGMP snooping is the process of listening to Internet Group Management Protocol (IGMP) network traffic. IGMP snooping, as implied by the name, is a feature that allows a Layer 2 switch to listen in on the IGMP conversation between hosts and routers. By listening to these conversations the switch maintains a map of links that need specific IP multicast streams. Multicasts may then be sent only to the links that need them.All IGMP snooped traffic rides on the fabric multicast tree. See Figure26.

All multicast traffic known and unknown is sent to all RBridges through the multicast tree All unknown multicast groups are flooded devices attached to the fabric If traffic for a known multicast group does not have any receivers on that RBridge then the traffic is

dropped32 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 26 IGMP Snooping in a VCS Fabric

VCS Fabric Edge Loop DetectionEdge-loop detection (ELD) detects and disables loops that would cause broadcast storms. These loops are generally caused by misconfigurations. See Figure27.

ELD is configured and enabled on Brocade VCS Fabrics ELD is not supported on switches in standalone mode ELD requires configuration at the global and interface levels 2013 Brocade Communications Systems, Inc. All Rights Reserved. 33

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 27 VCS Fabric Edge Loop Detection

How Edge Loop Detection WorksELD works by multicasting Protocol Data Unit (PDU) packets on edge ports. If a port is blocked by STP or another L2 protocol, ELD pdus are not processed or flooded on that port. A RBridge recognizes a loop when it receives a PDU that it initiated. If an ELD pdu is received on a port/VLAN that is not configured with ELD, the pdu will be dropped. Once an RBridge recognizes that a loop exists, it shuts down a port and breaks the loop.

In the case of LAG ports, ELD pdu are sent out on the active primary port of a lag group. ELD protocol will shut down all the member ports of the lag if an ELD pdu is received on any member port.

In case of vLAG ports, ELD pdu are sent out on the active primary port of the vLAG. ELD protocol will shut down all the member ports of the vLAG on the RBridge receiving the pdu. If there are other member ports exist on any other RBridge, they will still be up. This way loop will be broken and connectivity will still be maintained.

VLAN Configuration Guidelines and RestrictionsFollow these guidelines and restrictions when configuring VLANs:34 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionIn an active topology, MAC addresses can be learned, per VLAN, using Independent VLAN Learning (IVL) only. A MAC address ACL always overrides a static MAC address entry. In this case, the MAC address is the forwarding address and the forwarding entry can be overwritten by the ACL. The Brocade DCB switch supports Ethernet DIX frames and 802.2 LLC SNAP encapsulated frames only. You must configure the same native VLAN on both ends of an 802.1q trunk link. Failure to do so can cause bridging loops and VLAN leaks. All switches in a Brocade VCS Fabric cluster must be configured with the same VLAN number.

The default VLAN id is 1.

VCS Fabric and STPIn VCS mode the VDX switch operates in transparent mode with respect to spanning tree.

STP BPDUs are tunneled across the multicast tree All edge switches see each others BPDU Meaning the VCS Fabric will just forward the BPDU on edge ports

Enabling Port Fast (STP)From the DCB interface, use this command to enable port fast on an interface to allow the interface to quickly transition to the forwarding state. Port fast immediately puts the interface into the forwarding state without having to wait for the standard forward time.

If you enable the portfast bpdu-guard option on an interface and the interface receives a BPDU, thesoftware disables the interface and puts the interface in the ERR_DISABLE state.

BUM Storm Control A traffic storm occurs when packets flood the LAN, creating excessive traffic and degrading network performance. Broadcast, unicast and unknown multicast (BUM) storm control can prevent disruptions on Layer 2 physical ports. BUM storm control allows you to limit the amount of broadcast, unknown unicast, and multicast ingress traffic on a specified interface or on the entire system. All traffic received in excess of the configured rate gets discarded. You also have the option to specify whether to shutdown an interface if the maximum defined rate is exceeded within a five-second sampling period. When a port is shutdown, you receive a log message. You must then manually re-enable the interface using the no shut command.

Considerations BUM storm control is supported in VCS or standalone mode. BUM storm control replaces the multicast rate-limit feature for VDX 8770-4 and 8770-8, and later

platforms. This command is not supported on VDX 6XXX modules, such as the VDX 6710, 6720, and 6730.

BUM storm control must be configured on one of the following physical interface:- tengigabitethernet- gigabitethernet- fortygigabitethernet 2013 Brocade Communications Systems, Inc. All Rights Reserved. 35

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionL3: Routing

VCS Fabric - Layer 3 RoutingL3 functionality in a VCS Fabric operates on an individual switch basis as a separate router

L3 commands need to be issued on a per switch basisRouting is enabled on an a per RBridge basis

From a routing perspective VCS provides a flat L2 topology- Meaning that all routers enabled on a given VLAN appear as if they are on the same subnet

VCS Fabric L3 Routing ExampleIn the example, See Figure28 we have a 3 node VCS Fabric. Only RB1 has L3 routing enabled. RB2 and RB3 are acting as L2 switches. There are two VLANs active. VLAN 10 (red) and VLAN 20 (blue). The router (RB1) has Ves for VLAN 1 and 2 configured.

We assume that ARP has already been resolved and H1 knows how to reach H2 via RB1. The router RB1 has proxy-arp turned on so that it can respond to ARP requests for either VLAN 10 or 20 on any interface that has routing enabled.

H1 now wants to talk to H2.

Stage 1:

Before the packet enters the VCS Fabric notice:

The destination MAC address for the packet as M3 (RB1s Ve MAC address) The IP Dst Addr is that of H2. As in a regular stand alone switch all Ves within a single RBridge share the same MAC address. This can

be noted by observing that both VLAN 10 and 20 on RB1 have the same MAC address

The packet has an IP TTL of 63 The packet is tagged on VLAN 10 (red) Stage 2:

The packet first hits RB2 which is a pure L2 switch.

RB2 looks at its L2 forwarding tables and determines that the frame needs to be forwarded to destined MAC address M3 on VLAN 10 (red)

RB2 will encapsulate the frame in a TRILL frame and forward it along to its next hop (RB1). The outer header has the destination MAC address of Mc (Te 1/0/1 on RB1) The outer header has a source MAC address of Ma (Te 2/0/1 on RB2) The TRILL header has a Egress of RB1 and a Ingress of RB2 The inner header is untouched at this point Stage 3:36 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First Edition At this point the packet has reached RB1

RB1 determines the packet is destined to itself looking at the Egress RBridge address RB1 and the outer MAC address of Mc.

It decapsulates the packet and looks at the inner VLAN tag of VLAN 10 and inner MAC address M3 and realizes that M3 is its own (Ve) MAC address as well as the fact that routing is turned on VLAN 10

It looks at the inner packet and performs an IP lookup Based on the IP lookup it determines the next hop of the packet It changes the inner destination MAC address to M2 (the address of H2) It changes the inner source MAC address to M3 (Its own Ve MAC address) It decrements the TTL by 1 The router also realizes that the destination is in the VCS Fabric It will now encapsulate this packet into a TRILL header The TRILL encapsulation will include the egress of RB3 and ingress RB1 (itself) The outer header has the destination MAC address of Mb which is the MAC address of Te 3/0/1 (on RB3) The outer header has the source MAC address of Md which is the MAC address for Te 1/0/2 (on RB1) Stage 4:

At RB3, the switch looks at the outer MAC address Mb and egress RB3 realizes the packet is destined for itself

It decapsulates the packet and looks at the inner VLAN tag of 20 (blue) and inner destination mac address M2 and realizes that it needs to do a L2 lookup

Assuming the L2 table is already populated it will know that it needs to forward the packet to host H2 It will forward the decapsulated packet towards H2 2013 Brocade Communications Systems, Inc. All Rights Reserved. 37

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionFIGURE 28 VCS Fabric L3 Routing Example

How OSPF works in VCS FabricEach RBridge running OSPF in VCS Fabric exchanges routes with neighbors

OSPF neighbor can be inside or outside VCSAn exchange between switches within the fabric encapsulates the frame inside a TRILL frame. Within VCS Fabric, OSPF must be enabled on Ve ports.

OSPF neighbor outside VCS can be connected over:

Ve Router (edge) port Port Channel

To create the VLAN used for OSPF the VLAN must be previously created and match the configured Ve number.38 2013 Brocade Communications Systems, Inc. All Rights Reserved.

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionAssigning OSPF areasOnce OSPF is enabled on the system, you can assign areas. Assign an IP address or number as the area ID for each area. The area ID is representative of all IP addresses (subnets) on a router port. Each port on a router can support one area.

An area can be normal, a stub, or a Not-So-Stubby Area (NSSA):

Normal - OSPF routers within a normal area can send and receive External Link State Advertisements (LSAs).

Stub - OSPF routers within a stub area cannot send or receive External LSAs. In addition, OSPF routers in a stub area must use a default route to the area's Area Border Router (ABR) or Autonomous System Boundary Router (ASBR) to send traffic out of the area.

NSSA - The ASBR of an NSSA can import external route information into the area. ASBRs redistribute (import) external routes into the NSSA as type 7 LSAs. Type-7 External LSAs are a special type of LSA generated only by ASBRs within an NSSA, and are flooded to all the routers within only that NSSA.

ABRs translate type 7 LSAs into type 5 External LSAs, which can then be flooded throughout the AS. You can configure summary-addresses on the ABR of an NSSA so that the ABR converts multiple type-7 External LSAs received from the NSSA into a single type-5 External LSA.

When an NSSA contains more than one ABR, OSPF elects one of the ABRs to perform the LSA translation for NSSA. OSPF elects the ABR with the highest router ID. If the elected ABR becomes unavailable, OSPF automatically elects the ABR with the next highest router ID to take over translation of LSAs for the NSSA. The election process for NSSA ABRs is automatic.

Configuring a Summary-Address for the NSSAIf you want the ABR that connects the NSSA to other areas to summarize the routes in the NSSA before translating them into Type-5 LSAs and flooding them into the other areas, configure a summary-address. The ABR creates an aggregate value based on the summary-address. The aggregate value becomes the address that the ABR advertises instead of advertising the individual addresses represented by the aggregate. You can configure up to 32 ranges in an OSPF area.

To configure a summary-address in NSSA 2.2.2.2 (This example assumes that you have already configured NSSA 2.2.2.2.):

1. In privileged EXEC mode, issue the configure command to enter global configuration mode.

2. Enter the rbridge-id command followed by the RBridge ID to enter Rbridge sub-configuration mode.

3. Enter the router ospf command to enable OSPF on the router and to enter router configuration mode.

4. Enter the area operand followed by the area ID, then enter the nssa operand followed by the nssa ID.

5. Enter the summary-address command followed by the IP address and mask for the summary route.

switch# configureswitch(config)# rbridge-id 101switch(config-rbridge-id-101)# router ospfswitch(conf-ospf-router)# area 2.2.2.2 nssa 10switch(conf-ospf-router)# summary-address 209.157.1.1 255.255.0.0 2013 Brocade Communications Systems, Inc. All Rights Reserved. 39

Brocade Certified Ethernet Fabric Professional in a Nutshell First EditionUsing Designated RoutersIn an OSPF broadcast network, OSPF elects one router to serve as the designated router (DR) and another router on the segment to act as the backup designated router (BDR). This minimizes the amount of repetitive information that is forwarded on the network. OSPF forwards all messages to the designated router. Backup designated routers forward updates throughout the network.

On broadcast networks such as LAN links, all routers on the LAN other than the DR and BDR form full adjacencies with the DR and BDR and pass LSAs only to them. The DR forwards updates received from one neighbor on the LAN to all other neighbors on that same LAN. One of the main functions of a DR is to ensure that all the routers on the same LAN have identical LSDBs. Therefore, on broadcast networks, an LSDB is synchronized between a DROther (a router that is not a DR or a BDR) and its DR and BDR.

If the DR goes off-line, the BDR automatically becomes the DR. The router with the next highest priority becomes the new BDR.

If two neighbors share the same priority, the router with the highest

brocade

Documents