b cisco nx-os fcoe configuration guide
DESCRIPTION
CISCOTRANSCRIPT
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500First Published: July 12, 2011
Last Modified: November 01, 2012
Americas HeadquartersCisco Systems, Inc.170 West Tasman DriveSan Jose, CA 95134-1706USAhttp://www.cisco.comTel: 408 526-4000 800 553-NETS (6387)Fax: 408 527-0883
Text Part Number: OL-25809-02
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITEDWARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITHTHE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain versionof the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDINGANYOTHERWARRANTYHEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS"WITH ALL FAULTS.CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OFMERCHANTABILITY, FITNESS FORA PARTICULAR PURPOSEANDNONINFRINGEMENTORARISING FROMACOURSEOFDEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUTLIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERSHAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnershiprelationship between Cisco and any other company. (1110R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shownfor illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental.
© 2011-2012 Cisco Systems, Inc. All rights reserved.
C O N T E N T S
P r e f a c e Preface vii
Audience vii
Document Conventions vii
Related Documentation viii
Obtaining Documentation and Submitting a Service Request ix
P r e f a c e New and Changed Information xi
Cisco Nexus 7000: New and Changed Information xi
Cisco MDS 9500: New and Changed Information xii
C H A P T E R 1 Overview 1
Information About Fibre Channel Over Ethernet 1
Fibre Channel Forwarder 1
Fibre Channel Bridge 2
C H A P T E R 2 FCoE 3
Information About FCoE 3
FCoE and FIP 3
FCoE Initiation Protocol 3
FIP Virtual Link Instantiation 4
FCoE Frame Format 4
VLAN Tagging for FCoE Frames 4
FIP Ethernet Frame Format 5
DCBX 5
Data Center Bridging Exchange Protocol 5
Lossless Ethernet 6
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 iii
Logical Link Up/Down 6
Converged Network Adapters 6
STP Lite 7
Default Settings for FCoE 7
Licensing Requirements for FCoE 7
Guidelines and Limitations 8
Cisco Nexus 7000: Guidelines and Limitations 8
Cisco MDS 9500: Guidelines and Limitations 9
Configuring FCoE 10
Enabling FCoE 10
Cisco Nexus 7000: Enabling FCoE 10
Cisco MDS 9500: Enabling FCoE 11
Disabling FCoE 13
Cisco Nexus 7000: Disabling FCoE 13
Cisco MDS 9500: Disabling FCoE 14
Configuring the FC-Map 15
Configuring the Fabric Priority 16
Setting the Advertisment Interval 17
Disabling LAN Traffic on an FCoE Link 18
Verifying the FCoE Configuration 19
Additional References for FCoE 20
C H A P T E R 3 Configuring FCoE VLANs and Virtual Interfaces 21
Information About Virtual Fibre Channel (VFC) Interfaces 21
VF Port 21
VE Port 22
Default Settings for FCoE 22
Licensing Requirements for FCoE 22
Guidelines and Limitations 23
Cisco Nexus 7000: Guidelines and Limitations 23
Cisco MDS 9500: Guidelines and Limitations 24
Configuring Virtual Interfaces 25
Mapping a VSAN to a VLAN 25
Creating a Virtual Fibre Channel Interface 26
Creating a Virtual Fibre Channel Port Channel Interface 28
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500iv OL-25809-02
Contents
Associating a Virtual Fibre Channel Interface to a VSAN 29
Enabling VE Loopback Configuration 31
Verifying the Virtual Interface 32
Example:Mapping VSANs to VLANs 33
Additional References for FCoE 34
C H A P T E R 4 Storage VDC on Cisco Nexus 7000 Series Switches 37
Information About Storage VDC 37
Licensing Requirements for FCoE 38
Guidelines and Limitations 38
Cisco Nexus 7000: Guidelines and Limitations 38
Cisco MDS 9500: Guidelines and Limitations 39
Configuring FCoE VDCs 40
Creating a Dedicated Storage VDC 40
Cisco Nexus 7000:Licensing an FCoE Module 42
Allocating the FCoE VLAN Range 43
Allocating Shared Interfaces 44
Example: Storage VDC Configuration 46
A P P E N D I X A Feature History by Platform 49
Cisco Nexus 7000:Supported Features 49
Cisco MDS 9500:Supported Features 49
A P P E N D I X B Guidelines and Limitations 51
Verified Limits for FCoE for the Cisco Nexus 7000 Series and Cisco MDS 9500 51
Cisco Nexus 7000: Guidelines and Limitations 51
Cisco MDS 9500: Guidelines and Limitations 53
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 v
Contents
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500vi OL-25809-02
Contents
Preface
This preface describes the audience, organization, and conventions of theCisco NX-OS FCoE ConfigurationGuide. It also provides information on how to obtain related documentation.
• Audience, page vii
• Document Conventions, page vii
• Related Documentation, page viii
• Obtaining Documentation and Submitting a Service Request, page ix
AudienceThis publication is for experienced users who configure and maintain Cisco NX-OS devices.
Document ConventionsCommand descriptions use the following conventions:
DescriptionConvention
Bold text indicates the commands and keywords that you enter literallyas shown.
bold
Italic text indicates arguments for which the user supplies the values.Italic
Square brackets enclose an optional element(keyword or argument).[x]
Square brackets enclosing keywords or arguments separated by a verticalbar indicate an optional choice.
[x | y]
Braces enclosing keywords or arguments separated by a vertical barindicate a required choice.
{x | y}
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 vii
DescriptionConvention
Nested set of square brackets or braces indicate optional or requiredchoices within optional or required elements. Braces and a vertical barwithin square brackets indicate a required choice within an optionalelement.
[x {y | z}]
Indicates a variable for which you supply values, in context where italicscannot be used.
variable
A nonquoted set of characters. Do not use quotation marks around thestring or the string will include the quotation marks.
string
Examples use the following conventions:
DescriptionConvention
Terminal sessions and information the switch displays are in screen font.screen font
Information you must enter is in boldface screen font.boldface screen font
Arguments for which you supply values are in italic screen font.italic screen font
Nonprinting characters, such as passwords, are in angle brackets.< >
Default responses to system prompts are in square brackets.[ ]
An exclamation point (!) or a pound sign (#) at the beginning of a lineof code indicates a comment line.
!, #
This document uses the following conventions:
Means reader take note. Notes contain helpful suggestions or references to material not covered in themanual.
Note
Means reader be careful. In this situation, you might do something that could result in equipment damageor loss of data.
Caution
Related DocumentationThe Cisco Nexus 7000 Series documentation is available at the following URL:
http://www.cisco.com/en/us/products/ps9402/tsd_products_support_series_home.html
The Cisco MDS 9500 Series documentation is available at the following URL:
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500viii OL-25809-02
PrefaceRelated Documentation
http://www.cisco.com/en/US/products/ps5989/tsd_products_support_series_home.html
Obtaining Documentation and Submitting a Service RequestFor information on obtaining documentation, submitting a service request, and gathering additional information,see the monthlyWhat's New in Cisco Product Documentation, which also lists all new and revised Ciscotechnical documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
Subscribe to theWhat's New in Cisco Product Documentation as a Really Simple Syndication (RSS) feedand set content to be delivered directly to your desktop using a reader application. The RSS feeds are a freeservice and Cisco currently supports RSS version 2.0.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 ix
PrefaceObtaining Documentation and Submitting a Service Request
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500x OL-25809-02
PrefaceObtaining Documentation and Submitting a Service Request
New and Changed Information
This chapter includes the following sections:
• Cisco Nexus 7000: New and Changed Information, page xi
• Cisco MDS 9500: New and Changed Information, page xii
Cisco Nexus 7000: New and Changed InformationThis section provides release-specific information for each new and changed features documented for theCisco Nexus 7000 Series switches.
The latest version of this document is available at the following Cisco website:
http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html
To check for additional information about Cisco Nexus 7000 software, see theCisco Nexus 7000 Series NX-OSRelease Notes available at the following Cisco website:
http://www.cisco.com/en/US/products/ps9670/prod_release_notes_list.html
Table 1: New and Changed Features
Changed in ReleaseDescriptionFeature
6.1(2)FCoE support on F2e Series modulesF2e Series Module support (SPF+ only)
6.1(1)FCoE support on F2 Series modulesF2 Series Module support
6.0(1)—No changes in this release
5.2(1)Initial FCoE supportFCoE
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 xi
Cisco MDS 9500: New and Changed InformationThis section provides release-specific information for each new and changed features documented for theCisco MDS 9500 Series switches.
The latest version of this document is available at the following Cisco website:
http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html
To check for additional information about Cisco MDS 9500 software, see the Cisco MDS 9000 Series NX-OSRelease Notes available at the following Cisco website:
http://www.cisco.com/en/US/products/ps9670/prod_release_notes_list.html
Table 2: New and Changed Features
Where DocumentedChanged in ReleaseDescriptionFeature
———There are no changes sinceRelease 5.2(1)
This document5.2(1)Initial FCoE supportFCoE
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500xii OL-25809-02
New and Changed InformationCisco MDS 9500: New and Changed Information
C H A P T E R 1Overview
This chapter includes the following sections:
• Information About Fibre Channel Over Ethernet, page 1
• Fibre Channel Forwarder, page 1
• Fibre Channel Bridge, page 2
Information About Fibre Channel Over EthernetFibre Channel over Ethernet (FCoE) allows Fibre Channel traffic to be encapsulated over a physical Ethernetlink. FCoE and FIP frames use a unique EtherType so that FCoE traffic and standard Ethernet traffic can becarried on the same link.
Classic Ethernet is a best-effort protocol, which means that in a congested network, Ethernet discards packetsand relies on higher level protocols to provide retransmission and other reliability mechanisms.
Fibre Channel traffic requires a lossless transport layer; as a data storage protocol, it is unacceptable to losea single data packet. Native Fibre Channel implements a lossless service at the transport layer using abuffer-to-buffer credit system.
Ethernet links on Cisco NX-OS switches provide twomechanisms to ensure lossless transport for FCoE traffic:link-level flow control and priority flow control.
IEEE 802.3x link-level flow control allows a congested receiver to signal the far end to pause the datatransmission for a short period of time. The pause functionality is applied to all traffic on the link.
The priority flow control (PFC) feature on Cisco NX-OS platforms applies pause functionality to specificclasses of traffic on the Ethernet link. For example, PFC can provide lossless service for the FCoE traffic andbest-effort service for the standard Ethernet traffic using IEEE 802.1p traffic classes.
Fibre Channel ForwarderFibre Channel traffic requires a lossless transport layer, which means that it requires a data storage protocolthat does not lose a single data packet. A FCoE Forwarder (FCF) communicates with FCoE end devices, suchas converged network adapters (CNAs). The FCF accepts the Fibre Channel frame that is encapsulated in anEthernet packet and forwards that packet over a VLAN across an Ethernet network to a remote FCoE end
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 1
device. The FCF has a unique FC-MAC address in the FCoE network. An FCF can also assign FC addressIDs to the CNAs. An FCF can assign Fabric Provide MAC Addresses (FPMA) to the CNAs consisting of theFC-Map Value for the Fabric and the Fibre Channel IDs (FC IDs) assigned during Fabric Login.
Fibre Channel BridgeA Fibre Channel Bridge connects an FCoE network to a Fibre Channel network. A Fibre Channel Bridgedecapsulates an FCoE frame and sends the Fibre Channel frame to the Fibre Channel network. A Fibre ChannelBridge also encapsulates FC frames from a Fibre Channel network and forwards them to the FCoE network.
Cisco MDS 9500 switches support Fibre Channel Bridge functions.
The Cisco Nexus 7000 series switches do not offer native Fibre Channel ports and do not support FCBridge functions.
Note
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 95002 OL-25809-02
OverviewFibre Channel Bridge
C H A P T E R 2FCoE
This chapter includes the following sections:
• Information About FCoE, page 3
• Default Settings for FCoE, page 7
• Licensing Requirements for FCoE, page 7
• Guidelines and Limitations, page 8
• Configuring FCoE, page 10
• Verifying the FCoE Configuration, page 19
• Additional References for FCoE, page 20
Information About FCoEFCoE provides a method of transporting Fibre Channel traffic over a physical Ethernet connection. FCoErequires the underlying Ethernet to be full duplex and to provide lossless behavior for Fibre Channel traffic.
Lossless behavior on Ethernet is provided by using a priority flow control (PFC) mechanism that preventspacket loss during congestion conditions.
Note
The Cisco NX-OS software supports T11-compliant FCoE on all 10-Gigabit Ethernet interfaces.
FCoE and FIP
FCoE Initiation ProtocolThe FCoE Initialization Protocol (FIP) allows the switch to discover and initialize FCoE-capable entities thatare connected to an Ethernet LAN. Cisco NX-OS switches support the Converged Enhanced Ethernet DataCenter Bridging Exchange (CEE-DCBX) protocol for T11-compliant Generation 2 CNAs.
The following switches do not support Pre-FIP on Generation 1 CNAs:
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 3
• Cisco Nexus 7000
• Cisco MDS 9500
FIP Virtual Link InstantiationFIP is used to perform device discovery, initialization, and link maintenance. FIP performs the followingtasks:
• FIP VLAN discovery— Discovers the FCoE VLAN that will be used by all other FIP protocols as wellas by the FCoE encapsulation for Fibre Channel payloads on the established virtual link. FIP VLANdiscovery occurs in the native VLAN used by the initiator or target to exchange Ethernet traffic. TheFIP VLAN discovery protocol is the only FIP protocol running on the native VLAN; all other FIPprotocols run on the discovered FCoE VLANs.
• FIP FCF discovery—When a FCoE device is connected to the fabric, it sends a Discovery Solicitationmessage. A Fibre Channel Forwarder (FCF) or a switch responds to the message with a SolicitedAdvertisement that provides an FCF MAC address to use for subsequent logins.
• FCoE virtual link instantiation— FIP defines the encapsulation of fabric login (FLOGI), fabric discovery(FDISC), logout (LOGO), and exchange link parameter (ELP) frames along with the correspondingreply frames. The FCoE devices use these messages to perform a fabric login.
• FCoE virtual link maintenance— FIP periodically sends maintenance messages between the switch andthe CNA to ensure that the connection is still valid.
FCoE Frame FormatFCoE is implemented when the switch encapsulates a Fibre Channel frame in an Ethernet packet with adedicated Ethernet type, 0x8906. The packet has a 4-bit version field. The other header fields in the frame(the source and destination MAC addresses, VLAN tags, and frame markers) are all standard Ethernet fields.Reserved bits pad the FCoE frame to the IEEE 802.3 minimum packet length of 64 bytes.
A Fibre Channel frame consists of 36 bytes of headers and up to 2112 bytes of data for a total maximum sizeof 2148 bytes. The encapsulated Fibre Channel frame has all the standard headers, which allow it to be passedto the storage network without further modification. To accommodate the maximum Fibre Channel frame inan FCoE frame, the class-fcoe is defined with a default Maximum Transmission Unit (MTU) of 2240 bytes.
VLAN Tagging for FCoE FramesThe Ethernet frames that are sent by the switch to the adapter include the IEEE 802.1Q tag. This tag includesa field for the class of service (CoS) value used by the priority flow control (PFC). The IEEE 802.1Q tag alsoincludes a VLAN field.
The switch expects frames from a FIP T11-compliant CNA to be tagged with the VLAN tag for the FCoEVLAN. Frames that are not correctly tagged are discarded.
You cannot map VLAN 1 or the native VLAN to an FCoE VSAN.Note
The following switches do not support Pre-FIP on Generation 1 CNAs:
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 95004 OL-25809-02
FCoEFCoE and FIP
• Cisco Nexus 7000
• Cisco MDS 9500
FIP Ethernet Frame FormatFIP is encapsulated in an Ethernet packet with a dedicated EtherType, 0x8914. The packet has a 4-bit versionfield. Along with the source and destination MAC addresses, the FIP packet also contains a FIP operationcode and a FIP operation subcode. The following table describes the FIP operation codes and subcodes.
Table 3: FIP Operation Codes
FIP OperationFIP SubcodeFIP Operation Code
Discovery Solicitation0x010x0001
Discovery Advertisement0x02
Virtual Link Instantiation Request0x010x0002
Virtual Link Instantiation Reply0x02
FIP Keepalive0x010x0003
FIP Clear Virtual Links0x02
FIP VLAN Request0x010x0004
FIP VLAN Notification0x02
DCBX
Data Center Bridging Exchange ProtocolThe Data Center Bridging Exchange (DCBX) protocol is an extension of the Link Layer Discovery Protocol(LLDP). DCBX end points exchange request and acknowledgment messages. For flexibility, parameters arecoded in a type-length-value (TLV) format. Cisco NX-OS switches support the Converged Enhanced EthernetData Center Bridging Exchange (CEE-DCBX) is supported on all T11-compliant Generation 2 CNAs.
DCBX runs on the physical Ethernet link between the switch and the CNA. By default, DCBX is enabled onEthernet interfaces. When an Ethernet interface is brought up, the switch automatically starts to communicatewith the CNA.
During the normal operation of FCoE between the switch and the CNA, DCBX provides link-error detection.
DCBX is also used to negotiate capabilities between the switch and the CNA and to send configuration valuesto the CNA.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 5
FCoEDCBX
The CNAs that are connected to a switch are programmed to accept the configuration values that are sent bythe switch, allowing the switch to distribute configuration values to all attached CNAs, which reduces thepossibility of configuration errors and simplifies CNA administration.
Lossless EthernetStandard Ethernet is a best-effort medium which means that it lacks any form of flow control. In the event ofcongestion or collisions, Ethernet drops packets. The higher level protocols detect the missing data andretransmit the dropped packets.
To properly support Fibre Channel, Ethernet has been enhanced with a priority flow control (PFC) mechanism.
Logical Link Up/DownOn a native Fibre Channel link, some configuration actions (such as changing the VSAN) require that youreset the interface status. When you reset the interface status, the switch disables the interface and thenimmediately reenables the interface.
Caution:If an Ethernet link provides FCoE service, do not reset the physical link because this action is disruptive toall traffic on the link.
The logical link up/down feature allows the switch to reset an individual virtual link. The logical link downis signaled with a FIP Clear Virtual Link message.
Caution:If the CNA does not support the logical link level up/down feature, the CNA resets the physical link, whichmeans that all traffic on the Ethernet interface is disrupted.
Converged Network AdaptersCisco NX-OS switches support the following CNA types:
• Hardware adapter
◦Works with the existing Fibre Channel host bus adapter (HBA) driver and Ethernet NetworkInterface Card (NIC) driver in the server.
◦Server operating system view of the network is unchanged; the CNA presents a SAN interface anda LAN interface to the operating system.
• FCoE software stack
◦Runs on existing 10-Gigabit Ethernet adapters.
The following Cisco NX-OS series and platforms support Generation 2 CNAs that use the FIP to exchangeinformation about its available capabilities and to negotiate the configurable values with the switch:
• Cisco Nexus 2232
• Cisco Nexus 5000
• Cisco Nexus 5500
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 95006 OL-25809-02
FCoELossless Ethernet
• Cisco Nexus 7000
• Cisco MDS 9500
To reduce configuration errors and simplify administration, the switch distributes the configuration data toall the connected adapters.
STP LiteFCoE does not require full Spanning Tree Protocol (STP) because FCoE has no bridging functionality, whichmeans that no STP loops are created in the network. STP Lite on FCoE interfaces ensures rapid convergenceacross the network by sending an agreement Bridge Protocol Data Unit (BPDU)whenever it receives a proposalBPDU. The FCoE link sends the identical agreement BPDU in response to either an Multiple Spanning Tree(MST) or a Per VLANRapid Spanning Tree Plus (PVRST+) proposal BPDU.Additionally, STP Lite suppressesthe MAC address flushing function for FCoE VLANs.
STP Lite is enabled automatically by default across the entire device for FCoE VLANs as soon as the firstFCoE VLAN comes up. At the same time, the system automatically converts all FCoE links as the STP-typenormal ports. This feature runs only in FCoE VLANs.
Default Settings for FCoEThis table lists the default settings for FCoE parameters.
Table 4: Default FCoE Parameter Settings
DefaultParameters
Not installed, disabledFCoE feature
0E.FC.00FC-Map
128Fabric priority
8 secondsAdvertisement interval
Licensing Requirements for FCoELicenseProduct
Each F Series module that runs FCoE requires anFCoE license. FCoE enabled in a nondefault VDCdoes not require the Advanced Services License. Fora complete explanation of the Cisco NX-OS licensingscheme and how to obtain and apply licenses, see theCisco NX-OS Licensing Guide.
Cisco Nexus 7000 Series
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 7
FCoESTP Lite
LicenseProduct
FCoE does not require a license. For a completeexplanation of the Cisco NX-OS licensing schemeand how to obtain and apply licenses, see the CiscoNX-OS Licensing Guide.
Cisco MDS 9500 Series
Guidelines and Limitations
Cisco Nexus 7000: Guidelines and Limitations• You cannot enable FCoE on default VLAN.
• The QoS policy must be the same on all Cisco FCoE switches in the network.
• Beginning with Cisco NX-OS Release 6.1, FCoE is supported on F2 and F2e Series modules.
◦FCoE supports only F2e (SFP+) modules.
◦FCoE does not support F2e (Copper) modules.
FCoE VDC
FCoE in a dedicated storage VDC has the following guidelines:
• Enable the FCoE feature set in only one VDC.
• Create VLANs in the FCoE allocated VLAN range.
• Do not enable any other features other than storage-related features in the dedicated FCoE VDC.
• Allocate resources for the dedicated FCoE VDC from an F Series module, such as the 32-port 1- and10-Gigabit Ethernet I/O module (PID N7K-F132XP-15) .
• Rollback is not supported in a storage VDC.
• FCoE support on F2 and F2e Series modules, SF248XP-25 requires Supervisor 2 module.
• F2 and F2e series modules cannot exist in the same VDC with any other module type. This applies toboth LAN and storage VDCs.
Shared Interfaces
Any change in protocol state that flaps the parent port of a shared interface because of any port featurealso affects the FCOE traffic on the storage vdc.
Note
The following interface config modes are not allowed while sharing an interface from Ethernet vdc to a storagevdc:
• SPAN destination
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 95008 OL-25809-02
FCoEGuidelines and Limitations
• Private VLAN mode
• Port-channel interfaces
• Access mode
• mac-packet-classify
• Interfaces that are part of a VLAN that has an associated QoS policy
Shared Ethernet interfaces must be in trunk mode and only shared with one other VDC.
Storage VDC
Configuring a VDC for the OOB management interface mgmt0 is accomplished with the vrf contextmanagement command. However, a storage VDC does not support VRF, so configuring mgmt0 requires adifferent approach.
The following table shows how to configure mgmt 0 for a VDC and for a storage VDC:
Configuring mgmt 0 for storage VDCConfiguring mgmt 0 for VDC
interface mgmt 0ip address mgmt0_ip_address mgmt0_subnet_maskno shutip route 0.0.0.0/0 default_gateway
The ip route command specifies the defaultroute that points to the default gateway.
Note
vrf context managementip route 0.0.0.0/0 default_gateway
where
• mgmt0_ip_address is the mgmt0 IPv4 address.
• mgmt0_subnet_mask is the mgmt0 IPv4 netmask.
• default_gateway is the IPv4 address of the default-gateway.
Formore information about VDC, see theCisco Nexus 7000 Series NX-OSVirtual Device Context ConfigurationGuide.
Cisco MDS 9500: Guidelines and Limitations• You cannot enable FCoE on the default VLAN.
• You can only enable FCoE on an FCoE module.
• The QoS policy must be the same on all Cisco FCoE switches in the network.
• Cisco MDS 9500 does not support LAN traffic and only accepts and processes FCoE frames.
• You must install a Supervisor-2A module to configure FCoE.
• You can install an FCoE module in a fabric that uses FC-Redirect services but do not install the FCoEmodule in the same switch where FC-Redirect services application modules are enabled.
• SME does not support any FCoE connected devices including devices connected through theMDS FCoElinecard (DX-X9708-K9)
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 9
FCoECisco MDS 9500: Guidelines and Limitations
• In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is runningDMM, SME, or IOA.
Configuring FCoE
Enabling FCoE
Cisco Nexus 7000: Enabling FCoEYou must install the FCoE feature set in the default VDC and enable dependent features in order to enableFCoE in a storage VDC.
Before You Begin
• Ensure you are in the default VDC.
• Ensure you have installed the correct license for your switch.
• Do not enable FCoE on VLAN 1 or the native VLAN.
SUMMARY STEPS
1. configure terminal2. install feature-set fcoe3. feature lldp4. (Optional) feature lacp5. system qos6. service-policy type network-qos policy-name7. (Optional) show feature8. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Installs the FCoE feature set in the default VDC.install feature-set fcoe
Example:switch(config)# install feature-set fcoe
Step 2
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950010 OL-25809-02
FCoEConfiguring FCoE
PurposeCommand or Action
Enables the Link Layer Discovery Protocol (LLDP) featurein the default VDC. This feature is required for FCoEoperation.
feature lldp
Example:switch(config)# feature lldp
Step 3
(Optional)Enables the Link Aggregation Control Protocol (LACP)feature in the default VDC. This feature is considered a bestpractice for FCoE operation.
feature lacp
Example:switch(config)# feature lacp
Step 4
Enters Quality of service (QoS) configuration mode.system qos
Example:switch(config)# system qos
Step 5
Enables the QoS policy that supports FCoE traffic. Thepolicy-name default is default-nq-8e-policy.
service-policy type network-qos policy-name
Example:switch(config-sys-qos)# service-policy typenetwork-qos default-nq-7e-policy
Step 6
(Optional)Displays information about the enabled features.
show feature
Example:switch(config-sys-qos)# show feature
Step 7
(Optional)Copies the running configuration to the startupconfiguration.
copy running-config startup-config
Example:switch(config-sys-qos)# copy running-configstartup-config
Step 8
What to Do Next
You must create a storage VDC and allocate resources to finish enabling FCoE.
Cisco MDS 9500: Enabling FCoEThe CiscoMDS 9500 switch automatically enables the FCoE feature when an FCoE-capable module is presentin the chassis. You must enable supporting features and the appropriate QoS policy for FCoE.
Before You Begin
• Do not enable FCoE on VLAN 1 or the native VLAN.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 11
FCoEEnabling FCoE
SUMMARY STEPS
1. configure terminal2. (Optional) feature lacp3. system qos4. service-policy type network-qos policy-name5. (Optional) show feature6. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
(Optional)Enables the Link Aggregation Control Protocol (LACP)feature. This feature is considered a best practice for FCoEoperation.
feature lacp
Example:switch(config)# feature lacp
Step 2
Enters Quality of Service (QoS) configuration mode.system qos
Example:switch(config)# system qos
Step 3
Enables the QoS policy that supports FCoE traffic. Thepolicy-name default is default-nq-6e-policy.
service-policy type network-qos policy-name
Example:switch(config-sys-qos)# service-policy typenetwork-qos default-nq-7e-policy
Step 4
(Optional)Displays information about the enabled features.
show feature
Example:switch(config-sys-qos)# show feature
Step 5
(Optional)Copies the running configuration to the startupconfiguration.
copy running-config startup-config
Example:switch(config-sys-qos)# copy running-configstartup-config
Step 6
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950012 OL-25809-02
FCoEEnabling FCoE
Disabling FCoE
Cisco Nexus 7000: Disabling FCoEYou can disable or uninstall the FCoE feature set. You can also disallow the FCoE feature set in a VDC.
Before You Begin
Ensure you are in the correct VDC.
SUMMARY STEPS
1. configure terminal2. vdc vdc_id type storage3. (Optional) no allow feature-set fcoe4. no feature-set fcoe5. exit6. no install feature-set fcoe7. (Optional) show feature-set8. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Enters VDC configuration mode. The vdc_id can be anycase-sensitive, alphanumeric string up to 32 characters.
vdc vdc_id type storage
Example:switch(config)# vdc fcoe type storageswitch(config-vdc)#
Step 2
(Optional)Disallows any FCoE in this VDC. The default is allow.
no allow feature-set fcoe
Example:switch(config-vdc)# no allow feature-set fcoeswitch(config-vdc)#
Step 3
Disables the FCoE feature set. The no feature-set fcoecommand may take some time to complete if the size of the
no feature-set fcoe
Example:switch(config-vdc)# no feature-set fcoe
Step 4
configuration is very large. The command must clean up allof the configuration associated with the FCoE feature set.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 13
FCoEDisabling FCoE
PurposeCommand or Action
Exits VDC configuration mode.exit
Example:switch(config-vdc)# exitswitch(config)#
Step 5
Uninstalls the FCoE feature set. Use this command in thedefault VDC after you disable the FCoE feature set.
no install feature-set fcoe
Example:switch(config)# no install feature-set fcoe
Step 6
(Optional)Displays information about the feature sets.
show feature-set
Example:switch(config)# show feature-set
Step 7
(Optional)Copies the running configuration to the startup configuration.
copy running-config startup-config
Example:switch(config)# copy running-configstartup-config
Step 8
Cisco MDS 9500: Disabling FCoEAfter you disable the FCoE feature, all FCoE commands are removed from the CLI and the FCoE configurationis deleted.
SUMMARY STEPS
1. configure terminal2. no feature-set fcoe
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Disables the FCoE feature set. The no feature-set fcoe commandmay take some time to complete if the size of the configuration is
no feature-set fcoe
Example:switch(config)# no feature-set fcoe
Step 2
very large. The command must clean up all of the configurationassociated with the FCoE feature set.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950014 OL-25809-02
FCoEDisabling FCoE
This example shows how to disable FCoE on the switch:switch# configure terminalswitch(config)# no feature-set fcoe
Configuring the FC-MapYou can prevent data corruption due to cross-fabric talk by configuring an FC-Map that identifies the FibreChannel fabric for this switch. When the FC-Map is configured, the switch discards the MAC addresses thatare not part of the current fabric. An FCF can assign Fabric Provide MAC Addresses (FPMA) to the CNAsconsisting of the FC-Map Value for the Fabric and the Fibre Channel ID (FCID) assigned during Fabric Login
For Cisco MDS 9500 switches, start at configure terminal below .Note
Before You Begin
For Cisco Nexus 7000 Series, you must be in the storage VDC to configure this feature.
SUMMARY STEPS
1. (Cisco Nexus 7000) switchto vdc vdc-id type storage2. configure terminal3. fcoe fcmap fabric-map
DETAILED STEPS
PurposeCommand or Action
Switches to the storage VDC. The step is required for CiscoNexus 7000 Series only.
(Cisco Nexus 7000) switchto vdc vdc-id type storage
Example:switch# switchto vdc fcoe type storagefcoe#
Step 1
Enters configuration mode.configure terminal
Example:fcoe# configure terminalfcoe(config)#
Step 2
Configures the global FC-Map. The default value is0x0EFC00. The range is from 0x0EFC00 to 0x0EFCFF. Usethe no fcoe map command to reset to the default value.
fcoe fcmap fabric-map
Example:fcoe(config)# fcoe fcmap 0x0efc2a
Step 3
This example shows how to configure the global FC-Map on a Cisco Nexus 7000 Series switch:switch# switchto vdc fcoe type storagefcoe# configure terminal
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 15
FCoEConfiguring the FC-Map
fcoe(config)# fcoe fcmap 0x0efc2a
This example shows how to configure the global FC-Map on a Cisco MDS 9500 switch:switch# configure terminalswitch(config)# fcoe fcmap 0x0efc2a
Configuring the Fabric PriorityThe FCoE switch advertises its priority. The priority is used by the CNAs in the fabric to determine the bestswitch to connect to.
For Cisco MDS 9500 switches, start at configure terminal below .Note
Before You Begin
For Cisco Nexus 7000 Series, you must be in the storage VDC to configure this feature.
SUMMARY STEPS
1. (Cisco Nexus 7000) switchto vdc vdc-id type storage2. configure terminal3. fcoe fcf-priority fabric-priority
DETAILED STEPS
PurposeCommand or Action
Switches to the storage VDC. The step is required for CiscoNexus 7000 Series only.
(Cisco Nexus 7000) switchto vdc vdc-id type storage
Example:switch# switchto vdc fcoe type storagefcoe#
Step 1
Enters configuration mode.configure terminal
Example:fcoe# configure terminalfcoe(config)#
Step 2
Configures the global fabric priority. The default value is 128.The range is from 0 (higher) to 255 (lower). Use the no fcoe
fcoe fcf-priority fabric-priority
Example:fcoe(config)# fcoe fcf-priority 42
Step 3
fcf-priority command to reset the global fabric priority to thedefault value.
This example shows how to configure the global fabric priority for a Cisco Nexus 7000 Series switch:switch# switchto vdc fcoe type storagefcoe# configure terminalfcoe(config)# fcoe fcf-priority 42
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950016 OL-25809-02
FCoEConfiguring the Fabric Priority
This example shows how to configure the global fabric priority for a Cisco MDS 9500 switch:switch# configure terminalswitch(config)# fcoe fcf-priority 42
Setting the Advertisment IntervalYou can configure the interval for Fibre Channel fabric advertisement on the switch.
For Cisco MDS 9500 switches, start at configure terminal below .Note
Before You Begin
For Cisco Nexus 7000 Series, you must be in the storage VDC to configure this feature.
SUMMARY STEPS
1. (Cisco Nexus 7000) switchto vdc vdc-id type storage2. configure terminal3. fcoe fka-adv-period interval
DETAILED STEPS
PurposeCommand or Action
Switches to the storage VDC. The step is required forCisco Nexus 7000 Series only.
(Cisco Nexus 7000) switchto vdc vdc-id type storage
Example:switch# switchto vdc fcoe type storagefcoe#
Step 1
Enters configuration mode.configure terminal
Example:fcoe# configure terminalfcoe(config)#
Step 2
Configures the advertisement interval for the fabric. Thedefault value is 8 seconds. The range is from 4 to 60seconds.
fcoe fka-adv-period interval
Example:fcoe(config)# fcoe fka-adv-period 8fcoe#
Step 3
This example shows how to configure the advertisement interval for the fabric on a Cisco Nexus 7000 Seriesswitch:switch# switchto vdc fcoe type storagefcoe# configure terminalfcoe(config)# fcoe fka-adv-period 42
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 17
FCoESetting the Advertisment Interval
This example shows how to configure the advertisement interval for the fabric on a Cisco MDS 9500 switch:switch# configure terminalswitch(config)# fcoe fka-adv-period 42
Disabling LAN Traffic on an FCoE LinkYou can disable LAN traffic on an FCoE link.
DCBX allows the switch to send a LAN Logical Link Status (LLS) message to a directly-connected CNA.Enter the shutdown lan command to send an LLS-Down message to the CNA. This command causes allVLANs on the interface that are not enabled for FCoE to be brought down. If a VLAN on the interface isenabled for FCoE, it continues to carry SAN traffic without any interruption.
SUMMARY STEPS
1. configure terminal2. interface ethernet slot/port3. shutdown lan4. (Optional) show interface5. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Specifies an interface to configure, and enters interfaceconfigurationmode. Use ? to view a list of supported interfaces.
interface ethernet slot/port
Example:switch(config)# interface e 2/1switch(config-if)#
Step 2
Shuts down Ethernet traffic on the interface. If the interface ispart of an FCoE VLAN, the shutdown has no impact on the
shutdown lan
Example:switch(config-if)# shutdown lan
Step 3
FCoE traffic. Use no shutdown lan to reenable Ethernet trafficon this interface.
(Optional)Displays information about the interface.
show interface
Example:switch(config-if)# show interface
Step 4
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950018 OL-25809-02
FCoEDisabling LAN Traffic on an FCoE Link
PurposeCommand or Action
(Optional)Copies the running configuration to the startup configuration.
copy running-config startup-config
Example:switch(config-if)# copy running-configstartup-config
Step 5
Verifying the FCoE ConfigurationTo display FCoE configuration information, perform one of these tasks:
PurposeCommand
Displays whether FCoE is enabledon the switch.
show fcoe
Displays the contents of the FCoEdatabase.
show fcoe database
Displays information about thevFC interfaces.
show interface vfc [number]
Displays the FCoE settings for aninterface or all interfaces.
show interface [interface number] fcoe
This example shows how to verify that the FCoE capability is enabled:switch# show fcoeGlobal FCF details
FCF-MAC is 00:0d:ec:6d:95:00FC-MAP is 0e:fc:00FCF Priority is 128FKA Advertisement period for FCF is 8 seconds
This example shows how to display the FCoE database:switch# show fcoe database------------------------------------------------------------------------------INTERFACE FCID PORT NAME MAC ADDRESS------------------------------------------------------------------------------vfc3 0x490100 21:00:00:1b:32:0a:e7:b8 00:c0:dd:0e:5f:76
This example shows how to display the FCoE settings for an interface.switch# show interface ethernet 1/37 fcoeEthernet1/37 is FCoE UP
vfc3 is UpFCID is 0x490100PWWN is 21:00:00:1b:32:0a:e7:b8MAC addr is 00:c0:dd:0e:5f:76
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 19
FCoEVerifying the FCoE Configuration
Additional References for FCoERelated Documents
Document TitleRelated Topic
Cisco NX-OS FCoE CommandReference for Cisco Nexus 7000and Cisco MDS 9500
Command reference
Cisco NX-OS Licensing GuideCisco NX-OS licensing
Standards and RFCs
TitleStandard/RFC
Fibre Channel Backbone 5T11 FC BB-5
MIBs
MIBs LinkMIB
To locate and downloadMIBs for selected platforms,Cisco IOS releases, and feature sets, use Cisco MIBLocator found at the following URL:
http://www.cisco.com/go/mibs
Technical Assistance
LinkDescription
http://www.cisco.com/supportThe Cisco Support website provides extensive onlineresources, including documentation and tools fortroubleshooting and resolving technical issues withCisco products and technologies.
To receive security and technical information aboutyour products, you can subscribe to various services,such as the Product Alert Tool (accessed from FieldNotices), the Cisco Technical Services Newsletter,and Really Simple Syndication (RSS) Feeds.
Access to most tools on the Cisco Support websiterequires a Cisco.com user ID and password.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950020 OL-25809-02
FCoEAdditional References for FCoE
C H A P T E R 3Configuring FCoE VLANs and Virtual Interfaces
This chapter includes the following sections:
• Information About Virtual Fibre Channel (VFC) Interfaces, page 21
• Default Settings for FCoE, page 22
• Licensing Requirements for FCoE, page 22
• Guidelines and Limitations, page 23
• Configuring Virtual Interfaces, page 25
• Verifying the Virtual Interface , page 32
• Example:Mapping VSANs to VLANs , page 33
• Additional References for FCoE, page 34
Information About Virtual Fibre Channel (VFC) InterfacesFibre Channel over Ethernet (FCoE) allows Fibre Channel and Ethernet traffic to be carried on the samephysical Ethernet connection between the switch and the servers.
The Fibre Channel portion of FCoE is configured as a virtual Fibre Channel interface. Logical Fibre Channelfeatures (such as interface mode) can be configured on virtual Fibre Channel interfaces.
A virtual Fibre Channel (vfc) interface must be bound to an interface before it can be used. The binding is toa physical Ethernet interface when the converged network adapter (CNA) is directly connected to the switchor port channel when the CNA connects to the Fibre Channel Forwarder (FCF) over a virtual port channel(vPC).
VF PortA virtual fabric (VF) port in an FCoE network acts as a fabric port that connects to a peripheral device (hostor disk) operating as an N port. A VF port can be attached to only one N port.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 21
VE PortA virtual expansion (VE) port acts as an expansion port in an FCoE network. VE ports can connect multipleFCoE switches together in the network. You can bind a VE port to a physical ethernet port or a port channel.
Traffic is load balanced across equal cost E_Ports and VE_Ports based on SID, DID, and OXID. Traffic acrossmembers of a port channel that a VE_Port is bound to is load balanced based on SID, DID, and OXID.
Default Settings for FCoEThis table lists the default settings for FCoE parameters.
Table 5: Default FCoE Parameter Settings
DefaultParameters
Not installed, disabledFCoE feature
0E.FC.00FC-Map
128Fabric priority
8 secondsAdvertisement interval
Licensing Requirements for FCoELicenseProduct
Each F Series module that runs FCoE requires anFCoE license. FCoE enabled in a nondefault VDCdoes not require the Advanced Services License. Fora complete explanation of the Cisco NX-OS licensingscheme and how to obtain and apply licenses, see theCisco NX-OS Licensing Guide.
Cisco Nexus 7000 Series
FCoE does not require a license. For a completeexplanation of the Cisco NX-OS licensing schemeand how to obtain and apply licenses, see the CiscoNX-OS Licensing Guide.
Cisco MDS 9500 Series
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950022 OL-25809-02
Configuring FCoE VLANs and Virtual InterfacesVE Port
Guidelines and Limitations
Cisco Nexus 7000: Guidelines and Limitations• You cannot enable FCoE on default VLAN.
• The QoS policy must be the same on all Cisco FCoE switches in the network.
• Beginning with Cisco NX-OS Release 6.1, FCoE is supported on F2 and F2e Series modules.
◦FCoE supports only F2e (SFP+) modules.
◦FCoE does not support F2e (Copper) modules.
FCoE VDC
FCoE in a dedicated storage VDC has the following guidelines:
• Enable the FCoE feature set in only one VDC.
• Create VLANs in the FCoE allocated VLAN range.
• Do not enable any other features other than storage-related features in the dedicated FCoE VDC.
• Allocate resources for the dedicated FCoE VDC from an F Series module, such as the 32-port 1- and10-Gigabit Ethernet I/O module (PID N7K-F132XP-15) .
• Rollback is not supported in a storage VDC.
• FCoE support on F2 and F2e Series modules, SF248XP-25 requires Supervisor 2 module.
• F2 and F2e series modules cannot exist in the same VDC with any other module type. This applies toboth LAN and storage VDCs.
Shared Interfaces
Any change in protocol state that flaps the parent port of a shared interface because of any port featurealso affects the FCOE traffic on the storage vdc.
Note
The following interface config modes are not allowed while sharing an interface from Ethernet vdc to a storagevdc:
• SPAN destination
• Private VLAN mode
• Port-channel interfaces
• Access mode
• mac-packet-classify
• Interfaces that are part of a VLAN that has an associated QoS policy
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 23
Configuring FCoE VLANs and Virtual InterfacesGuidelines and Limitations
Shared Ethernet interfaces must be in trunk mode and only shared with one other VDC.
Storage VDC
Configuring a VDC for the OOB management interface mgmt0 is accomplished with the vrf contextmanagement command. However, a storage VDC does not support VRF, so configuring mgmt0 requires adifferent approach.
The following table shows how to configure mgmt 0 for a VDC and for a storage VDC:
Configuring mgmt 0 for storage VDCConfiguring mgmt 0 for VDC
interface mgmt 0ip address mgmt0_ip_address mgmt0_subnet_maskno shutip route 0.0.0.0/0 default_gateway
The ip route command specifies the defaultroute that points to the default gateway.
Note
vrf context managementip route 0.0.0.0/0 default_gateway
where
• mgmt0_ip_address is the mgmt0 IPv4 address.
• mgmt0_subnet_mask is the mgmt0 IPv4 netmask.
• default_gateway is the IPv4 address of the default-gateway.
Formore information about VDC, see theCisco Nexus 7000 Series NX-OSVirtual Device Context ConfigurationGuide.
Cisco MDS 9500: Guidelines and Limitations• You cannot enable FCoE on the default VLAN.
• You can only enable FCoE on an FCoE module.
• The QoS policy must be the same on all Cisco FCoE switches in the network.
• Cisco MDS 9500 does not support LAN traffic and only accepts and processes FCoE frames.
• You must install a Supervisor-2A module to configure FCoE.
• You can install an FCoE module in a fabric that uses FC-Redirect services but do not install the FCoEmodule in the same switch where FC-Redirect services application modules are enabled.
• SME does not support any FCoE connected devices including devices connected through theMDS FCoElinecard (DX-X9708-K9)
• In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is runningDMM, SME, or IOA.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950024 OL-25809-02
Configuring FCoE VLANs and Virtual InterfacesCisco MDS 9500: Guidelines and Limitations
Configuring Virtual Interfaces
Mapping a VSAN to a VLANA unique, dedicated VLAN must be configured at every converged access switch to carry traffic for eachvirtual fabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 10, VLAN 1003 for VSAN 2, andso on). If you enableMST, youmust use a separateMultiple Spanning Tree (MST) instance for FCoEVLANs.
Youmust exit VLANmode to execute the configured commands on the Cisco Nexus 7000 Series Switches.Note
Before You Begin
• Ensure you have installed the correct license for FCoE.
• Ensure you have enabled FCoE.
• For Cisco Nexus 7000 Series Switches, ensure that you are in the storage VDC.
SUMMARY STEPS
1. configure terminal2. vsan database3. vsan vsan-id4. vlan vlan-id5. fcoe [vsan vsan-id]6. exit7. (Optional) show vlan fcoe8. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Enters VSAN database configuration mode.vsan database
Example:switch(config)# vsan databaseswitch(config-vsan-db)#
Step 2
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 25
Configuring FCoE VLANs and Virtual InterfacesConfiguring Virtual Interfaces
PurposeCommand or Action
Defines the VSAN. The VSAN number range is from 1 to4094.
vsan vsan-id
Example:switch(config-vsan-db)# vsan 200
Step 3
Enters VLAN configuration mode. The VLAN number rangeis from 1 to 4096.
vlan vlan-id
Example:switch(config-vsan-db)# vlan 200switch(config-vlan)#
Step 4
Enables FCoE for the specified VLAN and configures themapping from this VLAN to the specified VSAN. If you do
fcoe [vsan vsan-id]
Example:switch(config-vlan)# fcoe vsan 200
Step 5
not specify a VSAN number, a mapping is created from thisVLAN to the VSAN with the same number.
Exits VLAN configuration mode. You must exit this mode toexecute the configured commands on the Cisco Nexus 7000Series Switches.
exit
Example:switch(config-vlan)# exitswitch(config)#
Step 6
(Optional)Displays information about the FCoE configuration for aVLAN.
show vlan fcoe
Example:switch(config-vlan)# show vlan fcoe
Step 7
(Optional)Copies the running configuration to the startup configuration.
copy running-config startup-config
Example:switch(config-vlan)# copy running-configstartup-config
Step 8
This example shows how to map VLAN 200 to VSAN 200 on a Cisco MDS 9500 Series Switch:switch(config)# vlan 200switch(config-vlan)# fcoe vsan 200
This example shows how to map VLAN 300 to VSAN 300 on a Cisco Nexus 7000 Series Switches:switch(config)# switchto vdc fcoe_vdcswitch-fcoe_vdc# configure terminalswitch-fcoe_vdc(config)# vlan 300switch-fcoe_vd(config-vlan)# fcoe vsan 300
Creating a Virtual Fibre Channel InterfaceYou can create a virtual Fibre Channel interface. You must bind the virtual Fibre Channel interface to aphysical interface before it can be used.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950026 OL-25809-02
Configuring FCoE VLANs and Virtual InterfacesCreating a Virtual Fibre Channel Interface
Before You Begin
• Ensure you have installed the correct license for FCoE.
• Ensure you have enabled FCoE.
SUMMARY STEPS
1. configure terminal2. interface vfc vfc-id3. switchport mode mode4. bind {interface {ethernet slot/port}}5. (Optional) show interface vfc6. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Creates a virtual Fibre Channel interface (if it does not alreadyexist) and enters interface configuration mode. The vfc-id rangeis from 1 to 8192.
interface vfc vfc-id
Example:switch(config)# interface vfc 4switch(config-if)#
Step 2
Configures the switchport mode for a virtual Fibre Channelinterface. The mode is E or F. The default is F mode.
switchport mode mode
Example:switch(configif)# switchport mode e
Step 3
Binds the virtual Fibre Channel interface to the specifiedinterface. Use ? to see the supported interfaces and port channels.
bind {interface {ethernet slot/port}}
Example:switch(config-if)# bind interface ethernet1/4
Step 4
Use the no form of this command to unbind the virtual FibreChannel interface from the specified interface.
(Optional)Displays information about the virtual Fibre Channel interfaces.
show interface vfc
Example:switch(config-if)# show interface vfc
Step 5
(Optional)Copies the running configuration to the startup configuration.
copy running-config startup-config
Example:switch(config)# copy running-configstartup-config
Step 6
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 27
Configuring FCoE VLANs and Virtual InterfacesCreating a Virtual Fibre Channel Interface
This example shows how to bind a virtual Fibre Channel interface to an Ethernet interface:switch# configure terminalswitch(config)# interface vfc 4switch(config-if)# bind interface ethernet 1/4
This example shows how to delete a virtual Fibre Channel interface:switch# configure terminalswitch(config)# no interface vfc 4
Creating a Virtual Fibre Channel Port Channel InterfaceYou can create a virtual Fibre Channel port channel interface that automatically binds to the port channel withthe same interface number.
Before You Begin
• For the Cisco Nexus 7000 Series, ensure that you create the port channel interface before you create thevirtual Fibre Channel port channel interface.
• For the Cisco MDS 9500 switches, ensure that you create the Ethernet port channel interface before youcreate the virtual Fibre Channel port channel interface.
SUMMARY STEPS
1. configure terminal2. interface vfc-port-channel int-number3. switchport mode mode4. (Optional) show interface vfc-port-channel int-number5. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Creates a virtual Fibre Channel interface (if it does not alreadyexist) that is bound to the port channel with the same interface
interface vfc-port-channel int-number
Example:switch(config)# interface vfc-port-channel2switch(config-if)#
Step 2
number and enters interface configuration mode. The int-numberrange is from 1 to 4096 (for Cisco Nexus 7000) or from 257 to4095 (for Cisco MDS 9500). The default switchport mode forthis interface is E.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950028 OL-25809-02
Configuring FCoE VLANs and Virtual InterfacesCreating a Virtual Fibre Channel Port Channel Interface
PurposeCommand or Action
Configures the switchport mode for a virtual Fibre Channelinterface. The mode is E or F. The default is F mode.
switchport mode mode
Example:switch(config-if)# switchport mode e
Step 3
(Optional)Displays information about the virtual Fibre Channel interfacesbound to port channel interfaces.
show interface vfc-port-channel int-number
Example:switch(config-if)# show interfacevfc-port-channel 2
Step 4
(Optional)Copies the running configuration to the startup configuration.
copy running-config startup-config
Example:switch(config)# copy running-configstartup-config
Step 5
Associating a Virtual Fibre Channel Interface to a VSANYoumust configure unique, dedicated VLAN at every converged access switch to carry traffic for each VirtualFabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 1, VLAN 1003 for VSAN 2, and so on). Ifyou enable MST, you must use a separate MST instance for FCoE VLANs.
Before You Begin
For Cisco Nexus 7000 Series Switches, ensure that you are in the storage VDC.
SUMMARY STEPS
1. configure terminal2. vsan database3. (Optional) vsan vsan-id4. Enter one of the following commands:
• vsan vsan-id interface vfc vfc-id
• vsan vsan-id interface vfc-port-channel vfc-id
5. (Optional) show vsan6. (Optional) copy running-config startup-config
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 29
Configuring FCoE VLANs and Virtual InterfacesAssociating a Virtual Fibre Channel Interface to a VSAN
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Enters VSAN configuration mode.vsan database
Example:switch(config)# vsan databaseswitch(config-vsan-db)#
Step 2
(Optional)Creates the VSAN. The vsan-id range is from 1 to 4094 and mustmap to a VLAN on the physical Ethernet interface that is bound tothe virtual Fibre Channel interface.
vsan vsan-id
Example:switch(config-vsan-db)# vsan 2
Step 3
Configures the association between the VSAN and virtual FibreChannel interface or virtual Fibre Channel port channel. The vsan-id
Enter one of the following commands:Step 4
• vsan vsan-id interface vfc vfc-id range is from 1 to 4094 and must map to a VLAN on the physicalEthernet interface or port channel that is bound to the virtual Fibre• vsan vsan-id interface vfc-port-channel
vfc-id Channel interface or virtual Fibre Channel port channel. The vfc-idrange is from 1 to 8192. Us the no form of this command todissassociate the connection between the VSAN and virtual FibreChannel interface or virtual Fibre Channel port channel.Example:
switch(config-vsan-db)# vsan 2 interfacevfc 4
(Optional)Displays information about the VSAN.
show vsan
Example:switch(config-vsan-db)# show vsan
Step 5
(Optional)Copies the running configuration to the startup configuration.
copy running-config startup-config
Example:switch(config-vsan-db)# copy running-configstartup-config
Step 6
This example shows how to associate a virtual Fibre Channel interface to a VSAN:switch# configure terminalswitch(config)# vsan databaseswitch(config-vsan-db)# vsan 2 interface vfc 4
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950030 OL-25809-02
Configuring FCoE VLANs and Virtual InterfacesAssociating a Virtual Fibre Channel Interface to a VSAN
Enabling VE Loopback ConfigurationThe VFID check verifies that the VSAN configuration is correct on both ends of a VE link. You can turn offthe VFID check for VE ports to allow VE loopback configuration between to VE ports on the same switch.
For Cisco MDS 9500 switches, start at Step 2, on page 31 .Note
Before You Begin
For Cisco Nexus 7000 Series, you must be in the storage VDC to configure this feature.
SUMMARY STEPS
1. (Cisco Nexus 7000) switchto vdc vdc-id type storage2. configure terminal3. fcoe veloopback
DETAILED STEPS
PurposeCommand or Action
Switches to the storage VDC. The step is requiredfor Cisco Nexus 7000 Series only.
(Cisco Nexus 7000) switchto vdc vdc-id type storage
Example:# switchto vdc fcoe type storagefcoe#
Step 1
Enters configuration mode.configure terminal
Example:fcoe# configure terminalfcoe(config)#
Step 2
Enables the VFID check for all VE ports.fcoe veloopback
Example:fcoe(config)# fcoe veloopback
Step 3
This example shows how to enable VE loopback for a Cisco Nexus 7000 Series switch:switch# switchto vdc fcoe type storagefcoe# configure terminalfcoe(config)# fcoe veloopbackThis example shows how to enable VE loopback for a Cisco MDS 9500 switch:switch# configure terminalswitch(config)# fcoe veloopback
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 31
Configuring FCoE VLANs and Virtual InterfacesEnabling VE Loopback Configuration
Verifying the Virtual InterfaceTo display configuration information about virtual interfaces, perform one of the following tasks:
PurposeCommand
Displays the detailed configuration of the specifiedFibre Channel interface.
show interface vfc vfc-id
Displays the status of all interfaces.show interface brief
Displays the mapping of FCoE VLANs to VSANs.show vlan fcoe
This example shows how to display a virtual Fibre Channel interface bound to an Ethernet interface:switch# show interface vfc 3vfc3 is up
Bound interface is Ethernet1/37Hardware is Virtual Fibre ChannelPort WWN is 20:02:00:0d:ec:6d:95:3fAdmin port mode is F, trunk mode is onsnmp link state traps are enabledPort mode is F, FCID is 0x490100Port vsan is 9311 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec0 frames input, 0 bytes0 discards, 0 errors
0 frames output, 0 bytes0 discards, 0 errors
Interface last changed at Thu May 21 04:44:42 2009
This example shows how to display the status of all the interfaces on the switch (some output has been removedfor brevity):switch# show interface brief-------------------------------------------------------------------------------Interface Vsan Admin Admin Status SFP Oper Oper Port
Mode Trunk Mode Speed ChannelMode (Gbps)
-------------------------------------------------------------------------------fc3/1 1 auto on trunking swl TE 2 --fc3/2 1 auto on sfpAbsent -- -- --...fc3/8 1 auto on sfpAbsent -- -- ---------------------------------------------------------------------------------Interface Status IP Address Speed MTU Port
Channel-------------------------------------------------------------------------------Ethernet1/1 hwFailure -- -- 1500 --Ethernet1/2 hwFailure -- -- 1500 --Ethernet1/3 up -- 10000 1500 --...Ethernet1/39 sfpIsAbsen -- -- 1500 --Ethernet1/40 sfpIsAbsen -- -- 1500 ---------------------------------------------------------------------------------Interface Status IP Address Speed MTU-------------------------------------------------------------------------------mgmt0 up 172.16.24.41 100 1500-------------------------------------------------------------------------------
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950032 OL-25809-02
Configuring FCoE VLANs and Virtual InterfacesVerifying the Virtual Interface
-------------------------------------------------------------------------------Interface Vsan Admin Admin Status SFP Oper Oper Port
Mode Trunk Mode Speed ChannelMode (Gbps)
-------------------------------------------------------------------------------vfc 1 1 F -- down -- -- --...
This example shows how to display the mapping between the VLANs and VSANs on the switch:switch# show vlan fcoeVLAN VSAN Status-------- -------- --------15 15 Operational20 20 Operational25 25 Operational30 30 Non-operational
Example:Mapping VSANs to VLANsThe following example shows how to configure the FCoE VLAN and a virtual Fibre Channel interface:
SUMMARY STEPS
1. Enable the associated VLAN and map the VLAN to a VSAN. For Cisco Nexus 7000, ensure you are inthe storage VDC.
2. Configure the VLAN on a physical Ethernet interface.3. Create a virtual Fibre Channel interface and bind it to a physical Ethernet interface. For Cisco Nexus 7000,
ensure you are in the storage VDC.4. Associate the virtual Fibre Channel interface to the VSAN. For Cisco Nexus 7000, ensure you are in the
storage VDC.5. (Optional) Display membership information for the VSAN.6. (Optional) Display the interface information for the virtual Fibre Channel interface.
DETAILED STEPS
Step 1 Enable the associated VLAN and map the VLAN to a VSAN. For Cisco Nexus 7000, ensure you are in the storage VDC.switch(config)# vlan 200switch(config-vlan)# fcoe vsan 200switch(config-vlan)# exit
Step 2 Configure the VLAN on a physical Ethernet interface.switch# configure terminalswitch(config)# interface ethernet 1/4switch(config-if)# spanning-tree port type edge trunkswitch(config-if)# switchport mode trunkswitch(config-if)# switchport trunk allowed vlan 1,200switch(config-if)# exit
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 33
Configuring FCoE VLANs and Virtual InterfacesExample:Mapping VSANs to VLANs
Step 3 Create a virtual Fibre Channel interface and bind it to a physical Ethernet interface. For Cisco Nexus 7000, ensure youare in the storage VDC.switch(config)# interface vfc 4switch(config-if)# bind interface ethernet 1/4
By default, all virtual Fibre Channel interfaces reside on VSAN 1. If the VLAN to VSANmapping is to a VSANother than VSAN 1, then proceed to Step 4.
Note
Step 4 Associate the virtual Fibre Channel interface to the VSAN. For Cisco Nexus 7000, ensure you are in the storage VDC.switch(config)# vsan databaseswitch(config-vsan)# vsan 200 interface vfc 4switch(config-vsan)# exit
Step 5 (Optional) Display membership information for the VSAN.switch# show vsan 200 membershipvsan 200 interfaces
vfc 4
Step 6 (Optional) Display the interface information for the virtual Fibre Channel interface.switch# show interface vfc 4
vfc4 is upBound interface is Ethernet1/4Hardware is Virtual Fibre ChannelPort WWN is 20:02:00:0d:ec:6d:95:3fPort WWN is 20:02:00:0d:ec:6d:95:3fsnmp link state traps are enabledPort WWN is 20:02:00:0d:ec:6d:95:3fAPort WWN is 20:02:00:0d:ec:6d:95:3fsnmp link state traps are enabledPort mode is F, FCID is 0x490100Port vsan is 2001 minute input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec1 minute output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec0 frames input, 0 bytes 0 discards, 0 errors0 frames output, 0 bytes 0 discards, 0 errorsInterface last changed at Thu Mar 11 04:44:42 2010
Additional References for FCoERelated Documents
Document TitleRelated Topic
Cisco NX-OS FCoE CommandReference for Cisco Nexus 7000and Cisco MDS 9500
Command reference
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950034 OL-25809-02
Configuring FCoE VLANs and Virtual InterfacesAdditional References for FCoE
Document TitleRelated Topic
Cisco NX-OS Licensing GuideCisco NX-OS licensing
Standards and RFCs
TitleStandard/RFC
Fibre Channel Backbone 5T11 FC BB-5
MIBs
MIBs LinkMIB
To locate and downloadMIBs for selected platforms,Cisco IOS releases, and feature sets, use Cisco MIBLocator found at the following URL:
http://www.cisco.com/go/mibs
Technical Assistance
LinkDescription
http://www.cisco.com/supportThe Cisco Support website provides extensive onlineresources, including documentation and tools fortroubleshooting and resolving technical issues withCisco products and technologies.
To receive security and technical information aboutyour products, you can subscribe to various services,such as the Product Alert Tool (accessed from FieldNotices), the Cisco Technical Services Newsletter,and Really Simple Syndication (RSS) Feeds.
Access to most tools on the Cisco Support websiterequires a Cisco.com user ID and password.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 35
Configuring FCoE VLANs and Virtual InterfacesAdditional References for FCoE
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950036 OL-25809-02
Configuring FCoE VLANs and Virtual InterfacesAdditional References for FCoE
C H A P T E R 4Storage VDC on Cisco Nexus 7000 SeriesSwitches
This chapter contains the following sections:
• Information About Storage VDC, page 37
• Licensing Requirements for FCoE, page 38
• Guidelines and Limitations, page 38
• Configuring FCoE VDCs, page 40
• Example: Storage VDC Configuration, page 46
Information About Storage VDCYou use a virtual device context (VDC) to separate LAN and SAN traffic on the same switch. A VDC allowsyou to maintain one physical infrastructure but separate logical data paths.
To achieve this configuration, you must perform the following tasks:
• Create a dedicated storage VDC.
• Allocate a VLAN range for FCoE.
• Optionally, share physical ports between the storage VDC and one other VDC.
For shared physical ports, the VDC where the ports are shared includes a VF Ethernet port that correspondsto the shared port. You cannot modify some details of that port because it must match the underlying sharedphysical port. If you move the source port to another VDC or delete the VDC, the shared ports are deletedand you must reconfigure them.
If the Ethernet VDC restarts or is suspended, any shared Ethernet ports are shut down in the correspondingVDC. These ports come up automatically once the Ethernet VDC is operational.
Note
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 37
Licensing Requirements for FCoELicenseProduct
Each F Series module that runs FCoE requires anFCoE license. FCoE enabled in a nondefault VDCdoes not require the Advanced Services License. Fora complete explanation of the Cisco NX-OS licensingscheme and how to obtain and apply licenses, see theCisco NX-OS Licensing Guide.
Cisco Nexus 7000 Series
FCoE does not require a license. For a completeexplanation of the Cisco NX-OS licensing schemeand how to obtain and apply licenses, see the CiscoNX-OS Licensing Guide.
Cisco MDS 9500 Series
Guidelines and Limitations
Cisco Nexus 7000: Guidelines and Limitations• You cannot enable FCoE on default VLAN.
• The QoS policy must be the same on all Cisco FCoE switches in the network.
• Beginning with Cisco NX-OS Release 6.1, FCoE is supported on F2 and F2e Series modules.
◦FCoE supports only F2e (SFP+) modules.
◦FCoE does not support F2e (Copper) modules.
FCoE VDC
FCoE in a dedicated storage VDC has the following guidelines:
• Enable the FCoE feature set in only one VDC.
• Create VLANs in the FCoE allocated VLAN range.
• Do not enable any other features other than storage-related features in the dedicated FCoE VDC.
• Allocate resources for the dedicated FCoE VDC from an F Series module, such as the 32-port 1- and10-Gigabit Ethernet I/O module (PID N7K-F132XP-15) .
• Rollback is not supported in a storage VDC.
• FCoE support on F2 and F2e Series modules, SF248XP-25 requires Supervisor 2 module.
• F2 and F2e series modules cannot exist in the same VDC with any other module type. This applies toboth LAN and storage VDCs.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950038 OL-25809-02
Storage VDC on Cisco Nexus 7000 Series SwitchesLicensing Requirements for FCoE
Shared Interfaces
Any change in protocol state that flaps the parent port of a shared interface because of any port featurealso affects the FCOE traffic on the storage vdc.
Note
The following interface config modes are not allowed while sharing an interface from Ethernet vdc to a storagevdc:
• SPAN destination
• Private VLAN mode
• Port-channel interfaces
• Access mode
• mac-packet-classify
• Interfaces that are part of a VLAN that has an associated QoS policy
Shared Ethernet interfaces must be in trunk mode and only shared with one other VDC.
Storage VDC
Configuring a VDC for the OOB management interface mgmt0 is accomplished with the vrf contextmanagement command. However, a storage VDC does not support VRF, so configuring mgmt0 requires adifferent approach.
The following table shows how to configure mgmt 0 for a VDC and for a storage VDC:
Configuring mgmt 0 for storage VDCConfiguring mgmt 0 for VDC
interface mgmt 0ip address mgmt0_ip_address mgmt0_subnet_maskno shutip route 0.0.0.0/0 default_gateway
The ip route command specifies the defaultroute that points to the default gateway.
Note
vrf context managementip route 0.0.0.0/0 default_gateway
where
• mgmt0_ip_address is the mgmt0 IPv4 address.
• mgmt0_subnet_mask is the mgmt0 IPv4 netmask.
• default_gateway is the IPv4 address of the default-gateway.
Formore information about VDC, see theCisco Nexus 7000 Series NX-OSVirtual Device Context ConfigurationGuide.
Cisco MDS 9500: Guidelines and Limitations• You cannot enable FCoE on the default VLAN.
• You can only enable FCoE on an FCoE module.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 39
Storage VDC on Cisco Nexus 7000 Series SwitchesCisco MDS 9500: Guidelines and Limitations
• The QoS policy must be the same on all Cisco FCoE switches in the network.
• Cisco MDS 9500 does not support LAN traffic and only accepts and processes FCoE frames.
• You must install a Supervisor-2A module to configure FCoE.
• You can install an FCoE module in a fabric that uses FC-Redirect services but do not install the FCoEmodule in the same switch where FC-Redirect services application modules are enabled.
• SME does not support any FCoE connected devices including devices connected through theMDS FCoElinecard (DX-X9708-K9)
• In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is runningDMM, SME, or IOA.
Configuring FCoE VDCs
Creating a Dedicated Storage VDC
Before You Begin
• Ensure you have installed the correct license for FCoE.
SUMMARY STEPS
1. configure terminal2. install feature-set fcoe3. system qos4. service-policy type network-qos policy-map name5. vdc vdc-name type storage6. system default switchport7. feature lldp8. (Optional) feature lacp9. (Optional) allocate interface ethernet int-numb10. (Optional) switchto vdc vdc-name11. (Optional) show feature-set12. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950040 OL-25809-02
Storage VDC on Cisco Nexus 7000 Series SwitchesConfiguring FCoE VDCs
PurposeCommand or Action
Installs the FCoE feature-set.install feature-set fcoe
Example:switch(config)# install feature-set fcoe
Step 2
Enters system qos mode.system qos
Example:switch(config)# system qosswitch(config-sys-qos)#
Step 3
Enables no drop queue for FCoE class.service-policy type network-qos policy-map name
Example:switch(config-sys-qos)#service-policy typenetwork-qos default-nq-7e-policy
Step 4
Creates a dedicated storage VDC and enters VDCconfiguration mode. You can only enable storage features
vdc vdc-name type storage
Example:switch(config)# vdc fcoe-vdc type storageswitch(config-vdc)#
Step 5
in a storage VDC. You do not need to allow the feature-setor enable it in the storage VDC because this process ishandled automatically for a storage VDC.
Set ethernet ports to default access port mode.system default switchport
Example:switch(config-vdc)# system default switchport
Step 6
Enables the LLDP feature in the storage VDC.feature lldp
Example:switch(config)# feature lldp
Step 7
(Optional)Enables the LACP feature in the storage VDC.
feature lacp
Example:switch(config)# feature lacp
Step 8
(Optional)Allocates interfaces to the storage VDC as a dedicatedFCoE port. You must allocate all interfaces in the port
allocate interface ethernet int-numb
Example:switch(config-vdc)# allocate interface ethernet2/1-2switch(config-if)#
Step 9
group. You must configure these interfaces in switchporttrunk mode as Spanning Tree Protocol (STP) edge ports.
(Optional)Switches to the storage VDC.
switchto vdc vdc-name
Example:switch(config-vdc)# switchto vdc fcoe-vdcswitch-fcoe-vdc#
Step 10
(Optional)Displays information about the feature-sets in this VDC.
show feature-set
Example:switch-fcoe-vdc# show feature-set
Step 11
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 41
Storage VDC on Cisco Nexus 7000 Series SwitchesCreating a Dedicated Storage VDC
PurposeCommand or Action
(Optional)Copies the running configuration to the startupconfiguration.
copy running-config startup-config
Example:switch-fcoe-vdc# copy running-configstartup-config
Step 12
Cisco Nexus 7000:Licensing an FCoE ModuleYou must associate an FCoE license with an FCoE module to configure FCoE. You need one license for eachmodule configured for FCoE.
Before You Begin
• Ensure you have installed the correct license for FCoE.
SUMMARY STEPS
1. configure terminal2. license fcoe module module-number3. (Optional) show license usage FCOE-N7K-F132XP4. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswtich(config)#
Step 1
Associates an FCoE license to a module.license fcoe module module-number
Example:swtich(config)# license fcoe module 2
Step 2
(Optional)Displays information about which FCoE modulesare using FCoE licenses.
show license usage FCOE-N7K-F132XP
Example:swtich(config)# show license usage FCOE-N7K-F132XP
Step 3
(Optional)Copies the running configuration to the startupconfiguration.
copy running-config startup-config
Example:swtich(config)# copy running-config startup-config
Step 4
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950042 OL-25809-02
Storage VDC on Cisco Nexus 7000 Series SwitchesCisco Nexus 7000:Licensing an FCoE Module
Allocating the FCoE VLAN Range
Before You Begin
• Ensure you have installed the correct license for FCoE.
• Ensure you are in the correct VDC.
SUMMARY STEPS
1. configure terminal2. vdc vdc-name type storage3. allocate fcoe-vlan-range vlan-range [from vdcs vdc-name]4. (Optional) show fcoe-vlan-range5. (Optional) copy running-config startup-config
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Enters VDC configuration mode. You can only enable storagefeature in a storage VDC. You do not need to allow the
vdc vdc-name type storage
Example:switch(config)# vdc fcoe-vdc type storageswitch(config-vdc)#
Step 2
feature-set or enable it in the storage VDC because this processis handled automatically for a storage VDC.
Allocates the VLANs that can be used for FCoE and mappedto a VSAN. You can optionally allocate the VLANs fromanother VDC.
allocate fcoe-vlan-range vlan-range [from vdcsvdc-name]
Example:switch(config-vdc)# allocate fcoe-vlan-range10-30
Step 3
(Optional)Displays information about the VLAN range allocated forFCoE.
show fcoe-vlan-range
Example:switch(config-vdc)# show fcoe-vlan-range
Step 4
(Optional)Copies the running configuration to the startup configuration.
copy running-config startup-config
Example:switch(config-vdc)# copy running-configstartup-config
Step 5
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 43
Storage VDC on Cisco Nexus 7000 Series SwitchesAllocating the FCoE VLAN Range
Related Topics
Cisco Nexus 7000: Guidelines and Limitations, on page 8
Allocating Shared InterfacesYou can share interfaces between a storage VDC and another VDC.
Before You Begin
• Ensure you have installed the correct license for FCoE.
• Ensure you are in the correct VDC.
• Ensure any shared interfaces are from an F-series module.
• Ensure you have allocated the FCoE VLAN range.
• Interfaces can only be shared between the storage VDC and one other VDC.
SUMMARY STEPS
1. configure terminal2. interface if-range3. switchport mode trunk4. spanning-tree port type edge trunk5. no shutdown6. vdc vdc-name type storage7. allocate shared interface if-range8. (Optional) show vdc shared membership9. (Optional) switchto vdc vdc-name10. configure terminal11. feature lldp12. interface if-range13. no shutdown14. (Optional) show interface if-range15. (Optional) copy running-config startup-config
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950044 OL-25809-02
Storage VDC on Cisco Nexus 7000 Series SwitchesAllocating Shared Interfaces
DETAILED STEPS
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch# configure terminalswitch(config)#
Step 1
Enters interface configuration mode for the interface inthe Ethernet VDC.
interface if-range
Example:switch(config)# interface ethernet 2/1
Step 2
Puts the Ethernet interface into trunk mode.switchport mode trunk
Example:switch(config-if)# switchport mode trunk
Step 3
Sets the interface to STP-type edge port to support STPLite for loop prevention.
spanning-tree port type edge trunk
Example:switch(config-if)# spanning-tree port type edgetrunk
Step 4
Administratively enables the Ethernet shared interface.no shutdown
Example:switch(config-if)# no shutdown
Step 5
Enters VDC configuration mode.vdc vdc-name type storage
Example:switch(config-if)# vdc fcoe-vdc type storageswitch(config-vdc)#
Step 6
Allocates interfaces that are shared with another VDCfor FCoE traffic. You must allocate the shared interfaces
allocate shared interface if-range
Example:switch(config-vdc)# allocate shared interfaceethernet 2/1
Step 7
to one of the VDC included in the FCoE VLANallocation. You can only use the shutdown or theswitchport trunk allowed vlan commands on sharedinterfaces in the storage VDC.
(Optional)Displays the interfaces that are shared for FCoE.
show vdc shared membership
Example:switch(config-vdc)# show vdc shared membership
Step 8
(Optional)Switches to the storage VDC.
switchto vdc vdc-name
Example:switch(config-vdc)# switchto vdc fcoe-vdcswitch-fcoe-vdc#
Step 9
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 45
Storage VDC on Cisco Nexus 7000 Series SwitchesAllocating Shared Interfaces
PurposeCommand or Action
Enters configuration mode.configure terminal
Example:switch-fcoe-vdc# configure terminalswitch-fcoe-vdc(config)#
Step 10
Enables the LLDP feature in the storage VDC.feature lldp
Example:switch-fcoe-vdc(config)# feature lldp
Step 11
Enters interface configuration mode for the sharedinterface in the storage VDC.
interface if-range
Example:switch-fcoe-vdc(config)# interface ethernet 2/1
Step 12
Administratively enables the FCoE shared interface.no shutdown
Example:switch-fcoe-vdc(config-if)# no shutdown
Step 13
(Optional)Displays information about the shared interface.
show interface if-range
Example:switch-fcoe-vdc(config-if)# show interfaceethernet 2/1
Step 14
(Optional)Copies the running configuration to the startupconfiguration.
copy running-config startup-config
Example:switch-fcoe-vdc(config-if)# copy running-configstartup-config
Step 15
Example: Storage VDC ConfigurationShared Interfaces in the Storage VDC!Enable associated features and interface to share:switch# configure terminalswitch(config)# feature lldpswitch(config)# feature lacpswitch(config)# system qosswitch(config-sys-qos)# service-policy type network-qos default-nq-7e-policy
switch(config-sys-qos)# interface ethernet 2/1switch(config-if)# switchport mode trunkswitch(config-if)# spanning-tree port type edge trunkswitch(config-if)# no shutdown
!Create Storage VDC and allocate resources:switch(config-if)# install feature-set fcoeswitch(config)# vdc fcoe_vdc type storage
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950046 OL-25809-02
Storage VDC on Cisco Nexus 7000 Series SwitchesExample: Storage VDC Configuration
switch(config-if)# allocate fcoe-vlan-range 10-20 from vdc switchswitch(config-vdc)# allocate shared interface ethernet 2/1
!Switch to storage VDC and bring up the shared interface:switch(config-vdc)# switchto vdc fcoe_vdcswitch-fcoe_vdc# configure terminalswitch-fcoe_vdc(config)# feature lldpswitch-fcoe_vdc(config)# interface ethernet 2/1switch-fcoe_vdc(config-if)# no shutdown
Ethernet 2/1 must be from an F-series module.Note
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 47
Storage VDC on Cisco Nexus 7000 Series SwitchesExample: Storage VDC Configuration
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950048 OL-25809-02
Storage VDC on Cisco Nexus 7000 Series SwitchesExample: Storage VDC Configuration
A P P E N D I X AFeature History by Platform
This chapter contains the following sections:
• Cisco Nexus 7000:Supported Features, page 49
• Cisco MDS 9500:Supported Features, page 49
Cisco Nexus 7000:Supported FeaturesFeature InformationReleaseFeature
Supports the T11-compliant FIP.5.2(1)FIP
This feature was introduced.5.2(1)FCoE
Supports F2 Series modules.6.1(1)FCoE
Supports F2e Series modules (SFP+ only).6.1(1)FCoE
Cisco MDS 9500:Supported FeaturesFeature InformationReleaseFeature
Supports the T11-compliant FIP.5.2(1)FIP
This feature was introduced.5.2(1)FCoE
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 49
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950050 OL-25809-02
Feature History by PlatformCisco MDS 9500:Supported Features
A P P E N D I X BGuidelines and Limitations
This chapter includes the following sections:
• Verified Limits for FCoE for the Cisco Nexus 7000 Series and Cisco MDS 9500 , page 51
• Cisco Nexus 7000: Guidelines and Limitations, page 51
• Cisco MDS 9500: Guidelines and Limitations, page 53
Verified Limits for FCoE for the Cisco Nexus 7000 Series andCisco MDS 9500
The configuration limits are documented in the Cisco Nexus 7000 Series NX-OS Verified Scalability Guide.
Cisco Nexus 7000: Guidelines and Limitations• You cannot enable FCoE on default VLAN.
• The QoS policy must be the same on all Cisco FCoE switches in the network.
• Beginning with Cisco NX-OS Release 6.1, FCoE is supported on F2 and F2e Series modules.
◦FCoE supports only F2e (SFP+) modules.
◦FCoE does not support F2e (Copper) modules.
FCoE VDC
FCoE in a dedicated storage VDC has the following guidelines:
• Enable the FCoE feature set in only one VDC.
• Create VLANs in the FCoE allocated VLAN range.
• Do not enable any other features other than storage-related features in the dedicated FCoE VDC.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 51
• Allocate resources for the dedicated FCoE VDC from an F Series module, such as the 32-port 1- and10-Gigabit Ethernet I/O module (PID N7K-F132XP-15) .
• Rollback is not supported in a storage VDC.
• FCoE support on F2 and F2e Series modules, SF248XP-25 requires Supervisor 2 module.
• F2 and F2e series modules cannot exist in the same VDC with any other module type. This applies toboth LAN and storage VDCs.
Shared Interfaces
Any change in protocol state that flaps the parent port of a shared interface because of any port featurealso affects the FCOE traffic on the storage vdc.
Note
The following interface config modes are not allowed while sharing an interface from Ethernet vdc to a storagevdc:
• SPAN destination
• Private VLAN mode
• Port-channel interfaces
• Access mode
• mac-packet-classify
• Interfaces that are part of a VLAN that has an associated QoS policy
Shared Ethernet interfaces must be in trunk mode and only shared with one other VDC.
Storage VDC
Configuring a VDC for the OOB management interface mgmt0 is accomplished with the vrf contextmanagement command. However, a storage VDC does not support VRF, so configuring mgmt0 requires adifferent approach.
The following table shows how to configure mgmt 0 for a VDC and for a storage VDC:
Configuring mgmt 0 for storage VDCConfiguring mgmt 0 for VDC
interface mgmt 0ip address mgmt0_ip_address mgmt0_subnet_maskno shutip route 0.0.0.0/0 default_gateway
The ip route command specifies the defaultroute that points to the default gateway.
Note
vrf context managementip route 0.0.0.0/0 default_gateway
where
• mgmt0_ip_address is the mgmt0 IPv4 address.
• mgmt0_subnet_mask is the mgmt0 IPv4 netmask.
• default_gateway is the IPv4 address of the default-gateway.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950052 OL-25809-02
Guidelines and LimitationsCisco Nexus 7000: Guidelines and Limitations
Formore information about VDC, see theCisco Nexus 7000 Series NX-OSVirtual Device Context ConfigurationGuide.
Cisco MDS 9500: Guidelines and Limitations• You cannot enable FCoE on the default VLAN.
• You can only enable FCoE on an FCoE module.
• The QoS policy must be the same on all Cisco FCoE switches in the network.
• Cisco MDS 9500 does not support LAN traffic and only accepts and processes FCoE frames.
• You must install a Supervisor-2A module to configure FCoE.
• You can install an FCoE module in a fabric that uses FC-Redirect services but do not install the FCoEmodule in the same switch where FC-Redirect services application modules are enabled.
• SME does not support any FCoE connected devices including devices connected through theMDS FCoElinecard (DX-X9708-K9)
• In Cisco MDS NX-OS Release 5.2(x), you cannot install a FCoE module in a switch that is runningDMM, SME, or IOA.
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 9500 OL-25809-02 53
Guidelines and LimitationsCisco MDS 9500: Guidelines and Limitations
Cisco NX-OS FCoE Configuration Guide for Nexus 7000 and MDS 950054 OL-25809-02
Guidelines and LimitationsCisco MDS 9500: Guidelines and Limitations