hp storageworks fibre channel over ethernet application note

42
HP StorageWorks Fibre Channel Over Ethernet Application Note Abstract This document describes support for HP FCoE products. It provides an overview of the product features and functionality, configuration rules, and support with existing HP Ethernet and FC SAN products. Part number: AA-RWQ5E-TE Seventh edition: April 2010

Upload: technical-dude

Post on 08-Jun-2015

1.516 views

Category:

Documents


5 download

TRANSCRIPT

Page 1: HP StorageWorks Fibre Channel Over Ethernet Application Note

HP StorageWorks Fibre Channel OverEthernet Application NoteAbstract

This document describes support for HP FCoE products. It provides an overview of the product features andfunctionality, configuration rules, and support with existing HP Ethernet and FC SAN products.

Part number: AA-RWQ5E-TESeventh edition: April 2010

Page 2: HP StorageWorks Fibre Channel Over Ethernet Application Note

Legal and notice information

© Copyright 2009-2010 Hewlett-Packard Development Company, L.P.

Intel® is a trademark of Intel Corporation in the U.S. and other countries.

Microsoft® and Windows® are U.S. registered trademarks of Microsoft Corporation.

Page 3: HP StorageWorks Fibre Channel Over Ethernet Application Note

DescriptionThe HP FCoE solution deploys FCoE technology in existing Ethernet and FC environments with thebenefit of convergence at the server and fabric edge using CNAs and CN switches.

The HP FCoE solution provides this benefit in addition to investment protection by allowing FCoE tobe integrated with existing HP Fibre Channel SANs. The FCoE product set consists of the following:

• HP B-series and C-series CN switches—Provide a bridge function from the CN to separate Ethernetand FC fabrics

• HP CN1000E CNAs, Emulex LP2100x CNAs, and QLogic 8100 Series CNAs—Converge Ethernetand FC technologies in the server over 10-GbE links to converged network switches

FCoE CN switches and HP CN1000E CNAs are available from HP. Emulex LP2100x CNAs andQLogic 8100 Series CNAs are available through the HP 3PO program or authorized resellers anddistributors as reference-sell products.

BenefitsThis section describes some of the benefits of FCoE technology. The convergence of Ethernet and FCprovides the same level of connectivity as previous technologies, but requires:

• 50% fewer switches in each server rack—Only two CN ToR switches, compared with four (twonetwork and two FC) switches per rack with separate Ethernet and FC switches

• 50% fewer adapters per server• 75% fewer cable connections (see Table 1)

Figure 1 illustrates a rack with 16 servers using FCoE technology compared to a rack using separateFC and Ethernet with the same level of connectivity. Similarly, Table 1 provides a comparison of cablerequirements for these same racks.

Table 1 Rack cable requirements comparison

Converged FCoE network1Separate FC and Ethernet networks

(2) 10-GbE cablesnl

(2) cables at 20 Gb per server

(4) 4-Gb FC cablesnl

(4) 1-GbE NIC cablesnl

(8) cables at 20 Gb per server

Total of 32 cables for 320 Gb of connectivityTotal of 128 cables for 320 Gb of connectivity

1Using copper cables for FCoE connections further reduces the per-port connection cost versus the cost of optical SFP+nl

transceivers.

HP StorageWorks Fibre Channel Over Ethernet Application Note 3

Page 4: HP StorageWorks Fibre Channel Over Ethernet Application Note

Figure 1 Rack cable comparison.

Converged network switchesThe first generation of FCoE CN switches enable Ethernet and FC to coexist in a fabric and aredesigned as edge switches or ToR switches. Edge/ToR switches are typically deployed in redundantpairs installed at the top of a server rack (see Figure 1). By using FCoE CN switches, you reduce thenumber of required switches by replacing separate Ethernet and FC switches with a convergededge/ToR switch. From the edge/ToR switch, ISL connections to the EoR FC and EoR Ethernet/IPswitches provide access to the separate core layers of the FC fabric and Ethernet/IP network.

An FCoE blade that resides in a DC or DC04 SAN Director can also be used to integrate FCoEsolutions with existing or new FC fabrics.

4

Page 5: HP StorageWorks Fibre Channel Over Ethernet Application Note

The following HP FCoE CN switches and blades are available:

• HP StorageWorks 2408 FCoE Converged Network Switch• HP StorageWorks DC SAN Director Switch 10/24 FCoE Blade• HP C-series Nexus 5010 Converged Network Switch• HP C-series Nexus 5020 Converged Network Switch

Converged network switch portsHP FCoE CN switches have two types of physical ports:

• 10-GbE ports• 4-Gb or 8-Gb FC ports

FC ports can be used for either ISL connections to an existing FC fabric, or connection directly to anHP storage array.

Table 2 lists the number of ports for each HP FCoE CN switch.

Table 2 Number of ports per switch

Number of FC portsNumber of 10-GbE portsHP switch

8 (8 Gb/s)242408 FCoE Converged Network Switch

0 external portsnl

32 (8 Gb/s) backplane ports24DC SAN Director Switch 10/24 FCoE Blade

0 to 8 (4 Gb/s)nl

0 to 6 (8 Gb/s)20 to 26C-series Nexus 5010 Converged Network

Switch

0 to 16 (4 Gb/s)nl

0 to 12 (8 Gb/s)40 to 52C-series Nexus 5020 Converged Network

Switch

NOTE:All FCoE CN switch models are by design CEE/IEEE DCB switches and are enabled for Ethernet bydefault. In order to be used for FCoE they must be enabled to forward FC frames to the FC ports.

HP does not support interoperability between different vendors' CN and FC switches:

• 2408 FCoE Converged Network Switch operate with B-series FC switches only.• C-series Nexus 5010/5020 Converged Network Switches operate with other C-series MDS FC

switches only.

NOTE:The FCoE CN switch firmware update process will result in momentary loss of Ethernet and FibreChannel port connectivity. HP recommends implementing a dual path configuration with redundantFCoE CN switches to minimize disruption during firmware updates.

HP StorageWorks Fibre Channel Over Ethernet Application Note 5

Page 6: HP StorageWorks Fibre Channel Over Ethernet Application Note

HP StorageWorks 2408 FCoE Converged Network SwitchThe 2408 FCoE Converged Network Switch is a rebranded version of the Brocade 8000 FCoE Switch(Figure 2).

Figure 2 2408 FCoE Converged Network Switch.

Features• Next-generation Ethernet L2/L3 switch and next-generation FC switch merged into one product• CEE support

• Full industry-standard implementation• Supports FCoE and FCoE Initialization Protocol (FIP)

• 24 10-GbE and 8 8-Gb/s FC ports• Uses 1 RU of space• Leverages existing Brocade fabric operating system (FOS)• Supports link aggregation (LAG) on DCB ports• Supports Brocade ISL trunking on FC ports

Considerations• The port types are fixed. You cannot use 10-GbE ports for FC connections, and you cannot use

FC ports for 10-GbE connections.• L3 routing features are not currently supported.• 10-GbE ports support virtual F_Ports only (virtual E_Ports are not supported).

6

Page 7: HP StorageWorks Fibre Channel Over Ethernet Application Note

HP StorageWorks DC SAN Director Switch 10/24 FCoE BladeThe HP DC SAN Director Switch 10/24 FCoE Blade is a rebranded version of the Brocade FCOE10-24blade (Figure 3).

26608a

10 GbE ports

Figure 3 HP DC SAN Director Switch 10/24 FCoE Blade.

Features• 24 10-GbE ports• 32 8-Gb/s FC ports on the backplane (DC and DC04 SAN Directors only)• Hot pluggable• Blade power and status LEDs• Link status LEDs for each port• FCoE switching• CEE switching• L2 Ethernet protocols STP/MSTP/RSTP, VLAN tagging, and link aggregation• Standard Ethernet encapsulation

Considerations• The DC SAN Director Switch 10/24 FCoE Blade has 24 FCoE ports and can be installed in either

a DC or DC04 SAN director.• The DC SAN Director Switch 10/24 FCoE Blade enables server edge connectivity by connecting

CNAs directly to any of its 24 Ethernet ports.• Storage can be connected to:

• Any other Fibre Channel blade in the same director• Any Fibre Channel switch that is in the same fabric as the DC or DC04 SAN Director that

contains the DC SAN Director Switch 10/24 FCoE Blade

• The DC SAN Director Switch 10/24 FCoE Blade supports optical cabling and SFP+ transceiversonly.

• There are no licensing requirements for this blade.

HP StorageWorks Fibre Channel Over Ethernet Application Note 7

Page 8: HP StorageWorks Fibre Channel Over Ethernet Application Note

HP C-series Nexus 5010/5020 Converged Network SwitchesHP resells the Cisco Nexus 5010 and 5020 Switches (Figure 4).

10-GbE portsFC ports 26651a

Figure 4 C-series Nexus 5010 and 5020 Converged Network Switches.

Features• L2 access:

• IEEE DCB and FCoE supportIEEE DCB is a Cisco unified fabric product with additional proprietary features.

• NX-OS with combined features from Cisco IOS and Cisco MDS 9000 SAN-OS/NX-OS

• Cisco 5020 (2 RU):• 40 10-GbE ports• Two optional expansion module slots• Up to 52 10-GbE ports, or a combination of 40 10-GbE ports and 16 FC ports

• Cisco 5010 (1 RU):• 20 10-GbE ports• One optional expansion module slot• Up to 26 10-GbE ports, or a combination of 20 10-GbE ports and 8 FC ports

• Optional expansion modules:• 6-port 10-GbE expansion module• 6-port 8-Gb/s FC expansion module• 8-port 4-Gb/s FC expansion module• 4-port 4-Gb/s FC and 4-port 10-GbE expansion module

8

Page 9: HP StorageWorks Fibre Channel Over Ethernet Application Note

Converged network adaptersServers with CNAs require fewer adapters per server and, therefore, 75% fewer cable connectionsfor network and FC attachments.

HP offers the HP CN1000E CNA (Figure 5) and also supports the Emulex LightPulse LP21000 familyof CNAs and QLogic 8100 Series CNAs in certain ProLiant servers. For a list of servers that supportCNAs, see HP ProLiant server support on page 22.

HP CNAsTable 3 lists the supported HP CNA available from HP.

Table 3 HP CNA part numbers

DescriptionHP modelnl

numberHP part number

Dual port converged network adapter (no cable or transceiver isincluded)CN1000EAW520A

Figure 5 shows an HP CN1000E CNA.

Figure 5 HP CN1000E CNA.

HP StorageWorks Fibre Channel Over Ethernet Application Note 9

Page 10: HP StorageWorks Fibre Channel Over Ethernet Application Note

Figure 6 shows a diagram of the internal components of an HP CN1000E CNA.

Figure 6 HP CNA internal components.

FeaturesThe HP CN1000E CNA has the following features:

• Ships with half-height and full-height brackets• Dual ports for redundancy• Full 10-Gb/s bandwidth on both ports• Each port can operate as a NIC and/or FCoE port• 2 SFP+ connectors• Supports optical or copper cables

Considerations• x8 PCI Express Gen2 card• Requires 14.5 W of power• 1 GbE is not supported

Emulex CNAsTable 4 lists the supported Emulex CNAs available as a reference sell from HP 3PO.

Table 4 Emulex CNA part numbers

DescriptionEmulex partnumber

HP 3PO partnumber1

FCoE and 10-GbE single channel optical CNA (includes one SFP+SR transceiver)LP21000-MA711956

FCoE and 10-GbE single channel copper CNA (includes one 3-mSFP+ Twinax copper cable)2

LP21000-CA711954

10

Page 11: HP StorageWorks Fibre Channel Over Ethernet Application Note

DescriptionEmulex partnumber

HP 3PO partnumber1

FCoE and 10-GbE dual channel optical CNA (includes two SFP+SR transceivers)LP21002-MA711957

FCoE and 10-GbE dual channel copper CNA (includes two 3-mSFP+ Twinax copper cables)2

LP21002-CA712947

FCoE and 10-GbE single channel copper CNA (no cable ortransceiver is included)LP21000-NCA745955

FCoE and 10-GbE dual channel copper CNA (no cables or trans-ceivers are included)LP21002-NCA745954

1HP sales representatives use the HP 3PO part numbers to order these parts from an Emulex-authorized distributor through theHP Vista procurement process. The parts are not labeled with an HP part number or identified as HP parts. HP supports theuse of these products in HP FCoE configurations. However, customer service support for CNAs is provided by Emulex.2The copper cable included with this CNA is supported with the C-series Nexus 5010/20 Converged Network Switch only.It is not supported with the 2408 FCoE Converged Network Switch or the DC SAN Director Switch 10/24 FCoE Blade.

Figure 7 shows an Emulex LP21002-M CNA. Other Emulex CNAs are similar in appearance.

Figure 7 Emulex LP21002-M CNA.

HP StorageWorks Fibre Channel Over Ethernet Application Note 11

Page 12: HP StorageWorks Fibre Channel Over Ethernet Application Note

Figure 8 shows a diagram of the internal components of an Emulex CNA.

Figure 8 Emulex CNA internal components.

FeaturesThe Emulex CNAs have the following features:

• Emulex 4-Gb/s FC dual-port controller• Dual-port 10-GbE Intel Opland 82598 NIC• 10-GbE and FC multiplexed through a Cisco Menlo ASIC

Considerations• Requires a full-height/full-length x8 PCIe slot• Requires 25 W of power• 1 GbE is not supported

QLogic CNAsTable 5 lists the supported QLogic CNAs available as a reference sell from HP 3PO.

Table 5 QLogic CNA part numbers

DescriptionQLogic partnl

numberHP 3PO partnumber1

10-GbE dual copper (no transceivers, copper cables sold separately)QLE8152-CU-CKA755983

10-GbE single copper (no transceivers, copper cables sold separately)QLE8150-CU-CKA765750

10-GbE dual optical (ships with two SR optical transceiver)QLE8142-SR-CKA765749

10-GbE single optical (ships with one SR optical transceiver)QLE8140-SR-CKA765751

1HP sales representatives use the HP 3PO part numbers to order these parts from a QLogic-authorized distributor through theHP Vista procurement process. The parts are not labeled with an HP part number or identified as HP parts. HP supports theuse of these products in HP FCoE configurations. However, customer service support for CNAs is provided by QLogic.

12

Page 13: HP StorageWorks Fibre Channel Over Ethernet Application Note

Figure 9 shows a QLogic QLE8142 CNA. Other QLogic CNAs are similar in appearance.

Figure 9 QLE8142 CNA.

Figure 10 shows a diagram of the internal components of a QLogic CNA.

Figure 10 QLogic CNA internal components.

FeaturesThe QLogic CNAs have the following features:

• Full hardware offload for FCoE protocol processing• Full support for TCP/IP and Ethernet performance enhancements, such as priority-based flow

control (802.1Qbb), jumbo frames, checksum offloads, and segmentation offloads

HP StorageWorks Fibre Channel Over Ethernet Application Note 13

Page 14: HP StorageWorks Fibre Channel Over Ethernet Application Note

Considerations• Low profile PCI Express card• Requires 6.9 W of power• 1 GbE is not supported

Configuration rulesThis section describes configuration rules for the HP FCoE solution. The rules are defined for the CNAsand each of the FCoE CN switch series.

Figure 11 shows a converged SAN fabric with servers using CNAs only.

26647a

Serverwith CNA

IP network

Serverwith CNA

X-seriesFCoE switch

FC attachedHP storage

Fabric A

X-seriesFC switch

Serverwith CNA

Serverwith CNA

X-seriesFCoE switch

FC attachedHP storage

10-GbE

Fabric B

X-seriesFC switch

X-seriesFCoE switch

X-seriesFCoE switch

10-GbE/FCoE A connection10-GbE/FCoE B connectionFabric A Fibre Channel connectionFabric B Fibre Channel connection10-GbE connection

Figure 11 Converged SAN fabric configuration.

14

Page 15: HP StorageWorks Fibre Channel Over Ethernet Application Note

Figure 12 shows a converged SAN fabric with servers using CNAs connected to a B-series CN switchand a DC SAN Director Switch 10/24 FCoE Blade.

26648a

Serverwith CNA

IP network

Serverwith CNA

B-seriesCN switch

FC attachedHP storage

Fabric A

B-seriesFCoEblade

Serverwith CNA

Serverwith CNA

FC attachedHP storage

10-GbE

Fabric B

B-seriesCN switch

B-seriesFCoEblade

10-GbE/FCoE A connection10-GbE/FCoE B connectionFabric A Fibre Channel connectionFabric B Fibre Channel connection10-GbE connection

Figure 12 Converged SAN fabric with an FCoE blade.

HP StorageWorks Fibre Channel Over Ethernet Application Note 15

Page 16: HP StorageWorks Fibre Channel Over Ethernet Application Note

Figure 13 shows a converged fabric that has servers with CNAs and servers with FC HBAs and NICs.

26649a

Serverwith FC HBA

and NIC

IP network

Serverwith FC HBA

and NIC

X-seriesCN switch

FC attachedHP storage

Fabric A

X-seriesFC switch

Serverwith CNA

Serverwith CNA

FC attachedHP storage

10-GbE

X-seriesCN switch

Fabric B

X-seriesFC switch

10-GbE/FCoE A connection10-GbE/FCoE B connectionFabric A Fibre Channel connectionFabric B Fibre Channel connection10-GbE connection

Figure 13 FCoE integrated with FC SAN fabric.

16

Page 17: HP StorageWorks Fibre Channel Over Ethernet Application Note

CNA configuration rulesTable 6 HP CN1000E requirements

DescriptionItem

2Number of CNAs per server(maximum)

• SFP+ optical• CopperCable types

2.701.374.8Firmware

• Windows: 2.30.018• SLES11: 8.2.8.32.1p• SLES10 and RHAS 5.4: 8.2.0.63.1p

FC driver versions (minimum)

• Windows: 2.101.374.3• SLES11: 2.101.374.0• SLES10 and RHAS 5.4: 2.101.374.0

NIC driver versions (minimum)

Table 7 Emulex LP21000/LP21002 requirements

DescriptionItem

• DL580-G5 and DL585-G5 support 2 CNAs• All others support 1 CNA

Number of CNAs per server(maximum)

• SFP+ optical• CopperCable types

0.5a5 or laterCEE firmware

1.00x1 or laterFC firmware

• Intel NIC Boot 13.5 or later• Emulex FC Universal Boot Version 5.03a5/x86 BootBIOS 2.02a2 or

laterBoot versions

• Windows 2003/2008 (x86, x64) 2.20.006• RHEL 5 Update 3,1 Update 2 (x86, x64) 8.2.0.33.3p• SUSE 11,1 10 Update 2 (x86, x64) 8.2.0.29• ESX 3.5 (x86) 7.4.0.41

FC driver versions (minimum)

HP StorageWorks Fibre Channel Over Ethernet Application Note 17

Page 18: HP StorageWorks Fibre Channel Over Ethernet Application Note

DescriptionItem

• Windows 2003/2008 (x86, x64) 1.3.10.0• RHEL 5 Update 2 (x86, x64) 1.3.16.1• SUSE 10 Update 2 (x86, x64) 1.3.16.5• ESX 3.5 (x86) In Box Driver

NIC driver versions (minimum)

1RHEL 5.3 and SLES 11 are supported with the B-series CN switch and FCoE blade only.

Table 8 QLogic 8100 Series requirements

DescriptionItem

Up to 2Number of CNAs per server(maximum)

• SFP+ optical• CopperCable types

1.01.54 or later—Which includes:• RISC FW: 5.02.01• MPI FW: 1.35.11• BIOS: 2.14• EFI: 3.16• PXE BOOT: 1.05• NIC EFI: 1.06

Multiboot package

• Windows 2003/2008 (x86, x64) 9.1.8.19• RHEL 5 Update 4 (x86, x64) 8.03.00.10.05.04-k• RHEL 5 Update 3 (x86, x64) 8.03.00.09.05.04-k• SUSE SLES 11 (x86, x64) 8.03.00.08.11.0-k4• SUSE SLES 10 SP3 (x86, x64) 8.03.00.10.10.3-k4• ESX/ESXi 4.0 831.k1.23vmw

FC driver versions (minimum)

• Windows 2003 NDIS5 32-bit, x64 NIC 1.0.1.0• Windows 2008 NDIS6 32-bit, x64 NIC 1.0.1.0• RHEL 5 Update 4, Update 3 (x86, x64) NIC 1.0.0.22• SUSE SLES 11, 10 SP3 (x86, x64) NIC 1.0.0.22• ESX/ESXi 4.0 NIC 1.0.0.36

NIC driver versions (minimum)

18

Page 19: HP StorageWorks Fibre Channel Over Ethernet Application Note

FCoE CN switch configuration rulesTable 9 DC SAN Director Switch 10/24 FCoE Blade and 2408 FCoE Converged Network Switch fabricrules

DescriptionRule number

A 2408 FCoE Converged Network Switch or DC SAN Director Switch 10/24 Blade can bea standalone switch or an edge switch in a Fibre Channel fabric. To attach the switch to anexisting Fibre Channel fabric as an edge switch, at least one Fibre Channel port on the FCoECN switch must be connected to a Fibre Channel switch in the fabric (E_Port). There cannotbe any other FCoE or 10-GbE CEE switches in the path to the Fibre Channel switch.Similarly, a DC or DC04 SAN Director with a 10/24 FCoE Blade installed can be astandalone switch, an edge switch, or a core switch in a Fibre Channel fabric. If it is an edgeswitch or core switch, other switches in the fabric can be attached to any available FC port(E_Port) on other FC blades in the director.For FCoE E_Port connectivity, see the appropriate Fabric OS release notes for the minimumand recommended Fibre Channel switch firmware versions and the supported Fibre Channelswitch models. HP recommends Brocade FOS 6.3.0b or later.

1

Server CNAs must be directly connected to the CEE ports on a 2408 FCoE Converged NetworkSwitch or DC SAN Director Switch 10/24 FCoE Blade.

2

Fibre Channel storage can be directly connected to the 2408 FCoE Converged NetworkSwitch (one of the eight FC F_Ports) or to any Fibre Channel switch that is part of the samefabric as the 2408 FCoE Converged Network Switch.With the DC SAN Director Switch 10/24 FCoE Blade, storage can be connected to any ofthe other FC blades in the same director or to any Fibre Channel switch that is part of thesame fabric as the DC SAN Director that contains the DC SAN Director Switch 10/24 FCoEBlade.

3

Supports a maximum of seven switch hops (eight switches) between a CNA and storage.4

Supports the following CEE media types:• HP B-series hot pluggable, 10-GbE SFP+ Short Reach (SR) and Long Reach (LR) optical

transceivers• HP B-series SFP+ 1 m, 3 m, or 5 m copper Twinax active cables (2408 FCoE Converged

Network Switch only)

5

Up to 4,000 VLANs, but only one FCoE VLAN is currently supported6

Only FPMA is currently supported.7

Supports existing B-series Fibre Channel rules and features except for the following:• Advanced Zoning (port-based and QoS)• Extended Fabrics• Adaptive Networking with QoS• Virtual Fabrics• Integrated Fibre Channel Routing• FICON• Access Gateway• Admin Domains• TI Zones• M-EOS Interop• Nondisruptive firmware upgrade

8

HP StorageWorks Fibre Channel Over Ethernet Application Note 19

Page 20: HP StorageWorks Fibre Channel Over Ethernet Application Note

DescriptionRule number

A maximum of two DC SAN Director Switch 10/24 FCoE Blades per chassis are supportedin DC and DC04 SAN Directors.9

The DC SAN Director Switch 10/24 FCoE Blade does not support Twinax copper cables(SFP+).10

DC and DC04 SAN Directors do not currently support concurrent installation of the DC SANDirector Switch 10/24 FCoE Blade and other blades, except for the FC8-xx or FC10-6 blades.11

Table 10 C-series FCoE Converged Network switch fabric rules

DescriptionRule number

A C-series FCoE CN switch can be a standalone switch or an edge switch in a Fibre Channelfabric. To attach the switch to an existing Fibre Channel fabric as an edge switch, at leastone Fibre Channel port on the FCoE CN switch must be connected to a Fibre Channel switchin the fabric (E_Port). There cannot be any other FCoE or 10-GbE IEEE DCB switches in thepath to the Fibre Channel switch.For FCoE E_Port connectivity, the Fibre Channel switch minimum firmware version is NX-OS4.1(3)N1(1). All C-series 4 Gb/s or 8 Gb/s switch models are supported when using theminimum firmware version.MDS switches running SAN-OS 3.x can be in the Fibre Channel SAN but cannot be connecteddirectly to an FCoE switch.

1

Server CNAs must be connected directly to FCoE CN switch IEEE DCB ports (pre-FIP supportonly).

2

Fibre Channel storage can be connected directly to the FCoE CN switch (F_Port) or to a FibreChannel switch (F_Port) in the same fabric as the FCoE CN switch. SAN boot is not supportedwhen storage systems are connected directly to FCoE Fibre Channel ports.

3

Supports a maximum of three switch hops (four switches) between a CNA and storage.4

Supports the following IEEE DCB media types:• HP C-series hot pluggable, 10-GbE SFP+ Short Reach (SR) optical transceivers• HP C-series SFP+ 3m or 5m copper Twinax passive cables• Emulex SFP+ copper Twinax passive cables1

5

Supports up to 50 switches in a single physical fabric or VSAN6

Supports up to 40 domains per VSAN7

Supports a maximum of 250 VLANs per switch8

Supports a maximum of 32 VSANs per switch9

Supports a maximum of 32 zones per virtual or physical F_Port (includes all VSANs)10

Supports a maximum of 500 zone sets per switch (includes all VSANs)11

Supports a maximum of 8,000 zone members per physical fabric (includes all VSANs)12

Supports a maximum of 8,000 zones per switch (includes all VSANs)13

Supports up to 52 virtual Fibre Channel interfaces per switch14

1Supported when used with Emulex LP2100x CNAs only.

20

Page 21: HP StorageWorks Fibre Channel Over Ethernet Application Note

American National Standard of Accredited Standards Committee INCITS FibreChannel Backbone - 5 revision 2.0 conformance

Earlier revisions of the FCoE Initialization Protocol were called FIP0. The Fibre Channel Backbone -5 (FC-BB-5) revision 2.0 standard is identified as FIP1. The switches supported by HP support bothFIP0 and FIP1 on the DCB ports. The Emulex LP21000 CNA supports FIP0 and the QLogic 8100Series CNA supports FIP1.

Table 11 FIP support

Support for FIP1Support for FIP0Device

YesNoHP CN1000E CNA

NoYesEmulex LP2100 CNA

YesNoQLogic 8100 Series CNA

Yes1YesHP 2408 FCoE Converged Network Switch

Yes1YesHP DC SAN Director Switch 10/24 FCoE Blade

YesYesHP C-series Nexus 5010 Converged Network Switch

YesYesHP C-series Nexus 5020 Converged Network Switch

1FOS 6.3.0b or later is required for FIP1 support.

HP StorageWorks Fibre Channel Over Ethernet Application Note 21

Page 22: HP StorageWorks Fibre Channel Over Ethernet Application Note

HP ProLiant server supportFCoE is supported with ProLiant G5 servers with a full-height/full-length x8 PCIe slot using the EmulexLP21000 family of CNA modules and with ProLiant G5 and G6 servers with a x8 PCIe slot using theQLogic 8100 Series of CNA modules. The server models that HP supports for use with CNAs are asfollows:

Models supported with EmulexLP21000 family of CNAs

Models supported with QLogic8100 Series of CNAs

Models supported with HPCN1000E CNAs

• DL120-G5• DL160-G5• DL160-G5p• DL165-G5• DL165-G5p• DL185-G5• DL320-G5p• DL360-G5• DL365-G5• DL380-G5• DL385-G5• DL385-G5p• DL580-G5• DL585-G5• DL785-G5

• DL160 G5• DL160 G5p• DL160 G6• DL165 G6• DL180 G6• DL360 G5• DL360 G6• DL365 G5• DL370 G6• DL380 G5• DL380 G6• DL385 G5• DL385 G5p• DL385 G6• DL580 G5• DL585 G5• DL585 G6• DL785 G5• DL785 G6• ML350 G6• ML370 G6

• DL120 G5• DL120 G5p• DL160 G5• DL160 G5p• DL160 G6• DL165 G5• DL165 G5p• DL165 G6• DL180 G5• DL180 G5p• DL180 G6• DL185 G5• DL185 G5p• DL320 G5p• DL320 G6• DL360 G5• DL360 G5p• DL360 G6• DL365 G5• DL365 G5p• DL380 G5• DL380 G5p• DL380 G6• DL385 G5p• DL385 G6• DL580 G5• DL580 G5p• DL585 G5• DL585 G5p• DL585 G6• DL785 G5• DL785 G5p• DL785 G6• ML350 G6• ML370 G6

22

Page 23: HP StorageWorks Fibre Channel Over Ethernet Application Note

Storage system supportThis section describes storage system support for access from CNA-based servers. These storagesystems can be attached to Fibre Channel switches in the fabric or connected to the Fibre Channelports of an FCoE CN switch.

SAN boot is not supported when storage systems are connected to the Fibre Channel ports of anFCoE CN switch. SAN boot is supported for storage attached to the Fibre Channel ports in a SANDirector that contains a 10/24 FCoE blade and based on the current support listed for Fibre Channelswitches.

Table 12 Storage systems supported with HP CN1000E CNAs

Controller software version (minimum)Storage array

M110R21MSA2000fc G2 (MSA2312fc/MSA2324fc)

J200P39MSA2000fc (MSA2012fc/MSA2212fc)

TS100R023P2000 G3

09522000EVA4400

EVA4400 with embedded switch1

6.220EVA4100/6100/8100

09522000EVA6400/8400

50.09.86.00/00XP10000/12000

60.05.14.00/02XP20000/24000

1The EVA4400 with the embedded switch is not supported for connectivity to C-series Fibre Channel switches or Nexus Con-verged Network Switches.

HP StorageWorks Fibre Channel Over Ethernet Application Note 23

Page 24: HP StorageWorks Fibre Channel Over Ethernet Application Note

Table 13 Storage systems supported with Emulex LP21000/LP21002 CNAs

Controller software version (minimum)Storage array

M100R18MSA2000fc G2 (MSA2312fc/MSA2324fc)

J200P30MSA2000fc (MSA2012fc/MSA2212fc)

09006000EVA4400

EVA4400 with embedded switch1

6.200EVA4100/6100/8100

09501100EVA6400/8400

50.09.78.00XP10000/12000

60.04.04.00XP20000/24000

1The EVA4400 with the embedded switch is not supported for connectivity to C-series Fibre Channel switches or Nexus Con-verged Network Switches.

Table 14 Storage systems supported with QLogic 8100 Series CNAs

Controller software version (minimum)Storage array

M110R21MSA2000fc G2 (MSA2312fc/MSA2324fc)

J200P39MSA2000fc (MSA2012fc/MSA2212fc)

09522000EVA4400

EVA4400 with embedded switch1

6.220EVA4100/6100/8100

09522000EVA6400/8400

50.09.86.00/00XP10000/12000

60.05.14.00/02XP20000/24000

1The EVA4400 with the embedded switch is not supported for connectivity to C-series Fibre Channel switches or Nexus Con-verged Network Switches.

24

Page 25: HP StorageWorks Fibre Channel Over Ethernet Application Note

Operating system supportFCoE is supported with the following operating systems:

• Microsoft Windows 2003 SP2 (x86, x64)• Microsoft Windows 2008 SP2, R2 (x86, x64)• Red Hat Enterprise Linux 5, Update 4 (x86, x64)• SUSE Linux Enterprise Server 10, SP3 (x86, x64)• SUSE Linux Enterprise Server 11 (x86, x64)

HP CN1000E CNA

• Microsoft Windows 2003 (x86, x64)• Microsoft Windows 2008 (x86, x64)• Red Hat Enterprise Linux 5, Update 2 (x86, x64)• Red Hat Enterprise Linux 5, Update 3 (2408 FCoE Converged

Network Switch and DC SAN Director Switch 10/24 FCoEBlade only)

• SUSE Linux Enterprise Server 10, Update 2 (x86, x64)• SUSE Linux Enterprise Server 11 (2408 FCoE Converged Net-

work Switch and DC SAN Director Switch 10/24 FCoE Bladeonly)

• VMware ESX 3.5 (x86)

Emulex LP21000/LP21002 CNAs

• Microsoft Windows 2003 SP2 (x86, x64)• Microsoft Windows 2008 SP2, R2 (x86, x64)• Red Hat Enterprise Linux 5, Update 3 (x86, x64)• Red Hat Enterprise Linux 5, Update 4 (x86, x64)• SUSE Linux Enterprise Server 10, SP3 (x86, x64)• SUSE Linux Enterprise Server 11 (x86, x64)• VMware ESX 4.0, 4.0 U1 (x86)

QLogic 8100 Series CNAs

Boot from SAN supportBFS is not supported for FC arrays attached to 2408 FCoE Converged Network Switches or C-seriesNexus 5000 Converged Network Switches at this time. BFS is supported for FC arrays attached tostandard FC switches in a fabric that includes CN switches.

DC SAN Director Switch 10/24 FCoE Blade supports BFS from the storage attached to any of theother FC blades in the same director or to any Fibre Channel switch that is part of the same fabric asthe DC SAN Director that contains the 10/24 FCoE Blade.

This section provides information about installing BFS software.

HP StorageWorks Fibre Channel Over Ethernet Application Note 25

Page 26: HP StorageWorks Fibre Channel Over Ethernet Application Note

Windows 2003 (x86, x64) and 2008 (x86, x64)• BFS is supported on all supported HP servers.• The information in HP StorageWorks Fibre Channel Host Bus Adapters Software Guide for Windows

(available on http://h18006.www1.hp.com/storage/networking/bootsan.html) is accurate forWindows BFS on a CNA based server for Windows 2003 with the following exceptions:• You must install one of the following BFS kits:

• An HP CNA driver kit (available for download at the HP website:nl

http://www.hp.com/support/downloads).• A 2.20.006 driver kit (available for download at the Emulex website

nl

http://www.emulex.com/downloads/emulex/cnas-and-hbas/drivers/windows/storport-miniport-driver.html).

• A BFS driver kit (available for download at the QLogic websitenl

http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/default.aspx).You can download drivers by selecting your CNA Model and Operating System.

• For Windows 2003, press F6 to prompt for the driver disk• For Windows 2008, click Load Driver when prompted for the location in which to install

Windows (see Figure 14).

Figure 14 Install Windows 2008 (x86, x64).

26

Page 27: HP StorageWorks Fibre Channel Over Ethernet Application Note

Red Hat Enterprise Linux

NOTE:Red Hat Enterprise Linux 5 Update 2 is supported with x86 and x64. Red Hat Enterprise Linux 5,Update 3 is supported with the 2408 FCoE Converged Network Switch and DC SAN Director Switch10/24 FCoE Blade only.

• BFS is supported on all supported HP servers.• The information in HP StorageWorks Fibre Channel Host Bus Adapters Software Guide for Linux

(available at http://h18006.www1.hp.com/storage/networking/bootsan.html) is accurate forBFS on a CNA based server with the following exceptions:• One of the following driver kits is required:

• An HP CNA driver kit (available for download at the HP website:nl

http://www.hp.com/support/downloads).• An Emulex driver kit (available for download at the Emulex website http://

www.emulex.com/downloads/emulex/cnas-and-hbas/drivers/linux.html).• A QLogic BFS driver kit (available for download at the QLogic website

nl

http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/default.aspx). You candownload drivers by selecting your CNA Model and Operating System.

• Use the linux dd mpath rather than the linux mpath command to prompt to load adriver and enable multipath for the installation.

SUSE Linux Enterprise Server

NOTE:SUSE Linux Enterprise Server 10 Update 2 is supported with x86 and x64. SUSE Linux EnterpriseServer 11 is supported with the 2408 FCoE Converged Network Switch and DC SAN Director Switch10/24 FCoE Blade only.

• BFS is supported on all supported HP servers.• The information in HP StorageWorks Fibre Channel Host Bus Adapters Software Guide for Linux

(available at http://h18006.www1.hp.com/storage/networking/bootsan.html) is accurate forBFS on a CNA based server with the following exceptions:• One of the following driver kits is required

• An Emulex driver kit (available for download at the Emulex website http://www.emulex.com/downloads/emulex/cnas-and-hbas/drivers/linux.html). Do not use theiso image for Red Hat.

• A QLogic BFS driver kit (available for download at the QLogic websitenl

http://driverdownloads.qlogic.com/QLogicDriverDownloads_UI/default.aspx). You candownload drivers by selecting your CNA Model and Operating System.

• Use the GUI to load the driver (Figure 15).

HP StorageWorks Fibre Channel Over Ethernet Application Note 27

Page 28: HP StorageWorks Fibre Channel Over Ethernet Application Note

Figure 15 SLES installation.

Multipathing software supportTable 15 Multipathing software supported with HP CN1000E CNAs

Version (minimum)Multipathingnl

softwareOperating system

4.00.00 (XP, EVA)nl

nl2.5.1.29 (MSA2K)HP MPIO FF DSMMicrosoft Windows 2003 SP2 (x86, x64)

4.00.00 (XP, EVA)HP MPIO FF DSMMicrosoft Windows 2008 SP2, R2 (x86, x64)

Embedded (MSA2K)Microsoft MPIO Basic

4.3.1Device Mapper

Red Hat Enterprise Linux 5, Update 3 (x86, x64)Red Hat Enterprise Linux 5, Update 4SUSE Linux Enterprise Server 10, SP3 (x86, x64)SUSE Linux Enterprise Server 11 (x86, x64)

Table 16 Multipathing software supported with Emulex LP21000/LP21002 CNAs

Version (minimum)Multipathingnl

softwareOperating system

3.02.00HP MPIO FF DSMMicrosoft Windows 2003 (x86, x64)

3.02.00HP MPIO FF DSMMicrosoft Windows 2008 (x86, x64)

Embedded (MSA2K)Microsoft MPIO Basic

28

Page 29: HP StorageWorks Fibre Channel Over Ethernet Application Note

Version (minimum)Multipathingnl

softwareOperating system

4.1.1Device Mapper

Red Hat Enterprise Linux 5, Update 2 (x86, x64)Red Hat Enterprise Linux 5, Update 3 (2408 FCoEConverged Network Switch and DC SAN Director Switch10/24 FCoE Blade only)SUSE Linux Enterprise Server 10, Update 2 (x86, x64)SUSE Linux Enterprise Server 11 (2408 FCoE ConvergedNetwork Switch and DC SAN Director Switch 10/24 FCoEBlade only)

Table 17 Multipathing software supported with QLogic 8100 Series CNAs

Version (minimum)Multipathingnl

softwareOperating system

4.00.00 (XP, EVA-XL)nl

2.5.1.29 (MSA2K)HP MPIO FF DSMMicrosoft Windows 2003 SP2 (x86, x64)

4.00.00 (XP, EVA-XL)HP MPIO FF DSMMicrosoft Windows 2008 SP2 R2 (x86, x64)

Embedded (MSA2K)Microsoft MPIO Basic

4.3.1Device Mapper

Red Hat Enterprise Linux 5, Update 3 (x86, x64)Red Hat Enterprise Linux 5, Update 4SUSE Linux Enterprise Server 10, SP3 (x86, x64)SUSE Linux Enterprise Server 11 (x86, x64)

Quick-setup instructionsFCoE CN switches have dual capabilities in that they serve as both an Ethernet switch and an FCswitch. You must perform a setup procedure to achieve the desired function.

If you are familiar with the FCoE CN switch vendor’s FC switches, then you are familiar with thecommands required to configure that portion of the FCoE CN switch setup. The procedures in thissection are intended as a quick-start guide to configuring the FCoE ports on an FCoE CN switch toaccess the FC ports on the FCoE CN switch and other attached FC switches.

2408 FCoE Converged Network Switch and DC SAN Director Switch 10/24FCoE Blade quick setup

This procedure is intended for users who are familiar with Brocade FC switches and have experiencemerging B-series FC switches into an existing FC fabric. Use this procedure to enable servers withCNAs attached to the 2408 FCoE Converged Network Switch or the DC SAN Director Switch 10/24FCoE Blade to access devices on the attached B-series FC fabric.

HP StorageWorks Fibre Channel Over Ethernet Application Note 29

Page 30: HP StorageWorks Fibre Channel Over Ethernet Application Note

IMPORTANT:If you are not familiar with Brocade FC switches or you do not have experience merging B-series FCswitches into an existing FC fabric, use the detailed instructions found in the switch user guide to setup your switch.

1. Configure LLDP for FCoE (common to all CEE ports).

switch:admin> cmshswitch# enableswitch# config terminalswitch(config)# protocol lldpswitch(conf-lldp)# advertise dcbx-fcoe-app-tlvswitch(conf-lldp)# advertise dcbx-fcoe-logical-link-tlvswitch(conf-lldp)# exit

2. Create a CEE map to carry LAN traffic (60%) and SAN traffic (40%) (common to all CEE ports).

switch(config)# cee-map defaultswitch(conf-ceemap)# priority-group-table 1 weight 40 pfcswitch(conf-ceemap)# priority-group-table 2 weight 60switch(conf-ceemap)# priority-table 2 2 2 1 2 2 2 2switch(conf-ceemap)# exit

3. Create an FCoE VLAN for traffic to and from the FC fabric (required for FCoE).

switch(config)# vlan classifier rule 1 proto fcoe encap ethv2switch(config)# vlan classifier rule 2 proto fip encap ethv2switch(config)# vlan classifier group 1 add rule 1switch(config)# vlan classifier group 1 add rule 2switch(config)# interface vlan 5 (Can be any VLAN number other than 1)

switch(conf-if-vl-5)# fcf forwardswitch(conf-if-vl-5)# exit

4. Configure interfaces (required for each port being configured).

switch(config)# interface tengigabitethernet 0/0 Change Port ID

switch(config-if-te-0/0)# switchportswitch(config-if-te-0/0)# switchport mode converged1

switch(config-if-te-0/0)# switchport converged allowed vlan add 51 FCoE Required (Step 3 VLAN)

switch(config-if-te-0/0)# vlan classifier activate group 1 vlan 51

switch(config-if-te-0/0)# cee default1

switch(config-if-te-0/0)# exit

1This command allows the port to access the FCoE VLAN. You can omit this command for non-FCoEports; however, both FCoE and non-FCoE ports might require a similar command for access toother VLANs.

5. Repeat Step 4 for each interface you are configuring.

30

Page 31: HP StorageWorks Fibre Channel Over Ethernet Application Note

6. Perform a port shutdown/no shutdown operation (required for each port for the configurationchange to be enabled).

switch(config)# interface tengigabitethernet 0/0 Change Port ID

switch(config-if-te-0/0)# shutdownswitch(config-if-te-0/0)# no shutdownswitch(config-if-te-0/0)# exit

7. Repeat Step 6 for each interface you are configuring.

8. Save the running configuration to boot flash.

switch(config)# exitswitch# copy running-config startup-configOverwrite the startup config file (y/n): yBuilding configuration...switch#

9. Verify that the CEE port link status and VLAN status are correct.

switch# show ip interface briefInterface IP-Address Status Protocol========= ========== ====== ========TenGigabitEthernet 0/0 unassigned up up

switch#show vlan brief

VLAN Name State Ports(u)-Untagged, (t)-Tagged

======= ================ ======= ===============================1 default ACTIVE Te 0/0(u)…5 VLAN0005 ACTIVE Te 0/0(u)…

switch# exitswitch:admin>

HP StorageWorks Fibre Channel Over Ethernet Application Note 31

Page 32: HP StorageWorks Fibre Channel Over Ethernet Application Note

10. Verify the status of the FC and FCoE virtual FC ports.

BR8000-01:admin> switchshowswitchName: BR8000-1switchType: 76.7switchState: OnlineswitchMode: NativeswitchRole: SubordinateswitchDomain: 4switchId: fffc04switchWwn: 10:00:00:05:1e:76:77:80zoning: ON (Brocade_East)switchBeacon: OFFFC Router: OFFFC Router BB Fabric ID: 1

Index Port Address Media Speed State Proto==============================================0 0 040000 id N4 Online FC E-Port 10:00:00:05:1e:36:2a:70 "BR48-02"1 1 040100 id N4 Online FC E-Port 10:00:00:05:1e:36:2a:70 "BR48-02"2 2 040200 id N4 Online FC E-Port 10:00:00:05:1e:36:2a:70 "BR48-02"3 3 040300 id N4 Online FC E-Port 10:00:00:05:1e:36:2a:70 "BR48-02" (upstream)4 4 040400 id N8 No_Light FC5 5 040500 id N8 No_Light FC6 6 040600 id N8 No_Light FC7 7 040700 id N8 No_Light FC8 8 040800 -- 10G Online FCoE VF-Port 1 VN-Port(s)9 9 040900 -- 10G Online FCoE VF-Port 1 VN-Port(s)10 10 040a00 -- 10G Online FCoE VF-Port 0 VN-Port(s)11 11 040b00 -- 10G Online FCoE VF-Port 1 VN-Port(s)12 12 040c00 -- 10G Online FCoE VF-Port 1 VN-Port(s)13 13 040d00 -- 10G Online FCoE VF-Port 1 VN-Port(s)14 14 040e00 -- 10G Online FCoE VF-Port 1 VN-Port(s)15 15 040f00 -- 10G Online FCoE VF-Port 1 VN-Port(s)16 16 041000 -- 10G Online FCoE VF-Port 1 VN-Port(s)17 17 041100 -- 10G Online FCoE VF-Port 0 VN-Port(s)18 18 041200 -- 10G Online FCoE VF-Port 0 VN-Port(s)19 19 041300 -- 10G Online FCoE VF-Port 0 VN-Port(s)20 20 041400 -- 10G Online FCoE VF-Port 1 VN-Port(s)21 21 041500 -- 10G Online FCoE VF-Port 1 VN-Port(s)22 22 041600 -- 10G Online FCoE VF-Port 0 VN-Port(s)23 23 041700 -- 10G Online FCoE VF-Port 0 VN-Port(s)24 24 041800 -- 10G Online FCoE VF-Port 0 VN-Port(s)25 25 041900 -- 10G Online FCoE VF-Port 0 VN-Port(s)26 26 041a00 -- 10G Online FCoE VF-Port 0 VN-Port(s)27 27 041b00 -- 10G Online FCoE VF-Port 0 VN-Port(s)28 28 041c00 -- 10G Online FCoE VF-Port 0 VN-Port(s)29 29 041d00 -- 10G Online FCoE VF-Port 0 VN-Port(s)30 30 041e00 -- 10G Online FCoE VF-Port 0 VN-Port(s)31 31 041f00 -- 10G Online FCoE VF-Port 0 VN-Port(s)

NOTE:Ports 8 through 31 are the virtual FC ports. To display the FCoE virtual FC devices connectedto those ports, enter the following command.

BR8000-01:admin> fcoe --loginshow ================================================================================Port Te port Device WWN Device MAC Session MAC================================================================================8 Te 0/0 10:00:00:00:c9:93:c9:13 00:00:c9:93:c9:13 0e:fc:00:04:08:019 Te 0/1 10:00:00:00:c9:93:c9:2b 00:00:c9:93:c9:2b 0e:fc:00:04:09:0111 Te 0/3 10:00:00:00:c9:93:c8:b3 00:00:c9:93:c8:b3 0e:fc:00:04:0b:0112 Te 0/4 10:00:00:00:c9:93:ca:8b 00:00:c9:93:ca:8b 0e:fc:00:04:0c:0113 Te 0/5 10:00:00:00:c9:93:c9:bf 00:00:c9:93:c9:bf 0e:fc:00:04:0d:0114 Te 0/6 10:00:00:00:c9:93:ca:e3 00:00:c9:93:ca:e3 0e:fc:00:04:0e:0115 Te 0/7 10:00:00:00:c9:93:ca:eb 00:00:c9:93:ca:eb 0e:fc:00:04:0f:0116 Te 0/8 10:00:00:00:c9:93:c9:1b 00:00:c9:93:c9:1b 0e:fc:00:04:10:0120 Te 0/12 10:00:00:00:c9:93:ca:93 00:00:c9:93:ca:93 0e:fc:00:04:14:0121 Te 0/13 10:00:00:00:c9:93:c8:43 00:00:c9:93:c8:43 0e:fc:00:04:15:01BR8000-1:admin>

32

Page 33: HP StorageWorks Fibre Channel Over Ethernet Application Note

HP C-series Nexus 5010/20 Converged Network Switch quick setupThis procedure is intended for users who are familiar with Cisco MDS FC switches and have experiencemerging C-series MDS FC switches into an existing MDS FC fabric. Use this procedure to enableservers with CNAs attached to the HP C-series Nexus 5010/20 Converged Network Switch to accessdevices on the attached C-series FC fabric.

IMPORTANT:If you are not familiar with Cisco FC switches or you do not have experience merging C-series FCswitches into an existing FC fabric, use the detailed instructions found in the switch user guide to setup your switch.

HP recommends that you use the VFC port assignments listed in Table 18.

Table 18 Recommended VFC port assignments

VFC port assignmentsnl

(recommended)Number of FCoE portsnl

(maximum)SlotModel

VFC ports 1 through 20201 (Standard)HP C-seriesNexus 5010ConvergedNetworkSwitch

VFC ports 41 through 4662 (Optional expansionnl

module slot1)

VFC ports 1 through 40401 (Standard)HP C-seriesNexus 5020ConvergedNetworkSwitch

VFC ports 41 through 4662 (Optional expansionnl

module slot1)

VFC ports 47 through 5263 (Optional expansionnl

module slot1)

1Optional expansion module slots can contain either 10-GbE ports or FC ports.

In the following examples, FCoE is enabled on a C-series Nexus 5010 Converged Network Switch,VLAN 200 is created, and the Ethernet ports are bound to a virtual SAN (VSAN 2) from an 8-portFC expansion module. The commands are the same for the HP C-series Nexus 5020 Switch.

To establish CNA connectivity and enable login to the HP C-series Nexus 5000 Converged NetworkSwitch, configure the IEEE DCB ports as follows:

HP StorageWorks Fibre Channel Over Ethernet Application Note 33

Page 34: HP StorageWorks Fibre Channel Over Ethernet Application Note

1. Enable FCoE (disabled by default).

NOTE:The C-series Nexus 5000 Converged Network Switch will require a reload (reboot).

Nexus5010# configure terminalNexus5010(config)# feature fcoeNexus5010(config)# feature npivNexus5010(config)# 2009 Apr 1 12:05:06 Nexus5010 %$ VDC-1 %$ %PLATFORM-2-FC_LICENSE_DESIRED: FCoE/FC feature will be enabled after the configurationis saved followed by a rebootNexus5010(config)# exitNexus5010# copy running-config startup-config[########################################] 100%Packaging and storing to flash: \Packaging and storing to flash: |Packaging and storing to flash: /

NOTE:Depending on the NX-OS version, a reload may be required when enabling features. Whenyou execute the feature command, a message appears to indicate if a reload is required.

2. Reload the system if you are directed to do so. Otherwise, proceed to Step 3.

Nexus5010# reload

WARNING: This command will reboot the systemDo you want to continue? (y/n) [n] yThe system is going down for reboot NOW!

34

Page 35: HP StorageWorks Fibre Channel Over Ethernet Application Note

3. Create a new VLAN for FCoE.

By default, all ports are in VLAN 1, however, you must use a different VLAN for FCoE. In thefollowing example, VLAN 200 is created with access to Ethernet ports 1/1 to 1/20, and VFCports 1–20 are also created.

NOTE:In the last section of this example, 1/1-20 indicates that the commands that follow applyto multiple ports (in this case, ports 1/1 through 1/20). All 20 ports are set for switchportmode trunk, and switchport trunk allowed is set for VLANs 1 and 200.

Nexus5010# configure terminalNexus5010(config)# vlan 200Nexus5010(config-vlan)# exitNexus5010(config)# exitNexus5010# show vlan briefVLAN Name Status Ports---- -------------------------------- ------- -------------------------1 default active Eth1/1, Eth1/2, Eth1/3,Eth1/4, Eth1/5, Eth1/6,Eth1/7, Eth1/8, Eth1/9,Eth1/10, Eth1/11, Eth1/12,Eth1/13, Eth1/14, Eth1/15,Eth1/16, Eth1/17, Eth1/18,Eth1/19, Eth1/20

200 VLAN0200 active

Nexus5010# configure terminalNexus5010(config)# interface ethernet 1/1-20Nexus5010(config-if-range)# switchport mode trunkNexus5010(config-if-range)# switchport trunk allowed vlan 1, 2001

Nexus5010(config-if-range)# interface vfc 1-20Nexus5010(config-if-range)# exit

1This command allows the port to access the FCoE VLAN (VLAN 200 in this example). Fornon-FCoE ports, you can omit the FCoE VLAN from this command; however, both FCoE andnon-FCoE ports might require access to other VLANs.

HP StorageWorks Fibre Channel Over Ethernet Application Note 35

Page 36: HP StorageWorks Fibre Channel Over Ethernet Application Note

4. Create a new VSAN that includes the FC and VFC ports.

By default, all ports are in VSAN 1. HP recommends that you use a different VSAN for SANconnectivity. In this example, VSAN 2 is created and includes FC ports 2/1 through 2/8 andVFC ports 1 through 20.

NOTE:VFC ports must be FCoE ports. FC ports cannot be VFC ports.

Nexus5010# configure terminalNexus5010(config)# vsan databaseNexus5010(config-vsan-db)# vsan 2Nexus5010(config-vsan-db)# vsan 2 interface fc2/1-8Nexus5010(config-vsan-db)# vsan 2 interface vfc 1-20Nexus5010(config-vsan-db)# exitNexus5010(config)# exitNexus5010# show vsan membershipvsan 1 interfaces:

vsan 2 interfaces:fc2/1 fc2/2 fc2/3 fc2/4 fc2/5 fc2/6 fc2/7 fc2/8vfc1 vfc2 vfc3 vfc4 vfc5 vfc6 vfc7 vfc8vfc9 vfc10 vfc11 vfc12 vfc13 vfc14 vfc15 vfc16vfc17 vfc18 vfc19 vfc20

vsan 4094(isolated_vsan) interfaces:

5. Associate the VLAN with the VSAN.

In this example, VLAN 200 is associated with VSAN 2.

Nexus5010# configure terminalNexus5010(config)# vlan 200Nexus5010(config-vlan)# fcoe vsan 2Nexus5010(config-vlan)# exitNexus5010(config)# exitNexus5010# show vlan fcoe

VLAN VSAN Status-------- -------- --------200 2 Operational

36

Page 37: HP StorageWorks Fibre Channel Over Ethernet Application Note

6. Bind each VFC port to a unique Ethernet port by issuing the following commands on each port:

interface vfc nbind interface ethernet slot/portexit

NOTE:Depending on your switch, up to 52 VFC ports may be available.

In Example 1, VFC ports 1 through 8 are created and are bound to Ethernet ports 1/1 through1/8, respectively. Example 2 shows a more complex configuration in which the VFC ports arenot sequential.

Example 1. Creating and binding consecutive VFC ports

Nexus5010# configure terminalNexus5010(config)# interface vfc 1Nexus5010(config-if)# bind interface ethernet 1/1Nexus5010(config-if)# exitNexus5010(config)# interface vfc 2Nexus5010(config-if)# bind interface ethernet 1/2Nexus5010(config-if)# exitNexus5010(config)# interface vfc 3Nexus5010(config-if)# bind interface ethernet 1/3Nexus5010(config-if)# exitNexus5010(config)# interface vfc 4Nexus5010(config-if)# bind interface ethernet 1/4Nexus5010(config-if)# exitNexus5010(config)# interface vfc 5Nexus5010(config-if)# bind interface ethernet 1/5Nexus5010(config-if)# exitNexus5010(config)# interface vfc 6Nexus5010(config-if)# bind interface ethernet 1/6Nexus5010(config-if)# exitNexus5010(config)# interface vfc 7Nexus5010(config-if)# bind interface ethernet 1/7Nexus5010(config-if)# exitNexus5010(config)# interface vfc 8Nexus5010(config-if)# bind interface ethernet 1/8Nexus5010(config-if)# exitNexus5010(config)# interface vfc 1-8Nexus5010(config-if)# no shutdownNexus5010(config-if)# exit

IMPORTANT:Example 2 is provided for reference only. It shows an alternate method of performing thisstep.

HP StorageWorks Fibre Channel Over Ethernet Application Note 37

Page 38: HP StorageWorks Fibre Channel Over Ethernet Application Note

In Example 2, VFC ports 1 through 6, 10, and 20 through 25 are created, and each VFC portis bound to an Ethernet port. Because the VFC ports are not sequential, multiple interfacevfc commands are required.

Example 2. Creating and binding nonconsecutive VFC ports

Nexus5020# configure terminalNexus5020(config)# interface vfc 1Nexus5020(config-if)# bind interface ethernet 1/1Nexus5020(config-if)# exitNexus5020(config)# interface vfc 2Nexus5020(config-if)# bind interface ethernet 1/2Nexus5020(config-if)# exitNexus5020(config)# interface vfc 3Nexus5020(config-if)# bind interface ethernet 1/3Nexus5020(config-if)# exitNexus5020(config)# interface vfc 4Nexus5020(config-if)# bind interface ethernet 1/4Nexus5020(config-if)# exitNexus5020(config)# interface vfc 5Nexus5020(config-if)# bind interface ethernet 1/5Nexus5020(config-if)# exitNexus5020(config)# interface vfc 6Nexus5020(config-if)# bind interface ethernet 1/6Nexus5020(config-if)# exitNexus5020(config)# interface vfc 10Nexus5020(config-if)# bind interface ethernet 1/10Nexus5020(config-if)# exitNexus5020(config)# interface vfc 20Nexus5020(config-if)# bind interface ethernet 1/20Nexus5020(config-if)# exitNexus5020(config)# interface vfc 21Nexus5020(config-if)# bind interface ethernet 1/21Nexus5020(config-if)# exitNexus5020(config)# interface vfc 22Nexus5020(config-if)# bind interface ethernet 1/22Nexus5020(config-if)# exitNexus5020(config)# interface vfc 23Nexus5020(config-if)# bind interface ethernet 1/23Nexus5020(config-if)# exitNexus5020(config)# interface vfc 24Nexus5020(config-if)# bind interface ethernet 1/24Nexus5020(config-if)# exitNexus5020(config)# interface vfc 25Nexus5020(config-if)# bind interface ethernet 1/25Nexus5020(config-if)# exitNexus5020(config)# interface vfc 1-6Nexus5020(config-if)# no shutdownNexus5020(config-if)# exitNexus5020(config)# interface vfc 10Nexus5020(config-if)# no shutdownNexus5020(config-if)# exitNexus5020(config)# interface vfc 20-25Nexus5020(config-if)# no shutdownNexus5020(config-if)# exit

38

Page 39: HP StorageWorks Fibre Channel Over Ethernet Application Note

7. Enable the FC ports.

In this example, FC ports 2/1 through 2/8 are enabled.

Nexus5010(config)# interface fc 2/1-8Nexus5010(config-if)# no shutdownNexus5010(config-if)# exitNexus5010(config)# exit

8. Copy the running configuration to the startup configuration location.

Nexus5010# copy running-config startup-config[########################################] 100%

9. Copy the running configuration to a backup location.

Nexus5010# copy running-config ftp://10.10.20.1/backup.txt

10. Verify the configuration.

Nexus5010# show interface brief

Nexus5010# show running-config

HP StorageWorks Fibre Channel Over Ethernet Application Note 39

Page 40: HP StorageWorks Fibre Channel Over Ethernet Application Note

Glossary of termsthird-party option3PO

application-specific integrated circuitASIC

boot from SANBFS

converged enhanced EthernetCEE

converged networkCN

converged network adapterCNA

Data Center BridgingDCB

device-specific moduleDSM

End of Row (switch)EoR

Fibre ChannelFC

Fibre Channel over EthernetFCoE

full-featuredFF

FCoE Initialization ProtocolFIP

Fabric Operating System (Brocade)FOS

Fabric Provided MAC AddressingFPMA

Gigabit EthernetGbE

host bus adapterHBA

Internetwork Operating System (Cisco)IOS

interswitch linkISL

link aggregation groupLAG

Link Layer Discovery ProtocolLLDP

Long Reach (optical)LR

network interface cardNIC

quality of serviceQoS

Red Hat Enterprise LinuxRHEL

rack unitRU

storage area networkSAN

SAN Operating System (Cisco)SAN-OS

small form-factor pluggable, plus (transceiver)SFP+

40

Page 41: HP StorageWorks Fibre Channel Over Ethernet Application Note

SUSE Linux Enterprise ServerSLES

Short Reach (optical)SR

Top of Rack (switch)ToR

IEEE DCB enabled network (Cisco)unified fabric

virtual Fibre ChannelVFC

virtual LANVLAN

HP StorageWorks Fibre Channel Over Ethernet Application Note 41

Page 42: HP StorageWorks Fibre Channel Over Ethernet Application Note

42