brocade fabric technology with ibm flashsystem 840 arrays ... · brocade fabric technology with ibm...
TRANSCRIPT
53-1003804-01May 2015
Brocade Fabric Technologywith IBM FlashSystem 840ArraysValidation Test Report
Supporting Fabric OS v7.3.1
© 2015, Brocade Communications Systems, Inc. All Rights Reserved.
ADX, Brocade, Brocade Assurance, the B-wing symbol, DCX, Fabric OS, HyperEdge, ICX, MLX, MyBrocade, OpenScript, The EffortlessNetwork, VCS, VDX, Vplane, and Vyatta are registered trademarks, and Fabric Vision and vADX are trademarks of BrocadeCommunications Systems, Inc., in the United States and/or in other countries. Other brands, products, or service names mentioned may betrademarks of others.
Notice: This document is for informational purposes only and does not set forth any warranty, expressed or implied, concerning anyequipment, equipment feature, or service offered or to be offered by Brocade. Brocade reserves the right to make changes to this documentat any time, without notice, and assumes no responsibility for its use. This informational document describes features that may not becurrently available. Contact a Brocade sales office for information on feature and product availability. Export of technical data contained inthis document may require an export license from the United States government.
The authors and Brocade Communications Systems, Inc. assume no liability or responsibility to any person or entity with respect to theaccuracy of this document or any loss, cost, liability, or damages arising from the information contained herein or the computer programs thataccompany it.
The product described by this document may contain open source software covered by the GNU General Public License or other opensource license agreements. To find out which open source software is included in Brocade products, view the licensing terms applicable tothe open source software, and obtain a copy of the programming source code, please visit http://www.brocade.com/support/oscd.
Contents
Preface..................................................................................................................................... 5Overview........................................................................................................... 5Purpose of this Document.................................................................................5Audience........................................................................................................... 5Objective........................................................................................................... 5Test Conclusions...............................................................................................6Related Documents...........................................................................................6About Brocade.................................................................................................. 6About IBM Storage............................................................................................7
Test Plan...................................................................................................................................8Scope................................................................................................................ 8Test Configuration.............................................................................................9DUT Descriptions.............................................................................................. 9DUT Specifications..........................................................................................10Test Equipment............................................................................................... 10
Configure DUT and Test Equipment..........................................................................................12Step 1: Brocade FC Fabric Configuration....................................................... 12Step 2: IBM FlashSystem 840 Array Configuration.........................................14Step 3: Host Setup.......................................................................................... 17
Test Cases.............................................................................................................................. 22Fabric Initialization – Base Functionality......................................................... 23
Storage Device – Physical and Logical Login with SpeedNegotiation.................................................................................... 23
Zoning and LUN Mapping................................................................... 24Storage Device Fabric I/O Integrity..................................................... 24Storage Device Multipath Configuration – Path Integrity.................... 25
Fabric – Advanced Functionality..................................................................... 25Storage Device Bottleneck Detection using MAPS FPI – w/
Congested Host.............................................................................25Storage Device Bottleneck Detection using MAPS FPI – w/
Congested Fabric.......................................................................... 27Storage Device - QoS Integrity........................................................... 28Storage Device – FC Protocol Jammer Test Suite............................. 28
Stress and Error Recovery with Device Multipath...........................................29Storage Device Fabric I/O Integrity – Congested Fabric.....................29Storage Device Integrity – Device Recovery from Port Toggle...........30Storage Device Integrity – Device Recovery from Device
Relocation......................................................................................31Storage Device Stress – Device Recovery from Device Port
Toggle – Extended Run.................................................................31Storage Device Recovery – ISL Port Toggle (Sequential).................. 32Storage Device Recovery – ISL Port Toggle (Entire Switch).............. 33Storage Device Recovery – Director Blade Maintenance................... 35
Storage Device Fibre Channel Routing (FCR) Internetworking Tests............ 36
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 353-1003804-01
Storage Device Internetworking Validation with FC host.................. 36Storage Device Internetworking Validation with FCoE using
VDX FlexPort...............................................................................37Storage Device Edge Recovery after FCR Disruptions.................... 38Storage Device Backbone Recovery after FCR Disruption.............. 39
Additional and Optional Tests....................................................................... 40Non-Disruptive Firmware Upgrade on Storage Device.....................40Synthetic I/O Workload Loop with Varying Block Sizes.................... 41VMware IOAnalyzer Workload Test..................................................41
4 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Preface
● Overview........................................................................................................................... 5● Purpose of this Document.................................................................................................5● Audience........................................................................................................................... 5● Objective........................................................................................................................... 5● Test Conclusions...............................................................................................................6● Related Documents...........................................................................................................6● About Brocade.................................................................................................................. 6● About IBM Storage............................................................................................................7
OverviewThe Solid State Ready (SSR) program is a comprehensive testing and configuration initiative to validatethe interoperability of Fibre Channel and IP flash storage with a Brocade network infrastructure. Thisprogram provides testing of multiple fabrics, heterogeneous servers, NICs and HBAs in a large port-count Brocade environment.
The SSR qualification program will help verify seamless interoperability and optimum performance ofsolid state storage in Brocade FC and Ethernet fabrics.
Purpose of this DocumentThe goal of this document is to demonstrate the compatibility of IBM FlashSystem 840 arrays in aBrocade FC SAN fabric running FOS v7.3.1. This document provides a test report on the SSRqualification test plan executed on the IBM FlashSystem 840 array.
AudienceThe target audience for this document includes storage administrators, solution architects, systemengineers, and technical development representatives.
Objective1. Test the IBM FlashSystem 840 array with Brocade FC fabrics, in single and routed configurations for
different stress and error recovery scenarios, to validate the interoperability and integration of theIBM FlashSystem array with Brocade FC fabrics.
2. Validate the performance of the Brocade FC fabric in a solid state storage environment for highthroughput and low latency applications.
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 553-1003804-01
Test Conclusions1. Achieved 100% pass rate on all the test cases in the SSR qualification test plan. The network and
the storage were able to handle the various stress and error recovery scenarios without any issues.2. Different I/O workload scenarios were simulated using Medusa and VMware I/OAnalyzer tools and
sustained performance levels were achieved across all workload types. The Brocade FC fabrichandled both the low latency and high throughput I/O workloads with equal efficiency without anyI/O errors or packet drops.
3. The results confirm that the IBM FlashSystem 840 array interoperates seamlessly with Brocade FCfabrics, and demonstrate high availability and sustained performance.
4. The Brocade Gen5 16Gb FC switches were able to handle the sustained throughput and latencyperformance requirements efficiently with fewer ISL trunks. Multiple ISLs to different switches in thefabric should be set up for providing path redundancy through the fabric.
5. It is recommended to enable the Monitoring and Alerting Policy Suite (MAPS) heath monitor on allswitches in the FC fabric to report fabric-wide events and traffic performance metrics. The additionalMAPS feature of Fabric Performance Impact monitoring should also be enabled to detectbottlenecks in the form of timeouts and latency.
a. For optimal availability and performance, consideration should be given to multipath configurationon the host side. Multipath configuration details for the hosts are provided in the host setupsection of "Configure DUT and Test Equipment".
Related Documents• Fabric OS Administrator's Guide, v.7.3.0• Monitoring and Alerting Policy Suite Administrator's Guide, v.7.3.0• Brocade SAN Design and Best Practices• Brocade SAN Fabric Administration Best Practices Guide• Network OS Layer 2 Switching Configuration Guide, v5.0
About BrocadeBrocade®networking solutions empower the world's leading organizations to transition smoothly to aworld where applications and information reside anywhere. By delivering agility and innovation forcloud-based environments, Brocade helps organizations modernize their networks and acceleratetheir journey to the New IP.
In particular, Brocade solutions for storage networking, data center routing, Software-DefinedNetworking (SDN), and Network Functions Virtualization (NFV) give organizations the power tocapitalize on the unique business opportunities driven by virtualization and the cloud.
To deliver a best-in-class solution, Brocade partners with world-class IT companies around the globe.www.brocade.com.
Test Conclusions
6 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
About IBM StorageBusinesses are reinventing themselves and looking for ways to gain value from their data. IBM'sstorage solutions enable customers to handle the growing trend of big data/analytics and clouddeployments. To help clients store and get insights from massive amounts of data, IBM offersenhancements to our software defined storage offerings code name: Elastic Storage and IBM SANVolume Controller along with the new TS1150 tape drive, and enhancements made to IBM Storwize®,FlashSystem™ V840 and the DS8870. The right storage infrastructure improves data economics whichdrives business success.
IBM FlashSystem 840 is an all-flash storage system that is used to make applications and data centersfaster and more efficient. IBM engineers FlashSystem 840 with a relentless focus on lowering latencyfor data accesses. The value your business gets from an all-flash system is directly related to itslatency. Where competing systems have settled on <1 millisecond response times, FlashSystem 840counters with latency closer to 100 microseconds. IBM calls this IBM MicroLatency™. For applicationsthat need to go fast, MicroLatency matters; OLTP, analytics, virtual desktops, and cloud infrastructuresall perform better with MicroLatency.
To learn more visit: http://www-03.ibm.com/systems/storage/flash/
About IBM Storage
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 753-1003804-01
Test Plan
● Scope..............................................................................................................................8● Test Configuration...........................................................................................................9● DUT Descriptions............................................................................................................9● DUT Specifications........................................................................................................10● Test Equipment.............................................................................................................10
The array is connected with one 16G Fibre Channel target port from each controller connected to theBrocade 16G FC fabric.
ScopeTesting focuses on interoperability of the IBM FlashSystem storage array and determining an optimalconfiguration for performance and availability.
Testing covers various I/O stress and error handling scenarios. Performance is observed within thecontext of best practice fabric configuration; however absolute maximum benchmark reporting ofstorage performance is beyond the scope of this test.
Details of the test steps are covered under "Test Case Descriptions" section. Standard test bed setupincludes IBM/HP/Dell servers with Brocade/QLogic/Emulex HBA's with two uplinks from every host tothe Brocade FC fabric. I/O generator tools used include Medusa Labs Test Tools and VMware I/OAnalyzer.
Test Plan
8 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Test Configuration
DUT Descriptions
The following lists the devices under test (DUT) and the test equipment used.
Storage arrayTABLE 1
DUT ID Model Vendor Description
IBMFlashSystem840
FlashSystem 840 IBM The IBM FlashSystem 840 is an eMLC based all-flash array with dual controllers in an active-activeconfiguration. Each controller node has 4x16Gb FCtarget ports for connectivity.
SwitchesTABLE 2
DUT ID Model Vendor Description
6510-1...9 BR-6510 Brocade 48 port 16Gb FC switch
DCX-1 DCX 8510-8 Brocade 8 slot 16Gb FC chassis
DCX-2 DCX 8510-4 Brocade 4 slot 16Gb FC chassis
VDX-1,2 VDX 6740 Brocade 48x10GbE and 4x40GbE ports switch
Test Configuration
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 953-1003804-01
DUT Specifications
Storage Version
IBM FlashSystem 840 array 1.1.2.7
Brocade Switches Version
DCX-8510-8 FOS v7.3.1
DCX-8510-4 FOS v7.3.1
6510 + Integrated Routing, Fabric Vision Licenses FOS v7.3.1
VDX 6740 NOS v5.0.1
Adapters Version
Brocade 1860 2-port 16Gb FC HBA/10GbE CNA Driver and firmware version 3.2.5.1
Brocade 825 2-port 8GB FC HBA Driver and firmware version 3.2.5.1
QLogic QLE2672 2-port 16GB FC HBA Driver 8.07.00.16, firmware 7.04.01
QLogic QLE2562 2-port 8GB FC HBA Driver 8.07.00.16, firmware 7.01.00
Emulex LPe 16202-X 2-port 16Gb FC HBA Driver 10.2.370.8, firmware 1.1.55.0
Emulex OCe14102-UM 2-port CNA adapter Driver 10.2.370.8, firmware 10.2.370.19
DUT ID Servers RAM Processor OS
SRV-1 HP Proliant DL380p Gen8 16GB Intel Xeon E5-2620 Windows Server 2012 R2
SRV-2 HP Proliant DL380p Gen8 32GB Intel Xeon E5-2690 RHEL 6.5 x86_64
SRV-3 IBM System x3650 M4 16GB Intel Xeon E5-2620 RHEL 7 x86_64
SRV-4 IBM System x3550 M3 24GB Intel Xeon E5645 SLES 12.0 x86_64
SRV-5 HP Proliant DL360 Gen7 24GB Intel Xeon E5645 VMware ESXi 5.5
SRV-6 HP Proliant DL360 Gen7 24GB Intel Xeon E5645 VMware ESXi 5.5
SRV-7 IBM System x3630 M4 24GB Intel Xeon E5-2420 Windows Server 2012 R2
SRV-8 Dell Poweredge R710 8GB Intel Xeon L5520 Windows Server 2012
Test Equipment
Device/Software Tools Version
Finisar 16Gb Analyzer/Jammer XGIG5K2001153
Medusa Labs Test Tools 6.0.1.148039
DUT Specifications
10 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Device/Software Tools Version
VMware I/O Analyzer 1.6.2
Test Plan
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 1153-1003804-01
Configure DUT and Test Equipment
● Step 1: Brocade FC Fabric Configuration..................................................................... 12● Step 2: IBM FlashSystem 840 Array Configuration.......................................................14● Step 3: Host Setup........................................................................................................17
Step 1: Brocade FC Fabric Configuration1. Enable MAPS and Fabric Performance Impact (FPI) monitoring on all switches in the fabric. MAPS
enables health monitoring on the switches to detect potential faults and create alerts. MAPS FPIallows fabric monitoring for performance impacts, including timeouts, latency, and throughput.Detailed information on MAPS configuration and setup can be found in the Monitoring and AlertingPolicy Suite Administrator's Guide.
• MAPS requires "Fabric Vision License" to be installed.• Enable the desired MAPS policy using any of the available default policies or create custom
policy.• Enable FPI monitoring. FPI monitoring requires "bottleneckmon" to be disabled.<==========>root> mapsconfig --enablemaps -policy dflt_aggressive_policyWARNING:This command enables MAPS and replaces all Fabric Watch configurations and monitoring. Once MAPS is enabled, the Fabric Watch configuration can't be converted to MAPS.If you wish to convert your Fabric Watch configuration into MAPS policies, select NO to this prompt and first issue the "mapsconfig --fwconvert" command. Once the Fabric Watch configuration is converted into MAPS policies, you may reissue the "mapsconfig --enablemaps" command to continue this process.If you do not use Fabric Watch or need the configuration, then select YES to enable MAPS now.Do you want to continue? (yes, y, no, n): [no] yEnabled dflt_aggressive_policy policy.root> mapsconfig --actions raslog,fence,snmp,email,sw_marginal,sw_criticalroot> bottleneckmon --disableroot> mapsConfig –enableFPImonroot> mapsconfig --showConfigured Notifications: RASLOG,SNMP,EMAIL,FENCE,SW_CRITICAL,SW_MARGINALMail Recipient: [email protected],[email protected] Monitoring: EnabledPaused members :===============PORT :CIRCUIT :SFP :root> mapsdb --show1 Dashboard Information:=======================DB start time: Wed Dec 17 20:59:09 2014Active policy: dflt_aggressive_policyConfigured Notifications: RASLOG,SNMP,EMAIL,FENCE,SW_CRITICAL,SW_MARGINALFenced Ports : NoneDecommissioned Ports : None2 Switch Health Report:
Configure DUT and Test Equipment
12 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
=======================Current Switch Policy Status: HEALTHY3.1 Summary Report:===================Category |Today |Last 7 days |--------------------------------------------------------------------------------Port Health |No Errors |Out of operating range |Fru Health |In operating range |In operating range |Security Violations |No Errors |No Errors |Fabric State Changes |No Errors |In operating range |Switch Resource |In operating range |In operating range |Traffic Performance |In operating range |In operating range |FCIP Health |Not applicable |Not applicable |Fabric Performance Impact|In operating range |In operating range |3.2 Rules Affecting Health:===========================Category(Rule Count)|RepeatCount|Rule Name |Execution Time |Object |Triggered Value(Units)|------------------------------------------------------------------------------------------------------------------------Port Health(4) |1 |defALL_TARGET_PORTSLOSS_SIG|12/29/14 15:39:54|Port 16 |1 LOS | | |NAL_0 | | | | |1 |defALL_TARGET_PORTSSTATE_CH|12/29/14 15:39:54|Port 16 |1 | | |G_0 | | | | |1 |defNON_E_F_PORTSLOSS_SIGNAL|12/29/14 15:39:54|Port 16 |1 LOS | | |_0 | | | | |1 |defALL_OTHER_F_PORTSLF_0 |12/29/14 15:31:48|Port 16 |1 |<==========>
2. Configure required zone sets to enable communication between the hosts and target.<==========>root> cfgactvshowzone: ssr067143_ibm 21:00:00:24:ff:59:4f:34; 21:00:00:24:ff:59:4f:35; 50:05:07:60:5e:80:76:52; 50:05:07:60:5e:80:76:71<==========>
3. Configure Fibre Channel Routing (Integrated Routing license required). Detailed information on FCRsetup can be found in the Brocade Fabric OS Administrator's Guide. Example FCR configuration isshown below:<==========>root> fcrconfigure –bbfid 100root> fosconfig --enable fcrroot> portcfgexport [port#] -a1 –m[0/5] -f 10 --- {-m 0-Brocade 5-Brocade NOS fabric}Example output of exported devicesroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 10 21:00:00:24:ff:48:b9:6a 02f001 20 551a00 Imported 10 21:00:00:24:ff:48:b9:6b 02f101 20 541e00 Imported 10 52:4a:93:7d:f3:5f:61:00 02f201 20 550e00
Configure DUT and Test Equipment
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 1353-1003804-01
Imported 10 52:4a:93:7d:f3:5f:61:01 02f401 20 540400 Imported<==========>
4. Configuring “FlexPort” on VDX 6740 switches. Flexport feature allows ports to transmit data aseither 10G Ethernet or Fibre Channel, and to be changed from one type to other without requiring areboot. Detailed information for FlexPort configuration can be found in the Network OS Layer 2Switching Configuration Guide.
Example FlexPort configuration is shown below:<==========>VDX6740# show running-config hardware flexporthardware flexport 119/0/48 type fibre-channel ! flexport 120/0/48 type fibre-channel !connector-group 119/0/6 speed HighMixed !connector-group 120/0/6 speed HighMixed !<==========>
Step 2: IBM FlashSystem 840 Array Configuration1. The array has an active-active controller node setup with four 16G Fibre Channel ports on each
controller node. One 16G FC port from each controller is connected to the Brocade 16G FC Fabricfor redundancy.
FIGURE 1 IBM FlashSystem 840 Array FC ports
Step 2: IBM FlashSystem 840 Array Configuration
14 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
FIGURE 2 IBM FlashSystem 840 Properties\
2. Create "Host Groups" and "Volume Groups" and map the volumes to the respective host.
Configure DUT and Test Equipment
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 1553-1003804-01
FIGURE 3 Create Host Group and Assign Host Ports
FIGURE 4 Create Volumes and Map Volumes to the Hosts
Configure DUT and Test Equipment
16 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Step 3: Host Setup1. Provision a minimum of two uplinks from the host to the FC fabric for redundancy and use native
multipath tools to manage the available paths and load-balance across them.2. Configuring the multipath settings allows for proper failover and load balancing across the available
links. Multipath settings used for Windows, Linux, and VMware are provided here.
• For Windows, the IBM 840 flash storage is discovered in MPIO. The load balance policy on all theLUNs is set to "Least Queue Depth" as recommended by IBM.
FIGURE 5 MPIO Discovery of IBM FlashSystem 840 Array
Step 3: Host Setup
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 1753-1003804-01
FIGURE 6 Windows MPIO Configuration
Configure DUT and Test Equipment
18 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
FIGURE 7 Windows Disk MPIO Properties
• For Linux, add the following to /etc/multipath.conf<==========>defaults { user_friendly_names yes find_multipaths yes polling_interval 30 checker_timeout 10}##devices { device { vendor "IBM" product "FlashSystem-9840" path_selector "queue-length 0" # Linux 6.2, if available path_grouping_policy multibus path_checker tur rr_min_io_rq 4 # Linux 6.x rr_weight uniform no_path_retry fail failback immediate dev_loss_tmo 300 fast_io_fail_tmo 25 }}Sample output:# multipath -llmpatho (36005076a018a900a0000000014000015) dm-4 IBM ,FlashSystem-9840
Configure DUT and Test Equipment
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 1953-1003804-01
size=50G features='0' hwhandler='0' wp=rw`-+- policy='queue-length 0' prio=1 status=active |- 0:0:0:2 sdc 8:32 active ready running |- 0:0:1:2 sdh 8:112 active ready running |- 5:0:0:2 sdm 8:192 active ready running `- 5:0:1:2 sdr 65:16 active ready running<==========>
• For VMware, the Path Selection policy is changed to "Round-Robin" as recommended by IBM.
FIGURE 8 Configure Path Selection policy for IBM 840 devices
FIGURE 9 IBM 840 devices discovered on VMware host
3. Apply any host tuning parameters recommended by IBM for better operation with the storage array.
• For Linux, the scheduling policy is changed to "NOOP" to provide the best write performancewith the FlashSystem array. The following udev rule is created to persistently set the schedulerpolicy to "NOOP" for IBM FlashSystem 840 devices.<==========># cat /etc/udev/rules.d/99-IBM-FlashSystem.rulesACTION=="add|change", SUBSYSTEM=="block",ATTRS{device/model}=="FlashSystem-9840",ATTR{queue/scheduler}="noop",ATTR{queue/rq_affinity}="1",ATTR
Configure DUT and Test Equipment
20 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
{queue/add_random}="0",ATTR{device/timeout}="5"ACTION=="add|change", KERNEL=="dm-*",PROGRAM="/bin/bash -c 'cat /sys/block/$name/slaves/*/device/model | grep FlashSystem-9840'",ATTR{queue/scheduler}="noop",ATTR{queue/rq_affinity}="1",ATTR{queue/add_random}="0"## cat /sys/block/sdi/queue/scheduler[noop] deadline cfq
• For Windows and VMware, all default values were used for disk and path timeouts and no othertuning parameters were applied.
Configure DUT and Test Equipment
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 2153-1003804-01
Test Cases
● Fabric Initialization – Base Functionality.......................................................................23● Fabric – Advanced Functionality...................................................................................25● Stress and Error Recovery with Device Multipath.........................................................29● Storage Device Fibre Channel Routing (FCR) Internetworking Tests.......................... 36● Additional and Optional Tests....................................................................................... 40
1.0 FABRIC INITIALIZATION - BASE FUNCTIONALITY
1.0.1 Storage Device - Physical and Logical Login withSpeed Negotiation
1.0.2 Zoning and LUN Mapping
1.0.3 Storage Device Fabric I/O Integrity
1.0.4 Storage Device Multipath Configuration - Path Integrity
1.1 FABRIC - ADVANCED FUNCTIONALTY
1.1.1 Bottleneck Detection using MAPS FPI - w/CongestedHost
1.1.2 Bottleneck Detection using MAPS FPI - w/CongestedFabric
1.1.3 Storage Device - QoS Integrity
1.1.4 Storage Device - FC Protocol Jammer Test Suite
1.2 STRESS and ERROR RECOVERY WITH DEVICEMULTI-PATH
1.2.1 Storage Device Fabric I/O Integrity - Congested Fabric
1.2.2 Storage Device Nameserver Integrity - DeviceRecovery with Port Toggle
1.2.3 Storage Device Nameserver Integrity - DeviceRecovery with Device Relocation
1.2.4 Storage Device Nameserver Stress - Device Recoverywith Device Port Toggle
1.2.5 Storage Device Recovery - ISL Port Toggle(Sequential)
1.2.6 Storage Device Recovery - ISL Port Toggle (entireswitch)
1.2.7 Storage Device Recovery - Director Blade Maintenance
1.2.8 Storage Device Recovery - Switch Offline
1.2.9 Storage Device Recovery - Switch Firmware Download
Test Cases
22 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
1.3 STORAGE DEVICE - FIBRE CHANNEL ROUTING(FCR) INTERNETWORKING TESTS
1.3.1 Storage Device InterNetworking Validation w/FC host
1.3.2 Storage Device InterNetworking Validation w/FCoEusing VDX FlexPort
1.3.3 Storage Device Edge Recovery after FCR Disruptions
1.3.4 Storage Device BackBone Recovery after FCRDisruptions
1.4 OPTIONAL/ADDITIONAL TESTS
1.4.1 Non-disruptive firmware upgrade on array
1.4.2 Synthetic I/O Workload Loop with Varying Block Sizes
1.4.3 VMware I/O Analyzer Workload Tests
Fabric Initialization – Base Functionality
Storage Device – Physical and Logical Login with Speed Negotiation
Test ObjectiveVerify device login to switch and name-server with all supported speed setting.
ProcedureTest Execution:
1. Set switch ports to 4/8/16/Auto_Negotiate speed settings.<==========>portcfgspeed <port> [4/8/16/0]<==========>
Result Validation1. Validate link states on the array and verify speed negotiation and device login at different speeds.2. Check switch port status and verify the "actual" and "configured" link speed. Check name server for
device.<==========># nscamshow# portshow Xroot> portshow 19portIndex: 19portName: port19portHealth: HEALTHYportState: 1 OnlineProtocol: FCportWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81Distance: normalportSpeed: N16Gbps ><-<==========>
Fabric Initialization – Base Functionality
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 2353-1003804-01
Result
PASS. Test Passed. Speed negotiation, device login and connectivity verified
Zoning and LUN Mapping
Test ObjectiveVerify host to LUN access exists with valid zoning.
ProcedureTest Execution:
1. Create FC zone on the fabric with the initiator and target WWNs.2. Create Host Groups and LUNs on the array with access to initiator WWN.
Result Validation:
1. Verify LUNs are discovered on the hosts with host specific tools.<==========>- Linux: Check output of ‘lsscsi’- Windows: Check output of ‘Computer Management’ -> ‘Storage” -> ‘Disk Management’- VMware: Check output at ‘Configuration’ -> ‘Storage’ -> ‘Devices’<==========>
Result
Pass. Test passes. Host has read/write access to presented LUNs
Storage Device Fabric I/O Integrity
Test ObjectiveValidate single path host-to-LUN I/O with write/read/verify testing. Include short device cable pulls/port-toggle to validate device recovery.
ProcedureTest Execution:
1. Set up read/write I/O to LUN using Medusa.2. Perform link disruptions by port-toggles, and cable pulls.
Result Validation:
Check Medusa I/O logs and verify I/O resumes after a short downtime. Medusa I/O will pause, butshould recover without errors.
Result:
Pass. Test passed. I/O resumes after the disruption
Zoning and LUN Mapping
24 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Storage Device Multipath Configuration – Path Integrity
Test ObjectiveVerify multipath configures successfully. Each Adapter and Storage port to reside in different switches.For all device paths, consecutively isolate individual paths and validate I/O integrity and path recovery.
ProcedureTest Execution:
1. Set up host with at least two initiator ports zoned with two target ports on array.2. Set up multipath on the host and start I/O.3. Perform sequential port toggles across initiator and target switch ports to isolate paths.
Result Validation:
1. Check host multipath properties to verify the toggled path recovers.<==========>- Windows: mpclaim –s –d - Linux: multipath –ll- VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>
2. Check the host and storage logs for any failures.3. Check the switch error logs and switch port status after toggle.
<==========># errdumpall# portstatsshow X# portshow Xroot> portshow 19portHealth: HEALTHY <-…………portState: 1 Online <-…………..portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 <-Distance: normalportSpeed: N16Gbps<==========>
4. Check I/O logs and verify I/O continues without any errors.
Result
PASS. Test passed. I/O fails over to remaining active paths and recovers when disrupted path isrestored.
Fabric – Advanced Functionality
Storage Device Bottleneck Detection using MAPS FPI – w/CongestedHost
Test ObjectiveVerify congestion on host ports is detected. Verify storage device and fabric behavior during congestion.
Storage Device Multipath Configuration – Path Integrity
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 2553-1003804-01
ProcedureTest Execution:
1. Verify congestion on host ports is detected. Verify storage device and fabric behavior duringcongestion.
2. Start I/O from single host initiator to multiple targets.3. Monitor switch logs for Congestion and Latency (IO_PERF_IMPACT/IO_FRAME_LOSS) warnings.
Result Validation:
Check switch error logs and MAPS dashboard for bottleneck warnings.<==========># errdumpall | grep IO_root> errdumpall | grep IO_2014/12/17-11:56:00:672622, [MAPS-1003], 117148/115018, FID 128, WARNING, B6510_066_088, Port 16, Condition=ALL_F_PORTS(DEV_LATENCY_IMPACT==IO_PERF_IMPACT), Current Value:[DEV_LATENCY_IMPACT,IO_PERF_IMPACT, 30.0% in 10 secs], RuleName=defALL_F_PORTS_IO_PERF_IMPACT, Dashboard Category=Fabric Performance Impact., actionHndlr.c, line: 755, comp:md, ltime:2014/12/17-11:56:00:671909# mapsdb --show allroot> mapsdb --show1 Dashboard Information:=======================Active policy: dflt_aggressive_policy…………………2 Switch Health Report:=======================Current Switch Policy Status: HEALTHY3.1 Summary Report:===================Category |Today |Last 7 days |--------------------------------------------------------------------------------Port Health |No Errors |Out of operating range |Fru Health |In operating range |In operating range |Security Violations |No Errors |In operating range |Fabric State Changes |No Errors |In operating range |Switch Resource |In operating range |In operating range |Traffic Performance |In operating range |In operating range |FCIP Health |Not applicable |Not applicable |Fabric Performance Impact|In operating range |In operating range |<==========>Result
PASS. Test passed. The bottlenecked ports are displayed on the MAPS dashboard and a raslogwarning is created.
Test Cases
26 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Storage Device Bottleneck Detection using MAPS FPI – w/CongestedFabric
Test ObjectiveCreate congestion on switch ISL port. Verify congestion in the fabric is detected. Verify storage deviceand fabric behavior during congestion.
ProcedureTest Execution:
1. Enable MAPS monitoring and MAPS FPI on all switches. Fabric Vision license required.2. Isolate single ISL in the fabric.3. Start I/O from multiple host initiator to multiple targets.4. Monitor switch logs for Congestion and Latency (IO_PERF_IMPACT/IO_FRAME_LOSS) warnings.
Result Validation:
Check switch error logs and MAPS dashboard for bottleneck warnings.<==========># errdumpall | grep IO_root> errdumpall | grep IO_2014/12/17-11:56:00:672622, [MAPS-1003], 117148/115018, FID 128, WARNING, B6510_066_088, Port 16, Condition=ALL_F_PORTS(DEV_LATENCY_IMPACT==IO_PERF_IMPACT), Current Value:[DEV_LATENCY_IMPACT,IO_PERF_IMPACT, 30.0% in 10 secs], RuleName=defALL_F_PORTS_IO_PERF_IMPACT, Dashboard Category=Fabric Performance Impact., actionHndlr.c, line: 755, comp:md, ltime:2014/12/17-11:56:00:671909# mapsdb --show allroot> mapsdb --show1 Dashboard Information:=======================Active policy: dflt_aggressive_policy…………………2 Switch Health Report:=======================Current Switch Policy Status: HEALTHY3.1 Summary Report:===================Category |Today |Last 7 days |--------------------------------------------------------------------------------Port Health |No Errors |Out of operating range |Fru Health |In operating range |In operating range |Security Violations |No Errors |In operating range |Fabric State Changes |No Errors |In operating range |Switch Resource |In operating range |In operating range |Traffic Performance |In operating range |In operating range |FCIP Health |Not applicable |Not applicable |Fabric Performance Impact|In operating range |In operating range |<==========>Result
Storage Device Bottleneck Detection using MAPS FPI – w/Congested Fabric
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 2753-1003804-01
PASS. Test passed. The bottlenecked ports are displayed on the MAPS dashboard and a raslogwarning is created
Storage Device - QoS Integrity
Test ObjectiveVerify storage device behavior and validate traffic characteristics with different QoS zones.
ProcedureTest Execution:
1. Set up initiator-target pairs with Low/Medium/High QoS zones in the fabric.2. Start I/O from all hosts and verify I/O statistics.
Result Validation:
1. Check I/O logs and verify I/O continues without any errors.2. Check switch error logs and switch port status for any errors.
<==========># porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>
Result
PASS. Test oassed. I/O from hosts in all QoS zones completed successfully without any errors
Storage Device – FC Protocol Jammer Test Suite
Test ObjectivePerform FC Jammer Tests including areas such as CRC corruption, packet corruption, missing frame,host error recovery, and target error recovery.
ProcedureTest Execution:
1. Insert Jammer device in the I/O path on the storage end.2. Execute the following Jammer scenarios:
• Delete one frame• Delete R_RDY• Replace CR of data frame• Replace EOF of data frame
Storage Device - QoS Integrity
28 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
• Replace "good status" with "check condition"• Replace IDLE with LR• Truncate frame• Create S_ID/D_ID error of data frame
Result Validation:
1. Check the host and storage logs for any errors.2. Check the switch logs and interface stats for any errors.
<==========># errdumpall# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>
3. Verify Jammer operations and recovery with Analyzer.
Result
PASS. Test passed. Host and target were able to recover from the errors and continue I/O operations
Stress and Error Recovery with Device Multipath
Storage Device Fabric I/O Integrity – Congested Fabric
Test Objective1. From all available initiators, start a mixture of READ/WRITE/VERIFY traffic with random data
patterns continuously to all their targets overnight.2. Verify no host application failover or unexpected change in I/O throughput occurs.3. Configure fabric & devices for maximum link and device saturation.
ProcedureTest Execution:
1. Start FC I/O to the storage array from multiple hosts.2. Set up a mix of READ/WRITE traffic.
Result Validation:
1. Check the host and storage logs for any errors.2. Verify the link congestion and check the switch logs for any errors.
<==========># errdumpall# portperfshow
Stress and Error Recovery with Device Multipath
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 2953-1003804-01
# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>
3. Check I/O generator tool logs to verify I/O runs without errors.
Result
PASS. All I/O completed without errors. All validation checks passed.
Storage Device Integrity – Device Recovery from Port Toggle
Test Objective1. With I/O running, perform a quick port toggle on every storage device and adapter port.2. Verify host I/O will recover.3. Sequentially performed for each storage device and adapter port.
ProcedureTest Execution:
1. Set up multipath on host and start I/O.2. Perform multiple iterations of sequential port toggles across initiator and target switch ports.
Result Validation:
1. Check switch port status after toggle and for any errors in the switch error logs.<==========># errdumpall# portstatsshow X# portshow Xroot> portshow 19portHealth: HEALTHY <-…………portState: 1 Online <-…………..portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 <-Distance: normalportSpeed: N16Gbps<==========>
2. Check host multipath status on hosts to verify the toggled path recovers.<==========>- Windows: mpclaim –s –d - Linux: multipath –ll- VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>
3. Check host and storage error logs and verify I/O continues without errors.
Result
PASS. I/O failed over and recovered successfully. All validation checks passed.
Storage Device Integrity – Device Recovery from Port Toggle
30 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Storage Device Integrity – Device Recovery from Device Relocation
Test Objective1. With I/O running, manually disconnect and reconnect port to different switch in same fabric.2. Verify host I/O will failover to alternate path and toggled path will recover.3. Sequentially performed for each storage device and adapter port.4. Repeat test for all switch types.
ProcedureTest Execution:
1. Set up multipath on host and start I/O.2. Move storage target ports to different switch ports in the fabric.
Result Validation:
1. Check for any errors in the switch error logs and the switch port status at the new switch port.<==========># errdumpall# portstatsshow X# portshow Xroot> portshow 19portHealth: HEALTHY <-…………portState: 1 Online <-…………..portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 <-Distance: normalportSpeed: N16Gbps<==========>
2. Check host multipath status on hosts to verify the toggled path recovers.<==========>- Windows: mpclaim –s –d - Linux: multipath –ll- VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>
3. Check host and storage error logs and verify I/O continues without errors.
Result
PASS. I/O failed over and recovered successfully. All validation checks passed.
Storage Device Stress – Device Recovery from Device Port Toggle –Extended Run
Test Objective1. Sequentially toggle each initiator and target port in fabric.2. Verify host I/O will recover to alternate path and toggled path will recover.3. Run for 24 hours.
Storage Device Integrity – Device Recovery from Device Relocation
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 3153-1003804-01
ProcedureTest Execution:
1. Set up multipath on host and start I/O.2. Perform multiple iterations of sequential port toggles across initiator and target switch ports.
Result Validation:
1. Check switch port status after toggle and for any errors in the switch error logs.<==========># errdumpall# portstatsshow X# portshow Xroot> portshow 19portHealth: HEALTHY <-…………portState: 1 Online <-…………..portWwn of device(s) connected: 10:00:8c:7c:ff:22:f7:81 <-Distance: normalportSpeed: N16Gbps<==========>
2. Check host multipath properties for iSCSI hosts to verify the toggled path recovers.<==========>On Windows: mpclaim –s –d On Linux: multipath –llOn VMware: Check the paths at ‘Configuration’ -> ‘Storage’ -> ‘Devices’ -> ‘Manage Paths’ <==========>
3. Check host and storage error logs and verify I/O continues without errors.
Result
PASS. I/O failed over and recovered successfully. All validation checks passed.
Storage Device Recovery – ISL Port Toggle (Sequential)
Test Objective1. Sequentially toggle each ISL path on all switches. Host I/O may pause, but should recover.2. Verify fabric ISL path redundancy between hosts and storage devices.3. Verify host I/O throughout test.
ProcedureTest Execution:
1. Set up host multipath with links on different switches in the FC fabric and start I/O.2. Ensure ISL redundancy by provisioning multiple ISLs connected to different switches to provide
multiple paths through the fabric.<==========># islshowroot> islshow 1: 0-> 0 10:00:00:05:33:5b:7d:06 97 B6510_066_097 sp: 16.000G bw: 16.000G TRUNK CR_RECOV FEC 2: 1-> 1 10:00:00:27:f8:0f:b9:f0 91 B6510_066_091 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 3: 4-> 1 10:00:00:05:33:5b:10:e8 100 B6510_066_100 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 4: 34-> 34 10:00:00:27:f8:66:f3:81 92 B6520_066_92 sp: 16.000G bw: 32.000G TRUNK QOS CR_RECOV FEC 5: 40-> 40 50:00:53:31:39:65:ae:28 160 fcr_fd_160 sp: 16.000G
Storage Device Recovery – ISL Port Toggle (Sequential)
32 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
bw: 16.000G TRUNK QOS CR_RECOV FEC 6: 41-> 41 50:00:53:35:b1:d1:de:28 1 fcr_fd_1 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC<==========>
3. Perform multiple iterations of sequential ISL toggles across the fabric.
Result Validation:
1. Check FC fabric status after ISL toggle. Verify all nodes are online.<==========># fabricshowroot> fabricshowSwitch ID Worldwide Name Enet IP Addr FC IP Addr Name------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083"The Fabric has 5 switches <-Fabric Name: SSR<==========>
2. Check the switch logs for any errors and verify I/O failed over to alternate ISL path in the fabric.<==========># errdumpall# portperfshow# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>
3. Check host and storage error logs and verify I/O continues without errors.
Result
I/O re-routes to available paths in the fabric and recovers when the link is restored. All validationschecks passed.
Storage Device Recovery – ISL Port Toggle (Entire Switch)
Test Objective1. Sequentially, and for all switches, disable all ISLs on the switch under test.2. Verify fabric switch path redundancy between hosts and storage devices.3. Verify switch can merge back in to the fabric.4. Verify host I/O path throughout test.
Storage Device Recovery – ISL Port Toggle (Entire Switch)
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 3353-1003804-01
ProcedureTest Execution
1. Set up host multipath with links on different switches in the FC fabric and start I/O.2. Ensure ISL redundancy by provisioning multiple ISLs connected to different switches to provide
multiple paths through the fabric.<==========># islshowroot> islshow 1: 0-> 0 10:00:00:05:33:5b:7d:06 97 B6510_066_097 sp: 16.000G bw: 16.000G TRUNK CR_RECOV FEC 2: 1-> 1 10:00:00:27:f8:0f:b9:f0 91 B6510_066_091 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 3: 4-> 1 10:00:00:05:33:5b:10:e8 100 B6510_066_100 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 4: 34-> 34 10:00:00:27:f8:66:f3:81 92 B6520_066_92 sp: 16.000G bw: 32.000G TRUNK QOS CR_RECOV FEC 5: 40-> 40 50:00:53:31:39:65:ae:28 160 fcr_fd_160 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC 6: 41-> 41 50:00:53:35:b1:d1:de:28 1 fcr_fd_1 sp: 16.000G bw: 16.000G TRUNK QOS CR_RECOV FEC<==========>
3. Perform multiple iterations of sequentially disabling all ISLs on a switch in the fabric.
Result Validation:
1. Check FC fabric status after ISL toggle. Verify all nodes are online.<==========># fabricshowroot> fabricshowSwitch ID Worldwide Name Enet IP Addr FC IP Addr Name------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083"The Fabric has 5 switches Fabric Name: SSR<==========>
2. Check the switch logs for any errors and verify I/O failed over to alternate ISL path in the fabric.<==========># errdumpall# portperfshow# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>
3. Check host and storage error logs and verify I/O continues without errors.
Result
PASS. I/O failed over to alternate path and recovered once the switch merged back in the fabric. Allvalidations checks passed.
Test Cases
34 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Storage Device Recovery – Director Blade Maintenance
Test Objective1. Toggle each blade on the director in sequential order.2. Include blade enable/disable, power on/off, and reboot testing.
Test ProcedureTest Execution:
1. Uplink edge switch ISLs to different blades on the directors.2. Sequentially perform firmware upgrades on all switches in the fabric.
Result Validation:
1. Verify firmware upgrade completes successfully on each switch node and they merge back in the FCfabric.<==========># versionroot> versionKernel: 2.6.14.2Fabric OS: v7.3.1 <-<-Made on: Thu Dec 11 14:30:38 2014Flash: Wed Dec 17 09:53:59 2014BootProm: 1.0.11# fabricshowroot> fabricshowSwitch ID Worldwide Name Enet IP Addr FC IP Addr Name------------------------------------------------------------------------- 1: fffc01 50:00:53:35:b1:d3:df:1b 0.0.0.0 0.0.0.0 "fcr_xd_1_40" 3: fffc03 10:00:00:05:33:13:95:9a 10.38.66.73 0.0.0.0 "B6510_066_073" 19: fffc13 10:00:00:05:33:a5:bf:86 10.38.66.74 0.0.0.0 >"B6510_066_074" 82: fffc52 10:00:00:05:33:13:96:5a 10.38.66.82 0.0.0.0 "B6510_066_082" 83: fffc53 10:00:00:05:33:5b:1d:1d 10.38.66.83 0.0.0.0 "B6510_066_083"The Fabric has 5 switches <-Fabric Name: SSR<==========>
2. Check I/O generator tool logs to verify I/O runs without errors throughout the firmware upgrade.3. Check the switch logs for any errors and verify I/O resumes on the node after the firmware upgrade is
complete.<==========># errdumpall# portperfshow<==========>
Result
PASS. I/O operations completed without any errors. I/O failed over to alternate path during the switchreload after firmware upgrade and resumed after the switch was online. All validation checks passed.
Storage Device Recovery – Director Blade Maintenance
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 3553-1003804-01
Storage Device Fibre Channel Routing (FCR) Internetworking Tests
Storage Device Internetworking Validation with FC host
Test Objective1. Configure two FC fabrics with FCR.2. Verify that edge devices are imported into adjacent name servers and hosts have access to their
routed targets after FC routers are configured.
ProcedureTest Execution:
1. Set up FCR in an Edge-Backbone-Edge configuration.2. Set up LSAN zoning and verify host access to target LUNs and start I/O.
Result Validation:
1. Verify name server and FCR fabric state.<==========># fcrfabricshowroot> fcrfabricshowFC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82,Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92"FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83,Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088"# fcrproxydevshowroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 ImportedTotal devices displayed: 20<==========>
2. Verify I/O runs successfully without any errors.
Result
PASS. I/O completed successfully. Both Edge fabrics have the corresponding proxy name serverentries for the host and target ports.
Storage Device Fibre Channel Routing (FCR) Internetworking Tests
36 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Storage Device Internetworking Validation with FCoE using VDXFlexPort
Test Objective1. Configure a FC fabric with FCR connected to an FCoE fabric.2. Verify that edge devices are imported into adjacent name servers and hosts have access to their
routed targets after FC routers are configured.
ProcedureTest Execution:
1. Add FCoE VCS fabric to FCR setup.2. Set up LSAN zoning and verify host access to target LUNs and start I/O.
Result Validation:
1. Verify name server and FCR fabric state.<==========># fcrfabricshowroot> fcrfabricshowFC Router WWN: 10:00:00:05:33:5b:7d:06, Dom ID: 97,Info: 10.38.66.97, "B6510_066_097" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 18 110 10.38.66.120 10:00:50:eb:1a:62:8c:33 "VDX6740_066_120"FC Router WWN: 10:00:00:05:33:5b:10:e8, Dom ID: 100,Info: 10.38.66.100, "B6510_066_100" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 8 110 10.38.66.119 10:00:50:eb:1a:62:83:7b "VDX6740_066_119"# fcrproxydevshowroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 110 50:05:07:60:5e:80:76:52 02f001 120 612800 Imported 110 50:05:07:60:5e:80:76:71 02f101 120 642300 Imported 120 10:00:8c:7c:ff:00:48:00 03f101 110 781000 Imported 120 10:00:8c:7c:ff:00:48:01 03f001 110 771000 ImportedTotal devices displayed: 4<==========>
2. Verify I/O runs successfully without any errors.
Result
PASS. I/O completed successfully. Both Edge fabrics have the corresponding proxy name serverentries for the host and target ports.
Storage Device Internetworking Validation with FCoE using VDX FlexPort
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 3753-1003804-01
Storage Device Edge Recovery after FCR Disruptions
Test Objective1. Configure FCR for Edge-Backbone-Edge configuration.2. With I/O running, validate device access and pathing.3. Perform reboots, switch disables, and port-Toggles on Edge connections to disrupt device pathing
and I/O.4. Verify path and I/O recovery once switches and ports recover.
ProcedureTest Execution:
1. Set up FCR in an Edge-Backbone-Edge configuration.2. Setup LSAN zoning and verify host access to target LUNs and start I/O.3. Perform sequential reboots, switch disables, and ISL port toggles on the switches in the edge fabric.
Result Validation:
1. Verify FCR fabric state throughout the disruptions.<==========># fcrfabricshowroot> fcrfabricshowFC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82,Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92"FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83,Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088"# fcrproxydevshowroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 ImportedTotal devices displayed: 20<==========>
2. Check the switch logs for any errors.<==========># errdumpall# portperfshow<==========>
3. Check host and storage logs, and verify I/O runs without any errors.
Result
Storage Device Edge Recovery after FCR Disruptions
38 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
PASS. I/O fails over to available switch path and recovers when disrupted switch is restored.
Storage Device Backbone Recovery after FCR Disruption
Test Objective1. Configure FCR for Backbone-Edge configuration.2. With I/O running, validate device access and pathing.3. Perform reboots, switch disables, and port-Toggles on Backbone connections to disrupt device
pathing and I/O.4. Verify path and I/O recovery once switches and ports recover.
ProcedureTest Execution:
1. Connect array target ports to backbone fabric in an Edge-Backbone configuration.2. Set up LSAN zoning and verify host access to target LUNs and start I/O.3. Perform sequential reboots, switch disables, and ISL port toggles on the switches in the backbone
fabric.
Result Validation:
1. Verify FCR fabric state throughout the disruptions.<==========># fcrfabricshowroot> fcrfabricshowFC Router WWN: 10:00:00:05:33:13:96:5a, Dom ID: 82,Info: 10.38.66.82, "B6510_066_082" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088" 41 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92"FC Router WWN: 10:00:00:05:33:5b:1d:1d, Dom ID: 83,Info: 10.38.66.83, "B6510_066_083" EX_Port FID Neighbor Switch Info (enet IP, WWN, name) ------------------------------------------------------------------------ 40 40 10.38.66.92 10:00:00:27:f8:66:f3:81 "B6520_066_92" 41 40 10.38.66.88 10:00:00:27:f8:06:23:28 "B6510_066_088"# fcrproxydevshowroot> fcrproxydevshow Proxy WWN Proxy Device Physical State Created PID Exists PIDin Fabric in Fabric---------------------------------------------------------------------------- 40 10:00:00:05:1e:60:b4:6b 02ff02 100 132400 Imported 40 10:00:00:05:1e:60:b4:6c 02fe02 100 032400 Imported………………. 100 50:05:07:60:5e:80:76:52 01fe02 40 612800 Imported 100 50:05:07:60:5e:80:76:71 01ff02 40 642300 ImportedTotal devices displayed: 20<==========>
2. Check the switch logs for any errors.<==========># errdumpall
Storage Device Backbone Recovery after FCR Disruption
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 3953-1003804-01
# portperfshow<==========>
3. Check host and storage logs, and verify I/O runs without any errors.
Result
PASS. I/O fails over to available switch path and recovers when disrupted switch is restored.
Additional and Optional Tests
Non-Disruptive Firmware Upgrade on Storage Device
Test Objective1. Perform firmware maintenance procedure on the storage device.2. Verify Host I/O will continue (with minimal disruption) through the "firmware download" and device
pathing will remain consistent.
ProcedureTest Execution:
1. Set up host multipath with links on different switches in the FC fabric and start I/O.2. Perform firmware update on all nodes of the storage array.
Result Validation:
1. Check the I/O generator tools logs to verify I/O completes without any errors.2. Check the host and storage logs for any errors throughout the I/O operations.3. Check the switch error logs and port stats for any errors or I/O drops.
<==========># errdumpall# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>
Result
PASS. I/O completed successfully throughout the firmware upgrade process.
Additional and Optional Tests
40 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01
Synthetic I/O Workload Loop with Varying Block Sizes
Test Objective1. Validate Storage/Fabric behavior while running a workload simulation test suite.2. Areas of focus may include random and sequential data patterns of various block sizes and database
simulation.
ProcedureTest Execution:
1. Set up 4 standalone hosts with two multipathed initiator ports for I/O generation.2. Use Medusa I/O tool for generating I/O and simulating workloads.3. Run random and sequential I/O in a loop at block transfer sizes of 512, 4k, 8k, 16k, 32k, 64k, 128k,
256k, 512k, and 1m. Include a nested loop of 100% read, 100% write, and 50% read/write.4. Run File Server simulation workload.5. Run Microsoft Exchange Server simulation workload.
Result Validation:
1. Check the I/O generator tools logs to verify I/O completes without any errors.2. Check the host and storage logs for any errors throughout the I/O operations.3. Check the switch error logs and port stats for any errors or I/O drops.
<==========># errdumpall# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>
Result
PASS. All workload runs were monitored at the host, storage and fabric and verified they completedwithout any I/O errors or fault.
VMware IOAnalyzer Workload Test
Test Objective1. Validate Storage/Fabric behavior while running a virtual workload simulation test suite.2. Areas of focus include VM environments running de-duplication/compression data patterns, and
database simulation.
ProcedureTest Execution:
Synthetic I/O Workload Loop with Varying Block Sizes
Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report 4153-1003804-01
1. Set up an ESX cluster of two hosts with four worker VMs per host.2. Use VMware IOAnalyzer tool for generating I/O and simulating workloads.
• Run random and sequential IO at large and small block transfer sizes• Run SQL Server simulation workload• Run OLTP simulation workload• Run Web Server simulation workload• Run Video on Demand simulation workload• Run Workstation simulation workload• Run Exchange srrver simulation workload
Result Validation:
1. Check the I/O generator tools logs to verify I/O completes without any errors.2. Check the host and storage logs for any errors throughout the I/O operations.3. Check the switch error logs and port stats for any errors or I/O drops.
<==========># errdumpall# porterrshowroot> porterrshow frames enc crc crc too too bad enc disc link loss loss frjt fbsy c3timeout pcs tx rx in err g_eof shrt long eof out c3 fail sync sig tx rx err 0: 1.7g 2.7g 0 0 0 0 0 0 0 946 1 0 1 0 0 0 0 0 1: 431.1m 431.4m 0 0 0 0 0 0 0 30 1 0 1 0 0 0 0 0 2: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0<==========>
Result
PASS. All workload runs were monitored at the host, storage and fabric and verified they completedwithout any I/O errors or faults.
Test Cases
42 Brocade Fabric Technology with IBM FlashSystem 840 Arrays Validation Test Report53-1003804-01