white paper emulex’s lpe16000 family accelerating the ... · emulex’s lpe16000 family-...

White Paper

Performance and Scalability for the

Virtualized and Cloud-Ready Data

Center

Emulex’s LPe16000 family- Accelerating the Transition to 16Gb Fibre Channel Storage

Table of contents

Introduction .................................................................................................................................................. 3

Development and Architecture of 16GFC .............................................................................................. 3

Growth Drivers for 16GFC ........................................................................................................................ 4

Accelerated Industry Transition to 16GFC ........................................................................................... 11

Benefits of 16GFC .................................................................................................................................... 11

Selecting an Optimum 16GFC HBA Solution ...................................................................................... 13

Summary ................................................................................................................................................... 16

Accelerating the Transition to 16Gb Fibre Channel

3 Emulex White Paper | Accelerating the Transition to 16Gb Fibre Channel

Introduction

Market research firm IDC forecasts continued growth for Fibre Channel (FC) ports and revenue

over the next four years to 20161 with a dominant, total FC adapter and storage infrastructure

market valued at over $13.5 billion.

Storage giants EMC and NetApp in a joint VMworld 2012 session2 indicated, based on various

surveys, FC as the dominant storage protocol, deployed by 57%-67% of respondents, ahead of

other technologies. Market research firm Gartner estimates the asset value of FC installed

infrastructure at over $50 billion3 - making it a virtual certainty that customers will want to protect

their investment in trusted FC technology, and move to new 16Gb FC (16GFC) technology to

meet new IT demands.

Overall, this significant investment in its installed base, and anticipated storage growth, point to

a strong FC market for the foreseeable future.

This paper examines FC growth drivers, the state of FC innovation, performance advantages of

16GFC and concludes with a discussion of the criteria that inform the selection of world class

FC adapter solutions.

Development and Architecture of 16GFC

Ratified in late 2010, and announced by Emulex in May 2011, 16GFC, consistent with the Fibre

Channel Industry Association’s (FCIA) market requirements, delivers on key capabilities

including:

Backward auto-negotiation compatibility to 8GFC and 4GFC

Compatibility with existing 4GFC and 8GFC cable assemblies

Comparable cable lengths to existing 4GFC and 8GFC options

Backward rate compatibility ensures data centers can be upgraded to 16GFC while

maintaining investments in existing 4GFC and 8GFC infrastructure.

Rate compatibility has been achieved with new dual-codec architecture, shown in Figure 1,

necessitated by the change in line coding rates from 8b/10b 4GFC/8GFC to 64b/66b for 16GFC.

This change at 16GFC allows more efficient data transmission with 97% of the link speed

delivering usable bandwidth (64 bits out of every 66, 2 for overhead) compared to 80%

efficiency (8 out 10 bits) for 4/8GFC.

1 IDC WW Enterprise Storage Systems Forecast May 2012

2 http://download3.vmware.com/vmworld/2012/top10/sto2980.pdf

3 SearchStorage, FC technology still dominates enterprise storage networking, March 2012

http://download3.vmware.com/vmworld/2012/top10/sto2980.pdf

http://searchstorage.techtarget.com/tip/FC-technology-still-dominates-enterprise-storage-networking



As a result, 16GFC delivers double the bandwidth (throughput) of 8GFC at a line rate of only

14.025Gbps, less than double the 8.5Gbps line rate of 8GFC Host Bus Adapters (HBAs),

resulting in lower overall cost for associated electronics and optics.

Figure 1

Of equal importance, existing cabling can continue to carry 16GFC with only a marginal

performance (distance) impact (Table 1)4 for 16GFC.

Cable Type/Rate 4GFC 8GFC 16GFC

OM2 150m 50m 35m

OM3 380m 150m 100m

OM4 400m 190m 125m

Table 1 – Cable Types, Rate and Lengths

Growth Drivers for 16GFC

The confluence of multiple and interrelated IT trends are driving the growth and accelerating the

transition to 16GFC. These include:

1. Corporate IT priorities

2. Application use cases

3. Advances in the IT ecosystem

4 Demartek Storage Interface Comparision, August 2012

http://www.demartek.com/Demartek_Interface_Comparison.html



4. Advances in server virtualization, specifically VMware vSphere® and Microsoft Windows

Server 2012® Hyper-V

1. Corporate IT Priorities

A recent survey by market research firm ESG5 indicated the top two IT priorities for 2012-2013,

both of which are interrelated and directly drive demand for higher storage performance and the

need to migrate to 16GFC.

Improved Backup and Recovery

As applications drive increased storage growth, there is a resulting need for robust and

rapid backup to minimize both data loss and downtime. Deploying 16GFC with its ability

to deliver twice the bandwidth of 8GFC is well suited to deliver improved backup and

recovery performance.

Increased Virtualization

Virtualization has matured from an esoteric concept to pervasive adoption, with

respondents in ESG’s survey indicating:

60 percent will deploy 251 or more virtual machines (VMs) in their data centers by

the end of 2013, compared to 38% in 2011.

65% will consolidate 11 or more VMs per host server by the end of 2013 compared

to 31% in 2011.

Multiple virtualized workloads and higher virtualization density (number of VM workloads per

physical host server) demand increased storage input-output (I/O) bandwidth and improved

latency performance.

While these two investment priorities have been highlighted in ESG’s research, other

applications use cases and ecosystem infrastructure advances also fuel the growth of 16GFC.

2. Application Use Cases

Application use cases that can immediately accrue the benefits of 16GFC include:

OLTP Database Servers

E-commerce transactions and email are classic high volume online transaction processing (OLTP) solutions, characterized by small block size random I/O that requires

5 ESG Webinar, 16G Fibre Channel, August 2012



the high input/output operations per second (IOPS) performance of 16GFC to ensure optimum performance.

Also, densely virtualized servers, carrying heterogeneous workloads generating random I/O can achieve optimum performance with high IOPS 16GFC connectivity.

Virtual Desktop Infrastructure (VDI)

Delivering desktop applications to hundreds or thousands of end users as a managed

centralized service places heavy demands on centralized storage best addressed with

16GFC connectivity for the back-end servers.

Streaming Media/Video On Demand

Consumption of digital content requiring exceptionally large block size sequential I/O

performance is well suited for the high throughput capability of 16GFC.

Comparative performance analysis by ESG, highlighted in Figure 2, clearly highlights the

relative throughput and latency benefits delivered by Emulex’s LPe16000 family of

16GFC solutions.



Figure 2 – Comparative FC Storage Application Performance Analysis

3. IT Ecosystem

Storage and Computing advances driving 16GFC demand include:

Enterprise Solid State Disk (SSD) Storage & Cached Storage Area Networks

(SANs)

SSD storage arrays, either all flash-based or as a front-end tier 0 cache for traditional

disk based arrays (in a tiered storage architecture), deliver superior performance

with10X-40X higher IOPS performance and improved latency, are demanding superior

FC I/O performance to keep up with the blistering pace.



T-10 Protection Information (T10 PI) compliance

Emulex FC HBAs support T10 PI, formerly called T10-DIF, which has been developed in

collaboration with Oracle to prevent silent data corruption and help ensure the integrity

and regulatory compliance of user data as it is transferred from the application to the

SAN.

Emulex has implemented the T10 PI standard on its LightPulse® family of FC HBAs with

its exclusive BlockGuard™ feature. Traditionally, protecting the integrity of customers'

data has been done with multiple discrete solutions, including Error Correcting Code

(ECC) and Cyclic Redundancy Check (CRC), but there have been coverage gaps

across the I/O path from the operating system to the storage.

The implementation of the T10 PI standard via Emulex's BlockGuard feature,

especially with it fully offloaded in LPe16000B 16GFCHBAs, in conjunction with

other industry players’ server and array implementations, ensures that data is

validated as it moves through the data path, from the application, to the HBA, to

storage, enabling seamless end-to-end integrity.

Server architecture

In addition to multi-core processors, larger memory configurations and support for PCI

Express (PCIe) 2.0 (and emerging PCIe 3.0) have established the server ecosystem

driving the need for deploying higher bandwidth 16GFC.

4. Server Virtualization

VMware vSphere® 5.1 and Microsoft Windows Server 2012® Hyper-V include new and enhanced storage capabilities driving the need to enhance storage infrastructure to 16GFC. Emulex’s LPe16000 family of 16GFC adapter drivers are “in-box” for both virtualization platforms (exclusively, at launch, for vSphere 5.1), enabling users to immediately accrue the benefits of deploying 16GFC infrastructure.

The enhanced FC storage capabilities in these leading virtualization platforms include:

VMware vSphere 5.1

Full 16GFC support

Unlike vSphere 5.0 which supported 16GFC at 8GFC performance, vSphere 5.1 delivers double the throughput of 8GFC for large block sizes leading to significant performance improvements. The VMware data below6 is an example for sequential read operations (NOTE: similar performance improvements are also noted for write operations):

o Better Throughput (Please see Figure 3 below)

6 http://www.vmware.com/files/pdf/techpaper/VMware-vSphere-16Gb-StorageIO-Perf.pdf

http://www.vmware.com/files/pdf/techpaper/VMware-vSphere-16Gb-StorageIO-Perf.pdf



25% better throughput at 4KB block size 90%-100% better throughput at 16KB-256KB block sizes

Figure 3 – Comparative Throughput Performance

o Better CPU efficiency per I/O - CPU is a critical resource that determines VM density in a host and CPU efficiency is a measure of the "CPU cost" to generate IOPS (Please see Figure 4) 16GFC is 7% to 15% more efficient for READ operations across multiple block

sizes.

Figure 4 – Comparative CPU Efficiency

Note: All measurements were taken with Emulex LightPulse FC HBAs



VMs in vSphere 5.1 scale to 64 vCPUs More powerful virtualized workloads can drive more storage I/O requiring 16GFC's higher IOPS capabilities.

vSphere Data Protection (VDP) Consistent with ESG’s research of IT priorities, VMware launched a new backup and recovery solution for vSphere 5.1. It is included with vSphere 5.1 and while ideally suited to protect smaller environments of up to 2TB or 100VMs, larger environments can be protected by scaling out multiple VDP appliances. vSphere 5.1's enhanced backup capability, now supported by higher throughput 16GFC, improves the overall solution efficiency.

Microsoft Windows Server 2012 Hyper-V

Windows Server 2012 delivers innovative FC related capabilities that underscore the continuing importance of FC architecture in virtualized and cloud-ready data centers. These include:

Virtual Fibre Channel (Virtual FC)

Virtual FC extends Fibre Channel functionality to VMs enabling native SAN access

directly from the VMs. Using Virtual FC, also known as Synthetic FC, Hyper-V now

enables virtualization of workloads that require direct access to SAN storage Logical Unit

Numbers (LUNs). Higher performing 16GFC HBAs ensure optimized performance when

these workloads demand higher storage IOPS or higher bandwidth.

Virtual FC N_Port ID Virtualization (NPIV)

Complementary to Virtual FC, NPIV technology maps multiple virtual FC ports to a single

physical FC port enabling VM storage I/O isolation. A new virtual NPIV port is created on

the host each time a virtual HBA configured VM is started, which then logs in to a

proximate switch. The Emulex LPe16000 16GFC family, with up to 8,192

simultaneous logins, up to 4X more than other HBAs, maximizes VM density in the

host.

Enhanced Queue Depth

Windows Server 2012 allows pending I/O depth per port (queue depth) to scale beyond

the older limit (on Windows Server 2008 and Windows Server 2008R2) of 1000,

enabling a maximum queue depth per port of 3,000 on Emulex’s LPe16000 series of

16GFC HBAs. This increases the ability to manage a larger number of I/O by adjusting



the queue depth, enabling the LPe16000 series to even out traffic to an array that may

not be able to handle traffic peaks.

Accelerated Industry Transition to 16GFC

Collectively, unlike with past transitions, the significant industry trends discussed above will make a rapid cut-over to 16GFC an imperative for data center IT management. Not surprisingly, Crehan Research forecasts 16GFC port shipments will exceed 8GFC by around mid-20137, (please see Figure 5) a faster pace than was experienced at the 4GFC to 8GFC transitions.

Figure 5 – FC adapter port shipment forecast

Benefits of 16GFC

Nominally, the availability of 16GFC appears to be an evolutionary bandwidth and IOPS

enhancement. However, it delivers significant and timely benefits supporting the current data

center trends discussed above. These include:

Doubled Throughput, Lower Latency and Higher IOPS

16GFC delivers up to1600 MBps unidirectional throughput (double 8GFC’s 800 MBps),

6X the IOPS (please see Figure 6) and 75% lower latency (please see Figures 6 and 7)8

- supporting deployments of densely virtualized servers, increased scalability and

7Crehan Research, July 2012

8 ESG Lab Validation Report, February 2012

2011A 2012E 2013E 2014E 2015E

4GFC

8GFC

16GFC

http://www.emulex.com/artifacts/f55b5cd7-04cd-416c-8602-253188ece5d0/elx_ar_all_hba_lpe16000_esg_lab_validation_report.pdf



matching the performance of multi-core processors and SSD based storage

infrastructure. The performance data was measured on Windows Server® 2008 R2.

Figure 6- Comparative IOPS Performance (8GFC & 16GFC)

Figure 7- Comparative Latency Performance (8GFC & 16GFC)

Reduced Capital Expenditures (CAPEX)



Higher speed 16GFC reduces the number of HBAs and switch ports required to achieve

similar performance, resulting in lower front-end CAPEX. Fewer links also mean less

structured cabling investment, running at hundreds of dollars per port.

Reduced Operating Expenditures (OPEX)

A 16GFC HBA delivers 100 percent more bandwidth for about a 40 percent increase in

power consumption9. Installing two 8GFC HBAs for doubling bandwidth would result in

100 percent increase in power consumption while installing a 16FC HBA can deliver the

same result for a 40 percent increase in power. Reduced power per bit transferred

reduces OPEX.

Reduced port counts also simplify manageability, improving IT productivity and reduction

in associated OPEX.

Dedicated, Future-Proofed Storage Networking

Mission critical applications workloads have long relied on a dedicated FC network to

deliver the required performance, resiliency and serviceability. Deploying 16GFC not

only preserves this architecture, but it is also future-proofed. Even as 16GFC

proliferates, standardization of 32GFC is in process, ensuring the continuity and

scalability of FC technology.

Selecting an optimum 16GFC HBA Solution

Storage I/O is a mission critical element of IT infrastructure. Selecting the best 16GFC product

from one of the many FC HBA vendors should take into consideration multiple factors that

address the whole solution.

Stability of the FC driver software stack.

Data centers can include a vast mix of storage and server infrastructure of varied

vintage. Nominally, all products are “standards-based” but operational “quirks” can

create a headache for the storage administrator.

A broadly deployed HBA driver stack, such as Emulex’s, with over 12 million installed

ports, ensures the reliability needed to avoid performance and interoperability issues that

can sap IT staff productivity.

Breadth of Operating System (OS) support

Comprehensive OS support is vital to ensure usability across heterogeneous workloads.

Emulex’s LPe16000 family of 16GFC HBAs not only support the major OS platforms

9 SDC 2011, 16GFC sets the pace for storage networks, SNIA presentation, August 30, 2011

http://www.snia.org/sites/default/files2/SDC2011/presentations/monday/ScottKipp_16GFC_Sets_Pace_StorageRev3.pd



such as VMware®, Linux®, Oracle Solaris® and Windows Server® but also the multiple

releases that accompany them. This allows a single FC HBA platform to service the

myriad server OS platforms in a heterogeneous data center – simplifying IT

management and reducing ongoing management related costs.

Manageability

Management costs (OPEX) can outweigh equipment acquisition costs (CAPEX) leading

to a higher total cost of ownership. Manageability of the HBA is a critical attribute that

can be overlooked in a discussion of “speeds and feeds”.

HBA management extends beyond providing high quality, bundled, vendor-specific

software. As the industry moves towards “single pane management” of all data center

infrastructure, management platforms such as VMware vCenter™ Server and

Microsoft® System Center have gained popularity.

Emulex delivers “plug in” software such as a “CIM Management Provider” for VMware

vCenter Server. This integrates the in-depth management of Emulex’s FC HBAs into the

VMware vCenter Server management console via a separate tab (please see Figure 8).

Figure 8 – Emulex vCenter CIM Management Provider Software



Emulex also delivers OneCommand® Vision software, which takes an applications view

of performance and providing and end-to-end view of I/O performance (Please see

Figure 9). This tool assists storage administrators perform true “forensic” analysis to

understand storage performance bottlenecks.

Figure 9 – End-to-End Storage I/O Performance with Emulex OneCommand Vision

Breadth of Server CPU Hardware support

Emulex’s LPe16000 family includes broad support for CPU platforms including x86, x64,

Intel® Itanium®-64 bit, IBM® PowerPC® and Oracle® SPARC, delivering similar

benefits discussed above vis-à-vis multiple OS support.



Summary

FC infrastructure continues to grow and dominate the enterprise storage networking market with

time-tested reliability, performance and enterprise-class features. Proliferating virtualization

deployments and growing use of flash SSD based storage arrays is driving the need for higher

bandwidth and higher IOPS 16GFC, ratified in 2010 and introduced to the market in 2011.

The other benefits of 16GFC include seamless integration into existing FC networks, reduced

CAPEX and OPEX on a future-proof technology platform. Multiple applications including data

replication/backup, video streaming and VDI benefit from deploying 16GFC for optimized

storage networking.

The selection of the right 16GFC solution should go beyond “speeds and feeds” and include an

understanding of the product’s driver stack, manageability and breadth of OS and hardware

support. Emulex is the only 16GFC vendor delivering FC HBA solutions that comprehensively

address all these attributes – making the LPe16000 family a superior choice for today’s data

centers.

© Copyright 2012 Emulex Corporation. The information contained herein is subject to change without notice. The only warranties for Emulex products and services are set

forth in the express warranty statements accompanying such products and services. Emulex shall not be liable for technical or editorial errors or omissions contained

herein.

World Headquarters 3333 Susan Street, Costa Mesa, California 92626 +1 714 662 5600 Bangalore, India +91 80 40156789 | Beijing, China +86 10 68499547 Dublin, Ireland+35 3 (0)1 652 1700 | Munich, Germany +49 (0) 89 97007 177 Paris, France +33 (0) 158 580 022 | Tokyo, Japan +81 3 5322 1348 Wokingham, United Kingdom +44 (0) 118 977 2929

white paper emulex’s lpe16000 family accelerating the ... · emulex’s lpe16000 family-...

Documents