august network announcement v3
TRANSCRIPT
© 2012 IBM Corporation
IBM Flex System Fabric SI4093 System Interconnect Module
Heather Richardson - Product MarketingHector Sanchez – Product Manager
Aug 2013
© 2012 IBM Corporation2 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Agenda
IBM Flex System Fabric SI4093 Module EN4093R Update IBM Flex System EN6131 40Gb Switch
Infrastructure ComponentsBeyond Blades
© 2012 IBM Corporation3 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Data Center Challenges
Complexity Too many layers to manage Virtual or logical management
Multiple Management points Physical and Virtual Management Manage Server, Storage and I/O as one entity
Increase infrastructure utilization IT budgets are flat or decreasing Virtualization / Cloud and new applications demand more throughput and
performance Manage risk and plan for next decade
Reduce data center foot print – space, power and cost Focus on all aspects to increase hardware utilization Management is one of the biggest area of focus followed by hardware
spending and power
© 2012 IBM Corporation4 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Intelligence is moving to the edge of the network
WANWAN
Layer2
SAN
Access Layer
Core Layer
Aggregation Layer
> 75%> 75%
< 2
5%<
25%
Servers
Network
Storage
Software
Intelligence is moving from the core to the edge– Opposite of what some of our competitors want!
Why this is happening:– Performance and latency demands
– Server and networking optimization is critical
– Convergence of LAN and SAN
– Virtualization of more east-west traffic
Customer Impact: – High Frequency Trading, Database intensive apps,
Messaging (MQ) and Highly Virtualized environments
© 2012 IBM Corporation5 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
NEW v 1.3 NEW v 1.3 NEW NEW
NEW POWER 7+NEW POWER 7+
NEWNEW
p270 p460
x222
p24L p260
The Flex System Portfolio Continues to Grow
EN4093RSI4093
Flex System Manager
Flex System V7000
Storwize V7000
x220 x240 x440 PCiE Expansion
Storage Expansion
EN6131 EN2092 FC5022 FC3171CN4093
© 2012 IBM Corporation6 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Networking
IBM Flex System Fabric SI4093 System Interconnect ModuleN
etw
ork
ing
In
fras
tru
ctu
re
Simplifies chassis connectivity without
compromising performance◊
Suited for Clients looking for ‘unmanaged’ device in the
chassis◊
“Pay as you grow” scalability ◊
Seamless interoperability
• Designed for easy connectivity and management• Pre-configured from factory – enhances ‘out of box’
experience for Clients• Rapid network provisioning without configuration
errors• Investment Protection – scalable pay-as-you-grow
design reduces cost and complexity• Intra chassis switching for optimal performance
(Ex. vMotion)• Virtual Fabric capability reduces cost and improves
IO utilization• Seamless interoperability with other vendors
switches like Cisco, Juniper and Brocade
Base 10 x 10G
bE
SF
P+
#2 = 4x10G
bE
#1 = 2x40G
bE
1Gb
E M
gm
t
Total Ports
10Gb
to Server
10Gb
Uplinks
40Gb
Uplinks
Base System 14 10 0
Upgrade #1 28 10 2
Upgrade #2 42 14 2
Unmanaged 10Gb module with10/40Gb Uplinks
Unmanaged 10Gb module with10/40Gb Uplinks
© 2012 IBM Corporation7 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Networking
IBM Flex System Fabric EN4093R with OpenFlow SupportSoftware Defined Networking
Net
wo
rkin
g I
nfr
astr
uct
ure
Increased IT efficiency with programmable networks
Increased IT efficiency with programmable networks
OpenFlow support enables Software Defined Networks
(SDN)◊
Programmable network enables easy customization
and innovation ◊
Create SDN-ready network architecture based on Open
Standards
• Reduce complexity by building the underlying network once
• Dynamic and rapid application deployment
• Automate network provisioning and interoperability
• Intelligent and dynamic multipath routing based on business policy
• Customize network architecture based on business policies and needs
• Centrally configure and enforce multi-tenant networks
• Available to new and existing Clients as feature update at no extra cost
• Reduce complexity by building the underlying network once
• Dynamic and rapid application deployment
• Automate network provisioning and interoperability
• Intelligent and dynamic multipath routing based on business policy
• Customize network architecture based on business policies and needs
• Centrally configure and enforce multi-tenant networks
• Available to new and existing Clients as feature update at no extra cost
© 2012 IBM Corporation8 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Benefits Key Features
Reduced cost and complexity • Easy connectivity with preconfigured settings• Eliminates the need to configure each device or individual ports
thus reducing the number of management points• Lower TCO - Up to 42% less expensive than the competition*• Lower cabling and power costs
“Pay as you grow” scalability • Supports up to forty-two downlink 10GbE connections, fourteen uplink 10GbE SFP+ connections plus two 40GbE QSFP+ connections
• 2-3 X more bandwidth than the competition**
Simple Management • Default profile end host mode – plug and play just add cables• No Spanning Tree, loop free, port aggregation System Interconnect• End Host configuration viewed as a large compute node to
upstream Network
“System administrator friendly” System Interconnect Module
• Minimal system admin dependency on network admin for most compute node provisioning actions
• Can never default into a switch mode – protects against network misconfigurations
Simplify connectivity and interoperability challenges with a preconfigured interconnect module
IBM Flex System Fabric SI4093 System Interconnect Module Highlights
Base - 95Y3313, ESWAUpgrade 1 - 95Y3318, ESW8Upgrade 2 - 95Y3320, ESW9
Claim * Based on SI4093 base plus upgrade 1 and 2 list price x 2 = $53.194, HP Virtual Connect Flex-10/10D x 6 = $75,450 => 41.8% http://h30094.www3.hp.com/product.aspx?sku=10440121&pagemode=ca **SI4093 data sheet 64x10Gb ports (640Gb) vs HP VC 10 data sheet 26x10Gb ports (260Gb) => 2.1x - http://h30094.www3.hp.com/product/sku/10440121 SI4093 42 Downlinks + 22 Uplinks (64 10GbE) vs HP FEX 16 downlinks + 8 Uplinks (24 10GbE) => 2.6x - http://h18004.www1.hp.com/products/quickspecs/14138_div/14138_div.html
© 2012 IBM Corporation9 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
I/O In
fras
tru
ctu
re
Ethernet Connectivity
Simple Management◊
Optimized for performance ◊
“Pay as you grow” scalability ◊
Lowest TCO◊
Easy interoperability
IBM Flex System Fabric SI4093 System Interconnect Module
Overview•Easy to deploy simple connectivity– upgrade in order
Important Features:•Exceptional price/performance•FCoE convergence as a transit switch•Easy connectivity to reduce deployment time•Low Touch – default mode or VLAN mode•Easy interoperability with other vendors switches•No need for network administrators to manage•Warranty & software upgrade licenses match the chassis•Alternative to: Pass-thru, HP Cisco FEX or HP VC Flex
Base 10 x 10G
bE
SF
P+
#2 = 4x10G
bE
#1 = 2x40G
bE
1Gb
E M
gm
t
Total Ports
10Gb
Downlinks
10Gb
Uplinks
40Gb
Uplinks
Base System (95Y3313, ESWA)
14 10 0
w/ Upgrade #1 (95Y3318, ESW8)
28 10 2
w/ Upgrade #2 (95Y3320, ESW9)
42 14 2
1. Default transparent end host mode2. VLAN aware mode for multi-tenancy environments or environments
requiring more control of VLAN Layer 2 forwarding. Also enables FCoE FIPS Snooping Bridge support enhancing the local FCoE operation in the chassis.
© 2012 IBM Corporation10 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Connectivity options and distances for the SI4093Networking Cable and Transceiver Questions
Will your 1Gb connections exceed 100m (less than 550m)? = 1Gb Short Range Transceivers + Optical Cable
Will your 10Gb connections exceed 5m (less than 300m)? = 10 Gb Short Range Transceivers + Optical Cable
Will your 40Gb connections exceed 3m? = 40Gb Short Range QSFP+ Trans+ Optical Cable
Will your 1 or 10Gb connections exceed 300m (less than 10km)? = Long Range Transceiver + Optical Cable
• DAC cables are ideal within a rack or adjacent racks (lower cost & power)• Transceivers are best for longer distances
• DAC cables are ideal within a rack or adjacent racks (lower cost & power)• Transceivers are best for longer distances
1GbE
10GbE
5 Meters
DAC/TwinAxOptical Cable
300 Meters
10GBase-SRTransceiver
100 Meters
550 Meters
10KM
RJ-45
1000Base-SXTransceiver
1000Base-LX Transceiver
Optical Cable Optical Cable
40GbE
3 Meters
QSFP+
QSFP+ MTP Optical Cable
100Meters
40GBase-SR Transceiver
1000Base-TTransceiver
Optical Cable
10GBase-LRTransceiver
© 2012 IBM Corporation11 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Portfolio Positioning
© 2012 IBM Corporation12 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Low latency/ high bandwidth apps
Telecommunication
Enterprise applications
high PerformanceScalability
VirtualizationCloud computing
Simple management
Simple connectivity and interop with
upstream network
Infrastructure Applications
(file/ print/ collaboration)
Flex System Ethernet Module Positioning
EN2092
1Gb Scalable Switch
10Gb UplinksFor easy
Transition
Ethernet, iSCSIsupport
1Gb Ethernet
CN4093Convergence within the
chassisConnect to existing LAN and
SAN networkScalable fabric
Easy transition to 40GbEL2/3 function
Enhanced Virtual Fabric for reduce I/O cost and complexity
EN4093RHigh performance 10GbE
connectionL2/3 function
Scalable fabricEasy transition to 40Gb
Enhanced Virtual Fabric for reduce I/O cost and complexity
OpenFlow SDN Support
SI4093Simple setup and management
Scalable fabricEasy transition to 40Gb
Virtual Fabric for reduce I/O cost and complexity
EN6131High performance 40Gb
Low latency
EN20921Gb connectivity
Easy transition to 10Gb
EN409110Gb connectivityUnmanaged device
NEW Aug 6thNEW Aug 6th
SI4093
Simple Low touch
ConnectivityModule
10Gb performanceScalable
40Gb uplinksEthernet, iSCSIFCoE support
NEW Aug 6thNEW Aug 6thEN4091
10Gb Pass thru
Unmanaged Device
Non blockingConnectivity to
UpstreamNetwork
Ethernet, FCoE
EN6131
40Gb Ethernet
Eighteen uplinkHigh bandwidth
Low latency
Ethernet support
CN4093
10Gb Scalable Converged
Switch
40Gb uplinks
Native FC portsFCF support for
Storage node
Ethernet, FCoE
EN4093R
10Gb ScalableSwitch
40Gb uplinks
Ethernet, iSCSIFCoE support
OpenFlow Enabled
10Gb iSCSI, EthernetFCoE(transit)
10Gb ConvergedIn chassis
40Gb Ethernet
Updated Aug 6th
Updated Aug 6th
© 2012 IBM Corporation13 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Features 10Gb Converged Switch CN4093
10Gb Ethernet Switch EN4093R
10Gb System Interconnect Module
SI4093
10Gb Pass ThruEN4091
Summary FCoE Convergence Multi mode flexibility Preconfigured end host mode 1:1 direct connect
Layer 2/3 Support Yes Yes (No L3 in Easy Connect) L2 aware Pass thru only
Multi Tenancy (SPAR)
Yes Yes Yes N/A
Transparent / End host mode
Yes – Easy connect Yes – Easy Connect Yes – default profile N/A
FCoE/CEE support Yes Yes Yes Yes
FC ports Yes – Omni ports No No No
Spanning Tree Yes (No, disabled in Easy Connect)
Yes (No, disabled in Easy Connect)
No N/A
Max through – put* 1280Gbps 1280Gbps 1280Gbps 280 Gbps
Stacking Yes2 Yes No N/A
Interoperability Open Open Open Open
Network Virtualization
VMready, Virtual Fabric (vNIC2 or Unified Fabric Port*)
VMready, Virtual Fabric (vNIC2 or Unified Fabric Port)
Virtual Fabric (vNIC2) N/A
Max Ports base + upgrades
42x downlinks10 GbE†2x uplinks 10 GbE SFP+12x uplinks SFP+ Omni 2x uplinks 40 GbE QSFP+
42x downlinks 10 GbE† 14x uplinks 10 GbE SFP+2x uplinks 40 GbE SFP+
42x downlinks 10 GbE† 14x uplinks 10 GbE SFP+2x uplinks 40 GbE SFP+
14 x downlinks 10GbE
14 x uplinks 10GbE SFP+
Open Flow Support No Yes (v7.7) No N/A
SDN VE Support Yes Yes Yes Yes† This configuration leverages six of the eight ports on the CN4058 adapter available for IBM Power Systems™ compute nodes 2 - Hybrid mode only 2 CN4093 + 2-6 EN4093R
Feature Compare of IBM System Networking offerings for IBM Flex System
© 2012 IBM Corporation14 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Decision Tree: Storage Connectivity Requirements
Connect to
storage?
Compute Node: Ethernet: •LOM •CN4054 (with SW upgrade for iSCSI HW offload•EN4132/EN4054 for Power nodes
Chassis: Ethernet: EN4093R* or SI4093
Compute Node: Ethernet: •LOM or CN4054•EN4132/EN4054 for Power nodesFibre Channel: FC3172 or FC3052 (8Gb)FC5022 (16Gb)
Chassis: Ethernet: EN4093R*/SI4093 Fibre Channel: FCXXXX
Compute Node: Ethernet: •LOM or CN4054 (with SW upgrade for CNA function)•CN4058 (Power)**
Chassis: Converged Switch: CN4093*
Compute Node: Ethernet: •LOM or CN4054 (with SW upgrade for CNA function)•CN4058 (Power)**
Chassis: Ethernet Transit: EN4093R*/SI4093
TOR Converged Switch: G8264CS or Cisco/Brocade
NAS / iSCSI FCoE
Separate Fibre Channel
FCoE to TORBroken out to
IBM, Brocade or Cisco
FC directly out of chassis to FC SAN switch or FCoE to
storage device
Chassis Top-of-RackBreak out of
FC
© 2012 IBM Corporation15 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Technical Overview and Use Cases
© 2012 IBM Corporation16 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Simplify chassis connectivity using IBM System Interconnect Module
Overview•Default profile enables transparent integration
• No Spanning tree• Built in logic for loop prevention• Minimal setup or management required
•Key operating modes 1.Transparent mode:
• Presents an end host, (compute node) view to the upstream network• VLAN communication between compute node NIC and edge switch runs
transparently through the module (VLAN agnostic operation)• Enables server to server communication for optimum performance (i.e. vMotion)• Provides traffic consolidation in the chassis to minimize TOR port utilization• Supports FCoE pass-thru to a transit switch or FCoE gateway device
2.VLAN aware mode (Local Domain)• Provides greater security and separation for multi-tenant environments• Enables forwarding decisions on client VLANs (ACL support)• Can re-use IP addresses in multi-tenancy configurations• Supports FCoE transit switch operation, FSB operation
© 2012 IBM Corporation17 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Hardware Description
RJ45 Mgmt Port
Serial PortSFP+ Connector (1Gb/10Gb) QSFP+ Connector (10Gb/40Gb)
© 2012 IBM Corporation18 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Switch Partition (SPAR) – How it works:1) Creates separate virtual switching contexts by segmenting the data plane of the switch.
2) The management and control plane partition of a physical switch is NOT addressed within the SPAR context and are still common across the partitions.
3) Implemented as a dedicated VLAN which is generally defined by a set of internal server ports and a single external uplink port or LAG (static or LACP).
4) Multiple uplink ports or LAGs are not allowed within a SPAR, eliminating the possibility of misconfiguring the device and creating a loop with the upstream network.
5) Port memberships are mutually exclusive between SPARs.
6) By default, a SPAR is a private domain that operates as a Layer-2 broadcast network.
7) Hosts on the same VLAN attached to a SPAR are allowed to communicate with each other and with the upstream switch.
8) Hosts attached to the same VLAN but on different SPARs don’t communicate with each other within the embedded interconnect. Traffic needs to be bridged by the upstream switch in this situation since there is no communication between SPARs within each broadcast domain.
9) SPAR can be configured to operate in two basic domain processing modes: Transparent and VLAN aware (Local).
© 2012 IBM Corporation19 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
SPAR support for SI4093 is a superset containing select features from SPAR Phase 1 & 3 plus customizations specific only to SI4093
Key differences in SPAR on SI4093
•Addition of “Host Mode”• Enables multiple SPARs to
connect to the same upstream network
• Disables MAC learning on upstream ports and applies ACLs to avoid MAC Flapping
•FCoE support within a SPAR context• Transparent mode support –
pass-thru• Local mode – Transit / FSB
•Increase of local VLAN support to 256
© 2012 IBM Corporation20 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
EXT-1
SPAR-1 (Pass-Through)Default VLAN-4081
Port: {INTA1, INTA2, INTA3, EXT1, EXT2}
SPAR-2 (Pass-Through)Default VLAN-4082
Port: {INTB1, INTB2, INTB3, EXT3, EXT4}
EXT-2
PacketData
C-VLAN10
S-VLAN4081
PacketData
C-VLAN10
INTA1 INTA2 INTA3
S-VLAN4081
PacketData
C-VLAN10
S-VLAN4081
PacketData
C-VLAN10
S-VLAN4081
PacketData
C-VLAN10
PacketData
C-VLAN10
PacketData
C-VLAN10
PacketData
C-VLAN10
EXT-3 EXT-4
PacketData
C-VLAN10
S-VLAN4082
PacketData
C-VLAN10
INTB1 INTB2 INTB3
S-VLAN4082
PacketData
C-VLAN10
PacketData
C-VLAN10
PacketData
C-VLAN10
PacketData
C-VLAN10
S-VLAN4082
PacketData
C-VLAN10
S-VLAN4082
PacketData
C-VLAN10
SI4093 Packet forwarding in Transparent Mode
© 2012 IBM Corporation21 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
EXT-1
SPAR-1 (Local-Domain)Default VLAN-4081
Port: {INTA1, INTA2, INTA3, EXT1, EXT2}
SPAR-2 (Local-Domain)Default VLAN-4082
Port: {INTB1, INTB2, INTB3, EXT3, EXT4}
EXT-2
PacketData
C-VLAN10
PacketData
C-VLAN10
INTA1 INTA2 INTA3
PacketData
C-VLAN10
PacketData
C-VLAN10
PacketData
C-VLAN10
PacketData
C-VLAN10
EXT-3 EXT-4
INTB1 INTB2 INTB3
PacketData
C-VLAN10
PacketData
C-VLAN10
VLAN-10{EXT1, EXT2, INTA1, INTA2}
VLAN-20{EXT1, EXT2, INTA2, INTA3} VLAN-10
{EXT3, EXT4, INTB1, INTB2}
VLAN-20{EXT3, EXT4, INTB2, INTB3}
PacketData
C-VLAN10
SI4093Packet forwarding in VLAN aware Mode (Local Domain)
© 2012 IBM Corporation22 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
SPAR-1 SPAR-2
SI4093
vSwitch-1 vSwitch-2
ITE-1
vSwitch-1 vSwitch-2
ITE-2
vSwitch-1 vSwitch-2
ITE-3
DC Network
SI4093 Multi-Tenancy Host Mode
© 2012 IBM Corporation23 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
SPAR-1 SPAR-2
SI4093
vSwitch-1
ITE-1 ITE-2 ITE-3
DC Network
PG-1 PG-2
vSwitch-1
PG-1 PG-2
vSwitch-1
PG-1 PG-2
V10, V11, V12 V20, V21, V22
SI4093 Multi-Tenancy with VLAN aware
Each port group within vSwitch is using a specific uplink with VLAN separation among them. Each port group mapped to a SPAR so they can use separate uplink port set from the SI4093. In this kind of topology the SI4093 can run in VLAN aware mode and learn MAC address on uplink port. No dependency on centralize controller for server/VM MAC learning
© 2012 IBM Corporation24 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Hypervisor
MGMT
Heart Beat
LiveMigration
StorageiSCSINFS
FCoE
Supervisor-SwitchHypervisor-Firewall
VM VM
P0 P1 P3P2
Hypervisor
Hypervisor
MGMT
Heart Beat
LiveMigration
StorageiSCSINFS
FCoE
Supervisor-SwitchHypervisor-Firewall
VM VM
P0 P1 P3P2
Hypervisor
Partition 1MGMT, Migration
Storage, Heart Beat
Partition 2VM Data
SI4093 1
INT1A
INT2A
INT14A
INT1B
INT2B
EXT1
INT14B
EXT2
EXT3
EXT4
EXT6
EXT8
EXT5
EXT7
Partition 1MGMT, Migration
Storage, Heart Beat
Partition 2VM Data
SI4093 2
INT1A
INT2A
INT14A
INT1B
INT2B
EXT1
INT14B
EXT2
EXT3
EXT4
EXT6
EXT8
EXT5
EXT7
Nexus 5K Nexus 5K
Flex Chassis
14 X ½ wide ITE
40 Gb 40 Gb40 Gb 40 Gb
14 X 10Gb
14 X 10Gb
14 X 10Gb
14 X 10Gb
© 2012 IBM Corporation25 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Recommended configuration using end to end IBM offering
Adapter Integrated Device FCoE Top Of Rack Switch
SAN Switch Storage Target OS levels
LOM & CN4054 4-port adapter (BE3) pNIC and vNIC modes
SI4093 10Gb Module G8264CS (IBM Converged ToR) in NPIV mode
Cisco & Brocade SAN FCoE: Storage node, V7K, FC: SVC, DS3K/5K, DS8K, Tape, XIV
Win2008, ESX 4/5, RHEL 5/6, SLES 10/11
LOM or CN4054 (pNIC, vNIC
mode)
G8264CS
G8264CSV7000
SI4093 Flex System Fabric Interconnect
Key value:•Integrated FCoE on compute node•High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis•IBM 10Gb System Interconnect Module aggregates all FCoE traffic and passes to LAN and SAN.
Black line is FCOERed line is Fibre Channel
Cisco or Brocade
SAN
IBM Converged ToR
© 2012 IBM Corporation26 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Connecting to Cisco network: Customer already owns Nexus ToR
LOM or CN4054 (pNIC, vNIC mode)
V7000
SI4093 Flex System Fabric Interconnect
Key value:•Integrated FCoE on compute node•High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis•Connects to existing or new Cisco network.
Black line is FCOERed line is Fibre Channel
Cisco SAN
Nexus 55XX
Nexus 55XX
Adapter Integrated Device FCoE Top Of Rack Switch
SAN Switch Storage Target OS levels
LOM & CN4054 4-port adapter (BE3) pNIC and vNIC modes
SI4093 10Gb Module Nexus 5548 / 5596 Cisco SAN FC: V7000, SVC Win2008 R2SP1, ESX 5.0 U1, RHEL 6.3
© 2012 IBM Corporation27 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Connecting to Brocade network: Customer already owns Brocade VDX or SAN
LOM or CN4054 (pNIC, vNIC
mode)
V7000
SI4093 Flex System Fabric Interconnect
Key value:•Integrated FCoE on compute node•High performance scalable 10Gb Flex System Fabric System Interconnect Module in chassis•Connects to existing or new Brocade network
Black line is FCOERed line is Fibre Channel
BrocadeSAN
Brocade VDX
Brocade VDX
Adapter Integrated Device FCoE Top Of Rack Switch
SAN Switch Storage Target OS levels
LOM & CN4054 4-port adapter (BE3) pNIC and vNIC modes
SI4093 10Gb Module Brocade VDX Brocade SAN FCoE: Storage node, V7K, FC: SVC, DS3K/5K, DS8K, Tape, XIV
Win2008, ESX 4/5, RHEL 5/6, SLES 10/11
© 2012 IBM Corporation
IBM Flex System EN6131 40Gb SwitchIBM Flex System EN6132 40Gb Adapter
Matthew Sheard - Mellanox
Aug 2013
© 2012 IBM Corporation29 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Networking
IBM Flex System 40Gb Ethernet Switch (EN61310) and adapter (EN6132)N
etw
ork
ing
In
fras
tru
ctu
re
End to end 40Gb enables highest performance
End to end 40Gb enables highest performance
Ultra low latency for HPC, High Frequency Trading and
Enterprise Clients◊
Up to four 40Gb ports per single wide node
◊Highest performance non
blocking architecture drives up to 2.8TB bidirectional
throughput
• High performance end to end 40Gb connectivity with ultra low latency
• Non blocking throughput that is highly suited for High-Performance Computing, High Frequency Trading and Enterprise Data Centers.
• Enables clients to maximize compute utilization by removing IO bottleneck
• Reduces cabling by 4 is to 1 for applications requiring 40Gb throughput
© 2012 IBM Corporation30 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Mellanox Value Proposition
Mellanox heritage in technical computing means that Mellanox switches have some of the lowest port to port latency and jitter, providing a stable high speed interconnect for todays demanding business and virtualization environments.
Financial
Cloud & Web 2.0
Clustered Database Weather Digital MediaComputational Aided
Engineering BioscienceLabs, ResearchUniversity, Academic Oil and Gas
Web 2.0 DB/EnterpriseHPC Cloud Financial Services
Storage
© 2012 IBM Corporation31 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Performance and “Green” Too !
Efficiency: Reduce components and maximize productivity a data center.
Energy: Reduce power and cooling expenses
Complete High-Performance Scalable Interconnect Solutions for Server and Storage
TCO
50+%Reduction
Energy Costs
65+%Reduction
Infrastructure
60+%Saving
Performance
100+%Increase
Opex: Reduce management overhead and costly real estate.
© 2012 IBM Corporation32 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
RDMA Technology Becoming Mainstream
Remote Direct Memory Access• Direct access of one computer’s memory by another computer without involving
either ones operating system
• Allows applications to transfer data directly to the buffer of a remote application
• Zero-copy, I/O offload, CPU bypass technology for data transfer
• Near wire speed data transfers with sub 1us latency
Standard RDMA Protocols• RDMA-over-Converged-Ethernet (RoCE)
•Up to 40Gb/s
• InfiniBand: •Up to 56Gb/s
IBM:Yuqung Gao, Internet Centric ComputingWXS RDMA ICPE 2013
© 2012 IBM Corporation33 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
I/O Offload Boosts Application and Business Performance
~88% CPU Available for Application
Use
r S
pa
ceS
yste
m S
pa
ce
~53% CPU Available forApplication
~47% CPU Overhead/Wait
~12% CPU Overhead/Wait
Without RDMA With RDMA and Offload
Use
r S
pa
ceS
yste
m S
pa
ce
I/O Offload:More
Efficient Application Processing
© 2012 IBM Corporation34 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Underlying Technologies Enabling Cloud 2.0
Cloud 2.0
Virtualization
Private Clouds
Cloud 1.0
Private Clouds
Overlay Networks
VXLAN, NVGRE
Interconnect Offload Engines
Reduce Application Cost
Open Platform
Open Ethernet, SDN
Scalability
Virtually Unlimited Tenants
Mobility
Resources Everywhere
Serviceability
Any Service, Any Time
Small Scale Clouds
Fixed Resource Allocation
© 2012 IBM Corporation35 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Triple the number of Virtual Machines per server: from 20 to 60 VMs Higher storage density: 2X throughput and I/O Operations per Second (IOPS)
• Atlantic.net achieves 2X cloud I/O performance vs. Amazon with less cost using InfiniBand
Drive Higher Dollars Per Square Foot for Cloud
Without Mellanox With Mellanox
VMVM
VMVM
VMVM
VMVM
VMVM
VMVM
VMVM
VMVM
VMVM
VMVM
VMVM
VMVM
© 2012 IBM Corporation36 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
RoCE delivers lower latency, lower CPU load and higher bandwidth• Latency as low as 1us compared to 2.1us on iWARP• Up to 6X in Throughput (64B) compared to iWARP
iWARP requires TCP, using packet drops to handle heavy traffic• TCP is handled by the CPU or a special and complicated TCP Offload (TOE) NIC
RoCE requires NO CPU involvement
RoCE vs. iWARP
© 2012 IBM Corporation37 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
EN6131 Switch Features
Management plane Data plane• 10/100/1000 ETH RJ45 MNG ports• USB Console port for Management• Dual SW image • Static IP • DHCP • SSH • Telnet • RADIUS \ TACACS+ \ LDAP• NTP• FTP\TFTP\SCP• SYSLOG • Auto Temperature control • Events history • Port counters• SNMP v1,2,3• Web UI • In-band Management
• Port mirroring• sFlow• Rapid Spanning Tree (802.1w)• Flow control (802.3x)• DCB
• PFC (802.1Qbb)• ETS (802.1Qaz)
• LLDP• LAG/LACP (802.3ad)• 48K Unicast MAC addresses• Cables/transceivers identification• IGMP v1,2• VLAN (802.1Q) – 4K• ACL – 12K rules• Jumbo frames 9K
© 2012 IBM Corporation38 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Leading throughput: 2.5X better
• 2.88Tb/s throughput on a single chip, running Full Wire Speed at any packet size
Leading L2 unicast/multicast latency for L2/L3 switches: 2X better
• 198-223ns for any packet size
Industry record power efficiency: 6X better than competition
• Sub 0.6Watt per 10GbE throughput with 100% load at Full Wire Speed
Shattering Ethernet Switching Performance Records
Packet Size (Bytes)
La
ten
cy
(n
s)
‘Zero’ Jitter!
© 2012 IBM Corporation39 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Performance Driven Applications From Server to Fabric
Storage Connectivity Big data applications Distributed DB VM Environment (fast VM Migration) HPC Disaster Recovery Add more capacity to strained network
It is not about the number of servers you run
It is about how you run them
The Price of Building a Machine to Process 1TB of Data in 1 Minute
Migration Time Over the Network (sec)
© 2012 IBM Corporation40 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
E2E 40GbE Lowest Latency Cut through Solution
Superior message rate
Lowest Latency 40GbE in Trading
40GbE NASDAQ connectivity at the lowest latency
Exchange
MC streams
UC AckTCP/UDP(UC)
10.10.0.0/30Direct interface (pseudo vlan 2) 10.10.0.2
10.10.0.1
10.10.10/2Single VLAN
SX1036 TOR
Exchange Entry
© 2012 IBM Corporation41 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
End-to-End 40GbE Live in US-based Exchange Co-Location
“Mellanox provided a 40 Gigabit Ethernet solution that delivered high density, sustainable and high throughput, low latency and low power consumption that was beyond our expectations and in ways no other vendor could. We applaud the Mellanox team for their great support and helping us break new latency performance records.”Ido Rosen, quantitative researcher, Strike Technologies
Breaking New Latency Records
© 2012 IBM Corporation42 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Mellanox ConnectX-3 Dual-Port 40GE NIC and Switch release in Q3/2013 Single wide chassis: 14x ITEs / Blade Servers support 2 adapters per server Double-wide chassis: 7x ITEs / Blade Servers support 4 adapters per server 14 ports Internal I/O; 18 ports External I/O Total System Compute up to 2.24Tbps System Front I/O up to 2.88Tbps
IBM Flex System Chassis with Mellanox 40Gb Interconnect
IBM PureFlex System
© 2012 IBM Corporation43 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
14 Compute Blades each using Single EN6132 Dual Port 40GbE NIC
2 Switch Blades each using EN6131 SwitchX-2 32 Ports 40GbE
Compute I/O 1.12 Tbps @40 Gbps Uplink I/O up to 1.44Tbps @40 Gbps
Dual Star Architecture Dual-Dual Star ArchitectureMellanox Interconnect in IBM PureFlex
10GbE
10GbE
40GbE
40GbE
22 x 10GbE
18 x 40GbE
40GbE
40GbE
40GbE
40GbE
18 x 40GbE
18 x 40GbE
22 x 10GbE
18 x 40GbE
18 x 40GbE
18 x 40GbE
EN4093R
EN4093R
EN6131
EN6131
EN6131
EN6131
EN6131
EN6131
ITESWITCH
ITESWITCH
• 14 Compute Blades each using Dual EN6132 Dual Port 40GbE NIC
• 4 Switch Blades each using EN6131 SwitchX-2 32 Ports 40GbE
• Compute I/O 2.24 Tbps @40 Gbps• Uplink I/O up to 2.88 Tbps @40 Gbps
© 2012 IBM Corporation44 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Mellanox based technology is available for IBM Blade, Rack, & Flex Communities
Heritage of technical computing being broadly adopted:• Cloud, • Database & Analytics• Big Data• Storage
Core RDMA Technology Becoming Mainstream• Web 2.0, Cloud, Analytics, and Storage• Out of Box support for Linux, Windows, Cloud, & Hadoop Distros• Storage: Windows Server 2012 with SMB 3.0
IBM has Broad Mellanox based Product Offering Today• How can we help to make your business grow?
Summary … and Next Steps
© 2012 IBM Corporation45 This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
This document is for IBM and IBM Business Partner use only. It is not intended for client distribution or use with clients.
Trademarks
Trademarks
The following are trademarks of the International Business Machines Corporation in the United States and/or other countries. For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml: AS/400, DBE, e-business logo, ESCO, eServer, FICON, IBM, IBM Logo, iSeries, MVS, OS/390, pSeries, RS/6000, S/30, VM/ESA, VSE/ESA, WebSphere, xSeries, z/OS, zSeries, z/VM
The following are trademarks or registered trademarks of other companies
Lotus, Notes, and Domino are trademarks or registered trademarks of Lotus Development CorporationJava and all Java-related trademarks and logos are trademarks of Sun Microsystems, Inc., in the United States and other countriesLINUX is a registered trademark of Linux TorvaldsUNIX is a registered trademark of The Open Group in the United States and other countries.Microsoft, Windows and Windows NT are registered trademarks of Microsoft Corporation.SET and Secure Electronic Transaction are trademarks owned by SET Secure Electronic Transaction LLC.Intel is a registered trademark of Intel Corporation* All other products may be trademarks or registered trademarks of their respective companies.
NOTES:
Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.
IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.
All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.
This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.
All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.
References in this document to IBM products or services do not imply that IBM intends to make them available in every country.
Any proposed use of claims in this presentation outside of the United States must be reviewed by local IBM country counsel prior to such use.
The information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.