10ge challenges, opportunities, vision
DESCRIPTION
Multiple communication technologies continue to evolve and will converge to create a Unified Fabric infrastructure based upon Ethernet.TRANSCRIPT
10GEChallenges, Opportunities, Vision
March, 2008
Robert Winter
Dell 2008 Interop 1
Interop 2008, Las Vegas
10GE
ChallengesProtocols Offloads and UtilizationProtocols, Offloads and UtilizationCost and Growth
OpportunitiesFabric ConvergenceUniversal Bit Conveyance
2Dell 2008 Interop
10GE Challenges: Protocols (Too Many)
iFC
IP iSCSI iWARP iSER FCoE FC IB PCIeFCIP
iFC
7
FTP iSCSI
SCSI
FC-4 IBApplication
FTP iSCSI
SCSI
Application PCIePnP User
6
5 SOCKET
RDMAP
DDP / MPA FC-3
FC-4
IBSession
IBPresentationPresentation
SessionPCIeTLP
PCIeSW Driver
4
3 IP
TCP/UDP/ICMP FC-3/2
IB
IBTransport
SCTPTransport
PCIeDLLP
IP
TCP
3
2 Ethernet – DL/MAC
IP
FC-2/1 IBDL/MAC
IBNetworkNetwork
Link Layer
PCIeDDLP
IP
1 Ethernet - Physical FC-0/1 IBPhysicalPhysical
PCIeLLP
Dell 2008 Interop
10GE Challenges: Offload (What Kind?)
ApplicationsOperating System
ApplicationsOperating SystemApplication Protocols
ApplicationsOperating System
ApplicationsOperating System
Application Protocols Application Protocols Application Protocolspp
API
TCPFLOWCONN
pp
API
TCPFLOWCONN
pp
API
pp
API
HardwareHardware Hardware Hardware
UDP
IP CSO
UDP
IP
DMA
LSO
TCP
UDP
IP
FLOWCONN
DMA
LSO
TCP
UDP
IP
DMA
LSO
API FLOWCONN
NO OFFLOAD STATELESS
MAC
PHY
STATEFUL (FULL TOE) STATEFUL (iWARP)
MAC
PHYCSO
SO
RSS MAC
PHY
IP
CSO
LSO
RSS MAC
PHY
IP
CSO
LSO
RSS
NO OFFLOADSOFTWARE PROTOCOLS
LAYER 2 HARDWARE
STATELESSSOFTWARE PROTOCOLS
LSO, RSS, CSO, etc.LAYER 2 HARDWARE
STATEFUL (FULL TOE)TCP FLOW and CONNLSO, RSS, CSO, etc.
LAYER 2 HARDWARE
STATEFUL (iWARP)RDMA Verbs API
TCP FLOW and CONNLSO, RSS, CSO, etc.
LAYER 2 HARDWARE
Dell 2008 Interop
10GE Challenges: Cost/Growth (Higher/Slower)TH 100Mb
10Gbps
1Gbps
GR
OW
T 100Mbps
40/100GbpsE p
REL
ATIV
E
T
10Gbps
40/100Gbpsx8
CO
S
100Mbps 1Gbps
x2
x4
TIME
Dell 2008 Interop
High Performance Cluster Networks
Complexity
Management
Infrastructure
Higher Cost
Segmented Networking
Metro/Wide Area Networks
Higher Cost
Multiple Adapters
Multiple Administration Silos
HPC(IB) Storage Area Networks(IB)
MAN/WAN(PoS)
Storage Area NetworksGWGW
SAN(FC)
LAN
(PoS)
CAN
GWGWGW
GW (EN)CAN (PCIe)
Local Area Networks
GW
Chassis Area Networks
Dell 2008 Interop
High Performance Cluster Networks
Unified FabricSimplicity
Management
Infrastructure
Lo er Cost
Metro/Wide Area Networks
Lower Cost
Single Type of Adapter
Common Administration
HPC(IB) Storage Area Networks(IB)
MAN/WAN(PoS)
Storage Area NetworksGWGW
SAN(FC)UNIFIED FABRIC
LAN
(PoS)
CAN
GWGWGW
GW
UNIFIED FABRIC
(EN)CAN (PCIe)
Local Area Networks
GW
Chassis Area Networks
Dell 2008 Interop
Unified Fabric Enablement: Speed, Latency, Cost
10 GE is “good enough” for Unified Fabric10 GE now; 40GE in under 2 years, 100GE soon after thatBoth FC and IB will have a presence for a considerable timeS d d l t ti t iSpeed and latency continue to improveCost will come down
50010 5
10 6
500
3504 4
UNIFIED FABRICSWEET SPOT
20
8
10 5
4 200SWEET SPOT 8
SPEED LATENCY COST
(Gbps) (µsec) ($)
POWER
(W)
(End-to-End) (NIC/HCA/HBA) (FIBER MAC/PHY)ETHERNET
INFINIBAND (Gbps) (µsec) ($) (W)FIBRE CHANNEL
Dell 2008 Interop
Unified Fabric Enablement: Trust
Non-technical factors will drive Unified FabricEthernet is widely known and trustedIP-based applications are well establishedV t f di i f d Eth tVenture funding is focused on Ethernet
HI HI HIHI
MED MEDLO
LO LO
TRUST UBIQUITY INVESTMENTETHERNET
INFINIBAND Well-known managementEthernet everywhere More Ethernet R&DWell-known protocols
SFIBRE CHANNEL
“Staying power”
Dell 2008 Interop
Universal Bit Conveyance
LAN
HPCC
SANiWARP, DCE,
10GE
CAN
10GE
iSCSI, DCE,10GE, FCoE
40GE, 100GE,
Dell 2008 Interop
, ,(Motherboard Ethernet?)
Summary
Multiple communication technologies continue to evolve and will converge to create a Unified Fabric ginfrastructure based upon Ethernet– Confusion in the near-term will dampen adoption, however
Offload virtualization new protocols and “better” fabrics– Offload, virtualization, new protocols and better fabrics
It may be possible for Ethernet to become a universal bit conveyance for both on-system as well as off-system usewell as off system use– “Motherboard” Ethernet?
Dell 2008 Interop