mellanox connectivity solutions for scalable hpc highest performing, most efficient end-to-end...

Mellanox Connectivity Solutions for Scalable HPC

Highest Performing, Most EfficientEnd-to-End Connectivity for Servers and Storage

April 2010

© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL - 2

Connectivity Solutions for Efficient Computing

Leading Connectivity Solution Provider For Servers and Storage

Enterprise HPC

High-end HPC

Leading Connectivity Solution Provider For Servers and Storage

HPC Clouds

Mellanox Technologies is a leading supplier of high-performance connectivity solutions for servers & storage

• End-to-End Ifniniband solutions• 10 GbE and Low Latency Ethernet NIC solutions

© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL -

Cluster Software Management

HPC Application Accelerations

Networking Efficiency/Scalability

Servers and Storage High Speed Connectivity

Complete End-to-End Highest Performance Solution

3

ICs Switches/GatewayAdapters Cables

CORE-Direct GPU-Direct

QoSCongestion Control

Adaptive Routing


Mellanox’s InfiniBand Leadership

HighestPerformance

Efficient, Scalable and

Flexible Networking

Converged Network

Efficient use ofCPUs and GPUs

CORE-Direct

GPU-Direct

Highest Throughput

Lowest Latency

CPU Availability

Virtual Protocol

Auto SensingHost

Hardware Bridging

Message Rate

Transport Offload

Adaptive Routing

CongestionControl

Multiple Topologies

Advanced QoS


ConnectX-2 Virtual Protocol Interconnect

StorageNFS, CIFS, iSCSI

NFS-RDMA, SRP, iSER,Fibre Channel, Clustered

NetworkingTCP/IP/UDP

Sockets

ClusteringMPI, DAPL, RDS, Sockets

ManagementSNMP, SMI-S

OpenView, Tivoli, BMC, Computer Associates

LLE

Consolidated Application Programming Interface

App1 App2 App3 App4 AppX…

Acceleration Engines

10GigE

Any Protocol over Any Convergence Fabric

Protocols

Applications

Networking VirtualizationClustering Storage RDMA

10/20/40 InfiniBand


Efficient Data Center Solutions

40Gb/s Network• InfiniBand • Eth over IB• FC over IB• FC over Eth*

Switches Bridge

Storage

ServersAdapters

Ethernet Storage

FC Storage

IB Storage

40G InfiniBand, FCoIB10G Ethernet, FCoE

IB to EthIB to FCEth to FC

* via ecosystem products

40G InfiniBand 10G Ethernet 8G Fibre Channel


Highest Performance

Highest throughput • 40Gb/s node to node and 120Gb/s switch to switch• Up to 50M MPI messages per second

Lowest latency• 1usec MPI end-to-end• 0.9us InfiniBand latency for RDMA operations• 100ns switch latency at 100% load• True zero scalable latency – flat latency up to 256 cores per node

7


Efficient use of CPUs and GPUs

GPU-direct• Works with existing NVIDIA Tesla and Fermi products• Enables fastest GPU-to-GPU communications• Eliminates CPU copy and write process in system memory• Reduces 30% of the GPU-to-GPU communication time

8

CPU

GPUChipset

GPUGPUMemorMemor

yy

MellanoxInfiniBand

System Memory1

2

CPU

GPUChipset

GPUGPUMemorMemor

yy

MellanoxInfiniBand

System

Memory

1


Efficient use of CPUs and GPUs

CORE-direct (Collectives Offload Resource Engine)• Collectives communication are communications used for system

synchronizations, data broadcast or data gathering• CORE-direct performs the collective on the HCA instead of the CPU• Eliminates system noise and jitter issue • Increases the CPU cycles available for applications

Transport Offload• Full transport offload maximizes CPU availability for user applications• Only solutions to achieve 40Gb/s throughput with ~5% CPU overhead

9


Efficient, Scalable and Flexible Networking

Congestion control• Eliminates network congestions (hot-spots) related to many senders and a single

receiver

10



Adaptive routing• Eliminated networks congestions related to point to point communications sharing

the same network path

11

Hot Spots Configuration

Average Bandwidth

Minimum Bandwidth

Average Bandwidth

Minimum Bandwidth

None 100.0% NA NA

2:41,3:4,4:1 79.4% 37.3% 99.7% 99.6%

2:35,3:5,4:1 80.3% 37.1% 99.7% 99.0%

2:30,3:10,4:1 75.9% 36.5% 99.7% 99.5%

2:41,3:5,4:1 82.4% 37.7% 99.7% 99.4%

No Adaptive Routing With Adaptive Routing

220 server node system, Mellanox InfiniBand HCAs and switches



Multiple Topologies• Fat-tree: non-blocking, oversubscription• Mesh, 3D-Torus• Hybrid solutions

Advanced Quality of Service• Fine grained QoS• Consolidation without performance degradation

12


Mellanox Performance/Scalability Advantage

Weather simulations at scale with Mellanox end-to-end

13

© 2010 MELLANOX TECHNOLOGIES - CONFIDENTIAL - 14

Top100 Interconnect Share Over Time

InfiniBand the natural choice for large scale computing• All based on Mellanox InfiniBand technology


The 20 Most Efficient Top500 Systems

15

InfiniBand solutions enables the most efficient system in the Top500 The only standard interconnect solution in the top 100 highest utilization systems

#55 US Army Research Laboratory

#22 The Earth Simulator Center

#93 National Institute for Materials Science #10 Jülich

Thank You

[email protected]

mellanox connectivity solutions for scalable hpc highest performing, most efficient end-to-end...

Documents

storagemellanox technologies

infiniband latency

end connectivity

scalable latency flat

efficient use of cpus

gbs node

gbs switch

highest performance