paving the road to exascale computing - hpc advisory council

Paving the Road to Exascale

Computing

Yossi Avni

[email protected]

© 2011 MELLANOX TECHNOLOGIES - MELLANOX CONFIDENTIAL - 2

Connectivity Solutions for Efficient Computing

Enterprise HPC

High-end HPC

Leading Connectivity Solution Provider For Servers and Storage

HPC Clouds

Host/Fabric

Software

Mellanox Interconnect Networking Solutions

ICs Switches/Gateways Adapter Cards Cables


Complete End-to-End Connectivity

- Collectives Accelerations (FCA/CORE-Direct)

- GPU Accelerations (GPUDirect)

- MPI/SHMEM

- RDMA

- Quality of Service

- Adaptive Routing

- Congestion Management

- Traffic aware Routing (TARA)

- UFM, FabricIT

- Integration with job schedulers

- Inbox Drivers

Server and Storage High-Speed Connectivity

Networking Efficiency/Scalability

Application Accelerations

Host/Fabric Software Management

- Latency

- Bandwidth

- CPU Utilization

- Message rate


Mellanox’s Interconnect Leadership

Highest

Performance

Complete

Eco-System

End-to-End

Quality

Advanced

HPC

MPI/SHMEM Offloads

GPU Acceleration

Adaptive Routing

Highest Throughput

Lowest Latency

CPU Availability

Auto Negotiation

Cable Reach

Signal Integrity

Power Management

Congestion Control

Message Rate

Topologies/Routing

From Silicon to

System

RDMA


2005 - 2006 - 2007 - 2008 - 2009 - 2010 - 2011

Ban

dw

idth

p

er

dir

ecti

on

(G

b/s

)

40G-IB-DDR

60G-IB-DDR

120G-IB-QDR

80G-IB-QDR

200G-IB-EDR

112G-IB-FDR

300G-IB-EDR

168G-IB-FDR

8x HDR

12x HDR

# of Lanes per direction

Per Lane & Rounded Per Link Bandwidth (Gb/s)

5G-IB DDR

10G-IB QDR

14G-IB-FDR (14.025)

26G-IB-EDR (25.78125)

12 60+60 120+120 168+168 300+300

8 40+40 80+80 112+112 200+200

4 20+20 40+40 56+56 100+100

1 5+5 10+10 14+14 25+25

x12

x8

x1

2014

12x NDR

8x NDR

40G-IB-QDR

100G-IB-EDR

56G-IB-FDR

20G-IB-DDR

4x HDR

x4

4x NDR

10G-IB-QDR

25G-IB-EDR

14G-IB-FDR

1x HDR

1x NDR

Market

Demand

InfiniBand Link Speed Roadmap


Next Generation InfiniBand Technology

Highest Throughput Connectivity for Server and Storage

Available: 2011 (end-to-end: adapters, switches, cables)


Offloading at the HCA

(CORE-Direct)

Offloading at the

network/switches

(iCPU)

Node

Scalable MPI Collectives Acceleration with FCA

Offloading/acceleration

management (FCA)

~20% Performance increase at 16 nodes!

Most Scalable Offloading for MPI Applications


Mellanox Message Rate Performance Results

PPN – process per node, or cores per node

Highest MPI Message Rate!

90 Million messages per Second

Highest IB Message Rate!

23 Million messages per Second


Network Utilization via Traffic Aware Routing

Maximizing Network Utilization

Job Submitted in Scheduler Matching Jobs Automatically

Application Level Monitoring

& Optimization Measurements

Fabric-wide Policy Pushed to Match

Application Requirements


Hardware Based Congestion Control

Network Latency % Improvement

Ping Pong Latency 88%

Natural Ring Latency 81.6%

Random Ring Latency 81.3%

Ping Pong bandwidth 85.5%

Applications (HPCC) % Improvement

PTRANS 76%

FFT 40%

For more performance examples:

“First Experiences with Congestion

Control in InfiniBand Hardware”;

Ernst Gunnar Gran, Magne Eimot,

Sven-Arne Reinemo, Tor Skeie,

Olav Lysne, Lars Paul Huse, Gilad

Shainer; IPDPS 2010

Congestion Free Network For Highest Efficiency


Highest Performance GPU Clusters with GPUDirect

GPUDirect

GPU computing mandates Mellanox solutions

GPUDirect: 35% application performance increase

3 nodes 3 nodes

Mellanox InfiniBand Accelerates GPU Communications

http://www.sandia.gov/


Superior InfiniBand Solutions

Performance: 45% Lower latency, highest throughput and 3x the message rate

Scalability: proven for Petascale computing, highest scalability through accelerations

Reliability :from silicon to system, highest signal integrity, two order of magnitude lower BER

Efficiency: Highest CPU/GPU availability through complete offloading, low power consumption

Certification: Complete ISVs support and qualification, MPI vendors, job schedulers

Return on investment: Most cost/effective, simple to manage, 40Gb/s end-to-end connectivity

Financial Mellanox Connectivity Solutions

Financial

Cloud &

Web 2.0 Clustered

Database

Weather

Digital Media

Computational Aided Engineering

Bioscience

Labs, Research

University, Academic

Oil and Gas


Complete High-Performance Scalable Interconnect

Solutions for Server and Storage

Bottom Line Mellanox Benefits for HPC

TCO

50+% Reduction

Energy Costs

65+% Reduction

Infrastructure

60+% Saving

Performance

100+% Increase

Entry-level HPC High-end HPC Enterprise HPC HPC Clouds


9 of Top 10 Automotive

Manufacturers

4 of Top 10 Pharmaceutical

Companies

7 of Top 10 Oil and Gas Companies

6 of Top 10 Global Banks

30%+ of Fortune-100 & top global-High Performance Computers

Performance Leadership Across Industries


Thank You

Thinking, Designing and Building Scalable HPC

[email protected]

paving the road to exascale computing - hpc advisory council

Documents