maximizing the compute power with mellanox infiniband connectivity gilad shainer wolfram technology...
TRANSCRIPT
Maximizing The Compute Power
With Mellanox InfiniBand Connectivity
Gilad Shainer
Wolfram Technology Conference2006
2 Mellanox Technologies - Maximizing The Compute Power
• A global leader in semiconductor
solutions for server and storage
connectivity
• Leading provider of low-latency and
high-bandwidth InfiniBand solutions
• Converges clustering, communications, management and storage onto a single link with Quality of Service
Mellanox Technologies
SERVERS STORAGE
INFINIBAND ADAPTERS
3 Mellanox Technologies - Maximizing The Compute Power
MellanoxInfiniBand
Off-the-ShelfStorage
CommodityServers
High Performance Computing
Proprietary Systems• $1M+• Expensive to scale• Not flexible
Clusters• One-tenth the cost• Affordable to scale • Very flexible
IndustryMegatrend
InfiniBand Clusters• Up to 40X more performance• 1/10 latency• Scalable to 10,000s of connections• Flexible and easy to manage
4 Mellanox Technologies - Maximizing The Compute Power
Increased Demand for Compute Power
• Automotive $500K per vehicle crash
test (GM) Design cycles reduced
from 4 years to 18 months
• Digital Content Creation $90M to produce “The
Incredibles” $630M gross income
• Oil and Gas Exploration Up to $4B for offshore
wells
• Weather Forecasting $1M per mile to evacuate
the coast
5 Mellanox Technologies - Maximizing The Compute Power
PAM-CRASH
Mellanox
Mellanox
Mellanox
GigE
GigE
GigE
0
5000
10000
15000
20000
25000
30000
35000
40000
16 CPUs 32 CPUs 64 CPUs
Ela
ps
ed
Tim
e (
se
c)
Mellanox GigE
Mellanox InfiniBand Performance
Bavarian Car-To-Car Model1.1 M elements, 145000 cycles
• InfiniBand price/performance advantage increases with cluster size• Gigabit Ethernet becomes ineffective with cluster size
LowerIs
better
6 Mellanox Technologies - Maximizing The Compute Power
Building High Performances Solution
• Best price/performance servers PCI Express servers Low power multi-core CPUs
• Best price/performance storage High performance file system
• Mellanox InfiniBand Highest bandwidth, low latency, low CPU overhead Parallel direct access from compute nodes to storage
HP c-Class Blade System
20Gb/s InfiniBand
20Gb/s InfiniBand
7 Mellanox Technologies - Maximizing The Compute Power
Top500 – Industry-wide Clustering Trends
• List of the 500 most powerful computers Published twice a year
• InfiniBand deployments increase 33% from Nov05 to Jun06 The only growing high speed interconnect solution
• Three Top10 high ranked industry-standard clusters use InfiniBand #4 NASA, 10K Itanium-2 CPUs, 51.8TFlops (2004) #6 Sandia National Laboratories, 4500 nodes, 9K CPUs,
38.2TFlops (2005) #7 Tokyo Institute of Technology, 1300 nodes, 10K
CPUs, 38.1TFlops (2006)
8 Mellanox Technologies - Maximizing The Compute Power
Mellanox Increases The Compute Power
• Reduces the high CPU processing overhead Full transport offload, Remote Direct Memory Access (RDMA)
• Allows performance-hungry applications to fully utilize CPU resources Overlapping I/O communication with CPU computation cycles
9 Mellanox Technologies - Maximizing The Compute Power
Mellanox Increases The Compute Power
• Dramatically increases overall compute cluster efficiency Eliminating memory bandwidth bottleneck (Zero-copy)
• Ensures I/O resource for multi-core systems 20Gb/s node to node (40Gb/s, 2007)
…
Memory Chipset CPU
Network AdapterReceiving Host
Memory Chipset CPU
Network AdapterReceiving Host
Memory Chipset CPU
Network Adapter
Memory Chipset CPU
Network AdapterReceiving Host
Sending Host
Memory Chipset CPU
Network Adapter
Memory Chipset CPU
Network AdapterReceiving Host
Sending Host
10 Mellanox Technologies - Maximizing The Compute Power
A Unified InfiniBand Fabric
• World-class performance• Simplified Management• Ultimate scalability• Optimal total cost of
ownershipMultiple Fabrics
High CapEx and High TCOSingle InfiniBand Fabric
Low CapEx and Optimal TCO
“One Wire”
11 Mellanox Technologies - Maximizing The Compute Power
Mellanox Cluster Center
http://www.mellanox.com/applications/clustercenter.php
• Neptune cluster 32 Node, Dual core AMD Opteron CPUs
• Helios cluster 32 node Dual core Intel Woodcrest CPUs
• Utilizing “Fat Tree” network architecture (CBB) Non-blocking switch topology Non-blocking bandwidth
• InfiniBand 20Gb/s
12 Mellanox Technologies - Maximizing The Compute Power
Air Pollution Simulation – The Problem
• Technological innovations increases our need for energy and materials but also increase air pollutions
• Worldwide air pollution is responsible for a large number of deaths and cases of respiratory disease The World Health Organization estimates 4.6 million people
die each year from causes directly attributable to air pollution
• Minimizing and managing the production of pollutants is critical to our environment
• Complex modeling and simulation requires a compute-intensive solution
13 Mellanox Technologies - Maximizing The Compute Power
Air Pollution Simulation – The Solution
• Personal supercomputing for scientific research, modeling and simulations
• High speed, low latency and low CPU overhead InfiniBand interconnect
• High performance low power multi-core CPUs
• Powerful Wolfram gridMathematica supercomputing environment for developing solutions
• Ease-of-use of Windows Compute Cluster Server operating system
14 Mellanox Technologies - Maximizing The Compute Power
Air Pollution Simulation - Performance
• Maximum utilization, efficiency and scalability
0
100
200
300
400
500
8 Cores 16 Cores 32 Cores
Time to Compute 10 Time StepsSeconds
15 Mellanox Technologies - Maximizing The Compute Power
Summary
• Wolfram gridMathematica with the performance of Mellanox InfiniBand
• Powerful environment for rapidly developing solutions for computationally challenging problems
Q&A
Gilad Shainer, [email protected]
“A leading supplier of semiconductor-based,high performance interconnect products”