radioss structural analysis solver performance · pdf file · 2014-12-19innovation...

Innovation Intelligence®

RADIOSS Structural Analysis Solver

Performance Optimizations

Eric Lequiniou

June 16, 2013

Copyright © 2013 Altair Engineering, Inc. Proprietary and Confidential. All rights reserved.

Contents

• About Altair

• About RADIOSS

• HPC Advisory Council Benchmark Testing and Results

• Introducing HyperWorks Unlimited

• Q&A / For More Information

“Our vision is to radically change

the way organizations design

products and make decisions.”

– James R. Scapa, Chairman & CEO, Altair


Overview

Founded ...

In 1985 as a product design consulting company

Today ...

A global software, services & technology leader

with over 40 offices in 19 countries and

3,200+ customers worldwide

‘85 ‘12

$100M

Est. $240M


Innovation Intelligence®

27+

Years of Innovation

40+

Offices in 19 Countries

1800+ Employees Worldwide


Customers

Automotive Aerospace Heavy Equipment Government Life/Earth Sciences Consumer Goods Energy

3,200+ customers worldwide


Altair Knows HPC

Altair is the only company that:

makes HPC tools…

develops HPC applications…

…and uses these to solve real HPC

problems

500 Altair engineers worldwide

use HPC every day for

real-world modeling

& simulation


HPC Advisory Council RADIOSS Performance Study

Research performed under the HPC Advisory Council activities

• Participating vendors: Intel, Dell, Mellanox

• Compute resource: HPC Advisory Council Cluster Center

Objectives

• Give an overview of RADIOSS Performance

• Compare different MPI libraries

• Compare different interconnects

• Understand RADIOSS communication patterns

• Provide Best Practices for running RADIOSS and increase productivity


About RADIOSS

Compute-intensive simulation software for Manufacturing

• For 20+ years an established standard for automotive crash and impact

• Differentiated by its high scalability, quality and robustness

• Supports multiphysics simulation and advanced materials

• Used across all industries to improve safety and manufacturability

Companies use RADIOSS to simulate real-world scenarios (crash

tests, climate effects, etc.) to test the performance of a product


RADIOSS Hybrid MPP Parallelization

Highly parallel code

• Multi-level parallelization

• Domain decomposition MPI parallelization

• Multithreading OpenMP

Enhanced performance

• Best scalability in the marketplace

• High efficiency on large HPC clusters

• Unique, proven method for rich scalability over thousands of cores for FEA

• Flexibility -- easy tuning of MPI & OpenMP

• Robustness -- parallel arithmetic allows perfect repeatability in parallel


HPC Advisory Council Test Configuration

Dell™ PowerEdge™ R720xd 32-node (512-core) “Jupiter” cluster

• Dual-Socket Eight-Core Intel E5-2680 @ 2.70 GHz CPUs

• Memory: 64GB memory, DDR3 1600 MHz

• OS: RHEL 6.2, OFED 1.5.3 InfiniBand SW stack

• Hard Drives: 24x 250GB 7.2 RPM SATA 2.5” on RAID 0

Mellanox Infiniband

• Mellanox ConnectX-3 FDR InfiniBand VPI adapters

• Mellanox SwitchX SX6036 InfiniBand VPI switch

Intel Cluster Ready certified cluster

MPI Libraries

• Intel MPI 4.1.0

• Platform MPI 8.2

Application: Altair RADIOSS 12.0 Single Precision

Benchmark dataset: Neon Refined 1 million elements 8ms


RADIOSS Performance – Interconnect (MPP)

FDR InfiniBand provides better scalability performance than Ethernet

• 743% better performance than 1GbE at 8 nodes


• 1GbE does not scale beyond 4 nodes with pure MPI

Higher is better

Platform MPI

16 Processes/Node

743%

214%


RADIOSS Performance – Hybrid MPP Version

Enabling Hybrid MPP unlocks RADIOSS

scalability

• At larger scale, productivity improves as more

threads involves

• As more threads involved, amount of

communications by processes are reduced

• At 32 nodes (or 512 cores), the best

configuration is 2 PPN with 8 threads each

The following environment setting and

tuned flags are used:

I_MPI_PIN_DOMAIN=auto

I_MPI_ADJUST_BCAST=1

I_MPI_ADJUST_REDUCE=2

KMP_AFFINITY=compact

KMP_STACKSIZE=400m

+ user environment: “ulimit -s unlimited”

Higher is better

Intel MPI

FDR InfiniBand


RADIOSS Performance – Interconnect (HMPP)

FDR InfiniBand provides better scalability performance than Ethernet • 290% better performance than 1GbE at 32 nodes


Higher is better Intel MPI

8 OpenMP Threads/MPI process

93% 290%


RADIOSS Performance – Processors (HMPP)

Intel E5-2680 (Sandy Bridge) cluster outperforms prior generations • Performs up to 50% better than X5670 cluster at 16 nodes

• Jupiter: 2-socket Intel E5-2680 @ 2.7GHz, 1600MHz DIMMs, FDR IB, 24 HDDs

• Janus: 2-socket Intel X5670 @ 2.93GHz, 1333MHz DIMMs, QDR IB, 1 HDD

Higher is better 8 OpenMP Threads/MPI process

Intel MPI

47%

50%

38%

42% 38%


RADIOSS Performance – OFA vs DAPL (HMPP)

“OFA provider” under Intel MPI delivers better scalability performance • Up to 15% better application performance than DAPL provider at 32 nodes

Higher is better 8 OpenMP Threads/MPI process

15%

Intel MPI


RADIOSS Profiling – % Time Spent on MPI

RADIOSS utilizes non-blocking

communications in most data transfers

• The most time MPI calls is MPI_Waitany() and

MPI_Wait()

MPI_Waitany(43%), MPI_Wait(22%), MPI_Bcast(18%),

MPI_Recv(12%)

• Time spent on MPI_Wait and Waitany are for

MPI_Isend/Irecv

Wait time are accounted for time spent on pending

non-blocking transfers

16 Processes/Node

Pure MPP


RADIOSS Profiling – MPI Message Sizes

RADIOSS uses small and medium MPI message sizes • Most message sizes are between 0B to 64B, and 257B to 1KB

Pure MPP

16 Processes/Node


Summary of the HPC Advisory Council Study

RADIOSS is designed to perform at large scale HPC environment

• Shows excellent scalability over 512 cores (32 nodes) and beyond with Hybrid MPP

• Hybrid MPP version enhanced RADIOSS scalability (1 MPI per socket, 8 threads each)

Intel Xeon E5-2600 series and FDR InfiniBand enable RADIOSS to scale

• The E5-2680 cluster outperforms X5670 cluster by up to 50%

Network and MPI comparisons

• Using fast Infiniband interconnect like Mellanox is the key to get good scalability

• Tuning environment parameters is important to maximize performance

• With Intel MPI it is easy to setup RADIOSS Hybrid MPP for optimal performance

A Big Thanks to the HPC Advisory Council for their competencies and great work to study

RADIOSS Scalability!

For more information, please visit

http://www.hpcadvisorycouncil.com/pdf/RADIOSS_Analysis_and_Profiling.ppt.pdf

http://www.hpcadvisorycouncil.com/pdf/RADIOSS_Analysis_and_Profiling.ppt.pdf


HPC

Hardware

HPC

Workload

Management

Introducing HyperWorks Unlimited

…for Infinite Exploration!

Lease Roll-in Power-Up Connect Simulate Innovate

Unlimited

HyperWorks

Software


Thanks for your time!

• Eric Lequiniou

• Director, High Performance Computing

• [email protected]

• Ravi Kunju

• Managing Director of Strategy and Marketing

• [email protected]

mailto:[email protected]

mailto:[email protected]

radioss structural analysis solver performance · pdf file · 2014-12-19innovation...

Documents