SPEC HPG Benchmarks for HPC Systems

Kumaran Kalyanasundaram

for

SPEC High-Performance Group

Kumaran Kalyanasundaram, PhDChair, SPEC HPGManager, SGI Performace Engineering

SPEC HPG’s Purpose

The High Performance Group focuses on the development of application benchmarks for high performance computers.

SPEC HPG

Founded in 1994 (Perfect Benchmarks initiative became HPG).

Members from industry and academia.

Two active benchmarks - SPEC OMP & SPEC HPC2002.

New MPI2006 benchmark currently under development.

OMPL2001

Founding of SPEC HPG

HPC96

OMP2001

HPC2002

MPI2006

Jan 1994 Oct 1995 June 2001 June 2002 Jan 2003 2006

SPEC HPG Benchmark Suites

SPEC OMP

Benchmark suite developed by SPEC HPG (High Performance Group)

Benchmark suite for performance testing of shared memory processor systems

Uses OpenMP versions of SPEC CPU2000 benchmarks and candidates

Why Did SPEC Choose OpenMP?

Benchmark suite is focused on SMP systems

OpenMP is a standard, and is applicable to Fortran, C, and C++.

Directive based OpenMP allows serial version to remain largely intact.

Quickest path to parallel code conversion.

OMP/CPU2000 Similarities

Same tools used to run the benchmarks

Similar run and reporting rulesUses geometric mean to calculate

overall performance relative to a baseline system

Similar output format

SPEC OMP Benchmark Principles

Source code basedLimited code and directive

modificationsFocused on SMP performanceRequires a base run

with no source modifications single set of compiler flags for all benchmarks

SPEC supplied tools required to run benchmark

OMPM2001 Benchmarks

Benchmark Lang What it does310.wupwise_m F90 SU(3) HE physics312.swim_m F90 Shallow water model from NCAR314.mgrid_m F90 Multigrid for EM problems316.applu_m F90 Fluid dynamics partial LU decomposition318.galgel_m F90 Fluid dynamics, Galerkin FE320.equake_m C Earthquake dynamics324.apsi_m F90 Lake weather model326.gafort_m F90 Genetic algorithm328.fma3d_m F90 Finite element mechanics330.art_m C Image recognition332.ammp_m C Molecular dynamics

OMP vs CPU2000

Characteristic CPU2000 OMPM2001 OMPL2001Max. working set 200 MB 1.6 GB 6.5 GBMemory needed 256 MB 2 GB 8 GBBenchmark runtime 30 min @ 300 MHz 5 hrs @ 300 MHz 9 hrs @ 300 MHzLanguage C, C++, F77, F90 C, F90, OpenMP C, F90, OpenMPFocus Single CPU < 16 CPU system > 16 CPU systemSystem type Cheap desktop MP workstation Engineering MP sysRuntime 24 hours 34 hours 75 hoursRuntime 1 CPU 24 hours 140 hours 1000 hoursRun modes Single and rate Parallel ParallelNumber benchmarks 26 11 9Iterations Median 3 or more Worst of 2, median of 3 or moreSource mods Not allowed Allowed AllowedBaseline flags Max of 4 Any, same for all Any, same for allReference system 1 CPU @ 300 MHz 4 CPU @ 350 MHz 16 CPU @ 300 MHz

Program Memory Footprints

OMPM2001(Mbytes)

OMPL2001(Mbytes)

wupwise 1480 5280swim 1580 6490mgrid 450 3490applu 1510 6450galgel 370equake 860 5660apsi 1650 5030gafort 1680 1700fma3d 1020 5210art 2760 10670ammp 160

SPEC HPC2002 Benchmark

Full Application benchmarks(including I/O) targeted at HPC platforms

Serial and parallel (OpenMP and/or MPI)

Currently three applications: SPECenv: weather forecast SPECseis: seismic processing, used

in the search for oil and gas SPECchem: comp. chemistry, used

in chemical and pharmaceutical industries (gamess)

All codes include several data sizes

An application benchmark suite that measures CPU, memory bw, interconnect, compiler, MPI performance.

Search program is open till March 31st, 06

Candidate codes in the areas of Comp. Chemistry, weather forecasting, HE Physics, Oceanography, CFD, etc.

SPEC MPI2006

Very large data sets for MPI2006.Follow onto SPEC OMPM(L)2001.Update SPEC HPC2002 suite.

Future Goals