what have we learned from the top500 lists? have we learned from the top500 lists? june 20, 2001...

What have we learned from the TOP500 lists? June 20, 2001

What have we learnedfrom the TOP500 lists?

Hans Werner MeuerUniversity of Mannheim

andPrometeus GmbH

Sun HPC Consortium MeetingHeidelberg, GermanyJune 19 - 20, 2001


� TOP500 Approach

� Snapshots in Time

� Development over Time

� Customer Types

� TOP500 Plans

Outlook


� Basis for analyzing the HPC market

� Quantification of observations

� Detection of trends

� Market� Architecture� Technology

TOP500 Motivation


� Listing of the 500 most powerfulcomputers in the world

� Yardstick: Rmax from LINPACK

Ax=b, dense problem

� Updated twice a year

TOP500 Procedure

Size

Ra

te

TPP performance


Fastest Computer Over Time

Cray Y-MP (8) Fujitsu

VP-2600

TMCCM-2(2048)

05

101520253035404550

1990 1992 1994 1996 1998 2000

GF

lop/

s

In 1980 a computation that took 1 full year tocomplete can now be done in 1 month!


Fastest Computer Over TimeHitachi

CP-PACS(2040)Intel

Paragon(6788)

FujitsuVPP-500

(140)TMC CM-5(1024)

NEC SX-3(4)

TMCCM-2(2048)

Fujitsu VP-2600

Cray Y-MP (8)

050

100150200250300350400450500

1990 1992 1994 1996 1998 2000

GF

lop/

s

In 1980 a computation that took 1 full year tocomplete can now be done in 4 days!


Fastest Computer Over Time

In 1980 a computation that took 1 full year tocomplete can now be done in 1 hour!

Intel ASCI Red Xeon

(9632)

Cray Y-MP (8)

Fujitsu VP-2600

TMCCM-2(2048)

NEC SX-3

(4)

TMC CM-5(1024)

FujitsuVPP-500

(140)

IntelParagon(6788)

HitachiCP-PACS

(2040)

Intel ASCI Red

(9152)SGI ASCI

Blue Mountain

(5040)

ASCI BluePacific SST

(5808)

ASCI White Pacific(8192)

0500

100015002000250030003500400045005000

1990 1992 1994 1996 1998 2000

GF

lop/

s


� 1st List in June 1993

� 16th List on November 3, 2000 in Dallas

� 17th List on June 21, 2001 in Heidelberg

� Accepted by users and manufacturers

TOP500 Status


- Authors since 06/1993 and 11/2000

Jack Dongarra Horst Simon

- Started in spring 1993 by:

Hans W. Meuer and Erich Strohmaier

TOP500 Authors


www.top500.org


� TOP500 Approach



� Customer Types

� TOP500 Plans

Outlook


� Manufacturer Manufacturer or vendor� Computer Type Indicated by manufacturer or vendor� Installation Site Customer� Location Location and country� Year Year of installation/last major update� Customer Segment Academic, Research, Industry, Vendor, Class.� # Processors Number of processors� Rmax Maximal LINPACK performance achieved� Rpeak Theoretical peak performance� Nmax Problemsize for achieving Rmax

� N1/2 Problemsize for achieving half of Rmax

� Nworld Position within the TOP500 ranking

TOP500 list - data shown


RANK MANU-FACTURER COMPUTER

RMAX

[TF/S]INSTALLATION SITE COUNTRY YEAR AREA OF

INSTALLATION#

PROC

1 IBMASCI WhiteSP Power3375 MHz

7.23 Lawrence LivermoreNational Laboratory USA 2000 Research

Energy 8192

2 IBM SP Power3375 MHz 16 way 2.53 NERSC/LBNL, Berkeley USA 2001 Research 2528

3 Intel ASCI Red 2.38 Sandia National Laboratory,Albuquerque USA 1999 Research 9632

4 IBMASCI Blue Pacific

SST, IBM SP604E

2.14 Lawrence LivermoreNational Laboratory USA 1999 Research

Energy 5808

5 Hitachi SR8000/MPP 1.71 University of Tokyo Japan 2001 Academic 1152

6 SGI ASCI BlueMountain 1.61 Los Alamos

National Laboratory USA 1998 Research 6144

7 IBM SP Power3375 MHz 1.42 Naval Oceanographic Office,

Bay St. Louis USA 2000 ResearchAerospace 1336

8 NEC SX-5/128 M33.2 ns 1.19 Osaka University Japan 2001 Academic 128

9 IBM SP Power3375 MHz 1.18 National Centers for

Environmental Predicition USA 2000 ResearchWeather 1104

10 IBM SP Power3375 MHz 1.18 National Centers for

Environmental Predicition USA 2001 ResearchWeather 1104

TOP10 06/2001


TOP10 - Sun 06/2001RANK MANU-

FACTURER COMPUTERRMAX

[TF/S]INSTALLATION SITE COUNTRY YEAR AREA OF

INSTALLATION#

PROC

57 Sun HPC 4500400 MHz Cluster 0.42 Defense, Stockholm Sweden 1999 Classified 896

58 Sun HPC 4500400 MHz Cluster 0.42 Service Provider USA 2000 Industry

WWW 896

59 Sun HPC 4500400 MHz Cluster 0.42 Service Provider USA 2000 Industry

WWW 896

60 Sun HPC 4500400 MHz Cluster 0.42 Sun, Burlington USA 2000 Vendor 896

61 Sun HPC 4500400 MHz Cluster 0.42 Sun, Sunnyvale USA 2000 Vendor 896

168 Sun HPC 10000450 MHz Cluster 0.14 Clearstream Services,

Grande Duchesse Luxemburg 2000 IndustryFinance 256

169 Sun HPC 10000450 MHz Cluster 0.14 Motorola, Scottsdale USA 2000 Industry

Electronics 256

170 Sun HPC 10000450 MHz Cluster 0.14 New York City

Human Resources USA 1999 Government 256

171 Sun HPC 10000450 MHz Cluster 0.14 Sun, Broomfield USA 2000 Industry

WWW 256

172 Sun HPC 10000450 MHz Cluster 0.14 US Army Research

Laboratory, Aberdeen USA 1999 Research 256


Industrial/Commercial 49 %

Classified 7 %

Research 22 %

Academic 18 %

Total 500

Systems installed 06/2001

Government 1 % Vendor 3 %


Industrial/Commercial 20 %

Classified 5 %

Research 38 %

Academic 31 %

Total 500

Systems installed 06/1995

Vendor 12 %


� TOP500 Approach



� Customer Types

� TOP500 Plans

Outlook


1.167 TF/s

88.0 TF/s4.94 TF/s

59.7 GF/s55.1 GF/s

0.4 GF/s

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

Intel XP/S140Sandia

Fujitsu'NWT' NAL

SNI VP200EXUni Dresden

Hitachi/TsukubaCP-PACS/2048

Intel ASCI RedSandia

IBMASCI White

LLNL

N=1

N=500

SUM

IBM SP PC604e130 processors

Alcatel

1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

Performance Development


ASCI

Earth Simulator

Jun 93

Jun 94

Jun 95

Jun 96

Jun 97

Jun 98

Jun 99

Jun 00

Jun 01

Jun 02

Jun 03

Jun 04

Jun 05

N=1

N=500

SUM

N=10

1 GFlop/s

1 TFlop/s

1 PFlop/s

100 MFlop/s

100 GFlop/s

100 TFlop/s

10 GFlop/s

10 TFlop/s

Performance Development


ASCI White

ASCI Red

Cray T3DTMC CM-5

TMC CM-2Cray 2

Cray X-MP

Cray 1

CDC 7600IBM 360/195

CDC 6600

IBM 7090

UNIVAC 1EDSAC 1

1950 1960 1970 1980 1990 2000

1 KFlop/s

1 MFlop/s

1 GFlop/s

1 TFlop/s

Moore�s Law and Peak Performance of Various Computers over Time


USA/Canada

Europe

Japanothers

0

100

200

300

400

500

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00

Continents


USA

JapanEurope

0

100

200

300

400

500

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00

Producers


Germany

UKFrance

ScandinaviaBenelux

Switzerlandothers

0

50

100

150

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

Europe - Countries


183111 101

90 89 7449 29

509

0

100

200

300

400

500

600US

A

Scan

dina

via

Ger

man

y

UK

Japa

n

Switz

erla

nd

Fran

ce

New

Zea

land

Luxe

mbo

urg

Kflops per Inhabitant

11/2000


0

100

200

300

400

500

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00

Sun

IBM

SGI

Cray

TMC

Convex/HPFujitsu

Intel

others HitachiNEC

Syst

em

s

Manufacturers


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

Sun

IBM SGI

Cray

TMC

Convex/HPFujitsu

Intel

others Hitachi

NEC

Manufacturers - Performance


��

��

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

Cray

Cray Computer

SGI

CDC/ETA

Fujitsu

NEC

Hitachi

Convex/HP

TMC

Intel

nCUBE

Alliant

FPS

Meiko

Parsytec

MasPar

DEC/Compaq

KSR

IBM

Sun


Scalar

VectorSIMD

0

100

200

300

400

500

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00

Processor Type


Alpha

PowerHP

intel

MIPS

Sparc

proprietary

0

100

200

300

400

500

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

other COTS

Chip Technology


Single Processor

SMP

MPP

SIMD Constellation Cluster - NOW

0

100

200

300

400

500

Jun-93

Nov-93

Jun-94

Nov-94

Jun-95

Nov-95

Jun-96

Nov-96

Jun-97

Nov-97

Jun-98

Nov-98

Jun-99

Nov-99

Jun-00

Nov-00

Architectures


� TOP500 Approach



� Customer Types

� TOP500 Plans

Outlook


0

100

200

300

400

500

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

Research

Academic

ClassifiedVendor

Industrial/Commercial

Customer Types


0%10%20%30%40%50%60%70%80%90%

100%

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

Research

Academic

Vendor


Classified

Customer Types - Performance


0

50

100

150

200

250

Jun 93

Nov 93

Jun 94

Nov 94

Jun 95

Nov 95

Jun 96

Nov 96

Jun 97

Nov 97

Jun 98

Nov 98

Jun 99

Nov 99

Jun 00

Nov 00

Unknown

Commercial

Engineering

Industrial/Commercial Applications


0%10%20%30%40%50%60%70%80%90%

100%

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00

Research

Academic

Other


Customer Types - USA


0%10%20%30%40%50%60%70%80%90%

100%

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00

Research

Academic

Other


Customer Types - Europe


0%10%20%30%40%50%60%70%80%90%

100%

Jun 93

Nov 93Ju

n 94Nov 94Ju

n 95Nov 95Ju

n 96Nov 96Ju

n 97Nov 97Ju

n 98Nov 98Ju

n 99Nov 99Ju

n 00Nov 00

Research

Academic

Other


Customer Types - Japan


Manufacturers

0%5%

10%15%20%25%30%35%40%45%

CompaqCray

Inc.

Fujitsu

Hitachi IBM

NEC SGI

SUNSelf

Made

Others

11/2000

Total 500


11/2000

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Com

paq

Cra

y In

c.

Fujit

su

Hita

chi

IBM

NEC SG

I

SUN

OtherAcademicResearchIndustrial/Commercial

Share of Customer Types 11 47 17 16 215 23 67 92 Σ 488


0

50

100

150

200

250

300

350

400

450

500

Japan

Inc 1

1/97

Japan

Inc 1

1/98

Japan

Inc 1

1/99

Japan

Inc 1

1/00

IBM 11/97

IBM 11/98

IBM 11/99

IBM 11/00

SUN 11/97

SUN 11/98

SUN 11/99

SUN 11/00

Total 11

/97Total

11/98

Total 11

/99Total

11/00

Industrial/Commercial Other Share of Customer Types

250

200

150

100

50

0

300

350

400

450

500


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Japan

Inc 1

1/97

Japan

Inc 1

1/98

Japan

Inc 1

1/99

Japan

Inc 1

1/00

IBM 11/97

IBM 11/98

IBM 11/99

IBM 11/00

SUN 11/97

SUN 11/98

SUN 11/99

SUN 11/00

Total 11

/97Total

11/98

Total 11

/99Total

11/00

Industrial/Commercial Other

50 %

40 %

30 %

20 %

10 %

0 %

60 %

70 %

80 %

90 %

100 % Share of Customer Types


0

50

100

150

200

250

300

350

400

450

500

SUN 11/97

SUN 11/98

SUN 11/99

SUN 11/00

SUN 06/01

Total 11

/97Total

11/98

Total 11

/99Total

11/00

Total 06

/01

Share of Customer Types - Sun

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

SUN 11/97

SUN 11/98

SUN 11/99

SUN 11/00

SUN 06/01

Total 11

/97Total

11/98

Total 11

/99Total

11/00

Total 06

/01

+ Industrial/Commercial+ Other


� TOP500 Approach



� Customer Types

� TOP500 Plans

Outlook


clusters.top500.org


� we are just finalizing the work with the 17th list� here are some speculations about the new list:

- we expect again IBM to dominate the list as far as thenumber of systems is concerned and as aperformance leader

- we believe that the Supercomputer System NEC SX5/128 M8 at Osaka University will make the TOP10 of the new list exceeding 1 Tflop/s Rmaxperformance

- we expect an entry level performance for the new listlower than 70 Gflop/s

17th TOP500 list to be published atSC2001 in Heidelberg


www.supercomp.de


� Snapshot in time

� LINPACK Rmax overestimates MPP systems

� Actual usage is unknown

TOP500 Limitations


00.10.20.30.40.50.60.70.80.9

1

Jun-93

Nov-93

Jun-94

Nov-94

Jun-95

Nov-95

Jun-96

Nov-96

Jun-97

Nov-97

Jun-98

Nov-98

Jun-99

Nov-99

Jun-00

Nov-00

VectorAllScalarSIMD

LINPACK �Efficiency�


0.01

0.1

1

10

Jun-93

Nov-93

Jun-94

Nov-94

Jun-95

Nov-95

Jun-96

Nov-96

Jun-97

Nov-97

Jun-98

Nov-98

Jun-99

Nov-99

Jun-00

Nov-00

Peak

Per

form

ance

[GFl

op/s

]

VectorScalar

Processor Performance


Problems with LINPACK

� Commercial DB systems� �don�t care about floating point�

� New architectures - �compute farms�� not designed to run LINPACK�

� �Self-made� cluster� no vendor to measure LINPACK� every system is different


TOP500 Plans

� Talks with experts since 3 years onhow to �resolve� these issues

� Q: �How can we provide betterinformation for different applicationand architecture domains?�

� A: �Make additional lists�


� Collaborate with research centers(NERSC, ICL, ...) which develop newbenchmarks

� Feedback about benchmarks� TOP500.org will provide

� free access to benchmark resultsand

� possibilities for �re-ranking�

TOP500 Plans


Thank you!

The PowerPoint slides of this presentationcan be downloaded at:

www.uni-mannheim.de/rum/dokus/talks/

what have we learned from the top500 lists? have we learned from the top500 lists? june 20, 2001...

Documents