![Page 1: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/1.jpg)
International conference“Modern problems of computational mathematics and mathematical modeling”,
dedicated to the 90th anniversary of academician G.I.Marchuk
AlgoWiki: an Open Encyclopedia of Parallel Algorithmic Features
Prof. Vladimir VoevodinDeputy Director RCC MSU
Head of Department on Supercomputers and Quantum Informatics CMC [email protected]
June 10th , 2015, INM RAS, MoscowJune 10th , 2015, INM RAS, Moscow
![Page 2: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/2.jpg)
1956 1959 1967
20092008
2008
2003
2000
103
1015
106
1010
Strela Setun BESM-6
BlueGene/P
“Chebyshev”
“Lomonosov”
Computing Facilities of Moscow State University(from 1956 up to now)
![Page 3: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/3.jpg)
Research on Algorithm and Application Structures(what are our algorithms?)
How often did we change programming paradigm?(How many times did we have to rewrite our applications completely?)
70s - Loop Vectorization (innermost)80s - Loop Parallelization (outer) + Vectorization (innermost)90s - MPImid 90s - OpenMPmid 2000s - MPI+OpenMP2010s - CUDA, OpenCL, MPI+OpenMP+accelerators…
Do you see the end of this rewriting process?
![Page 4: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/4.jpg)
Tablets, Smartphones…
PCs, Laptops…
Servers…
Supercomputers
![Page 5: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/5.jpg)
Tablets, Smartphones…
PCs, Laptops…
Servers…
Supercomputers
102
103
104
109
2025
1-4
4-8
12-64
106
2015
1
1
2-4
104
2005
Degree of parallelism
![Page 6: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/6.jpg)
15 years ago …(24 CPUs, Intel P-III/500 MHz, SCI network, 8 m2 , 12 Gflops)
15 years ago …(24 CPUs, Intel P-III/500 MHz, SCI network, 8 m2 , 12 Gflops)
![Page 7: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/7.jpg)
HPC Stages at MSUHPC Stages at MSU
![Page 8: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/8.jpg)
M.V.Lomonosov1711 – 1765
… and now(52000+ Intel cores, 2065 NVIDIA GPUs, QDR IB, 1000 m2 , 1.7 Pflops)
… and now(52000+ Intel cores, 2065 NVIDIA GPUs, QDR IB, 1000 m2 , 1.7 Pflops)
![Page 9: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/9.jpg)
Top500 the most powerful computers…(www.top500.org, November 2014)
Top500 the most powerful computers…(www.top500.org, November 2014)
![Page 10: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/10.jpg)
MSU Supercomputer “Lomonosov-2”
1 rack = 256 nodes: Intel (14c) + NVIDIA = 515 Tflop/s“Lomonosov-2” Supercomputer (5 racks) = 2.5 Pflop/s
MSU Supercomputer Center:
Users: 2511Projects: 1607
Organizations: 302
MSU Faculties/ Institutes : 20+
Computational science is everywhere…
![Page 11: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/11.jpg)
What do we need to know to control efficiency of supercomputer centers ?
LicensesProjects
Users
Software components
Quotas
Organizations
Hardware components
Statuses
ApplicationsPartitions
Queues Jobs
UsersSysAdminsManagement
Is it difficult to control few components ? A few ?..
![Page 12: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/12.jpg)
A few? Info on MSU “Lomonosov” Supercomputer :(1.7 Pflops, 6000 computing nodes, 12K CPUs, 2K GPUs…)
LicensesProjects
Users
Software components
Quotas
Organizations
Hardware components
Statuses
ApplicationsPartitions
Queues Jobs
400
100
100 300
60025 00015
202500
301000 per day
20
UsersSysAdminsManagement
Current trend: all these numbers grow extremely fast!
![Page 13: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/13.jpg)
Software Infrastructureof “Lomonosov” Supercomputer
(what about scalability?)
Software packages, systems, tools, libraries – 60+, and this number grows very fast:
Intel ICC/IFORT, GCC, PathScale, PGI, MPIs, Intel VTune Performance Analyzer, Intel Cluster Tools, RogueWave TotalView, RogueWave ThreadSpotter, Allinea DDT, ScaLAPACK, ATLAS, IMKL, AMCL, BLAS, LAPACK, FFTW, cuBLAS, cuFFT, MAGMA, cuSPARSE, CUSP, and cuRAND…
VASP, WIEN2k, CRYSTAL, Gaussian, MOLPRO, Turbomole,Accelrys Material Studio, MesoProp, MOLCAS, Gromacs, FireFly, LAMMPS, NAMD, GAMESS, Quantum ESPRESSO, ABINIT, Autodock, CP2K, NWChem, PRIRODA, SIESTA, Amber, CPMD, DL POLY, VMD, GULP, Aztec, Geant, OpenFOAM, PARMETIS, FDMNES, GSL, METIS, Msieve, Octave, OpenMX, PETSc, SMEAGOL, VisIt, VTK, WRF…
![Page 14: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/14.jpg)
Fine analysis of supercomputing applications:dynamic features
![Page 15: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/15.jpg)
Fine analysis of supercomputing applications:dynamic features
![Page 16: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/16.jpg)
Fine analysis of supercomputing applications:dynamic features
![Page 17: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/17.jpg)
I. Description of properties and structures: theoretical part (machine-independent properties)
II. Description of properties and structures: implementation issues(programming technologies, static and dynamic features, properties of computer architectures)
Algorithms: description of properties and structures (from mobile platforms to exascale supercomputers)
AlgoWiki
![Page 18: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/18.jpg)
AlgoWiki
http://AlgoWiki-Project.org
![Page 19: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/19.jpg)
AlgoWiki
http://AlgoWiki-Project.org
![Page 20: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/20.jpg)
AlgoWiki
http://AlgoWiki-Project.org
![Page 21: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/21.jpg)
AlgoWiki
http://AlgoWiki-Project.org
![Page 22: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/22.jpg)
AlgoWiki
http://AlgoWiki-Project.org
![Page 23: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/23.jpg)
I. Description of properties and structures: theoretical part
• General description of algorithms• Mathematical description of algorithms• Computational kernel• Macro structure of algorithms• A description of algorithms’ serial implementation• Serial complexity of algorithms • Information graph• Describing the resource parallelism of algorithms • Input/output data description • Algorithm’s properties• …
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (from mobile platforms to exascale supercomputers)
![Page 24: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/24.jpg)
AlgoWikiAlgorithms: description of properties and structures
(information graph)
![Page 25: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/25.jpg)
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (information graph)
![Page 26: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/26.jpg)
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (information graph)
![Page 27: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/27.jpg)
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (information graph)
![Page 28: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/28.jpg)
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (information graph)
![Page 29: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/29.jpg)
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures
![Page 30: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/30.jpg)
I. Description of properties and structures: theoretical part
• General description of algorithms• Mathematical description of algorithms• Computational kernel• Macro structure of algorithms• A description of algorithms’ serial implementation• Serial complexity of algorithms • Information graph• Describing the resource parallelism of algorithms • Input/output data description • Algorithm’s properties• …
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (from mobile platforms to exascale supercomputers)
![Page 31: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/31.jpg)
• Serial and parallel complexity ratio for the algorithm;• computational intensity of an algorithm (ratio of a number of arithmetic operations
to amount of data);• balance of different types of operations;• determinacy of the algorithm:
• number of iterations,• data structures (e.g., the structure of sparse matrices), • use of random number generators,• use of associative operations: the effect of round-off errors.
• “long” edges in the information graph and other similar features of families of edges;
• intensity of data usage;• known compact representations of the information graph;• …
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (algorithm’s properties)
![Page 32: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/32.jpg)
AlgoWiki
http://AlgoWiki-Project.org
MPI_Reduce( buf, …)
MPI_Reduce( buf, …)
MPI_Reduce( buf, …)
Algorithms: description of properties and structures (determinacy of algorithms: collective operations)
![Page 33: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/33.jpg)
II. Description of properties and structures: implementation issues
• Peculiarities of implementations of the serial algorithm;• Description of data and computation locality;• Possible methods and considerations for parallel implementation of the
algorithm;• Scalability of the algorithm and its implementations;• Dynamic characteristics and efficiency of algorithm implementations;• Conclusions for different classes of computer architectures;• Existing implementations of the algorithm;• …
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (from mobile platforms to exascale supercomputers)
![Page 34: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/34.jpg)
Description of data localityAlgoWiki
http://AlgoWiki-Project.org
Canonical analysis of algorithms: ops = timeMake algorithm faster = minimize ops, compares…
![Page 35: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/35.jpg)
Courtesy of Vad.Voevodin, MSU
FFT
Linpack
Random Access
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (description of data locality)
![Page 36: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/36.jpg)
Courtesy of Vad.Voevodin, MSU
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (data locality of Cholesky factorization)
![Page 37: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/37.jpg)
Courtesy of A.Teplov, MSU
AlgoWiki
http://AlgoWiki-Project.org
AlgoWiki and Top500:
• Top500 = “Linpack + performance + computers”.
• “Algorithms + performance + computers” is an elementof the Part II of AlgoWiki.
Algorithms: description of properties and structures (scalability of algorithms: performance)
![Page 38: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/38.jpg)
Courtesy of A.Teplov, MSU
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (scalability of algorithms: efficiency)
![Page 39: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/39.jpg)
Courtesy of A.Teplov, MSU
AlgoWiki
http://AlgoWiki-Project.org
Algorithms: description of properties and structures (dynamic features)
![Page 40: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/40.jpg)
Проведение анализа масштабируемости программ
40
Performance (Gflops) Performance of 1 core L1 cache misses (mln/s)
L3 cache-misses (mln/s)Number of Load ops (bln/s) Number of Store ops (mln/s)
Data transfer rate (MB/s) Average data transfer rate (MB/s) Data transfer rate (thousands packets/s)
Courtesy of A.Teplov, MSU
![Page 41: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/41.jpg)
AlgoWiki
http://AlgoWiki-Project.org
AlgoWiki is a project for the entire
computing community !
![Page 42: International conference “Modern problems of computational mathematics and mathematical modeling”, dedicated to the 90th anniversary of academician G.I.Marchuk](https://reader035.vdocuments.site/reader035/viewer/2022070323/56649dc95503460f94abfe0b/html5/thumbnails/42.jpg)
International conference“Modern problems of computational mathematics and mathematical modeling”,
dedicated to the 90th anniversary of academician G.I.Marchuk
Thank you !
June 10th, 2015, INM RAS, MoscowJune 10th, 2015, INM RAS, Moscow