![Page 1: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/1.jpg)
High Performance Computing Systems
Exascale Computing
Doug Shook
![Page 2: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/2.jpg)
2
Exascale Computing How many flops?
Why is this an important milestone?
What challenges would you anticipate?
![Page 3: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/3.jpg)
3
Challenges DOE has found 10 challenges to exascale computing
What must we do to meet these challenges?
When will these challenges be met?
Lucas, et. Al, “Top Ten Exascale Research Challenges” 2014
![Page 4: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/4.jpg)
4
#1 Energy Efficiency Current usage– Can we simply increase energy usage?
What parts of the system are affected by energy efficiency?
Evolutionary vs. revolutionary
![Page 5: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/5.jpg)
5
Near Threshold Voltage (NVT) What is the threshold voltage?
Energy advantages of operating near threshold voltage
Problems?
![Page 6: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/6.jpg)
6
Energy Efficient Architecture How much energy does it take to perform 1 FP operation?
![Page 7: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/7.jpg)
7
Energy Efficient Interconnects
![Page 8: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/8.jpg)
8
On Chip Power Management
![Page 9: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/9.jpg)
9
System Scale Power Management Power Distribution
Cooling
Packaging
![Page 10: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/10.jpg)
10
#2 Interconnect Technology Data Movement Energy and Bandwidth
![Page 11: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/11.jpg)
11
#2 Interconnect Technology On-die Interconnect Fabric
Inter-chip Network Integration
Photonics
![Page 12: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/12.jpg)
12
#3 Memory Technology Memory Capacity
![Page 13: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/13.jpg)
13
#3 Memory Technology Energy
Scaling
Resiliance
![Page 14: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/14.jpg)
14
#4 Scalable System Software Research Directions
Lightweight OS
Runtime Systems
Introspection
Energy Management
![Page 15: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/15.jpg)
15
#5 Programming Systems
![Page 16: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/16.jpg)
16
#5 Programming Systems Programming Models
Compilers
![Page 17: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/17.jpg)
17
#6 Data Management Offensive vs. Defensive I/O
General I/O Challenges
Offensive I/O Challenges
Defensive I/O Challenges
![Page 18: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/18.jpg)
18
#7 Exascale Algorithms Multicore friendly vs. Multicore aware
Communication Avoiding Algorithms
Synchronization Reduction
Multi-physics algorithms
Multi-scale algorithms
Energy Efficient Algorithms
![Page 19: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/19.jpg)
19
#8 Algorithms for Discovery, Design, and Decision Uncertainty Quantification
Optimization
![Page 20: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/20.jpg)
20
#9 Resiliance and Correctness Hardware Support
Programming Models
Algorithmic based fault tolerance
Correctness
![Page 21: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/21.jpg)
21
#10 Scientific Productivity Research Directions
![Page 22: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/22.jpg)
22
Co-Design and Integration Framework Execution Model
Architecture
Performance Metrics
![Page 23: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/23.jpg)
23
Integration Framework
![Page 24: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/24.jpg)
24
Design Process
![Page 25: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/25.jpg)
25
Modeling and Simulation
![Page 26: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/26.jpg)
26
Recommendations
![Page 27: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/27.jpg)
27
Current State Four countries currently have plans for exascale systems– Economic considerations?– Political considerations?
Reed, Dongarra, “Exascale Computing and Big Data: The Next Frontier” 2015
![Page 28: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/28.jpg)
28
Exascale Computing Project National Strategic Computing Initiative (2015)– Unite HPC and Big Data– Preserve US Dominance in HPC– Improve interoperability between supercomputers– Provide widespread access and training to
researchers– Develop post-silicon technologies
Messina, Lee, “The Exascale Computing Project” 2017
![Page 29: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/29.jpg)
29
Exascale Computing Project Lead Agencies– DoE, NSF, DoD
7 year project– $3.5-5.7 billion
![Page 30: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/30.jpg)
30
Four Key Challenges Parallelism
Memory and Storage efficiencies
Reliability
Energy Consumption
![Page 31: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/31.jpg)
31
Performance from Parallelism
![Page 32: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/32.jpg)
32
Goals Deliver two exascale systems by 2023– First in 2021 based on “advanced architecture”– 20-30 MW– Sufficiently resilient– Support a broad range of workloads
![Page 33: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/33.jpg)
33
Strategic Pillars National Security
Energy Security
Economic Security
Scientific Discovery
Earth System
Health Care
![Page 34: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/34.jpg)
34
Software and Hardware Goals Software
Hardware
![Page 35: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/35.jpg)
35
Schedule
![Page 36: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/36.jpg)
36
Current Status Recent Actions
Risks
![Page 37: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/37.jpg)
37
Alternative Architectures What does this mean?
Options?
![Page 38: High Performance Computing Systemsdshook/cse566/lectures/Exascale.pdf · Programming Models ... Deliver two exascale systems by 2023 – First in 2021 based on “advanced architecture”](https://reader033.vdocuments.site/reader033/viewer/2022052000/6012e2e729deaa19571c1373/html5/thumbnails/38.jpg)
38
Course Recap / Discussion