programming models for exascale systems: what, when and...
TRANSCRIPT
Programming Models for Exascale Systems: What, When and How?!
!Kathy Yelick!
Associate Laboratory Director for Computing Sciences!Lawrence Berkeley National Laboratory !
and !Professor of Electrical Engineering and Computer Science!
UC Berkeley
1. Lightweight cores will have all/most of the system performance – Need fine-‐grained parallelism; avoid unnecessary synchroniza8on – Cores not powerful enough for complex communica8on protocols ?
2. On-‐chip interconnect offers opportuni=es for performance – New models of communica8on may be essen8al
3. Hardware is heterogeneous: no single ISA – Portability and performance portability are challenging
4. New levels of memory hierarchy, possibly soIware-‐controlled – Locality and communica8on-‐avoidance paramount
5. Performance variability may increase – SoCware or hardware control clock speeds
6. Resilience will be paramount at scale – Failures grow with the number of components and connec8ons
Why is Change Necessary?
2 Programing Models and Environments
Don’t Overstake Claims – Prove them!
Programing Models and Environments 3
The Trade-Off Is With Us
Programing Models and Environments 4
Locality Load Balance
• Target new applica=ons, not rewri=ng old ones – The mo8va8on is higher (all that code to be wriIen) – The barrier to entry is lower (no code to throw away)
• Target parts of applica=ons, not full rewrites – Support for incremental adop8on is key – Need to leverage exis8ng libraries and parts of exis8ng codes
• Target new programming paOerns – Irregular-‐in-‐space communica8on (random access to large data) – Irregular-‐in-‐8me (one-‐sided access to shared state) – Irregular workload (dynamic load balancing)
• Look for low hanging performance fruit – Do you rooflining homework – What’s the threshold? 15% -‐ no, 15x -‐ yes
Approach to Adoption
5 Programing Models and Environments
• Target new applica=ons, not rewri=ng old ones – The mo8va8on is higher (all that code to be wriIen) – The barrier to entry is lower (no code to throw away)
• Target parts of applica=ons, not full rewrites – Support for incremental adop8on is key – Need to leverage exis8ng libraries and parts of exis8ng codes
• Target new programming paOerns – Irregular-‐in-‐space communica8on (random access to large data) – Irregular-‐in-‐8me (one-‐sided access to shared state) – Irregular workload (dynamic load balancing)
• Look for low hanging performance fruit – Do you rooflining homework – What’s the threshold? 15% -‐ no, 15x -‐ yes
Approach to Adoption
6 Programing Models and Environments
Work with applica8ons people to understand
what they need!
Random Access to large state
Dynamic load
balancing
Fusion of observa8on
into simula8on
Unstruct-‐ured DAG algorithms
Application Challenges (Opportunities?)
7 Programing Models and Environments
Infinite 15-‐20% Infinite 2x Speedup
Programming mode role: encourage a way of thinking
Lightweight mechanisms, flexible policies • Atomic remote memory opera=ons (compare&swap,…) • 1-‐way put + signal (2 addresses, data + increment) • SoIware-‐managed memory as an op#on • Fault detec=on and resilience job schedulers, etc. • Transparent performance informa=on
Hardware changes desired/required?
8 Programing Models and Environments
Programing Models and Environments 9
Give an applica=on some op=miza=ons and you feed it for a
machine.
Teach a computer to generate op=mized code for an applica=on
and you feed it for a life=me
Thank you!
• What kind of programming models will be good for Exascale systems? Will it be evolu=onary or revolu=onary?
• Will MPI+X (together with PGAS and Task-‐based models) will be best suited for Exascale systems?
• Will newer programming models like HPC++, Legion, HPX, etc. gain more trac=on?
• How will the upcoming hardware trend (accelerators, coprocessors, high bandwidth memory, on-‐chip networking, etc.) have impact on the suitability of programming models?
11 Programing Models and Environments