High Performance Computing Systems

and Enabling Platforms

Marco Vanneschi

Department of Computer Science, University of Pisa

Master Program (Laurea Magistrale) in Computer Science and Networking

Academic Year 2011-2012

Master Program (Laurea Magistrale) in Computer Science and Networking

High Performance Computing Systems and Enabling Platforms

Marco Vanneschi

Course Introduction

www.di.unipi.it/vannesch section: Teaching

SPA: course objectives

• Providing a solid framework of concepts and techniques in high-performance computing– Parallelization methodology and models

– Support to parallel programming models and software development tools

– Parallel Architectures

– Performance evaluation (cost models)

• Methodology for studying existing and future systems

• Technology: state-of-the-art and trends– Parallel processors

– Multiprocessors

– Multicore / manycore / … / GPU

– Shared vs distributed memory architectures

– Programming models and their support

3MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms

SPA: first course in HPC and enabling platforms

• Distributed Systems: Paradigms and Models (Prof. Danelutto), mandatory, 2nd semester

– assumes SPA as prerequisite

• Other courses in this area– Programming tools for parallel and distributed systems

– Complements of distributed enabling platforms

– Parallel and distributed algorithms

– …

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 4

Course approach: a Computer Science approach

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 5

methodologicalknowledge

technologicalknowledge

• Computer Science approach

o Parallel computing: computational and programming models

o Cost models of parallel computations

o Computing architecture has its own concepts, principles, models, and

techniques

o Conceptual framework in common with the other disciplines of Computer

Science:

• Programming languages, algorithms, computability and complexity, …

Basic background and prerequisites

• An undergraduate-level course on structured computer architecture– Firmware level structuring

– Assembler level, CPU architecture, compiling

– Memory hierarchies and caching

– Interrupt handling, exception handling

– Process level, addressing space, low level scheduling, interprocess

communication

– Input/Output processing

• In Pisa: the course “Computer Architecture” (UndergraduateProgram in Computer Science) adopts such an approach

• Precourse on Computer Architecture fundamentals

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 6

Precourse: subjects and approach

1. System structuring: level, modules, cooperation models

2. Firmware level fundamentals

3. Assembler level

4. Memory hierachies and caching

5. Process level fundamentals and run-time support

• Approach of Precourse:

– Main concepts and techniques on these subjects

– Exercises: proposed to students and discussed (class-level or individually)

• Recommendation: Precourse-level knowledge is of fundamentalimportance for SPA course and exam.

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 7

Precourse: teaching material

• Slides: used as course notes for the precourse– On my personal page

– Part 5 of Precourse (Process Level and Communication) is present in SPA Course Notes (Part 1, Section 3)

• Reference books:

– D.A. Patterson, J.H. Hennessy, “Computer Organization and Design:

the Hardware/Software Interface”, Morgan Kaufman Publishers Inc.

– A. Tanenbaum, “Structured Computer Organization”, Prentice-Hall.

– in Italian: M. Vanneschi, “Architettura degli Elaboratori”, PLUS, Pisa.

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 8

Working approach: Precourse and SPA

• As in any other course, it is fundamental to acquire skills and capabilities in concepts and principles, besides knowing the technologies.

• Critical aptitude must be properly developed.

• Interaction with the teacher is strongly encouraged– Questions during the lectures

– Presentation and discussion of exercises

– Question time (“orario di ricevimento”) (in Italian for Italians)

• … (to be fixed)

• and/or by appointment in case of collision with other courses, or other reasons.

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 9

SPA: Course Program

PART 1 (about 3 CFU)

1. Methodology for Parallel Application Structuring and Design

• Parallel computation models

• Parallelization methodology and techniques

• Parallel paradigms

• Programming models

• Cost models

• Static and dynamic optimizations

PART 2 (about 3 CFU)

2. Parallel Architectures

• Shared memory architectures : SMP, NUMA

• Distributed memory architectures: Clusters, MPP

• Interconnection networks,

• Support to concurrency mechanisms

• Cost models

• Parallel application benchmarks

• Multicore, multithreading, advanced technologies

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 10

SPA: Teaching Material

• Course notes

– available on my personal page

– version 2011-12

• Reference books:– Patterson: see Precourse references

– D.E. Culler, J.P. Singh, A. Gupta, “Parallel Computer Architecture: a

Hardware/Software Approach”, Morgan Kaufman Publishers Inc.

– Papers and reports

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 11

SPA: exam modality

• Written + oral exam (in English or in Italian)

• Intermediate tests (midterms) ?

• Exam approach: the student must be able to demonstrate a deep understanding of concepts and techniques and of theirinterrelationships, as well as to write / to present the answerin a clear and rigorous way.

• Registration to the exam on the Official Site of Corso di Laurea: http://compass2.di.unipi.it/didattica, section Laurea Magistrale in Informatica e Netwoking, subsection “calendar”

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 12

Architetture Parallele e Distribuite (ASE)

Esame di ASE (9 CFU), laurea specialistica vecchio ordinamento:

• SPA (6 CFU)

• + integrazione 3 CFU (Libro PLUS, Parte IV)

– implementazione di programmi paralleli in LC secondo forme di

parallelismo note

– architettura di CPU pipeline e valutazione di programmi

• Modalità di esame: scritto.

• All’atto dell’iscrizione: indicare ASE

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 13

MCSN - M. Vanneschi: High Performance Computing Systems and Enabling Platforms 14

Good Luck !