parallel & concurrent programming: zpl - umass amherst · zpl parallel array language...
TRANSCRIPT
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science
Parallel & Concurrent Programming:
ZPLEmery BergerCMPSCI 691WSpring 2006
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 2
Outline
Previously:MPI – point-to-point & collective
Complicated, far from problem abstraction
OpenMP - parallel directivesLanguage extensions to Fortran/C/C++Questionable semantics, error-prone
Today:Something way better: ZPL
lecture material from ZPL project, UW
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 3
ZPL
Parallel array languageImplicitly parallel
No parallel constructs per se
Very high levelAssignments work at array level, as inA := B + C
Machine independentCompiles to ANSI C +communication library calls (e.g., MPI)
Efficient
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 4
Comparison
Matrix-multiplication:C
triply-nested loop
ZPLdot-product of rows & columnsefficiently implemented on parallel machines
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 5
ZPL Outline
Language overviewRegionsDirectionsParallel array operationsHandling boundary conditions
ZPL programs & performance
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 6
Regions
Key abstraction in ZPL: regionsIndex sets (rows,cols) partition matricesOperate on regions, not indexed items!
rows
columns
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 7
Region Examples
Interior of matrix
Left-most column
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 8
Directions
Directions:Offset vectors used to manipulate regions & array data
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 9
Creating New Regions
Prepositions create new regions:in
Applies direction to select part of region
ofCreates new region outside existing region
atShifts a region by a direction
byCreates new region strided by direction
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 10
Applying Directions
Use “in” to apply direction to region
+ =
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 11
Create Region Outside
Use “of” to create regionoutside existing region
Extends region
+ ==
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 12
Shifting Regions
Use “at” to create new regionshifted by a direction
+ =
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 13
Striding Regions
Use “by” to create new regionstrided by a direction
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 14
Parallel Arrays
Parallel arrays declared over regions
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 15
Computing Over Arrays
Can use regions as modifiers that define computations over arrays:
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 16
Arrays & Communication
Most computations in ZPLdo not involve communication
Exceptions include:ShiftingReductionBroadcastAll-to-all
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 17
Shifting Arrays
@ operator shifts data in directionThis translation induces point-to-pointcommunication
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 18
Reduction
Op<< computes reductionsReduction (tree-style) communication
+<< (sum), *<< (times), min<<, max<<…
For prefix (scan), use op||
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 19
Broadcast (Flooding)
>> (flood) replicates data across dimensions of array
Triggers broadcast operation
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 20
Mapping
Remap (#) moves data between arraysSpecified by “map” arraysBuilt-in Index1, Index2
Index1 = row indices, Index2 = col indices
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 21
Boundary Conditions
Boundary conditions (“corner cases”)Usually tedious, error-proneVery simple in ZPL
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 22
Boundary Conditions
Periodic boundary conditions with wrap
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 23
ZPL Example
Jacobi iteration – replace elements in array with average of four nearest neighbors, until largest change < δ
Consider difficulty of parallelizing with MPI/OpenMP
boundary conditions, etc.
3
1
2
4
2.5 3
1
2
4
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 24
ZPL Example
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 25
ZPL Performance
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 26
ZPL Example: Life
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 27
ZPL Example: Life
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS AAMHERST MHERST •• Department of Computer ScienceDepartment of Computer Science 28
The End
Next time:Your turn!Occam & Multilisp