(c) p. h. welch, 20031 paradigms of parallelism chapter 2
TRANSCRIPT
![Page 1: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/1.jpg)
(C) P. H. Welch, 2003 1
Paradigmsof
Parallelism
Chapter 2
![Page 2: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/2.jpg)
(C) P. H. Welch, 2003 2
•Geometric (90%)•Algorithmic (70%)•Farming (99%)•???
+ mixtures of the above
Levels of efficiencythat should be obtained
Paradigms of Parallelism
![Page 3: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/3.jpg)
(C) P. H. Welch, 2003 3
Fox’s Wall
specification
•A large pile of bricks•One bricklayer – OK!•Three bricklayers???
need organizing
![Page 4: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/4.jpg)
(C) P. H. Welch, 2003 4
Geometric Distribution
• Task partitioned geometrically amongst workers
• For the main, each worker works away independently
• But needs to interact with neighbouring workers when working on boundaries of allotted task
![Page 5: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/5.jpg)
(C) P. H. Welch, 2003 5
Geometric Distribution
Green can work fastest(no half bricks to cut)
Red works as fast as it can
Blue is an apprentice(and slow)
![Page 6: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/6.jpg)
(C) P. H. Welch, 2003 6
Geometric Distribution
Green can work fastest(but is now held up!)
Red works as fast as it can
Blue is an apprentice(and slow)
![Page 7: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/7.jpg)
(C) P. H. Welch, 2003 7
Algorithmic (Pipeline)Distribution
• Functional distribution of tasks to workers
• Need to interact with neighbouring workers continually (to input data and pass on results)
• Like geometric distribution but work area is all boundary
![Page 8: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/8.jpg)
(C) P. H. Welch, 2003 8
Pipeline Distribution
• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the
“pipe”
![Page 9: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/9.jpg)
(C) P. H. Welch, 2003 9
Pipeline Distribution
• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the
“pipe”
![Page 10: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/10.jpg)
(C) P. H. Welch, 2003 10
Pipeline Distribution
• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the
“pipe”
![Page 11: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/11.jpg)
(C) P. H. Welch, 2003 11
Pipeline Distribution
• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the
“pipe”
![Page 12: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/12.jpg)
(C) P. H. Welch, 2003 12
Pipeline Distribution
• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the
“pipe”
![Page 13: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/13.jpg)
(C) P. H. Welch, 2003 13
Pipeline Distribution
• Some workers idle at the beginning and end• All workers busy when “pipe” is full• Must work at pace of slowest element in the
“pipe”
![Page 14: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/14.jpg)
(C) P. H. Welch, 2003 14
Farming Distribution
• Each worker gets work from a single source (“farmer”)
• Each worker sends completed work to a single “harvester”
• Each worker’s work is not dependent on any particular other worker
• Each worker’s rate of work may be different
![Page 15: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/15.jpg)
(C) P. H. Welch, 2003 15
Farming Distribution
• All workers always busy
• Work at your own pace
• Your work must not be (tightly) dependent on other work
![Page 16: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/16.jpg)
(C) P. H. Welch, 2003 16
• All workers always busy
• Work at your own pace
• Your work must not be (tightly) dependent on other work
Farming Distribution
![Page 17: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/17.jpg)
(C) P. H. Welch, 2003 17
• All workers always busy
• Work at your own pace
• Your work must not be (tightly) dependent on other work
Farming Distribution
![Page 18: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/18.jpg)
(C) P. H. Welch, 2003 18
Take n particles with different masses and initial positions and velocities in 3-space.
Assume some inter-particle forces (e.g., gravity).
Display their movements.
n-Body Problem
![Page 19: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/19.jpg)
(C) P. H. Welch, 2003 19
C0
C2
C1
Ci
Ci+1
Ci+2
Cn-1
C3
(positionvelocity)
Each cell knows all constants (e.g. themasses of all particles).Cell Ci knows the current position andvelocity of particle i(its variables).
Geometric Distribution
![Page 20: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/20.jpg)
(C) P. H. Welch, 2003 20
C0
C2
C1
Ci
Ci+1
Ci+2
Cn-1
C3
(positionvelocity)
Each cell knows all constants (e.g. themasses of all particles).Cell Ci knows the current position andvelocity of particle i(its variables).
graphics
Geometric Distribution
![Page 21: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/21.jpg)
(C) P. H. Welch, 2003 21
pump
<unsorted items>end.marker
<unsorted items>end.marker
<sorted items>end.marker
<sorted items>end.marker
Sort Pump
![Page 22: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/22.jpg)
(C) P. H. Welch, 2003 22
cell
<items>end.marker<items>
end.marker
<items> flow through the cell one at a time. The cell hangs on to the largest item is sees – passing smaller ones out. When the end.marker arrives, thecell finally outputs what it was holding (followed by the end.marker).
Sort Cell
![Page 23: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/23.jpg)
(C) P. H. Welch, 2003 23
• Rate of input of unsorted items.
• Cell cycle time.
• Inter-cell transfer rate.
(n-1)This starts out with up to n items. With (n-1) separate (silicon)cells, its rate of output of sorted items is:-
minimum
This performance is independent of n.
Pipeline Distribution
![Page 24: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/24.jpg)
(C) P. H. Welch, 2003 24
(0,2)
(-2,0) (2,0)
(0,-2)
c
Z0 = cZi+1 = Zi + c
Given c, find first |M such that|ZM| > 2.
Give up is M = 511.
Mandelbrot
![Page 25: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/25.jpg)
(C) P. H. Welch, 2003 25
Colour the point c according to the value of M such that:-
M
0 511
some colour spectrumwhite black
Mandelbrot
![Page 26: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/26.jpg)
(C) P. H. Welch, 2003 26
work packets
results
farmer
worker
worker
harvester
worker
Standard Process Farming
![Page 27: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/27.jpg)
(C) P. H. Welch, 2003 27
Work Packet
(x, y) a point in the complex plane
a horizontal resolutionn number of points to colour
Results
(x,y) (x + n.a,y)
Computer the Mandelbrot number Mfor each of these points.
Mandelbrot
![Page 28: (C) P. H. Welch, 20031 Paradigms of Parallelism Chapter 2](https://reader033.vdocuments.site/reader033/viewer/2022042702/56649cd05503460f9499c2eb/html5/thumbnails/28.jpg)
(C) P. H. Welch, 2003 28
in.work in.result
buffer
prompter
out.work
work
catch
mux
out.result
worker
Farm Worker Harness
* omitted from the last worker!* omitted from the last worker!
**
* omitted from the first worker!* omitted from the first worker!
**