parallelprocessing mt05 7 organization mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · lectured by:...
TRANSCRIPT
![Page 1: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/1.jpg)
Processor Organization
Lectured by: Pham Tran Vu
Prepared by: Thoai Nam
![Page 2: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/2.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Outline
� Criteria:
– Diameter, bisection width, number of edges per node,
maximum edge length.
� Processor Organizations:
– Mesh, binary tree, hypertree, pyramid, butterfly, hypercube, shuffle-exchange
![Page 3: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/3.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Criteria
� Diameter
– The largest distance between two nodes
– Lower diameter is better
� Bisection width
The minimum number of edges that must be removed in order to divide the network into two halves (within one)
� Number of edges per node
� Maximum edge length
![Page 4: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/4.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Mesh (1)
� Q-dimensional lattice
� Communication is allowed only between neighboring nodes. Interior nodes communicate with 2q other nodes.
![Page 5: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/5.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Mesh (2)
� Q-dimensional mesh with kq nodes
– Diameter: q(k-1)
– Bisection width: kq-1
– The maximum number of edges per node: 2q
– The maximum edge length is a constant
![Page 6: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/6.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Binary Tree
� Depth k-1: 2k-1 nodes
� Diameter: 2(k-1)
� Bisection width: 1
� Length of the longest edge: increasing
![Page 7: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/7.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Fat Tree
� Bandwidth problem on binary tree
![Page 8: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/8.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Hypertree (1)
� Hypertree of degree k and depth d: a complete k-ary tree of
height d.
![Page 9: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/9.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Hypertree (2)
� A 4-ary hypertree with depth d has 4d leaves and 2d(2d+1-1) nodes
in all
– Diameter: 2d
– Bisection width: 2d+1
– The number of edges per node ≤ 6
– Length of the longest edge: increasing
![Page 10: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/10.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Pyramid
� Size k2: base a 2D mesh network containing k2 processors, the
total number of processors=(4/3)k2 -1/3
� A pyramid of size k2:
– Diameter: 2logk
– Bisection width: 2k
– Maximum of links per node: 9
– Length of the longest edge: increasing
![Page 11: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/11.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Butterfly (1)
� (k+1)2k nodes divided into k+1 rows (rank), each contains n=2k
nodes.
� Ranks are labeled 0 through k
� Node(i,j): j-th node on the i-th rank
� Node(i,j) is connected to two nodes on rank i-1: node(i-1,j) and node (i-1,m), where m is the integer found by inverting the i-th
most significant bit in the binary representation of j
� If node(i,j) is connected to node(i-1,m), then node (i,m) is connected to (i-1,j)
� Diameter=2k
� Bisection width=2k-1
� Length of the longest edge: increasing
![Page 12: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/12.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Butterfly (2)
Rank 0
Rank 1
Rank 2
Rank 3
Node(1,5): i=1, j=5
j = 5 = 101 (binary)
i=1
001 = 1
Node(1,5) is connected to
node(0,1)
0 1 2 3 4 5 6 7
![Page 13: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/13.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Hypercube (1)
� 2k nodes form a k-dimensional hypercube
� Nodes are labeled 0, 1, 2,…, 2k-1
� Two nodes are adjacent if their labels differ in exactly one
bit position
� Diameter=k
� Bisection width= 2k-1
� Number of edges per node is k
� Length of the longest edge: increasing
![Page 14: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/14.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Hypercube (2)
6
7
3
2
1
0
4
5
3
2
1
0
1
00
![Page 15: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/15.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Hypercube (3)
6
7
3
2
1
0
4
5
14
15
11
10
9
8
12
13
� 5 = 0101
� 1 = 0001
� 4 = 0100
� 13 = 1101
![Page 16: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/16.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Others
� Cube-Connected cycles
� Shuffle-Exchange
� De Bruijn
![Page 17: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/17.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Topologies in Real Machines
ButterflyBBN Butterfly (really old)
Fat tree (approx)IBM SP
ArbitraryMyricom (Millennium)
HypercubeSGI Origin
Fat treeQuadrics (in HP Alpha
server clusters)
2D MeshIntel Paragon (old)
4D Hypercube*Cray X1
Fat treeSGI Altix
3D TorusBlue Gene/L
3D MeshRed Storm (Opteron +
Cray network, future)o
lde
r n
ew
er
![Page 18: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/18.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Mapping
� The process of embedding a data structure (represented by
a graph G) into the structure of processors (represented by graph G’)
� Dilation: the largest distance between any two adjacent nodes of G in G’ after mapping
![Page 19: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/19.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Mapping a Ring into a 2D Mesh
� Dilation: 1
![Page 20: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/20.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Mapping a 2D Mesh into a 2D Mesh
� Dilation: 1
![Page 21: ParallelProcessing MT05 7 Organization Mappingcse.hcmut.edu.vn/~ptvu/ppds/lec6.pdf · Lectured by: Pham Tran Vu Prepared by: Thoai Nam. Khoa Khoa Hc và K Thu t Máy Tính Outline](https://reader034.vdocuments.site/reader034/viewer/2022042917/5f58efddbedd5d4b41659513/html5/thumbnails/21.jpg)
Khoa Khoa Học và Kỹ Thuật Máy Tính
Mapping a binary tree into a 2D Mesh
� Dilation: 1