parallel decomposition methods
DESCRIPTION
Parallel Decomposition Methods. Introduction to Parallel Programming – Part 2. Review & Objectives. Previously: Defined parallel computing Explained why parallel computing is becoming mainstream Explained why explicit parallel programming is necessary - PowerPoint PPT PresentationTRANSCRIPT
INTEL CONFIDENTIAL
Parallel Decomposition MethodsIntroduction to Parallel Programming – Part 2
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
2
Review & Objectives
Previously:Defined parallel computingExplained why parallel computing is becoming mainstreamExplained why explicit parallel programming is necessary
At the end of this part you should be able to:Identify opportunities for parallelism in code
segments and applicationsDescribe three methods of dividing independent
work
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
3
Methodology
Study problem, sequential program, or code segmentLook for opportunities for parallelismTry to keep all cores busy doing useful work
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
4
Ways of Exploiting Parallelism
Domain decompositionTask decompositionPipelining
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
5
Domain Decomposition
First, decide how data elements should be divided among cores
Second, decide which tasks each core should be doing
Example: Vector addition
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
6
Domain Decomposition
Find the largest element of an array
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
7
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
8
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
9
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
10
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
11
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
12
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
13
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
14
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
15
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
16
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
17
Domain Decomposition
Find the largest element of an array
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
18
Task (Functional) Decomposition
First, divide problem into independent tasksSecond, decide which data elements are going to be
accessed (read and/or written) by which tasks
Example: Event-handler for GUI
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
19
Task Decomposition
s()
r()q()h()
g()
f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
20
Task Decomposition
s()
r()q()h()
g()
Core 0
Core 2
Core 1 f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
21
Task Decomposition
s()
r()q()h()
g()Core 2
Core 1 f() Core 0
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
22
Task Decomposition
s()
r()q()h()
g()Core 2
Core 1 Core 0f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
23
Task Decomposition
s()
r()q()h()
g()Core 2
Core 1 Core 0f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
24
Task Decomposition
s()
r()q()h()
g()Core 2
Core 1 Core 0f()
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
25
Pipelining
Special kind of task decomposition“Assembly line” parallelismExample: 3D rendering in computer graphics
RasterizeClipProjectModel
Input Output
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
26
Processing One Data Set (Step 1)
RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
27
Processing One Data Set (Step 2)
RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
28
Processing One Data Set (Step 3)
RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
29
Processing One Data Set (Step 4)
RasterizeClipProjectModel
The pipeline processes 1 data set in 4 steps
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
30
Processing Two Data Sets (Step 1)
RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
31
Processing Two Data Sets (Time 2)
RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
32
Processing Two Data Sets (Step 3)
RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
33
Processing Two Data Sets (Step 4)
RasterizeClipProjectModel
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
34
Processing Two Data Sets (Step 5)
RasterizeClipProjectModel
The pipeline processes 2 data sets in 5 steps
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
35
Pipelining Five Data Sets (Step 1)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
36
Pipelining Five Data Sets (Step 2)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
37
Pipelining Five Data Sets (Step 3)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
38
Pipelining Five Data Sets (Step 4)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
39
Pipelining Five Data Sets (Step 5)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
40
Pipelining Five Data Sets (Step 6)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
41
Pipelining Five Data Sets (Step 7)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
42
Pipelining Five Data Sets (Step 8)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
Core 0 Core 1 Core 2 Core 3
Copyright © 2009, Intel Corporation. All rights reserved. Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. * Other brands and names are the property of their respective owners.
43
References
Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006).
Robert L. Mitchell, “Decline of the Desktop,” Computerworld (September 26, 2005).
Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004).
Herb Sutter, “The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software,” Dr. Dobb’s Journal 30, 3 (March 2005).