introduction to parallel computingkarypis/parbook...decompose the graph a (adjacency matrix) and...

17
Introduction to Parallel Computing George Karypis Graph Algorithms

Upload: others

Post on 14-Dec-2020

8 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Introduction to Parallel Computing

George KarypisGraph Algorithms

Page 2: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

OutlineGraph Theory BackgroundMinimum Spanning Tree

Prim’s algorithmSingle-Source Shortest Path

Dijkstra’s algorithmAll-Pairs Shortest Path

Dijkstra’s algorithmFloyd’s algorithm

Maximal Independent SetLuby’s algorithm

Page 3: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Background

Page 4: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Minimum Spanning TreeCompute the minimum weight spanning tree of an undirected graph.

Page 5: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Prim’s AlgorithmPrim’s Algorithm

Θ(n2) serial complexity for dense graphs.

why?

How can we parallelize this algorithm?Which steps can be done in parallel?

Page 6: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Parallel Formulation of Prim’sAlgorithmParallelize the inner-most loop of the algorithm.

Parallelize the selection of the “minimum weight edge” connecting an edge in VT to a vertex in V-VT.Parallelize the updating of the d[] array.

What is the maximum concurrency that such an approach can use?How do we “implement” it on a distributed-memory architecture?

Page 7: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns.

Why columns?Assign each block of size n/p to one of the processors.How will lines 10 & 12—13 be performed?Complexity?

Parallel Formulation of Prim’sAlgorithm

Isoefficiency:

Page 8: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Single-Source Shortest Path

Given a source vertex s find the shortest-paths to all other vertices.Dijkstra’s algorithm.How can it be parallelized for dense graphs?

Page 9: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

All-pairs Shortest PathsCompute the shortest paths between all pairs of vertices.Algorithms

Dijkstra’s algorithmExecute the single-source algorithm n times.

Floyd’s algorithmBased on dynamic programming.

Page 10: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

All-Pairs Shortest PathDijkstra’s Algorithm

Source-partitioned formulationPartition the sources along the different processors.

Is it a good algorithm?Computational & memory scalabilityWhat is the maximum number of processors that it can use?

Source-parallel formulationUsed when p > n.Processors are partitioned into n groups each having p/n processors.Each group is responsible for one single-source SP computation.Complexity?

Page 11: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Floyd’s AlgorithmSolves the problem using a dynamic programming algorithm.

Let d(k)i,j be the shortest path distance

between vertices i and j that goes only through vertices 1,…, k.

Complexity: Θ(n3).Note: The algorithm can run in-place.

How can we parallelize it?

Page 12: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Distribute the matrix using a 2D block decomposition.Parallelize the double inner-most loop.

Communication pattern?Complexity?

Parallel Formulation of Floyd’s Algorithm

Page 13: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each
Page 14: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Comparison of All-Pairs SP Algorithms

Page 15: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Maximal Independent SetsFind the maximal set of vertices that are not adjacent to each other.

Page 16: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Serial Algorithms for MISPractical MIS algorithms are incremental in nature.

Start with an empty set.1. Add the vertex with the smallest degree.2. Remove adjacent vertices3. Repeat 1—2 until the graph becomes empty.These algorithms are impossible to parallelize.

Why?Parallel MIS algorithms are based on the ideas initially introduced by Luby.

Page 17: Introduction to Parallel Computingkarypis/parbook...Decompose the graph A (adjacency matrix) and vector d vector using a 1D block partitioning along columns. Why columns? Assign each

Luby’s MIS AlgorithmRandomized algorithm.

Starts with an empty set.1. Assigns random numbers to each vertex.2. Vertices whose random number are smaller

than all of the numbers assigned to their adjacent vertices are included in the MIS.

3. Vertices adjacent to the newly inserted vertices are removed.

4. Repeat steps 1—3 until the graph becomes empty.

This algorithms will terminate in O(log (n)) iterations.Why is this a good algorithm to parallelize?How will the parallel formulation proceed?

Shared memoryDistributed memory