A Pattern Language for Parallelizing

Irregular Algorithms

Pedro and Miguel MonteiroUniversidade Nova de Lisboa

A lot of ideas come from the Ph.D. Dissertation of Milind V. Kulkarni (Cornell 2008)

THE GALOIS SYSTEM: OPTIMISTICPARALLELIZATION OF IRREGULAR PROGRAMS

Also:Amorphous Data-parallelism in Irregular AlgorithmsKeshav Pingali, Milind Kulkarni, Donald Nguyen, Martin Burtscher, Mario Mendez-Lojo, Dimitrios Prountzos, Xin Sui and Zifei ZhongThe University of Texas at Austin, Dept of Comp SciReport# TR-09-05February, 2009

Irregular Algorithm

Unpredictability of Data Dependencies

MOST GENERAL FORM:

foreach element in dataStructure dodataElements = // identify neighborslock dataElements;compute(dataStructure.get(dataElements));unlock dataElements;

endForeach

Create a Taxonomy

KULKARNI DEFINITION:

• for each Element e in Set S { B(e) }The loop body B(e) is executed for each element e of Set S.Since set elements are not ordered, this construct asserts that ina serial execution of the loop, the iterations can be executed inany order. There may be dependences between the iterations,but any serial order of executing iterations is permitted. Whenan iteration executes, it may add elements to S.

• for each Element e in OrderedSet S { B(e) }This construct iterates over an ordered set S of active elements.It is similar to the Set iterator above, except that the executionorder must respect the order on elements of S.

“The focus of this pattern is not to provide specific implementation solutions, merely to allow us to understand how graph characteristics influence irregular parallel problems.”

Data-Parallel Graph

Paper proposes a set of steps for decisions that need to be made, but they are all abstract

Optimistic Iteration

Like speculative instruction execution

Graph graph;Worker worker;//worker threadScheduler scheduler;while (true){

try{Iteration it;it =

scheduler.newIteration(worker);scheduler.nextElement(it);<result,work> = compute(graph,it);graph.replaceSubgraph(it, result);if(work.isNotNull())

scheduler.addWork(work);

scheduler.commitIteration(it);}catch (violationException ve){

//do nothing//graph is only updated on commit

}//check for termination}

General optimistic implementation of a irregular algorithm.

In-order Iteration

When there are some ordering constraints on data

Don't want to reduce to sequential algorithm

Put some knowledge of ordering constraints into the Optimistic Iteration engine so rollbacks are avoided where possible

KULKARNI TAXONOMY:

How to exploit structure:

Zero-copy implementation Cautious operator, unordered

Data partitioning Scheduling

Run-timeJust-in-time (compute once, reuse)Compile-time

“Synthetic” approach, rather than a taxonomy.

Compare to

Mark Snir Berkeley “dwarfs”