routing in a parallel computer

Routing in a Parallel Computer


• A network of processors is represented by graph G=(V,E), where |V| = N.

• Each processor has unique ID between 1 and N.

• Each edge e=(v1,v2) represents a communication link between processors v1 and v2.


• All communication proceeds in a sequence of synchronous steps.

• Each link ca carry one packet in a step.

• During a step processor can send at most one message to each one of his neighbours.

Permutation Routing

Initially each processor contains packet destined for some processor in the network.

• i: processor ID.

• vi: packet that processor i contains.

• d(i):destination processor of the packet vi.

For 1 ≤ i ≤ N, d(i)’s form a permutation of {1,.2,…,N}.

Permutation Routing

Route: A sequence of edges packet follow from its source to destination.

Algorithm’s goal: Find a route for each packet.

Note:A packet may occasionally have to wait at an intermediate node because the next edge on its route is "busy" transmitting another packet.

We assume that each node contains one queue for each edge leaving the node.

Oblivious Algorithm

The route followed by vi depends on d(i) alone, and not on d(j) for any j ≠ i.

Oblivious routing algorithms are simple to implement.

Theorem:For any deterministic oblivious permutation routing algorithm on a network of N nodes each of out-degree d, there is an instance of permutation routing requiring

)(dN steps.

Boolean Hypercube Network

N=2n

• n: Hypercube dimensions.• N: Number of nodes.

Each node i represented as n-bits binary number:(i0,i1,…,in-1) ϵ {0,1}n

There is edge (i,j) iff i and j differs in exactly one bit.Example: (1001, 1101)


n=3N=8

101 111

100 110

001 011

000 010


Algorithm: sending vi from node i to node σ(i):

• Scan the bits of σ(i) from left to right, and compare them with the address of the current location of vi.

• Send vi out of the current node along the edge corresponding to the left-most bit in which the current position and σ(i) differ.

Example:Sending (1011) to (0000) in 4-dimensional hypercube.

Route: (1011) , (0011) , (0001), (0000)

Randomized Oblivious Routing Algorithm

Phase 1:Pick a random destination σ(i) from {1,...,n}. Packet vi travels to node σ(i).

Phase 2:Packet vi travels from σ(i) to its destination d(i).

In each phase, each packet uses the bit-fixing strategy to determine its route.


Notes:

• σ(i) are chosen randomly. So they are not a permutation.

• Each node maintains a queue for each outgoing edge, with packets leaving in FIFO order (ties should be broken arbitrarily).


How many steps elapse before packet vi reaches its destination?

Let us first consider this question for Phase 1:

• Let pi be route for vi in Phase 1.• The number of steps taken by vi: - The length of pi, which is at most n. - Number of steps it is queued.

Randomized Oblivious Routing Algorithm (Lemma 1)

Lemma 1:

• View routes in Phase 1 as a directed path.

• For two different packets’ routes in Phase 1: once they separate, they do not rejoin.


Proof:

• Each path may be seen as a sequence of binary numbers, representing nodes.• Let’s consider the paths parts starting from the first common node.• Mark: - v - the last common node. - v1 – the next node in the first path - v2 – the next node in the second path


b1

c2b2

c1V:

V1:

V2:

b1≠b2 ^ c1≠c2

The order of updating the bits is left-to-right.So the bit b1 in the first path and bit b2 in the second path will never change.And thus the paths will never pass the node with the same ID.

b2

c1


Lemma 2:• Let the route of vi follow the sequence of edges pi = (e1,e2,...,ek). • Let S be the set of packets (other than vi) whose routes pass through at least one of {e1,e2,...,ek}.

Then: the delay incurred by vi is at most |S|.


Proof:• A packet in S is said to leave pi at that time step at which it traverses an edge of pi for the last time.• For each packet v passing pi we define: lag(v) = t-j v is ready to follow edge ej at time t.• The lag of vi is initially zero, and the delay incurred by vi is its lag when it traverses ek.

We will show that each step at which the lag of vi increases by one can be charged to a distinct member of S.

So the time that takes vi to get to its intermediate target is:k+|S| ≤ k+N


e1 e2 e3 e4

Example: • Next step time is 4• pi = (e1,e2,e3,e4)

v

Lag(v) = t – j = 4 – 3 = 1


Claim:If the lag of vi reaches l + 1, some packet in S leaves pi with lag l.

Let t’ be the last time step at which any packet in S has lag l.

Claim:Some packet of S leaves pi at t’.


So for each such l there is a packet v such that lag(v)=l.

When the lag of vi increases from l to l + 1, there must be at least one packet (from S) that wishes to traverse the same edge as vi at that time step,


At time t’ there is a packet v ready to follow edge e j’ at t’ such that t’ – j’ = l.

Some packet u in S follows ej’ at t’ (maybe u=v).

Claim:u leaves pi at t’.

Proof:Let’s assume it’s not true. So u will want to follow the ej’+1 at time t’+1. lag(u) at time t’+1 is ((t’+1)-(j’+1)) = l.In contradiction that t’ is the last time when any packet in S has lag l.


When vi is delayed, one packet in S leaves pi.According to Lemma 1, once the paths are separated, they never rejoin.

After |S| delays no packet except vi travels thru pi. vi may be delayed up to |S| steps.

It takes vi up to k + |S| steps to get to it’s intermediate target.

Proof summary:


Random variables Hij: • Hij = 1 , if pi and pj share at least one edge• Hij = 0 , otherwise

The intermediate nodes for each packet are chosen independently. So Hij for some i are independent.

By lemma 2, delay of package vi is at most:

N

jijH

1


121)(/6for

2)6Pr(

1

6

1

eHEn

nH

N

jij

nN

jij

n6)1( :Let

12for 2),(

),(])1(Pr[)1(

eF

FX


Random variables T(e): The number of routes that pass through edge e.

For any route pi=(e1,e2,…,ek):

k

ll

N

jij eTH

11

)(

k

ll

N

jij eTEHE

11

)]([][


Lemma:

)]([)]([

:e and e edges ofpair each For ji

ji eTEeTE


Nn. is hypercube in the edges ofnumber The

.2

N is packets theallover summedlength route total theofn expectatio that theSo

i. allfor 2

is p oflength expected The i

n

n

Nn

ll

nNeTE1 2

)]([

Nn

ll

Nn

ll eTENneTEeTE

11

)]([*)]([)]([

e. edges allfor 21)/()

2()]([ NnnNeTE

22])([][

11

nkeTEHEk

ll

N

jij


121)(/6for

2)6Pr(

1

6

1

eHEn

nH

N

jij

nN

jij

1212

/6 enn

1211 e


The probability for package to wait in Phase 1 for more than 6n steps is less than 2-6n.

The probability for one of the packages to wait in Phase 1 for more than 6n steps is less than N * 2-6n =2n * 2-6n = 2-5n.

Add the length of the longest possible path and we get that the probability to finish Phase 1 in more than 7n steps is less than 2-5n.


Phase 2:

Phase 2 can be seen as reflection of Phase 1.

Paths of Phase 2 can intersect with paths of Phase 1.In order to prevent this we wait 7n steps to be sure all packets got to their intermediate destination, before starting Phase 2.


The probability for the process not to finish in 2 * 7n steps is less than 2 * 2-5n = 2-4n.

With probability at least 1 - (1/N), every packet reaches its destination in 14n or fewer steps.

Summary

The deterministic algorithm takes:

steps.

The randomized 14n steps.

The probability algorithm is exponentially better than the deterministic one.

)2()(nd

N n

routing in a parallel computer

Documents