ee382c lecture 6 - stanford universitycva.stanford.edu/classes/ee382c/lectures/l6.pdf · ee 382c -...

EE 382C - S11 - Lecture 6

1

EE382C

Lecture 6

Adaptive Routing

4/14/11

EE 382C - S11 - Lecture 6

2

Question of the day

What is the worst-case traffic pattern for CQR routing on a

torus?

EE 382C - S11 - Lecture 6

3

Review of Routing Basics

• Deterministic, Oblivious, or Adaptive Routing

• Deterministic – simple, maintains ordering, only choice if no

path diversity

• Oblivious

– VAL – achieves optimum throughput on worst-case traffic

• But destroys locality, same throughput on all traffic

– Linearity lets us find worst-case traffic pattern for arbitrary oblivious

algorithm

• Bipartite maximal-weight matching problem

EE 382C - S11 - Lecture 6

4

Report Card from Last Time

Algorithm Locality Uniform traffic Tornado traffic

Greedy A 1 (A) 0.33 (F)

Random F 0.5 (F) 0.4 (C)

RLB C 0.76 (C) 0.53 (A)

VAL F 0.5 (F) 0.5 (A-)

EE 382C - S11 - Lecture 6

5

Topology & Routing

EE 382C - S11 - Lecture 6

6

Adaptive routing

• Path from source A to destination B based on network

conditions

• Advantages

– Can provide low latency at low loads and good load balancing at

high loads

• Disadvantages

– Making good routing decisions with approximation information can

be difficult

– Packet reordering

EE 382C - S11 - Lecture 6

7

Minimal adaptive routing

• Two phases– Route “up” adaptively

towards a random top level node

– Route “down” deterministically to destination

• “Turn around” at first common ancestor– Keeps route minimal

EE 382C - S11 - Lecture 6

8

How is minimal adaptive on a Clos

different than minimal Oblivious?

EE 382C - S11 - Lecture 6

9

Transient Imbalance

0

2

4

6

8

10

12

14

16

Routers in the middle stage of the network

Ma

xim

um

bu

ffe

r siz

e

EE 382C - S11 - Lecture 6

10

With Adaptive Routing

0

2

4

6

8

10

12

14

16

Routers in the middle stage of the network

Ma

xim

um

bu

ffe

r siz

e

EE 382C - S11 - Lecture 6

11

Minimal adaptive routing

• Minimal Adaptive (MIN AD) [Gravano ’93]

– Always route in minimal quadrant

– Adaptively routing in each hop

• Optimal ben; Poor adv

EE 382C - S11 - Lecture 6

12

Minimal adaptive drawback

10 20 3000

13 23 3303

12 22 3202

11 21 3101

EE 382C - S11 - Lecture 6

13

Fully adaptive routing

10 20 3000

13 23 3303

12 22 3202

11 21 3101

• Any issues with misrouting?

EE 382C - S11 - Lecture 6

14

Livelock issues

10 20 3000

13 23 3303

12 22 3202

11 21 3101

EE 382C - S11 - Lecture 6

15

Global Adaptive Routing

• Global - Make overall routing decision (e.g., which quadrant

to traverse in a torus).

• Adaptive - Based on network state

• Need to estimate network state at packet source.

• Need a method to partition routes - so we can choose

between them based on network state.

EE 382C - S11 - Lecture 6

16

Reconsider an 8-node Ring

1 2 3 4 5 6 70

Two partitions: clockwise and counter-clockwise

Congested clockwise channels

Sender Receiver

Sender cannot sense congestion locally

EE 382C - S11 - Lecture 6

17

Channel Queue Routing CQR

3. There are 4 quadrants with hop

counts H1, H2, H3, H4

2. Compute quadrant congestion

for each quadrant - q1, q2, q3, q4

4. Choose quadrant i

corresponding to the smallest Hiqi

5. Route minimally adaptively

within chosen quadrant

1. Check queue lengths for

outgoing channels at source

EE 382C - S11 - Lecture 6

18

CQR: TOR Throughput

0

0.1

0.2

0.3

0.4

0.5

0.6

0 0.1 0.2 0.3 0.4 0.5 0.6

Offered Load (fraction of capacity)

Accepte

d T

hro

ughput

Minimal

Non-minimal

Overall

Switches to non-

minimal at 0.12

EE 382C - S11 - Lecture 6

19

CQR: TOR Latency

0

5

10

15

20

25

30

35

40

45

50

0 0.1 0.2 0.3 0.4 0.5 0.6

Offered Load

Avera

ge L

ate

ncy (

Cycle

s)

MinimalNon-MinimalOverall

EE 382C - S11 - Lecture 6

20

Report card

Algo Θ benign

UR traffic

Θ adv

traffic

Θ avg Tlow (0.2

UR)

Tint (0.4

TOR)

VAL 0.5 (F) 0.5 (A) 0.5 (C) 10.5 (F) 20.5 (B)

DOR 1.0 (A) 0.25 (F) 0.31 (F) 4.5 (A) (F)

RLB 0.76 (C) 0.31 (C) 0.51 (C) 6.5 (C) 5.5 (A)

MIN AD 1.0 (A) 0.33 (C) 0.63 (B) 4.5 (A) (F)

GOAL 0.76 (C) 0.5 (A) 0.68 (A) 5.5 (C) 5.5 (A)

CQR 1.0 (A) 0.5 (A) 0.73 (A) 4.5 (A) 5.5 (A)

Throughput (frac of capacity) Latency (Cycles)

EE 382C - S11 - Lecture 6

21

Routing Mechanics

EE 382C - S11 - Lecture 6

22

Routing Mechanics

• The mechanism used to implement any routing algorithm:

deterministic, oblivious, or adaptive

• Table-based routing

– Source routing

– Node table routing

• Algorithmic routing

EE 382C - S11 - Lecture 6

23

Source table routing

00 01 02

10 11 12

03

13

Advantages:

•Very low routing delay

•Routers can be used in any

topology.

Disadvantages:

•Can’t be used for adaptive

routing.

•Packet overhead

•Any others?

EE 382C - S11 - Lecture 6

24

Building routing tables

• There may be several routes for each destination– These may be selected using a pseudo random generator as a

selector

– For ordering, hash using a flow identifier

• Having multiple routes help in– Fault tolerance

• If routes are edge disjoint

– Load balance

• Distribute load over several network paths

– Distribution

• When destination is a logical destination to a “service” rather than a physical destination node. In this case, the multiple routes may in fact lead to multiple destinations

EE 382C - S11 - Lecture 6

25

Encoding variable length header

N N E N

E N N W

N W N N

- - X N

(a) At start of route

N N E

E N N W

N W N N

- - X N

P

(b) After first hop

- - F

E N N W

N W N N

- - X N

-

(c) After four hops

- - F

- - P E

N W N N

- - X N

-

(d) After seven hops

N W N N

- - X N

(e) After eight hops

- - X N

(f) After twelve hops

- - F-

Each line is a phit. Each rectangle is a flit. Read from right to left

EE 382C - S11 - Lecture 6

26

Impl variable length src table

Dest Route 1 Route 2

E X E X W W X N E N E X N N X

• Avoids the storage waste associated with rounding

up all routes to a maximum length

EE 382C - S11 - Lecture 6

27

Node table routing

- Livelock avoidance (00 to 11 takes S

(10) then N)

- Latency at each hop

+ Local adaptivity (bolded letters are

misroutes)

+ Space saving 00 01 02

10 11 12

03

13

EE 382C - S11 - Lecture 6

28

Hierarchical Node Table

Table for node --- of an 8 ary 3 cube

EE 382C - S11 - Lecture 6

29

Algorithmic routing in a torus

sx syx y

= 0

routing header

= 0

-y+y

-x+x

exitproductive

direction

vector

route selection

-y+y

-x+x

exitselected

direction

vector

queue lengths

ydone

xdone

•?Choose first productive

dimension from left to right

•?Randomly select a productive

dimension

•?Select a productive dimension

based on queue lengths

•?Allow selection of

unproductive dimensions if

productive dimension queue

lengths are above threshold

• CHOICES:

Min OB, Min Ad, Full Ad, DOR

EE 382C - S11 - Lecture 6

30

Question of the day

What is the worst-case traffic pattern for CQR routing on a

torus?

EE 382C - S11 - Lecture 6

31

Summary

• Adaptive routing

– Use network state

– Is non-linear

– Minimal adaptive - make local decisions

– Non-minimal

– Global adaptive (example: CQR)

• Sensing global state

• Partitioning routes

• Routing mechanics

– Tables

• Source, Node, Associative

– Algorithmic