![Page 1: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/1.jpg)
EE 382C - S11 - Lecture 6
1
EE382C
Lecture 6
Adaptive Routing
4/14/11
![Page 2: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/2.jpg)
EE 382C - S11 - Lecture 6
2
Question of the day
What is the worst-case traffic pattern for CQR routing on a
torus?
![Page 3: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/3.jpg)
EE 382C - S11 - Lecture 6
3
Review of Routing Basics
• Deterministic, Oblivious, or Adaptive Routing
• Deterministic – simple, maintains ordering, only choice if no
path diversity
• Oblivious
– VAL – achieves optimum throughput on worst-case traffic
• But destroys locality, same throughput on all traffic
– Linearity lets us find worst-case traffic pattern for arbitrary oblivious
algorithm
• Bipartite maximal-weight matching problem
![Page 4: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/4.jpg)
EE 382C - S11 - Lecture 6
4
Report Card from Last Time
Algorithm Locality Uniform traffic Tornado traffic
Greedy A 1 (A) 0.33 (F)
Random F 0.5 (F) 0.4 (C)
RLB C 0.76 (C) 0.53 (A)
VAL F 0.5 (F) 0.5 (A-)
![Page 5: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/5.jpg)
EE 382C - S11 - Lecture 6
5
Topology & Routing
![Page 6: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/6.jpg)
EE 382C - S11 - Lecture 6
6
Adaptive routing
• Path from source A to destination B based on network
conditions
• Advantages
– Can provide low latency at low loads and good load balancing at
high loads
• Disadvantages
– Making good routing decisions with approximation information can
be difficult
– Packet reordering
![Page 7: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/7.jpg)
EE 382C - S11 - Lecture 6
7
Minimal adaptive routing
• Two phases– Route “up” adaptively
towards a random top level node
– Route “down” deterministically to destination
• “Turn around” at first common ancestor– Keeps route minimal
![Page 8: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/8.jpg)
EE 382C - S11 - Lecture 6
8
How is minimal adaptive on a Clos
different than minimal Oblivious?
![Page 9: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/9.jpg)
EE 382C - S11 - Lecture 6
9
Transient Imbalance
0
2
4
6
8
10
12
14
16
Routers in the middle stage of the network
Ma
xim
um
bu
ffe
r siz
e
![Page 10: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/10.jpg)
EE 382C - S11 - Lecture 6
10
With Adaptive Routing
0
2
4
6
8
10
12
14
16
Routers in the middle stage of the network
Ma
xim
um
bu
ffe
r siz
e
![Page 11: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/11.jpg)
EE 382C - S11 - Lecture 6
11
Minimal adaptive routing
• Minimal Adaptive (MIN AD) [Gravano ’93]
– Always route in minimal quadrant
– Adaptively routing in each hop
• Optimal ben; Poor adv
![Page 12: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/12.jpg)
EE 382C - S11 - Lecture 6
12
Minimal adaptive drawback
10 20 3000
13 23 3303
12 22 3202
11 21 3101
![Page 13: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/13.jpg)
EE 382C - S11 - Lecture 6
13
Fully adaptive routing
10 20 3000
13 23 3303
12 22 3202
11 21 3101
• Any issues with misrouting?
![Page 14: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/14.jpg)
EE 382C - S11 - Lecture 6
14
Livelock issues
10 20 3000
13 23 3303
12 22 3202
11 21 3101
![Page 15: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/15.jpg)
EE 382C - S11 - Lecture 6
15
Global Adaptive Routing
• Global - Make overall routing decision (e.g., which quadrant
to traverse in a torus).
• Adaptive - Based on network state
• Need to estimate network state at packet source.
• Need a method to partition routes - so we can choose
between them based on network state.
![Page 16: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/16.jpg)
EE 382C - S11 - Lecture 6
16
Reconsider an 8-node Ring
1 2 3 4 5 6 70
Two partitions: clockwise and counter-clockwise
Congested clockwise channels
Sender Receiver
Sender cannot sense congestion locally
![Page 17: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/17.jpg)
EE 382C - S11 - Lecture 6
17
Channel Queue Routing CQR
3. There are 4 quadrants with hop
counts H1, H2, H3, H4
2. Compute quadrant congestion
for each quadrant - q1, q2, q3, q4
4. Choose quadrant i
corresponding to the smallest Hiqi
5. Route minimally adaptively
within chosen quadrant
1. Check queue lengths for
outgoing channels at source
![Page 18: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/18.jpg)
EE 382C - S11 - Lecture 6
18
CQR: TOR Throughput
0
0.1
0.2
0.3
0.4
0.5
0.6
0 0.1 0.2 0.3 0.4 0.5 0.6
Offered Load (fraction of capacity)
Accepte
d T
hro
ughput
Minimal
Non-minimal
Overall
Switches to non-
minimal at 0.12
![Page 19: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/19.jpg)
EE 382C - S11 - Lecture 6
19
CQR: TOR Latency
0
5
10
15
20
25
30
35
40
45
50
0 0.1 0.2 0.3 0.4 0.5 0.6
Offered Load
Avera
ge L
ate
ncy (
Cycle
s)
MinimalNon-MinimalOverall
![Page 20: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/20.jpg)
EE 382C - S11 - Lecture 6
20
Report card
Algo Θ benign
UR traffic
Θ adv
traffic
Θ avg Tlow (0.2
UR)
Tint (0.4
TOR)
VAL 0.5 (F) 0.5 (A) 0.5 (C) 10.5 (F) 20.5 (B)
DOR 1.0 (A) 0.25 (F) 0.31 (F) 4.5 (A) (F)
RLB 0.76 (C) 0.31 (C) 0.51 (C) 6.5 (C) 5.5 (A)
MIN AD 1.0 (A) 0.33 (C) 0.63 (B) 4.5 (A) (F)
GOAL 0.76 (C) 0.5 (A) 0.68 (A) 5.5 (C) 5.5 (A)
CQR 1.0 (A) 0.5 (A) 0.73 (A) 4.5 (A) 5.5 (A)
Throughput (frac of capacity) Latency (Cycles)
![Page 21: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/21.jpg)
EE 382C - S11 - Lecture 6
21
Routing Mechanics
![Page 22: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/22.jpg)
EE 382C - S11 - Lecture 6
22
Routing Mechanics
• The mechanism used to implement any routing algorithm:
deterministic, oblivious, or adaptive
• Table-based routing
– Source routing
– Node table routing
• Algorithmic routing
![Page 23: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/23.jpg)
EE 382C - S11 - Lecture 6
23
Source table routing
00 01 02
10 11 12
03
13
Advantages:
•Very low routing delay
•Routers can be used in any
topology.
Disadvantages:
•Can’t be used for adaptive
routing.
•Packet overhead
•Any others?
![Page 24: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/24.jpg)
EE 382C - S11 - Lecture 6
24
Building routing tables
• There may be several routes for each destination– These may be selected using a pseudo random generator as a
selector
– For ordering, hash using a flow identifier
• Having multiple routes help in– Fault tolerance
• If routes are edge disjoint
– Load balance
• Distribute load over several network paths
– Distribution
• When destination is a logical destination to a “service” rather than a physical destination node. In this case, the multiple routes may in fact lead to multiple destinations
![Page 25: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/25.jpg)
EE 382C - S11 - Lecture 6
25
Encoding variable length header
N N E N
E N N W
N W N N
- - X N
(a) At start of route
N N E
E N N W
N W N N
- - X N
P
(b) After first hop
- - F
E N N W
N W N N
- - X N
-
(c) After four hops
- - F
- - P E
N W N N
- - X N
-
(d) After seven hops
N W N N
- - X N
(e) After eight hops
- - X N
(f) After twelve hops
- - F-
Each line is a phit. Each rectangle is a flit. Read from right to left
![Page 26: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/26.jpg)
EE 382C - S11 - Lecture 6
26
Impl variable length src table
Dest Route 1 Route 2
E X E X W W X N E N E X N N X
• Avoids the storage waste associated with rounding
up all routes to a maximum length
![Page 27: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/27.jpg)
EE 382C - S11 - Lecture 6
27
Node table routing
- Livelock avoidance (00 to 11 takes S
(10) then N)
- Latency at each hop
+ Local adaptivity (bolded letters are
misroutes)
+ Space saving 00 01 02
10 11 12
03
13
![Page 28: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/28.jpg)
EE 382C - S11 - Lecture 6
28
Hierarchical Node Table
Table for node --- of an 8 ary 3 cube
![Page 29: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/29.jpg)
EE 382C - S11 - Lecture 6
29
Algorithmic routing in a torus
sx syx y
= 0
routing header
= 0
-y+y
-x+x
exitproductive
direction
vector
route selection
-y+y
-x+x
exitselected
direction
vector
queue lengths
ydone
xdone
•?Choose first productive
dimension from left to right
•?Randomly select a productive
dimension
•?Select a productive dimension
based on queue lengths
•?Allow selection of
unproductive dimensions if
productive dimension queue
lengths are above threshold
• CHOICES:
Min OB, Min Ad, Full Ad, DOR
![Page 30: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/30.jpg)
EE 382C - S11 - Lecture 6
30
Question of the day
What is the worst-case traffic pattern for CQR routing on a
torus?
![Page 31: EE382C Lecture 6 - Stanford Universitycva.stanford.edu/classes/ee382c/lectures/L6.pdf · EE 382C - S11 - Lecture 6 6 Adaptive routing • Path from source A to destination B based](https://reader033.vdocuments.site/reader033/viewer/2022050520/5fa3f5d9efc6f36b113a3f70/html5/thumbnails/31.jpg)
EE 382C - S11 - Lecture 6
31
Summary
• Adaptive routing
– Use network state
– Is non-linear
– Minimal adaptive - make local decisions
– Non-minimal
– Global adaptive (example: CQR)
• Sensing global state
• Partitioning routes
• Routing mechanics
– Tables
• Source, Node, Associative
– Algorithmic