1

Hi gh Pe rf orm a nceSwi tc hi ng and Routi ngTe lec om Ce nter W orks ho p: Sep t 4 , 19 97.

Performance Guarantees for Internet Routers

ISL Affiliates MeetingApril 4th 2002

Nick McKeownProfessor of Electrical Engineering and Computer Science, Stanford Universitynickm@stanford.eduwww.stanford.edu/~nickm

2

What a Router Looks LikeCisco GSR 12416 Juniper M160

6ft

19”

2ft

Capacity: 160Gb/sPower: 4.2kW

3ft

2.5ft

19”

Capacity: 80Gb/sPower: 2.6kW

3

Basic Architectural Components

of an IP Router

Control Plane

Datapathper-packet processing

SwitchingForwardingTable

Routing Table

Routing Protocols

4

Generic Router Architecture

LookupIP Address

UpdateHeader

Header ProcessingData Hdr Data Hdr

~1M prefixesOff-chip DRAM

AddressTable

IP Address Next Hop

QueuePacket

BufferMemory

~1M packetsOff-chip DRAM

5

Generic Router ArchitectureLookup

IP AddressUpdateHeader

Header Processing

AddressTable

LookupIP Address

UpdateHeader

Header Processing

AddressTable

LookupIP Address

UpdateHeader

Header Processing

AddressTable

BufferManager

BufferMemory

BufferManager

BufferMemory

BufferManager

BufferMemory

6

High Performance Networking Research Group

1. Adisak Mekkitikul: Crossbar scheduling algorithms that provide 100% throughput.

2. Pankaj Gupta: IP address lookup and classification algorithms.

3. Sundar Iyer: Parallel packet switches; High performance packet buffers; Distributed shared memory routers.

4. Isaac Keslassy, Shang-tse Chuang: Incorporating optics into routers.

5. Pablo Molinero Fernandez: The use of circuit switching in the Internet.

6. Nandita Dukkipati, Rui Zhang: Congestion control for short-lived flows.

7. Yashar Ganjali: Multipath routing.

Graduated

7

Performance metrics of routers

1. Capacity “maximize C, s.t. volume < 2m3 and power < 5kW”

2. Throughput Operators like to maximize usage of expensive long-

haul links. This would be trivial with work-conserving output-

queued routers3. Controllable Delay

Some users would like predictable delay. This is feasible with output-queueing plus weighted

fair queueing (WFQ).

WFQ( , ) ( , )

8

The Problem Output queued switches are impractical

R

R

RR

DRAM

NR NR

data

9

Memory BandwidthCommercial DRAM

1. It’s hard to keep up with Moore’s Law: The bottleneck is memory speed. Memory speed is not keeping up with Moore’s Law.

0.0001

0.001

0.01

0.1

1

10

100

10001980 1983 1986 1989 1992 1995 1998 2001

Acce

ss T

ime

(ns) DRAM

1.1x / 18months

Moore’s Law2x / 18 months

Router Capacity2.2x / 18months

Line Capacity2x / 7 months

10

Potted history1. [Karol et al. 1987] Throughput limited to by

head-of-line blocking for Bernoulli IID uniform traffic.

2. [Tamir 1989] Observed that with “Virtual Output Queues” (VOQs) Head-of-Line blocking is reduced and throughput goes up.

%5822

11

Potted history3. [Anderson et al. 1993] Observed analogy to maximum size

matching in a bipartite graph.

4. [M et al. 1995] (a) Maximum size match can not guarantee 100% throughput.(b) But maximum weight match can – O(N3).

5. [Mekkittikul and M 1998] A carefully picked maximum size match can give 100% throughput.

Matching

O(N2.5)

12

Potted history Speedup

5. [Chuang, Goel et al. 1997] Precise emulation of an output queued switch is possible with a speedup of two and a “stable marriage” scheduling algorithm.

6. [Prabhakar and Dai 2000] 100% throughput possible for maximal matching with a speedup of two.

13

Potted historyNewer approaches

7. [Tassiulas 1998] 100% throughput possible for simple randomized algorithm with memory.

8. [Giaccone et al. 2001] “Apsara” algorithms.

9. [Iyer and M 2000] Parallel switches can achieve 100% throughput and emulate an output queued switch.

10. [Chang et al. 2000] A 2-stage switch with a TDM scheduler can give 100% throughput.

11. [Iyer, Zhang and M 2002] Distributed shared memory switches can emulate an output queued switch.

14

Basic Switch Model

A1(n)

S(n)

N NLNN(n)

A1N(n)

A11(n)L11(n)

1 1

AN(n)

ANN(n)

AN1(n)

D1(n)

DN(n)

15

Some definitions

matrix. npermutatio a is and :where :matrix Service 2.

".admissible" is traffic the say we Ifwhere

:matrix Traffic 1.

SssS

nAE

ijij

jij

iij

ijijij

1,0],[

1,1

)]([:,

3. Queue occupancies:

Occupancy

L11(n) LNN(n)

16

Some definitions of throughput

( ) ,

. [ ( )] ,

[ ( )] ,

( )( )lim lim

1. Work conservation2. "100% throughput"3. 4 5.

6.

ij

ij

ij

ijijijn n

L n C n

E L n C

E L n

A nD nn n

When traffic is

admissible

17

Scheduling algorithms to achieve 100% throughput

1. When traffic is uniform (Many algorithms…)2. When traffic is non-uniform, but traffic matrix is known

• Technique: Birkhoff-von Neumann decomposition. [Chang ‘99]3. When matrix is not known.

• Technique: Lyapunov function. [M et al. ‘96]4. When algorithm is pipelined, or information is incomplete.

• Technique: Lyapunov function. [Keslassy & M ’01]5. When algorithm does not complete.

• Technique: Randomized algorithm. [Tassiulas ’00]6. When there is speedup.

• Technique: Fluid model. [Dai & Prabhakar ’00]7. When there is no algorithm.

• Technique: 2-stage load-balancing switch. [Chang ’01]• Technique: Parallel Packet Switch. [Iyer & M ’01]

18

When the traffic matrix is not known

( 1) ( ) ( ) 0

( ) ( ) ( ) ( ) | ( ) 0.

1. We would like to show that:

i.e.

2. I nstead we fi nd that if the scheduling algorithm is a maximumweight bipartite

ij

ij ijij ij

E V L n V L n | L n ,ij

E V L n S n A n V L n L n

( 1) ( 1) ( ) ( ) ( ) ( ) .

( ) ( ) ( ),

( )

matching, then

where: is our Lyapunov f unction.

3. Hence, if is large enough, there is an expected single-step downward drif t in

T T

T

E L n L n L n L n | L n c L n

V L n L n L n

L n

[ ( )] . occupancy, and so

E L n

19

Throughput resultsTheory:

Practice:

InputQueueing

(IQ)

InputQueueing

(IQ)

58% [Karol, 1987]

IQ + VOQ,Maximum weight matching

IQ + VOQ,Sub-maximal size matching

e.g. PIM, iSLIP.

100% [M et al., 1996]

Different weight functions,incomplete information, pipelining.

Randomized algorithms

100% [Tassiulas, 1998]

100% [Various]

Various heuristics, distributed algorithms,

and amounts of speedup

IQ + VOQ,Maximal size matching,

Speedup of two.100% [Dai & Prabhakar, 2000]