Fair queueing and congestion controlJim Roberts (France Telecom)

Joint work with Jordan Augé

Workshop on Congestion ControlHamilton Institute, Sept 2005

Fairness and congestion control

s fair sharing: an objective as old as congestion controlQcf. RFC 970, Nagle, 1985

Qnon-reliance on user cooperation

Qpainless introduction of new transport protocols

Qimplicit service differentiation

s fair queueing is scalable and feasibleQaccounting for the stochastics of traffic

Qa small number of flows to be scheduled

Qindependent of link speed

s performance evaluation of congestion controlQmust account for realistic traffic mix

Qimpact of buffer size, TCP version, scheduling algorithm

Flow level characterization of Internet traffic

s traffic is composed of flowsQan instance of some application

Q(same identifier, minimum packet spacing)

s flows are "streaming" or "elastic"Qstreaming SLS = "conserve the signal"

Qelastic SLS = "transfer as fast as possible"

inter-packet < T silence > T

start end

streaming elastic

Characteristics of flows

s arrival processQPoisson session arrivals: succession of flows and think times

s size/durationQheavy tailed, correlation

flowarrivals

start of session

end ofsession

think times

Characteristics of flows

s arrival processQPoisson session arrivals: succession of flows and think times

s size/durationQheavy tailed, correlation

s flow peak rateQstreaming: rate codec

Qelastic: rate exogenous limits (access link,...)

rate

duration

rate

duration

Three link operating regimes

needscheduling

excellent for elastic,some streaming loss

needoverload control

low throughput,significant loss

FIFOsufficient

negligible loss and delay

overallrate

"transparent" "congested""elastic"

flows

Performance of fair sharing without rate limit (ie, all flows bottlenecked)

s a fluid simulation:QPoisson flow arrivals

Qno exogenous peak rate limit flows are all bottlenecked

Qload = 0.5 (arrival rate x size / capacity)

high

low

averagerate

start end

20 seconds

linkrate

incoming flows

The process of flows in progress depends on link load

load 0.5

high

low

10

0

20

30flows in progress

The process of flows in progress depends on link load

10

0

20

30flows in progress

load 0.6

high

low

The process of flows in progress depends on link load

10

0

20

30flows in progress

load 0.7

high

low

The process of flows in progress depends on link load

10

0

20

30flows in progress

load 0.8

high

low

The process of flows in progress depends on link load

10

0

20

30flows in progress

load 0.9

high

low

Insensitivity of processor sharing: a miracle of queuing theory !s link sharing behaves like M/M/1

Qassuming only Poisson session arrivals

s if flows are bottlenecked, E [flows in progress] = Qi.e., average 9 for 0.9, but as 1

s but, in practice, < 0.5 and E [flows in progress] = O(104) !

0 .2 .4 .6 .8 load,

E [flows in progress]

0

5

10

15

20

Trace data

s an Abilene link (Indianapolis-Clevelend) – from NLANRQOC 48, utilization 16 %

Qflow rates (10 Kb/s, 10 Mb/s)

Q~7000 flows in progress at any time

10 sec

2.5 Gb/s

>2.5 Mb/s

< 250 Kb/s

Most flows are non-bottlenecked

s each flow emits packets rarely

s little queueing at low loadsQFIFO is adequate

Qperformance like a modulated M/G/1

s at higher loads, a mix of bottlenecked and non-bottlenecked flows...

~5 µs2.5 Gb/s

~7000 flows

~1 ms15 Mb/s

Fair queueing is scalable and feasible

s fair queueing deals only with flows having packets in queueQ<100 bottlenecked flows (at load < 90%)

QO(100) packets from non-bottlenecked flows (at load < 90%)

s scalable since number does not increase with link rateQdepends just on bottlenecked/non-bottlenecked mix

s feasible since max number is ~500 (at load < 90%)Qdemonstration by trace simulations and analysis (Sigmetrics 2005)

s can use any FQ algorithmQDRR, Self-clocked FQ,...

Qor even just RR ?

Typical flow mix

s many non-bottlenecked flows (~104)Qrate limited by access links, etc.

s a small number of bottlenecked flows (0, 1, 2,...)QPr [ i flows] ~ i with the relative load of bottlenecked flows

s exampleQ 50% background traffic

–ie, E[flow arrival rate] x E[flow size] / capacity = 0.5

Q0, 1, 2 or 4 bottlenecked TCP flows–eg, at overall load = 0.6, Pr [ 5 flows] 0.003

Simulation set up (ns2)

s one 50 Mbps bottleneckQRTT = 100ms

s 25 Mbps background trafficQPoisson flows: 1 Mbps peak rate

Qor Poisson packets (for simplicity)

s 1, 2 or 4 permanent high rate flowsQTCP Reno or HSTCP

s buffer sizeQ20, 100 or 625 packets (625 = b/w x RTT)

s schedulingQFIFO, drop tail

QFQ, drop from front of longest queue

Results:- 1 bottlenecked flow,- Poisson flow background

FIFO + Reno20 packets 625 packets

1000

1

100s 100s

cwnd(pkts)

utilization

0

0

FIFO + Reno1000

1

100s 100s

cwnd(pkts)

utilization

20 packets 100 packets

0

0

Severe throughput loss with small buffer: - realizes only 40% of available capacity

FIFO + 100 packet buffer1000

1

100s 100s

cwnd(pkts)

utilization

0

0

Reno HSTCP

HSTCP brings gain in utilization, higher loss for background flows

Reno + 20 packet buffer1000

1

100s 100s

cwnd(pkts)

utilization

0

0

FIFO FQ

FQ avoids background flow loss,little impact on bottlenecked flow

Results:- 2 bottlenecked flows,- Poisson packets background

FIFO + Reno + Reno1000

1

100s 100s

cwnd(pkts)

utilization

0

0

20 packets 625 packets

Approximate fairness with Reno

FIFO + HSTCP + HSTCP1000

1

100s 100s

cwnd(pkts)

utilization

0

0

20 packets 625 packets

FIFO + HSTCP + Reno1000

1

100s 100s

cwnd(pkts)

utilization

0

0

20 packets 625 packets

HSTCP is very unfair

Reno + HSTCP + 20 packet buffer1000

1

100s 100s

cwnd(pkts)

utilization

0

0

FIFO FQ

Reno + HSTCP + 625 packet buffer1000

1

100s 100s

cwnd(pkts)

utilization

0

0

FIFO FQ

Fair queueing is effective (though HSTCP gains more throughput)

Results:- 4 bottlenecked flows,- Poisson packet background

All Reno + 20 packet buffer1000

1

100s 100s

cwnd(pkts)

utilization

0

0

1 flow 4 flows

Improved utilization with 4 bottlenecked flows,approximate fairness

All Reno + 625 packet buffer1000

1

100s 100s

cwnd(pkts)

utilization

0

0

1 flow 4 flows

Approximate fairness

All HSTCP + 625 packet buffer1000

1

100s 100s

cwnd(pkts)

utilization

0

0

1 flow 4 flows

Poor fairness, loss of throughput

All HSTCP + 625 packet buffer1000

1

100s 100s

cwnd(pkts)

utilization

0

0

FIFO FQ

Fair queueing restores fairness, preserves throughput

Conclusions

s there is a typical traffic mixQsmall number of bottlenecked flows (0, 1, 2,...)

Qlarge number of non-bottlenecked flows

s fair queueing is feasibleQO(100) flows to schedule for any link rate

s results for 1 bottlenecked flow + 50% backgroundQsevere throughput loss for small buffer

QFQ avoids loss and delay for background packets

s results for 2 or 4 bottlenecked flows + 50% backgroundQReno approximately fair

QHSTCP very unfair, loss of utilization

QFQ ensures fairness for any transport protocol

s alternative transport protocols ?