![Page 1: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/1.jpg)
Router - assisted Congestion Control
RED, ECN, and XCP
2007
![Page 2: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/2.jpg)
2
Outline
Active Queue Mgmt. (AQM) and
Random Early Detection (RED) Explicit congestion notification (ECN) eXplicit Control Protocol (XCP)
![Page 3: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/3.jpg)
3
Router Support For Congestion Management
Traditional Internet
- Congestion control mechanisms
at end-systems, mainly
implemented in TCP
- Routers play little role Traditional routers
- FIFO
- Tail drop You need both end-host congestion control and
router support for congestion control
- End-host congestion control to adapt
- Router congestion control to protect / isolate
![Page 4: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/4.jpg)
4
What Would Router Do?
Router mechanisms affecting congestion management
Congestion Signaling:- Drop, mark, - send explicit messages
Buffer management:- Which packets to drop?- When to signal congestion?
Scheduling- If multiple connections, which one’s
packets to send at any given time?- FQ does not eliminate congestion it just
manages the congestion
![Page 5: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/5.jpg)
5
Congestion Signaling
Drops (we’ve covered) In-band marking
- One bit (congested or not): ECN
- Multiple bits (how congested / how much available): XCP
Out-of-band notification
- IP Source Quench
• Problem: It sends more packets when things are congested…
• Not widely used.
![Page 6: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/6.jpg)
6
When to mark packets? Drop-tail / FIFO:
- When the buffer is full- The de-facto mechanism today- Very easy to implement- Keeps average queue length high
• ½ full delay Drawbacks of FIFO with Tail-drop
- Buffer lock out by misbehaving flows- Synchronizing effect for multiple TCP flows- Burst or multiple consecutive packet drops
• Bad for TCP fast recovery
- Note relation to FIFO: a scheduling discipline, NOT a drop policy, but they’re often bundled
![Page 7: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/7.jpg)
7
FIFO Router with Two TCP Sessions
![Page 8: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/8.jpg)
8
Active Queue Mgmt. w/RED Explicitly tries to keep queue small
- Low delay, but still high throughput under bursts
- (This is “power”: throughput / delay) Assumes that hosts respond to lost packets Technique:
- Randomization to avoid synchronization
• (Recall that if many flows, don’t need as much buffer space!)
- Drop before the queue is actually full
- RED is “Random Early Detection”
• Could mean marking, not dropping
![Page 9: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/9.jpg)
9
RED
FIFO scheduling Buffer management:
- Discard probability is computed as a function of average queue length
Discard Probability
AverageQueue Length
0
1
min_th max_th queue_len
![Page 10: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/10.jpg)
10
RED Router with Two TCP Sessions
![Page 11: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/11.jpg)
11
RED RED
- Advantages• Absorb burst better• Avoids synchronization• Signal end systems earlier
- Problems with RED• No protection: if a flow misbehaves it will hurt
the other flows
Fair Queueing- Advantages: protection among flows
• Misbehaving flows will not affect the performance of well-behaving flows
• FIFO does not have such a property
![Page 12: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/12.jpg)
12
Simulation Example
10 Mbps)
UDP (#1)
TCP (#2)
TCP (#32)
...
UDP (#1)
TCP (#2)
TCP (#32)
...0
1
2
3
4
5
6
7
8
9
10
1 4 7 10 13 16 19 22 25 28 31Flow Number
Th
rou
gh
pu
t(M
bp
s)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
1 4 7 10 13 16 19 22 25 28 31Flow Number
Th
rou
gh
pu
t(M
bp
s)
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
1 4 7 10 13 16 19 22 25 28 31Flow Number
Th
rou
gh
pu
t(M
bp
s)
Stateful solution: Fair Queueing
Our Solution: Core-Stateless Fair Queueing
Stateless solution: RandomEarly Detection
1 UDP (10 Mbps) and 31 TCPs sharing a 10 Mbps link
![Page 13: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/13.jpg)
13
RED parameter sensitivity RED can be very sensitive to parameters
- Tuning them is a bit of a black art! One thing: “gentle” RED
- max_p <= pb <= 1 as
- maxthresh <= qa <= 2*maxthresh
- instead of “cliff” effect. Makes RED more robust to choice of maxthresh, max_p
But note: Still must choose wq, minthresh… RED is not very widely deployed, but testing against
both RED and DropTail is very common in research, because it could be.
![Page 14: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/14.jpg)
14
ECN : Explicit Congestion Notification
Explicit Congestion Notification- In IP-land marking- Router sets bit for congestion instead of
dropping a packet Receiver should copy bit from packet to ack If bit set, react the same way as if it had been dropped
(but you don’t have to retransmit or risk losing ACK clocking)
- Sender reduces cwnd when it receives ack with marking
Where does it help?- Delay-sensitive apps, particularly low-bw ones- Small window scenarios
![Page 15: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/15.jpg)
15
ECN Some complexity:
- How to send in legacy IP packets (IP ToS field)- Determining ECN support: two bits (one “ECN
works”, one “congestion or not”- How to echo bits to sender (TCP header bit)
More complexity: Cheating!- Receiver can clear ECN bit- Solution: Multiple unmarked packet states
• Sender uses multiple unmarked packet states• Router sets ECN mark, clearing original
unmarked state• Receiver must either return ECN bit or guess
nonce• More nonce bits less likelihood of cheating (1
bit is sufficient)
![Page 16: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/16.jpg)
16
TCP Problems When TCP congestion control was originally
designed in 1988:
- Key applications: FTP, E-mail
- Maximum link bandwidth: 10Mb/s
- Users were mostly from academic and government organizations (i.e., well-behaved)
- Almost all links were wired (i.e., negligible error rate)
Thus, current problems with TCP:
- High bandwidth-delay product paths
- Wireless (or any high error links)
- Selfish users
![Page 17: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/17.jpg)
18
High Delay High Bandwidth Challenges: AIMD
TCP lacks fast response In AIMD, spare bandwidth is available TCP
increases cwnd increases by 1 packet/ RTT even if spare
bandwidth is huge Time to reach 100% utilization is proportional
to available bandwidth- e.g., 2 flows share a 10Gb/s link, one flow
finishes available bandwidth is 5Gb/s- e.g., 5Gb/s available, 200ms RTT, 1460B
payload 17,000s- If High Delay (RTT > 200 ms), need more
time
![Page 18: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/18.jpg)
19
High Delay High Bandwidth Challenges: Slow Start
TCP lacks fast response Short TCP flows (majority) cannot acquire the
spare bandwidth faster than “slow start” In slow start, window increases exponentially e.g., 10Gb/s, 200ms RTT, 1460B payload,
assume no loss- Time to fill pipe: 18 round trips = 3.6 s- Throughput: 382MB / 3.6s = 850Mb/s- 8.5% utilization not very good- If High Delay (RTT > 200 ms), ?
Loose only one packet drop out of slow start into AIMD by 1 pkt/RTT (even worse), taking forever to grab the large bandwidth
![Page 19: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/19.jpg)
20
TCP congestion control performs poorly as bandwidth or delay increases
Round Trip Delay (sec)
Avg
. T
CP
Uti
l iza
tio
n
Bottleneck Bandwidth (Mb/s)
Avg
. T
CP
Uti
l iza
tio
n
• Inefficient as bandwidth or delay increases [Low02]
50 flows in both directions
Buffer = BW x DelayRTT = 80 ms
50 flows in both directions
Buffer = BW x DelayBW = 155 Mb/s
![Page 20: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/20.jpg)
21
Solution: Decouple Congestion Control from Fairness
Example: In TCP, Additive-Increase Multiplicative-Decrease (AIMD) controls both
Coupled because a single mechanism controls both
How does decoupling solve the problem?
1. To control congestion: use MIMD which shows fast response
2. To control fairness: use AIMD which converges to fairness
Congestion Control: High Utilization; Small Queues; Few Drops Fairness: Bandwidth Allocation Policy
![Page 21: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/21.jpg)
22
Efficient link utilization needs expressive feedback
In coupled systems, expressive feedback led to per-flow state (Unscalable!)
Efficiency Problem:
Solution: Use Decoupling• Decoupling looks at efficiency as a problem
about aggregate traffic
• Match aggregate traffic to link capacity and drain the queue
• Benefits: No need for per-flow information
Why Current Approaches Don’t Use Expressive Feedback?
![Page 22: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/22.jpg)
23
Router computes a flow’s fair rate explicitlyTo make a decision, router needs state of all flows
Unscalable
Shuffle bandwidth in aggregate to converge to fair rates
To make a decision, router needs state of this flow
Put a flow’s state in its packets
Scalable
Fairness Control
![Page 23: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/23.jpg)
24
XCP: An eXplicit Control Protocol
1. Congestion Controller2. Fairness Controller
![Page 24: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/24.jpg)
25
Feedback
Round Trip Time
Congestion Window
Congestion Header: RTT and congestion window are filled in by the sender and never modified in transit. Feedback is initialized by the sender. Routers along the path modify this field.
Feedback
Round Trip Time
Congestion Window
How does XCP Work?
Feedback = + 0.1 packet
![Page 25: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/25.jpg)
26
Feedback = + 0.1 packet
Round Trip Time
Congestion Window
Feedback = - 0.3 packet
How does XCP Work?
![Page 26: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/26.jpg)
27
Congestion Window = Congestion Window + Feedback
Routers compute feedback without any per-flow state at Router (using flow’s state ‘RTT and congestion window’ in Congestion Header)
Routers compute feedback without any per-flow state at Router (using flow’s state ‘RTT and congestion window’ in Congestion Header)
How does XCP Work?
XCP extends ECN and CSFQ
![Page 27: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/27.jpg)
28
Algorithm:
Aggregate traffic changes by ~ Spare Bandwidth ~ - Queue Size
So, = davg Spare - Queue
How Does an XCP Router Compute the Feedback (decoupling)?
Congestion Controller Fairness ControllerGoal: Divides between flows to converge to fairness
Looks at a flow’s state (RTT and congestion window) in Congestion Header
Algorithm:
If > 0 Divide equally between flows
If < 0 Divide between flows proportionally to their current rates
MIMD
AIMD
Goal: Matches input traffic to link capacity & drains the queue Aggregate feedback Looks at aggregate traffic & queue
![Page 28: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/28.jpg)
29
= davg Spare - Queue
224
0 2 and
Theorem: System converges to optimal utilization (i.e., stable) for any link bandwidth, delay, number of sources if:
(Proof based on Nyquist Criterion)
Details
Congestion Controller Fairness Controller
No Parameter Tuning
Algorithm:If > 0 Divide equally between flowsIf < 0 Divide between flows proportionally to their current rates
Need to estimate number of flows N
Tinpkts pktpkt RTTCwndTN
)/(1
RTTpkt : Round Trip Time in header
Cwndpkt : Congestion Window in header
T: Counting Interval
No Per-Flow State at Router
![Page 29: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/29.jpg)
34
Characteristics of Solution
1. Improved Congestion Control (in high bandwidth-delay & conventional environments):
• Small queues
• Almost no drops
2. Improved Fairness
3. Scalable (no per-flow state)
4. Flexible bandwidth allocation: min-max fairness, proportional fairness, differential bandwidth allocation,…
![Page 30: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/30.jpg)
35
Simulations Show XCP is Better
Extensive Simulations Compared with TCP over DropTail, RED,
REM, AVQ, CSFQ
XCP:• Better utilization• Near-zero drops• Fairer• Efficient & robust to increase in bandwidth • Efficient & robust to increase in delay
![Page 31: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/31.jpg)
36
Bottleneck
S1
S2
R1, R2, …, Rn
Sn
Subset of Results
Similar behavior over:
![Page 32: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/32.jpg)
37
XCP Remains Efficient as Bandwidth or Delay Increases
Bottleneck Bandwidth (Mb/s)
Utilization as a function of Bandwidth
Uti
liza
tion
Round Trip Delay (sec)
Uti
liza
tion
Utilization as a function of Delay
![Page 33: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/33.jpg)
38
XCP Shows Faster Response than TCP
XCP shows fast response
Start 40 Flows
Start 40 Flows
Stop the 40 Flows
Stop the 40 Flows
![Page 34: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/34.jpg)
39
XCP Deals Well with Short Web-Like Flows
Arrivals of Short Flows/sec
Average Utilization
Average Queue
Drops
![Page 35: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/35.jpg)
40
XCP is Fairer than TCP
Flow ID(all RTT = 40 ms )
Different RTTSame RTT
Av
g.
Th
rou
gh
pu
t
Flow ID
Av
g.
Th
rou
gh
pu
t
(RTT is 40 ms 330 ms )
![Page 36: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/36.jpg)
41
XCP Summary XCP
- Outperforms TCP
- Efficient for any bandwidth
- Efficient for any delay
- Scalable (no per flow state) Benefits of Decoupling
- Use MIMD for congestion control which can grab/release large bandwidth quickly
- Use AIMD for fairness which converges to fair bandwidth allocation
![Page 37: Router - assisted Congestion Control RED, ECN, and XCP](https://reader035.vdocuments.site/reader035/viewer/2022062321/56813221550346895d987f30/html5/thumbnails/37.jpg)
42
XCP benefits & issues
Requires “policers” at edge if you don’t trust hosts to report cwnd/rtt correctly
- Much like CSFQ…
Doesn’t provide much benefit in today’s common case
- But may be very significant for tomorrow’s.
- High bw*rtt environments (10GigE coming to a desktop near you…)
- Short flows, highly dynamic workloads
Cool insight: Decoupled fairness and congestion control
Pretty big architectural change VCP ?