copa: practical delay-based congestion control...copa gets higher throughput cubic flows are not...
TRANSCRIPT
Copa: Practical Delay-Based Congestion Control
Venkat Arun and Hari Balakrishnan
MIT, CSAILweb.mit.edu/copa
The Internet is more challenging than ever(Why are we still talking about congestion control in 2018?)
The Internet is more challenging than ever
Greater bandwidth ⇒Higher bandwidth-delay product
Lower tolerance for non-congestive loss
Greater flow-churn
(Why are we still talking about congestion control in 2018?)
The Internet is more challenging than ever
Greater bandwidth ⇒
Large flows (e.g. video streaming) co-exist with short-flows
Higher bandwidth-delay product
Lower tolerance for non-congestive loss
Greater flow-churn
(Why are we still talking about congestion control in 2018?)
The Internet is more challenging than ever
Greater bandwidth ⇒
Large flows (e.g. video streaming) co-exist with short-flows
Wireless links with variable bandwidths are commonplace
Higher bandwidth-delay product
Lower tolerance for non-congestive loss
Greater flow-churn
(Why are we still talking about congestion control in 2018?)
The Internet is more challenging than ever
Greater bandwidth ⇒
Large flows (e.g. video streaming) co-exist with short-flows
Wireless links with variable bandwidths are commonplace
Higher bandwidth-delay product
Lower tolerance for non-congestive loss
Greater flow-churn
Simultaneously, users are more sensitive to performance!
(Why are we still talking about congestion control in 2018?)
Loss-based schemes have long-standing problems
• Buffer-filling
• Vulnerable to non-congestive loss
• Loss is a coarse signalWorsens with increasing bandwidth
Delay-based congestion control?
Benefits Challenges
Delay-based congestion control?
Maintain low delay Not competitive with buffer-filling schemes
Benefits Challenges
Delay-based congestion control?
Maintain low delay
Robust to misleading loss
Not competitive with buffer-filling schemes
Delay can mislead too!
Benefits Challenges
Delay-based congestion control?
Maintain low delay
Robust to misleading loss
Not competitive with buffer-filling schemes
Delay can mislead too!
Benefits Challenges
Rich signal
Delay-based congestion control?
Maintain low delay
Robust to misleading loss
Not competitive with buffer-filling schemes
Delay can mislead too!
Finding true minimum RTTis hard
Benefits Challenges
Rich signal
Delay-based congestion control?
Not competitive with buffer-filling schemes
Delay is noisy too!
Finding true minimum RTTis hard
Our SolutionChallenges
Delay-based congestion control?
If buffer-fillers are present, give up on low delay
Not competitive with buffer-filling schemes
Delay is noisy too!
Finding true minimum RTTis hard
Our SolutionChallenges
Delay-based congestion control?
If buffer-fillers are present, give up on low delay
MIN: a more robust statistic for queuing delay
Not competitive with buffer-filling schemes
Delay is noisy too!
Finding true minimum RTTis hard
Our SolutionChallenges
Delay-based congestion control?
If buffer-fillers are present, give up on low delay
MIN: a more robust statistic for queuing delay
Not competitive with buffer-filling schemes
Delay is noisy too!
Finding true minimum RTTis hard
Our SolutionChallenges
Empty queues periodically
Basic Goals
Avoid congestion collapse+
Efficient and Fair allocation of bandwidth+
Low delay
Target rate = r( = )*+,-
Adjustable Parameterdefault = 0.5
Queuing delay
Target Rate ≡ Nash Equilibrium
"#$%$#&' = log #,-# − /'log(12)Assuming Poisson arrivals (more details in paper)
Selfishly optimize for:
Target Rate ≡ Nash Equilibrium
"#$%$#&' = log #,-# − /'log(12)Assuming Poisson arrivals (more details in paper)
Selfishly optimize for:
Unique and Efficient
Computing the Target Rate
Estimating queuing delay from RTT
Time
RTT
True minimum RTT
RTT
RTTminQueuing delay
Estimating queuing delay from RTT
Time
RTT
True minimum RTT
RTT
RTTminRTT – RTTmin = queuing delay
Queuing delay
Estimating queuing delay from RTT
Time
RTT
True minimum RTT
RTTmin
New flow starts here
Wrong!
Estimating queuing delay from RTT
Time
RTT
True minimum RTT
True minimum RTT = RTTmin for new flows!
A “noisy” cellular link: Stanford to AWSDe
lay
expe
rienc
ed b
y in
divi
dual
pac
kets
(ms)
Time (s)
Decoupling queuing delay from other delay variation
Time
Que
ue le
ngth Measured RTT
• Wireless links• Cross traffic• ACK compression• Bursty transmission• …
Decoupling queuing delay from other delay variation
Time
Que
ue le
ngth Measured RTT
• Wireless links• Cross traffic• ACK compression• Bursty transmission• …
Positive Additive Noise!
Take Min over last 0.5 RTT of samples
A “noisy” cellular link: Stanford to AWS
Using the MIN delay estimator improves throughput from0.5 Mbits/s to 3.9 Mbits/s
Dela
y ex
perie
nced
by
indi
vidu
al p
acke
ts
Time (s)
Attaining the Target
Calculate target rate = '( = )*+,
If current rate < r.: additively increase by 0* pkts/RTTElse: additively decrease by 0* pkts/RTT
The Copa Algorithm
Calculate target rate = '( = )*+,
If current rate < r.: additively increase by 0* pkts/RTTElse: additively decrease by 0* pkts/RTT
The Copa AlgorithmVelocity forfaster convergence
Calculate target rate = '( = )*+,
If current rate < r.: additively increase by 0* pkts/RTTElse: additively decrease by 0* pkts/RTT
22
24
26
28
0 0.2 0.4 0.6 0.8 120
22
24
26
28
30
Cw
nd (p
kts)
RTT
stan
ding
(ms)
Time (s)
CwndRTTstanding
The Copa Algorithm
RTTCwnd
Cwnd
(pkt
s)
RTT
(ms)
Time (s)
Velocity forfaster convergence
Steady-State Dynamics of CopaQ
ueue
Leng
th (i
n pk
ts)
Time
Equilibrium queue length
2.5 $%&
1 $%&
00 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT
Period: 5 RTTsAmplitude: 2.5 $%&
Steady-State Dynamics of CopaQ
ueue
Leng
th (i
n pk
ts)
Time
Equilibrium queue length
2.5 $%&
1 $%&
0
0.5 RTT
window
Switch point
Queue length corresponding toRTTstanding at the switch point
0 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT
Period: 5 RTTsAmplitude: 2.5 $%&
Steady-State Dynamics of CopaQ
ueue
Leng
th (i
n pk
ts)
Time
Equilibrium queue length
2.5 $%&
1 $%&
0
0.5 RTT
window
Switch point
Queue length corresponding toRTTstanding at the switch point
0 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT
Period: 5 RTTsAmplitude: 2.5 $%&
Steady-State Dynamics of CopaQ
ueue
Leng
th (i
n pk
ts)
Time
Equilibrium queue length
2.5 $%&
1 $%&
0
0.5 RTT
window
Switch point
Queue length corresponding toRTTstanding at the switch point
0 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT
Period: 5 RTTsAmplitude: 2.5 $%&
Steady-State Dynamics of CopaQ
ueue
Leng
th (i
n pk
ts)
Time
Equilibrium queue length
2.5 $%&
1 $%&
0
0.5 RTT
window
Switch point
0.5 RTTwindow
Queue length corresponding toRTTstanding at the switch point
Switch point
0 RTT 0.5 RTT. 1.5 RTT 2.5 RTT 3 RTT 4 RTT
Period: 5 RTTsAmplitude: 2.5 $%&
Detect buffer-filling TCP
Estimate true minimum RTT
Queue empties every 5 RTTs! ⇒
TCP-Competitiveness
Mode switching for TCP competitiveness
0
5
10
15
20
25
0 10 20 30 40 50 600
5
10
15
20
Thro
ughp
ut (M
bits
/s)
Que
uing
Del
ay (m
s)
Time (s)
Delay sensitive(! = 0.5)
TCP CubicCopa
Mode switching for TCP competitiveness
0
5
10
15
20
25
0 10 20 30 40 50 600
5
10
15
20
Thro
ughp
ut (M
bits
/s)
Que
uing
Del
ay (m
s)
Time (s)
Delay sensitive(! = 0.5)
TCP CubicCopa
Low Delay Low Throughput Low Delay
Mode switching for TCP competitiveness
0
5
10
15
20
25
0 10 20 30 40 50 600
5
10
15
20
Thro
ughp
ut (M
bits
/s)
Que
uing
Del
ay (m
s)
Time (s)
0
5
10
15
20
25
0 10 20 30 40 50 600
5
10
15
20
Thro
ughp
ut (M
bits
/s)
Que
uing
Del
ay (m
s)
Time (s)
Delay sensitive(! = 0.5)
TCP Competitive(AIMD on !&')
TCP CubicCopa
Low Delay Low Throughput
High ThroughputHigh Delay
Low Delay
High Delay
TCP CubicCopa
Mode switching for TCP competitiveness
0
5
10
15
20
25
0 10 20 30 40 50 600
5
10
15
20
Thro
ughp
ut (M
bits
/s)
Que
uing
Del
ay (m
s)
Time (s)
0
5
10
15
20
25
0 10 20 30 40 50 600
5
10
15
20
Thro
ughp
ut (M
bits
/s)
Que
uing
Del
ay (m
s)
Time (s)
Delay sensitive(! = 0.5)
TCP Competitive(AIMD on !&')
TCP CubicCopa
Low Delay Low Throughput
High ThroughputHigh Delay
Low Delay
High Delay
TCP CubicCopa
Mode switching for TCP competitiveness
0
5
10
15
20
25
0 10 20 30 40 50 600
5
10
15
20
Thro
ughp
ut (M
bits
/s)
Que
uing
Del
ay (m
s)
Time (s)
TCP Cubic
Copa
Low Delay High Throughput Low Delay
When queue doesn’t empty once every 5 RTTs, switch to TCP Competitive mode!
Best of both worlds!
0
0.5
1
1.5
2
2.5
Vegas Reno Cubic Copa PCC BBR
Avg.
(Tpt
/ Id
eal t
pt)
Scheme under test's throughputCubic's throughput
Copa gets higher throughput without hurting TCP Cubic!
0
0.5
1
1.5
2
2.5
Vegas Reno Cubic Copa PCC BBR
Avg.
(Tpt
/ Id
eal t
pt)
Scheme under test's throughputCubic's throughput
Copa gets higher throughput without hurting TCP Cubic!
0
0.5
1
1.5
2
2.5
Vegas Reno Cubic Copa PCC BBR
Avg.
(Tpt
/ Id
eal t
pt)
Scheme under test's throughputCubic's throughput
Copa gets higher throughput without hurting TCP Cubic!
Copa gets higher throughput Cubic flows are not hurt
Limitations
• Cannot ignore low frequency noise
• Queues don’t empty periodically if:• Propagation delay is much smaller than queuing delay• Flows with very different propagation delays share a bottleneck queue
• Needs precise RTT measurements
Consistent Performance on Real Paths
0
0.2
0.4
0.6
0.8
1
16326412825651210242048
BBR
Copa
CubicLEDBAT
PCC
Sprout
RemyVegasVerus
Vivace latency
Vivace lossVivace LTE
Avg
. Nor
mal
ized
Thr
ough
put
Avg. Queuing Delay (ms)
Cellular Networks
0
0.2
0.4
0.6
0.8
1
124816
BBRCopaCubic
LEDBAT
PCC
Sprout
Remy
VegasVerus Vivacelatency
Vivace lossVivace LTE
Avg.
Nor
mal
ized
Thr
ough
put
Avg. Queuing Delay (ms)
Wired Networks
Better⊳
Better⊳
Satellite link: High BDP, high loss
010203040
800 110100
CopaRemyPCC
Cubic Vegas
BBR
Thro
ughp
ut (M
bit/s
)
Median Queuing Delay (ms)
Better⊳
Fairness during flow-churn
1
10
1 3 5 7 9 11 13 15 17
1
10
1 3 5 7 9 11 13 15 171
10
1 3 5 7 9 11 13 15 17
1
10
1 3 5 7 9 11 13 15 17
Copa
BBR
Cubic
PCC
Thro
ughp
ut (M
bit/
s)Av
g±
Std.
Dev
.
Time (s) Time (s)
Fairness during flow-churn
1
10
1 3 5 7 9 11 13 15 17
1
10
1 3 5 7 9 11 13 15 171
10
1 3 5 7 9 11 13 15 17
1
10
1 3 5 7 9 11 13 15 17
Copa
BBR
Cubic
PCC
Thro
ughp
ut (M
bit/
s)Av
g±
Std.
Dev
.
Time (s) Time (s)
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
CopaCu
bicBBRPC
C
Jain index
Better⊳
Summary
NetworkEstimate queuing delay
and compute target
!
A practical delay-based congestion control algorithmhttps://web.mit.edu/copa
Summary
NetworkEstimate queuing delay
and compute target
!
Outer control loopMode switching andCompetitive mode
A practical delay-based congestion control algorithmhttps://web.mit.edu/copa
Summary
NetworkEstimate queuing delay
and compute target
Move toward target (AIAD-variant)
!
Outer control loopMode switching andCompetitive mode
A practical delay-based congestion control algorithmhttps://web.mit.edu/copa
Rate/cwnd