RESTRUCTURING ENDPOINT CONGESTION CONTROL

Akshay Narayan, Frank Cangialosi, Deep5 Raghavan, Prateesh Goyal, Srinivas Narayana, Radhika Mi>al, Mohammad Alizadeh, Hari Balakrishnan

CONGESTION CONTROL

APPLICATION

OS TCP

Data

Data

NETWORK

DATAPATHS

MTCP DPDK QUIC DCCPUDPSMARTNIC

FPGARDMA

APPLICATION

Data

Data

NETWORK

OS TCP

NEW ALGORITHMS

1987 2018

VegasWestwood Compound

PRRBIC

H-TCP

Cubic

FAST

EBCCBinomial

LEDBATVeno

BBRPCCRemy

Sprout

DCTCP

IllinoisNV

HyblaTIMELY

ABC

NewReno

Copa

RC3XCP RCP

DCQCN

1998 2001

TahoeReno

2010

VivaceIndigo

Nimbus

ALGORITHM COMPLEXITY

Sprout (NSDI 2013): Bayesian forecas5ng Remy (SIGCOMM 2013): Offline learning

PCC / PCC Vivace (NSDI 2015 / NSDI 2018): Online learning Indigo (Usenix ATC 2018): Reinforcement learning

[][CROSS PRODUCT OF SADNESS

Vegas

WestwoodCompound

PRRBIC

H-TCP

Cubic

FAST

EBCCBinomial

LEDBAT

Veno

BBRPCCRemy

Sprout

DCTCP

IllinoisNV

HyblaTIMELY

ABCNewReno

Copa

RC3

XCP

RCP

DCQCNTahoe

RenoVivaceIndigo

Nimbus

×] MTCP DPDK

QUIC

DCCP

USERSPACE

SMARTNIC FPGA

RDMA

OS TCP

CROSS PRODUCT OF SADNESS

NEW CAPABILITIES

APPLICATION

CCCC

CCCC

CC

INDEPENDENT CC

APPLICATION

CC

AGGREGATE CC

CURRENT DESIGN

NIC

ApplicaEon

TX RX

Datapath

Datapath State

CongesEon Control

CONGESTION CONTROL PLANE

NIC

ApplicaEon

TX RX

Datapath

Datapath State

CCP Agent

CCP Datapath

CONGESTION CONTROL PLANE

Write-once, run-anywhere

Sophis5cated algorithms

New capabili5es

NIC

ApplicaEon

TX RX

Datapath

Datapath State

CCP Agent

CCP Datapath

LATENCY TRADEOFF

Latency ( < 30 µs)

NIC

ApplicaEon

TX RX

Datapath

Datapath State

CCP Agent

CCP Datapath

Write-once, run-anywhere

Sophis5cated algorithms

New capabili5es

SPLIT IMPLEMENTATION

NIC

ApplicaEon

TX RX

DatapathCWND

RATEStaEsEcs

Datapath State

CCP Agent

CCP Datapath

Asynchronous CC Logic

Synchronous Measurement

Gathering

Split CC performs similarly to datapath-na5ve

SPLIT IMPLEMENTATION

NIC

ApplicaEon

TX RX

DatapathCWND

RATEStaEsEcs

Datapath State

CCP Agent

CCP DatapathStandardized Datapath Interface

MEASUREMENT PRIMITIVES

Measurement timestamp

In-order acked bytes

Out-of-order acked bytes

ECN-marked bytes

Lost bytes

Timeout occurred

RTT sample

Bytes in flight

Outgoing rate

Incoming rate

Demo

CUBIC WINDOW DYNAMICS

200

400

600

0 20 40 60Time (s)

Con

gest

ion

Win

dow

(Pkt

s)

CCP Kernel

96 Mbit/s, 20ms link RTT

WRITE-ONCE RUN-ANYWHERE

Link: 24 Mbit/s, 20ms RTT

Kernel QUIC mTCP

Copa

Cubi

c

Link: 12 Mbit/s, 20ms RTT

STRESS TEST

cubic reno

1 2 4 8 16 32 64 1 2 4 8 16 32 640.0

2.5

5.0

7.5

Flows

CPU

Util

izat

ion

(%)

ccp

kernel

Link: 10Gbit/s, 100µs RTT

5%

0.00

0.25

0.50

0.75

1.00

1.25

20us2RTT

50us5RTT

100us10RTT

200us20RTT

300us30RTT

400us40RTT

500us50RTT

Reporting Interval

RC

T/Ba

selin

e

10Pkts

100Pkts

1000Pkts

LOW-RTT SCENARIOS

Link: 10Gbit/s, 10 µs

1.15

DESIGN: FAST AND SLOW PATH

NIC

ApplicaEon

TX RX

DatapathCWND

RATEStaEsEcs

Datapath State

CCP Agent

CCP Datapath Shim

Asynchronous API

SPLIT IMPLEMENTATION

NIC

ApplicaEon

TX RX

Datapath

Datapath State

CCP Agent

CCP Datapath Shim

libccp

Datapath shim Expose datapath variables

SPLIT IMPLEMENTATION

libccp Per Packet Opera5ons Shared across datapaths

NIC

ApplicaEon

TX RX

Datapath

Datapath State

CCP Agent

CCP Datapath Shim

libccp

Datapath shim Expose datapath variables

Datapath Program:▸ Per ACK measurements ▸ Pulse:

Rate = 1.25 x bo>le rate ▸ Aeer 1 RTT:

Rate = 0.75 x bo>le rate ▸ Aeer 2 RTT: Rate = bo>le rate ▸ Aeer 8 RTT: repeat

▸ Every report ▸ Calculate new rate based

on measurements ▸ Handle switching between

modes

Asynchronous:

BBR SPLIT IMPLEMENTATION

Asynchronous

Datapath Program

SLOW START

0

200

400

600

0.0 0.2 0.4 0.6Time (s)

Con

gest

ion

Win

dow

(Pkt

s) CCP, 100ms Report

CCP, In−Fold

CCP, Rate−Based

In−Datapath

48Mbit/s, 100ms link RTT

NEW CAPABILITIES

Sophis5cated algorithms

Rapid prototyping

CC for flow aggregates

Applica5on-integrated CC

Dynamic, path-specific CC

NEXT STEPS

▸ More algorithms!

▸ Hardware datapaths

▸ Impact of new API on conges5on control algorithms

▸ New capabili5es using CCP plajorm

github.com/ccp-project

▸ Datapaths (libccp):

▸ Linux TCP

▸ QUIC

▸ mTCP/DPDK

▸ CCP Agent (portus)

CURRENT STATUS

Reproduce our results and build your own conges5on control at

Extra Slides

EBPF

▸ Event-driven seman5cs

▸ Explicit repor5ng model

Front-End (Language)

Back-End (Datapath)

▸ Conges5on control enforcement

▸ Direct access to socket state

(def (Report (acked 0))) (when true (:= Report.acked (+ Report.acked Ack.bytes_acked)) (:= Cwnd (+ Cwnd Report.acked)) (fallthrough)) (when (> Flow.lost_pkts_sample 0) (report))