end-to-end multicast congestion control and avoidance jiang li advisor: shivkumar kalyanaraman

End-to-End Multicast Congestion Control and Avoidance

Jiang Li

Advisor: Shivkumar Kalyanaraman

Outline

• Overview of multicast and multicast congestion management

• Solution series– LE-SBCC – ORMCC– GMCC– MCA+

• Simulations & Experiments

• Future work

Multicast

• Efficient One-to-Many Data Distribution– Unicast: one copy for EACH receiver– Multicast: one copy for ALL receivers

Unicast Multicast

Source

Receiver 1 Receiver 2 Receiver 1 Receiver 2

Source

Router Router

When / Where to Use Multicast?• Whenever / wherever efficient one-to-many delivery

of same data is needed• Applications:

– News/sports/stock/weather updates– Distance learning– Configuration, routing updates, service location– Teleconferencing (audio, video, shared whiteboard, text

editor)– Distributed interactive gaming or simulations– Email distribution lists– Content distribution; Software distribution– Web-cache updates – Database replication

Research on Multicast

• Routing

• Reliable transmission

• Congestion management

• Security

• Address allocation

• And more

Multicast Congestion Management

• Adapt sending rate to available bandwidth– Single-rate– Multi-rate

Receiver 1 Receiver 2

Source

Router

1Mbps 2Mbps

1Mbps 1Mbps2Mbps

Challenges

• Multiple paths– Complex congestion pattern– Heterogeneous bandwidth

• Dynamically changing

…

Bottlenecks

Source

Receivers

Router

`0.5Mbps

1Mbps

2Mbps

Potential Problems

• Drop-to-Zero– React to congestion more than necessary

0 Mbps

Sending rate

Congestion !Cut rate



Rate increase



4Mbps3Mbps

2Mbps1Mbps

2Mbps

1Mbps 0 Mbps

Source

Router

btnk1 btnk3btnk2

Potential Problems (cont’d)

• Feedback implosion

…

Source

Receivers

Router

Potential Problems (cont’d)

• Large number of states and large computation complexity– O(1) number of states wanted– O(1) computation time wanted

• Unfriendliness to other existent congestion control protocols (e.g. TCP)– Break others or be broken by others

End-to-end

Solution Categories

• End-to-end vs. third-party assisted– End: source, receiver– Third-party: router, proxy etc

Router

Proxy (uncontrollable by end users)

Source

Receiver

Receiver

(Third party)

(Third party)

(End)

(End)

(End)

Easy to deploy. Our choice.

Network

Congestion Control vs. Congestion Avoidance

• Congestion control– Reactive– Congestion is managed

when packets are lost

• Congestion avoidance– Proactive– Congestion is managed

when queue is being built up at bottlenecks– Low avg. queue length, high b/w utilization

Queue length

Buffer size

Time

Cong. Avoid. Cong. Ctrl

Packet drop begins

t0

A Series of Solutions

• LE-SBCC (single-rate congestion control)– Purely source-based, compatible with many

multicast transport protocols

• ORMCC (single-rate congestion control)– O(1) state and computation complexity– Scalable to large groups

• GMCC (multi-rate congestion control)– Adaptive to receiver heterogeneity

• MCA+ (single-rate congestion avoidance)– Responsive to incipient congestion

Outline



• Simulations

• Future work

LE-SBCC

• Loss-Event Oriented Source-Based Multicast Congestion Control

• Motivations– Compatible with all multicast transport

protocols (e.g. RMTP, PGM)– Easiest deployment

• Trade-offs– For small to medium sessions– Single-rate

Characteristics

• Based on source– Source does most of the work, e.g.

• Filtering feedback packets• Adjusting sending rate

• Minimum receiver support– Single-bit feedback

• Can be piggybacked by ACK or NAK• Commonly available in transport protocols

Can be deployed by upgrading source only

Problem

• How to react to congestion?

Source

Router

Rcvr 1 Rcvr 2 Rcvr 3 Rcvr 4 Rcvr 5

Idea

• React to the most congested receiver

Source

Router


7050 30 60 40

Reacts to 70 feedback packets

• React to a number of feedback packets approx. equal to those from the most congested receiver.

Feedback Filter Cascade

RateadaptationLI2LE MaxLPRF ATFFeedback

packets

Source

Router


7050 30 60 40

RTT (Round Trip Time):biased towards the most

congested receiver

RTTEstimator

Filter 1: LI2LE

• Pass at most one feedback packet per RTT per receiver• Loss indication (LI): an original feedback packet• Loss event (LE): a packet passing the filter

RTT

Filtering receiver 1’sfeedback

LI LE

Filtering receiver 2’sfeedback RTT

LI LE

t0t1

< RTT

Filter 2: Max-LPRF

• Passes every LE with the probability:

i

i

i

X

Xp

maxSource

Router


X1=70

X2=50 X3=30 X4=60 X5=40

250

705

1

1

i

iX

Xp

Approx. 70 LEs

Filter 3: ATF

• Enforce at most one rate cut per RTT.

RTT Time

Loss events

Sendingrate

Ignored

Rate cut

t0 t0+RTT

Accepted for rate cut

Rate Adaptation

• AIMD (additive increase/multiplicative decrease)

• Other (e.g. TFRC, binomial)

RTT RTT RTT RTT RTT RTT

Time

Sending rate

s/RTT

s: packet size

V

V/2

Outline




• Future work

ORMCC

• LE-SBCC– O(N) state at the source (N: # receivers)– Subject to feedback implosion

• ORMCC– O(1) state and computation complexity– Statistics-based feedback suppression– Proprietary support required from receivers

Source Functions

• Dynamically maintains a congestion representative (CR) (the slowest receiver)– Using TRAC (Throughput Rate At Congestio

n)• Receiving rate during congestion epochs

– Slowest receiver has the lowest average TRAC

• Only accepts CR’s input for rate adaptation (AIMD)

• Detect & recover from loss of CR– Omitted here

Receiver Functions

• Measures TRAC– Only during congestion epochs– Average over several packets or a short perio

d to avoid oscillation

• Sends feedback with TRAC– Suppresses if its average TRAC < average C

R TRAC – σ• σ : std. deviation of CR TRAC• Average CR TRAC andσare multicast by the sourc

e

Outline




• Future work

GMCC (Generalized Multicast Congestion Control)

• LE-SBCC & ORMCC– Single-rate

• GMCC: A multi-rate scheme– Various receivers receive data at

different rates. – Can run in single-rate mode– Leverage ORMCC and greatly simplify the

scheme


Source

Router

1Mbps 2Mbps

1Mbps 1Mbps2Mbps

Previous Multi-Rate Schemes• Receiver-based schemes

– The source sends data in each layer without regarding to network situation

– Receivers increase/decrease their receiving rates by constantly join/leave layers

• Coarse control, heavy router burden

Join interval

Leave

Leave intervalReceiverSource

Data

Layer 1Layer 2Layer 3Layer 4

~

Join

1 layer = 1 multicast group

(Very short) (Very short)

Previous Multi-Rate Schemes (cont’d)

• SMCC– Source adaptation– Static layering (pre-defined maximum rate for

each layer)

ReceiverSource Join interval Leave interval

~

Data

Layer 1Layer 2Layer 3Layer 4

(Dynamic) (Dynamic)

Join Leave

single-ratesingle-ratesingle-ratesingle-rate

GMCC

• Independent single-rate congestion control in each layer

• Dynamic layering (no rate limit for each layer)

Join interval Leave intervalSource Receiver

Data

Layer 1Layer 2

Layer 3Layer 4

(Dynamic) (Dynamic)

Join Leave

~

Key Ideas

• “Unsatisfied” receivers join a new layer.


Source

Router

1Mbps 2Mbps

1Mbps1Mbps

1Mbps=2Mbps

Unsatisfied: much less congested than the most congested one (in a laye

r)

Key Ideas (cont’d)

• No receiver is allowed to be the most congested in more than one layer.


Source

Router

1Mbps 2Mbps

1Mbps1Mbps

0.5Mbps0.5Mbps

0.33Mbps0.33Mbps

0.33Mbps0.5Mbps0.33Mbps

Receiver Operations

• Join– A receiver joins a higher layer if it sees much

less congestion than CR in its top joined layer.

• CR: The most congested receiver

• Leave– A receiver leaves its top joined layer if it is the

most congested in more than one layers.

JoinLess congested

Most congested

Leave

Layer 1Layer 2

Layer 1

Layer 3Layer 2

Layer 1

Layer 3Layer 2

Layer 1Layer 2

Source Operations

• Control on and off of each layer– Only the top layer can be deactivated– A previously unnoticed problem

• Do single-rate multicast congestion control in each layer– Very similar to ORMCC

Layer 1

Layer 3Layer 2

Receivers

Outline




• Future work

MCA+

• Multicast congestion avoidance with feedback suppression

• MCA+ = ORMCC + incipient congestion detection– Using accumulation to detect incipient

congestion

MCA+: Accumulation• A flow’s accumulation is a time-shifted,

distributed sum of the queued bits in all nodes along its path.

• Can be measured end-to-end– Showed theoretically by colleague’s work

Accumulations:

Red flow: 5 packets

Blue flow: 4 packets

Yellow flow: 9 packets

Routers

Packets in queues

t1

t2 t3 t4

Data

MCA+: Accumulation Measurement

• Use control packets as pivot points• (λ: input rate, μ: output rate)tt

Time

PropagationDelay

CP0

CP0

CPi

CPi

N packets sent (λt)

M packets received (μt)Accumulation

(N – M packets)

Sender side

Receiver side

MCA+: Accumulation & Congestion

• Accumulation ≥ 2 packets → congestion– By receivers.– Possible one packet accumulation even when

underloaded– More noise, higher threshold– Accumulation ≤ 1 packet, synchronize (begin

another measurement period).

• Feedback and rate adaptation– Very similar to ORMCC

Outline




• Future work

Problems to Solve -- Review

• Drop-to-Zero

• TCP-friendliness

• State and computation complexity

• Feedback implosion

Simulations: Drop-to-Zero Avoidance Test

…Bottlenecks

Source

Receivers

Router

`

• Topology

Drop-to-Zero Avoidance Test (cont’d)

Multicast flow rateThroughput rates of two randomly chosen

unicast flows

• LE-SBCC, using ns-2 – Multicast rate ≈ Unicast rate

Simulations: Drop-to-Zero Avoidance Test (cont’d)

• ORMCC, 10000 receivers, using ROSS

Multicast ORMCC

Unicast ORMCC


• ORMCC vs PGMCC & TFMCC, 500 receivers, using ROSS

ORMCC

PGMCCTFMCC

Unicast PGMCC


• MCA+

Bottleneck Utilization Improvement of MCA+

• Average queue length

Bottleneck Utilization Improvement of MCA+

• Bandwidth utilization

Throughput Improvement of GMCC

Throughput Improvement of GMCC(cont’d)

• 6 groups of bottlenecks: 0.2 Mbps ~ 1.2Mbps

Outline




• Future work

Future Work

• Application of GMCC to multicast video streaming

• Application of GMCC to bulk data transmission

• Multi-rate application layer multicast congestion control

Thank you

&

Estimated # of Multicast Receivers on the Internet

ORMCCDetect/Recover from CR Loss

• Recover from CR absence– Measure response time (TR)

since hypothetical bottleneck queue build-up (tq)

• If no feedback from CR for TR since tq

– Claim CR absence– Requests feedback from all

receivers

Buffer size

Queuelength

Packet loss begins

Time

Time

Rate

TRAC

TR

tq

Feedback from

CR arrives

GMCCCompare Degree of Congestion

• Throughput Attenuation Factor (TAF)– A: Individual throughput attenuation factor

• 1 - (output/input) ( )• Severity of congestion

– B: Congestion occurrence rate• Number of packet loss epochs / Total sent pack

ets• Frequency of congestion

– TAF = A • B. Higher TAF, more congested

Input

Output

Input - output

Packets sent

Packets arrived 10

2

at congestion

Epoch 1 Epoch 2

Probabilistic Inter-layer Bandwidth Switching

• Hidden bandwidth problem

2Mbps

1Mbps10

Mbps

10Mbps

Receivers

Rcvr 1 Rcvr 2 Rcvr 3

Source

Probabilistic Inter-layer Bandwidth Switching (cont’d)

Layer 2traffic volume

Layer 1traffic volume

2Mbps

1Mbps10

Mbps

10Mbps

Receivers

Rcvr 1 Rcvr 2 Rcvr 3

Source

Test period

• No change to total traffic volume

• Difference in test period

Posttest

period

Simulations: TCP-Friendliness Test

• Topology

…

…

TCP

Multicast

Simulations: TCP-Friendliness Test (cont’d)

• LE-SBCC

Simulations: TCP-Friendliness Test (cont’d)

• ORMCC

Effect of Feedback Packet Loss• ORMCC, 5% loss

CR Switch Test

• ORMCC, every CR stops response after 20 seconds

Linux Experiment (ORMCC)

• Topology

Linux Experiment (ORMCC)

• Result

Effectiveness of Layering

Source1Mbps

GMCC receiver 1

GMCC receiver 2

Unicast receivers

5Mbps

Layering & Throughput Rate (1)

Throughput rate of receiver 2Receiver 2 joined layer 0

Receiver 2 stayed in layer 1Receiver 2 joined layer 1

Effectiveness of Layering (cont’d)

Source1Mbps

GMCC receiver 1

GMCC receiver 2

Unicast receivers

10Mbps

Layering & Throughput Rate (2)

Throughput rate of receiver 2

Receiver 2 joined layer 0

Receiver 2 stayed in layer 1Receiver 2 joined layer 1

GMCC’s Response to Dynamic Traffic

• Topology

GMCC’s Response to Dynamic Traffic (cont’d)

• Throughput dynamics

GMCC PIBS Test

• Topology

GMCC PIBS Test Result

Related Research

• End-to-end single-rate– DeLucia’s scheme, PGMCC, TFMCC, MDP-C

C

• End-to-end multi-rate– MLDA, HALM– RLM,RLC,PLM,FLID-DL,FGLM,STAIR etc

• Third-party assisted– Single-rate: e.g. Chiu’s work– Multi-rate: e.g. Kar’s work

end-to-end multicast congestion control and avoidance jiang li advisor: shivkumar kalyanaraman

Documents

computation complexityscalable

networkcongestion control

routing updates

data distributionunicast

shared whiteboard

avoidancejiang liadvisor

thirdparty assistedend

raterate increasecongestion