improving tcp performance in a differentiated services network: investigations and solutions

Improving TCP Performance in a Differentiated Services Network:

Investigations and Solutions

Kaleelazhicathu R R Kumar

Centre for Internet Research

School Of Computing

National University of Singapore

Outline

Introduction Motivation

Background TCP DiffServ TCP in DiffServ : Issues

Related work Research Objective Thesis Contribution Memory-Based Marker (MBM) Memory-Based Three Color Marker (MBTCM) TSWTCM vs. MBTCM : A Comparison Congestion-Aware Traffic Conditioner (CATC) Linux Implementation of MBM Areas for Deployment Conclusion

Introduction: Motivation

An exponential growth in traffic resulted in deterioration of QoS.

Over provisioning of networks could be a solution. A better solution: An intelligent network service with better

resource allocation and management methods. DiffServ has emerged as a solution for providing QoS by

service differentiation. Recent measurements have shown TCP flows being in

majority (95% approx. of byte share). Inherent limitations of TCP is a hurdle for providing better

QoS.

Background: Transmission Control Protocol (TCP)

De facto Transport layer Protocol Common applications like Telnet,FTP and HTTP uses

TCP Provides a connection oriented,reliable,

byte stream service to the application. Control mechanisms

Slow start, Congestion Avoidance, Fast Retransmit and Recovery

Flow Control

Differentiated Services

Provides QoS for aggregate flows using different per hop forwarding behaviours at the core routers

Scalable The philosophy: simpler at the core (AQM), complex at the

edges Per-Hop behaviours

Expedited forwarding: Deterministic QoS Assured forwarding: Statistical QoS

RIO-based schemes proposed for AQM.


Basic building blocks of DiffServ Classifier Traffic Conditioner

Token Bucket (TB), Time Sliding Window (TSW) Meter Marker Shaper/Dropper


Classifier

Meter

MarkerShaper/Dropper

Packets

Logical View of a Packet Classifier and Traffic Conditioner

Drop

Forward

TCP in DiffServ: Issues

TCP flows are much more sensitive to transient congestion. Bias against connections with long Round Trip Time (RTT)

Reason:Long RTT flows takes longer time to ramp up. Bias against connections with smaller window sizes

Reason: Smaller windows mean smaller throughput Protection from unruly traffic like UDP traffic.

Reason: UDP traffic has no rate control mechanism and hence kills TCP traffic.

DiffServ issues Bandwidth assurance affected by size of target rate. Markers sensitive to its own parameters. Absence of Edge-to-Edge feedback. Existing markers fail to track TCP dynamics

Related Work

Clark et al came up with the RIO scheme. Nandy et al identified the factors affecting bandwidth

assurance. Kalyanaraman et al proposes a TCP-Friendly component. Kalyanaraman et al also proposed an edge-to-edge

feedback architecture based on ECN and ICMP messages. Sahu et al studied the influence of token bucket

parameters on providing assured service. Feng et al proposed an adaptive marker.

Research Objective

Markers, one of the building blocks of a trafficconditioner play a major role for resource allocationin a DiffServ network.

Design an Intelligent Marker Least sensitive to both the marker and TCP parameters Should be transparent to end hosts. Maintain optimum marking Tracks the TCP dynamics Minimize synchronizations. Be fair to different target sizes. Be congestion aware.

Design an Edge-to-Edge feedback architecure An early indication of congestion in a network helps to prioritize the packets in

advance.

Thesis Contribution

Two Approaches Memory-Based

Memory-Based Marker (MBM) Reduces influence of TCP’s limitations

Memory-Based Three Color Marker (MBTCM) Suitable for DiffServ with AF PHB

Feedback-basedCongestion-aware Traffic Conditioner

Provides edge-to-edge feedback to the marker

Memory-Based Marker (MBM)

Tracks TCP dynamics. Transparent to end hosts. Maintain optimum marking. Fairness Less sensitive to its own parameters.

Memory-Based Marker?

During the period when TCP flows experience congestion, either or both of the following occurs: a) The cwnd reduces reducing the value of W b) The RTT increases causing a decrease in throughput or rate of flow. The TCP window size W and the round trip time RTT are related to the

throughput by the equation:BW = ¾*(MSS*W)/(RTT) where W is expressed in

number of segments. Any variation in W or RTT is reflected as subsequent changes in BW, i.e., in our

case, the avg_rate. The parameter previous average rate (par) is compared with the present average

rate to track any change in the rate of flow and thus indirectly extract the variations in RTT or W.

MBM Algorithm

For each packet arrival If avg_rate cir then mp=mp+(1-avg_rate/cir)+ (par- avg_rate)/avg_rate; par = avg_rate; mark the packet using: cp 11 w.p. mp cp 00 w.p. (1-mp)

else if avg_rate > cir

then

mp= mp + (par – avg_rate)/avg_rate;

par=avg_rate;

mark the packet using:

cp 11 w.p. mp

cp 00 w.p. (1-mp)

MBM Algo. Cont’d..

where,avg_rate= the rate estimate upon each packet arrival

mp = marking probability (1)cir = committed information rate (i.e., the target rate)par = previous average ratecp denotes ‘codepoint’ and w.p. denotes ‘withprobability’.

MBM Algo. Explained

In the expression for the marking probability mp, (par – avgrate)/avgrate tracks the variations

in RTT and window size (W) and thus increases or decreases the marking probability according to the changes in the flow rate.

(1- avgrate)/cir constantly compares the average rate observed with the target rate to keep the rate closer to the target.

Experiments

We used FTP bulk data transfer for the TCP traffic in all our experiments.

NS (2.1b7a) simulator on Red Hat 7.0 Modified Nortel’s DiffServ module for our architecture implementation. Core routers use RIO like mechanism We conducted simulation studies for:

Assured service for aggregates with different target rates. Effect of different RTTs Effect of different window sizes Protection from best effort UDP flows Effect of UDP flows with target rates.

Topology

Results

Expt #Rt 1 Rt 2 BE TCP flow Link goodput

Total Marked Total Marked (Mbps) (Mbps)1 1 1 2.85 1.45 3.35 1.97 2.94 9.142 1 2 2.93 1.76 3.6 2.7 2.64 9.173 1 3 2.93 2.08 4.08 3.44 2.2 9.214 1 4 2.93 2.21 4.29 3.84 1.93 9.155 1 5 2.8 2.32 4.89 4.64 1.51 9.26 2 2 3.4 2.58 3.56 2.73 2.49 9.457 3 3 3.75 3.34 3.53 3.08 1.85 9.138 4 4 3.88 3.7 3.94 3.7 1.31 9.139 5 5 4.38 4.38 4.35 4.35 0.42 9.15

10 6 6 4.35 4.35 4.5 4.5 0.34 9.199.192 Average link utilization = 92% (approx.)

Ra1 Ra2 Target Rates (Mbps) Achieved Rates (Mbps)

Achieved Rates (Ra) for different Target Rates (Rt).

Results..

RTT (ms) Ra 1 Ra 2

60 1.82 3.81 5.6380 1.49 3.74 5.23

100 1.52 3.52 5.04120 1.38 3.58 4.96140 1.43 3.45 4.88

25.74

Achieved Rates ( Mbps)

Total link goodput

per source pair goodput(Mbps)

Achieved Rates (Ra) for different RTT values

window size(KB ) Without MBM With MBMAchieved Rates Achieved Rates Ra (Mbps) Ra (Mbps)

384 0.58 1.88768 3.1 3.06

1125 3.21 2.871536 2.76 3.071920 1.25 2.93

13.81Total Link utilization 10.90

Achieved Rates (Ra) for different window sizes

Results..

Target Rate (Mbps) Rt Ra(udp_be) Ra(tcp_be)

Total Marked2 3.83 2.03 2.95 2.64 4.85 4.13 2.91 1.666 5.76 5.6 2.84 0.818 7.13 7.13 2.22 0.04

10 7.94 7.94 1.4 0

Achieved Rates (Mbps) Ra(tcp_prio)

Achieved Rates in presence of BE UDP and TCP

Target Rate (Mbps)Ra(udp_prio) Ra(tcp_be)

Total Marked2 3.73 1.83 2.98 2.634 4.73 4.04 2.98 1.646 5.66 5.58 2.98 0.738 6.08 6.08 2.98 0.32


Achieved Rates in presence of AS UDP and BE TCP

Inference

MBM Achieves transparency from the end hosts, simplicity,

and least sensitivity to parameters of both TCP as well as its own parameters.

helps in achieving the target rate, with a better fairness in terms of sharing the excess bandwidth among flows.

provides the TCP flows, a greater degree of insulation

from differences in RTT and window sizes. The overall link utilization also seems to be much better.

Memory-Based Three Color Marker(MBTCM)

An Extension of MBM suitable for DiffServ

with AF PHB. Solves some issues in MBM. an improvement over TSWTCM

MBTCM Algorithm

For each packet arrival If avg_rate cir then mp=mp+(1-avg_rate/cir)+ (par- avg_rate)/avg_rate; par = avg_rate; mark the packet using: cp 10 (green) w.p. mp

cp 11 (yellow) w.p. (1-mp)

else if (avg_rate cir) && (avg_rate pir) then mp= mp + (par – avg_rate)/avg_rate – (avg_rate-cir)/pir; par=avg_rate; mark the packet using: cp 11 w.p. mp cp 00 (red) w.p. (1-mp) else cp 00 w.p 1

MBTCM Algo. Explained

(avg_rate-cir)/pir acts as the reduction factor for reducing the probability as the avg_rate increases towards pir. This component is particularly useful when the traffic stream has a constant avg_rate (e.g., UDP traffic) and is above cir. In such a scenario, mp doesn’t remain constant but reduces to zero.

Results…



10 6 6 3.83 3.82 4 3.99 1.8 9.63 Average link utilization 9.402 Average link utilization= 94 % (approx.)

Ra1 Ra2 Target Rates(Mbps) Achieved Rates (Mbps)


Results..

window size(KB ) Without MBTCM With MBTCMAchieved Rates Achieved Rates Ra (Mbps) Ra (Mbps)

384 0.58 2.07768 3.1 3.12

1125 3.21 3.521536 2.76 2.581920 1.25 3.85

15.14Total Link goodput 10.90

Achieved Rates (Ra) for different window sizes


Total Marked2 3.24 3 2.97 3.224 3.99 3.7 2.96 2.256 3.36 3.3 2.97 3.178 3.5 3.48 2.97 3.01

10 3.5 3.48 2.97 3.01



TSWTCM vs. MBTCM: A Comparison

TSWTCM 3 color TSW-TC based marker Marking based on two parameters- Committed

Target Rate (CTR), and Peak Target Rate (PTR).

Results..



10 6 6 3.11 3.11 4.25 4.25 1.65 9.01 Average link utilization 9.173 Average link utilization= 92 % (approx.)

Ra1 Ra2 Target Rates(Mbps) Achieved Rates (Mbps)


Results..

window size(KB ) Without TSWTCM With TSWTCMAchieved Rates Achieved Rates Ra (Mbps) Ra (Mbps)

384 0.58 3.23768 3.1 1.4

1125 3.21 0.011536 2.76 4.271920 1.25 4.47

13.38Total Link goodput 10.90

Achieved Rates (Ra) for different window sizes.


Total Marked2 2.92 2.7 2.97 3.394 3.12 3.12 2.97 2.976 3.18 3.18 2.97 2.938 3.25 3.25 2.97 3

10 3.25 3.25 2.97 3



The Comparison.

MBTCM achieves the target rates for priority flows with optimum

marking. helps in achieving consistency of goodput in cases of

flows with different window settings. performs better than TSWTCM in terms of protection

from BE UDP flows. has an overall link utilization much better than

TSWTCM.

Congestion-aware Traffic Conditioner(CATC)

Congestion-aware Least sensitive to the marker parameters. Transparent to end hosts. Maintain optimum marking.

Edge-to-Edge Feedback Architecture

Two edge routers Control sender (CS) and control receiver (CR)

Upstream: At CS:

CS sends control packets (CP) at regular interval of time, control packet interval (cpi). CPs are given highest priority.

At Core: Core routers maintain the status of drops of the best effort packets. Information maintained as a status flag to a max. of cpi time. CP’s congestion notification (CN) bit set or reset based on status flag.

At CR: Responds to the incoming CP with a CN bit set by setting the congestion echo (CE) bit of the

outgoing acknowledgement. Downstream

At CS: Maintains a parameter, congestion factor (cf). Cf is set to 1 or 0 based on status of the CE bit in acknowledgement received

CATC Algorithm

For each packet arrival If avg_rate cir then mp=mp+(1- avg_rate/cir)*(1+ cf*(cir/cir_max)); mark the packet using : cp 11 w.p. mp cp 00 w.p. (1-mp)

else if avg_rate > cir

then

mp=mp+ (1- avg_rate/cir)*(1- cf*(cir/cir_max));

mark the packet using :

cp 11 w.p. mp

cp 00 w.p. (1-mp)

CATC Algo. Explained

The effect on mp: i)Flow component (1- avg_rate/cir) constantly compares

the average rate observed with the target rate to keep the rate closer to the target.

ii)Network component cf*(cir/cir_max) provides a dynamic indication of congestion level status in the network. The marking probability increment is done in proportion to the target rate by multiplying cf with a weight factor cir/cir_max to mitigate the impact of the target rates.

Results

Expt #Rt 1 Rt 2 Ra1 Ra2 BE TCP flow Link goodput

(Mbps) (Mbps)1 1 1 2.54 2.58 3.76 8.882 1 2 2.54 2.58 3.76 8.883 1 3 2.41 2.93 3.46 8.84 2 3 2.36 2.89 3.58 8.835 3 3 2.8 2.8 3.21 8.816 3 4 2.73 3.49 2.59 8.81

8.835

Target Rates(Mbps) Achieved Rates (Mbps)

Average link bandwidth (Mbps)

Achieved Rates (Ra) for different Target Rates (Rt) -- under- and well-subscribed cases.

Results..

Expt #Rt 1 Rt 2 Ra1 Ra2 BE TCP flow Link goodput

(Mbps) (Mbps)1 2 6 1.83 4.85 2.06 8.742 3 5 2.5 4.04 2.05 8.593 3 6 2.4 4.6 1.53 8.534 1 8 1.2 6 1.28 8.485 4 6 3.17 4.5 0.11 7.786 2 8 1.55 6.16 0.72 8.43

8.425



Achieved Rates (Ra) for different Target Rates (Rt) -- over-subscribed cases

Results: Goodput vs Time Graph (2/6 Mbps target rate.)

-1

0

1

2

3

4

5

6

0 50 100 150 200 250

t ime (s)

go

od

pu

t (M

bp

s)

-2

0

2

4

6

8

10

0 50 100 150 200 250

time (s)

good

put (

Mbp

s)

Results..

Expt #Rt 1 Rt 2 Ra1 Ra2 BE TCP flow BE UDP Link goodput

(Mbps) (Mbps) (Mbps)1 2 6 1.52 4.18 0.46 3.54 6.162 3 5 2.08 3.41 0.44 2.52 5.933 3 6 2 4.42 0.13 2.12 6.554 1 8 0.66 6.34 0.01 1.87 7.015 4 6 2.65 4.6 0 1.5 7.256 2 8 1.21 6 0 1.6 7.21

6.685




Results..

Expt #Rt 1 Rt 2 Ra1 Ra2 BE TCP flow AS UDP Link goodput

(Mbps) (Mbps) (Mbps)1 1 1 1.7 1.77 2.61 2.99 6.082 2 2 1.92 1.88 2.27 2.99 6.073 3 3 2.37 2.47 1.18 2.99 6.024 4 4 2.92 2.98 0.13 2.98 6.035 5 5 3.12 2.83 0.1 2.97 6.05

6.05



Achieved Rates in presence of AS UDP and BE TCP

Inference

Achieves goodput close to the target rates. Succeeds in taking the share of BE TCP and UDP flows

in the worst case scenario. The average link utilization pretty good. The AS UDP flow gets its assured rate.

Linux Implementation of MBM

Incorporated with the existing traffic control functions of Linux.

Linux kernel version used was 2.2.14 , Redhat 6.2.

Traffic Control setup for Linux implementation of MBM

Areas for Deployment

Marker anywhere (lack of sensitivity to marker parameters).

MPLS over DiffServ.

Conclusion

transparency from the end hosts, simplicity, and least sensitivity to parameters of both TCP as well as marker parameters.

helps in achieving the target rate, with a better fairness in terms of sharing the excess bandwidth among flows.

provides the TCP flows, a greater degree of insulation from differences in RTT and window sizes.

overall link utilization also seems to be much better.

Conclusion…

Provides an architecture which is transparent to TCP sources and hence doesn’t require any modifications at the end hosts.

The edge-to-edge feedback control loop helps the marker to take proactive measures in maintaining the assured service effectively, especially during periods of congestion.

A single feedback control is used for an aggregated flow. Hence this architecture is scalable to any number of flows between the two edge gateways.

The architecture is adaptive to changes in load and network conditions.

The marking algorithm takes care of any bursts in the flows.

Acknowledgement

Dr. Lillykutty Jacob Prof. A.L.Ananda NS Community Rajesh,Boon Peng Michael, Srijith, Yong Xiang Saswat, Prashant, Sriram, RK All my dear friends My Family God Almighty

Papers published

Conferences: 1. K.R.R.Kumar, A.L.Ananda, Lillykutty Jacob,“A Memory-Based Approach

for a TCP-Friendly Traffic Conditioner in DiffServ Networks”, in Proc. of the 9 th IEEE International Conference on Network Protocols (ICNP 2001), Riverside, California.

2. K.R.R.Kumar, A.L.Ananda, Lillykutty Jacob, “Using Edge-To-Edge Feedback Control to make Assured Service More Assured in DiffServ Networks”, in Proc. of the 26th Annual IEEE Conference on Local Computer Networks (LCN 2001), Tampa, Florida.

Journal: 1. K.R.Renjish Kumar, A.L.Ananda, Lillykutty Jacob,“TCP-Friendly

Traffic Conditioning in DiffServ Networks : A Memory-Based Approach ”, accepted (invited paper) in Computer Networks, Elsevier Publications.

Thank You

improving tcp performance in a differentiated services network: investigations and solutions

Documents

tcp traffic

udp traffic

tcp parametersshould

tcp performance

unruly traffic

diffserv network

kalyanaraman et

intelligent marker