Synthesis of Robust Active Queue Management controllersin TCP Networks

Sabato Manfredi, Mario di Bernardo and Franco Garofalo

Abstract

This paper is concerned with the derivation of improved AQM control schemes to cope with unwanted variations ofcharacteristic parameters such as the the average round-trip time, the load and the link capacity. To achieve this aim, weuse novel robust control strategies for time-delay systems, which are based on the use of robust reduction methods and H∞

strategies resulting inmemoryandmemorylesscontrol laws respectively. To avoid the need for full state availability, difficultto satisfy in realistic network applications, the control strategy is complemented by a robust observer for time delay system.The control laws presented in the paper are validated and tested on different network scenarios through numerical simulationsin both Matlab/Simulink and Network Simulator-2.

I. BACKGROUND

Recently, novel congestion control strategies have been proposed to improve the performance of standard TCP-REDalgorithms (see for example [3] and references therein). It has been shown that control theory can offer an invaluable setof tools to improve the performance of existing AQM schemes which can be seen as particular types of feedback controlsystems [1], [3], [2], [25], [5], [8], [7], [6].For example, to deal with the disadvantages of a classical RED controller, a PI (proportional-integral) controller is proposedin [1] to obtain zero steady-state error, better responsiveness and robustness (see [1] for further details). The controlleris successful in achieving a better regulation of the queue length when compared with classical methods. Nevertheless, itperforms poorly in the presence of network parameter variations such as round-trip time, load and link capacity variationswhich are bound to occur in more realistic networks [8], [7], [12].Indeed in more realistic networks, the round-trip time (RTT) can also vary with respect to its nominal value because ofcongestion phenomena and other effects such as the presence of variations of responsive sources at different points inthe network (causing the variation of equivalent round trip time propagation delay), rerouting and additional unresponsivesource traffic. Also, the assumption of fixed long-lived TCP workload, often taken in the derivation of these models, isviolated in realistic network scenarios. Moreover, variations in the link capacity advertised from long-lived TCP flows canalso occur because of unresponsive traffic generated by UDP and short-lived TCP flows.To overcome these problems, robust control approaches have been recently presented in literature. In [13], [14] it ispointed out that time delays must be taken explicitly into account in the controller design. An H∞ controller for highspeed networks was designed in [15]. A drawback of the approach presented therein is that the controller was designedequalizing the nominal delays in the channels and therefore resulting in an under-utilization of the network as, in practice,the time delays of the channels can be significantly different from one another. Moreover, the robustness of the schemewas not guaranteed in the case of time varying delays. To overcome the drawbacks highlighted above, a robust controllerwas introduced in [16], based on the use of queue length information, which was shown to be robust against uncertainvarying multiple time delays. Then, to shorten the transient and achieve a faster steady state control, a different controlscheme was presented in [17]. Such scheme uses the outgoing link capacity in addition to the queue length informationin order to achieve steady state more rapidly.In [8], an H∞ Active Queue Management (AQM) controller is designed for a single bottleneck node supporting TCP flowsbased on the fluid model introduced in [4]. The controller uses only the queue length for feedback purposes. Other robustapproaches based on the fluid model developed in [4] have been introduced in [18], [19]. In particular in [18], a variablestructure technique was used to synthesise a sliding-mode controller which was shown to guarantee better robustness andresponsiveness than the classical PI strategy introduced in [1]. A drawback of the above controller is the large queueoscillations detected in the presence of a relatively small number of active TCP connections. Alternative AQM controllersbased on feedback compensation were presented in [19].In this paper, we will discuss a different approach to the design of robust AQM control strategies. Namely, we will detailthe synthesis of two alternative state feedback approaches (see [10], [9] for further details). A robust observer is used inthe control loop to avoid direct measurement of the transmission window, as this would be unpractical in applications.The controller is synthesised by using a robust reduction method and and an H∞ approach. The technique presented here

Sabato Manfredi and Franco Garofalo are with the Faculty of Engineering, University of Naples Federico II, Via Claudio 21, Napoli 80100, Italy.Email: {smanfred,fgarogal }@unina.itMario di Bernardo is with the Faculty of Engineering, University of Sannio, Benevento 82100, Italy.Email: dibernardo@unisannio.it

is shown to be applicable in practice through realistic simulations in NS-2 and experimentally. It will be shown that theimproved AQM control schemes can cope with unwanted variations of the average round-trip time, load and link capacity.It will be shown that the memoryless H∞ controller performs better that the one based on the use of reduction methods.Hence, this controller is further validated and tested through numerical simulations in NS-2 and experiments.

II. A FLUID MODEL OF TCP BEHAVIOR

The control design is based on the use of a fluid model of TCP dynamical behaviour, which was derived in [4] using thetheory of stochastic differential equations. The model describes the evolution of the average characteristic variables on thenetwork such as the average TCP window size and the average queue length. Extensive simulations inNetwork-Simulator(NS)-2 [20] have shown that the model captures indeed the qualitative behavior of TCP traffic flows. Under the assumptionof neglecting the TCP timeout, the model is described by the following set of nonlinear coupled ODEs:

Ẇ (t) =1

R(t)− W (t)

2W (t−R(t))R(t−R(t)) p(t−R(t)), (1)

q̇(t) =

{−C + N(t)R(t) W (t), q > 0max{0,−C + N(t)R(t) W (t)}, q = 0

(2)

whereW is the average TCP window size (packets),q the average queue length (packets),Tp the propagation delay (s),R the transmission round-trip time (R = qC + Tp), C the link capacity (packets/s),N the number of TCP sessions andpthe probability of a packet being marked. All variables are assumed non negative.If we assumeN(t) = N , R(t) = R0 then we can linearize the dynamic model (2) about the operating point (W0, q0, R0)obtaining the following set of linear time-delay ODEs [1]:

δẆ (t) = − NR20C

(δW (t) + δW (t−R0))

− 1R20C

(δq(t)− δq(t−R0))

−R0C2

2N2δp(t−R0), (3)

δq̇(t) =N

R0δW (t)− 1

R0δq(t),

whereδW.= W −W0, δq .= q − q0 andδp .= p− p0.

In what follows, for the sake of clarity, we will consider the same parameter values considered in [1], i.e.C = 3750packets,R0 = 0.246 s,N = 60 sessions,qmax = 600 packets corresponding to the steady-state operating regimeW0 = 15packets,q0 = 200 packets,p0 = 0.008.We observe that as shown in [3], the delayR0 in the state termδW (t−R0) in (3) can be neglected whenW0 À 1. Thisis a realistic assumption for a typical network operating condition and so we obtain the simplified model:

δẆ (t) = − 2N0R20C0

δW (t)− R0C20

2N20δp(t−R0)

δq̇(t) =N0R0

δW (t)− 1R0

δq(t). (4)

Note that the linear model (3) is valid under some restricting assumptions. In particular: (1) the model captures only thebehavior of long-lived TCP connections neglecting slow start and timeout mechanisms; (2) different source behaviors arenot taken into account such as unresponsive flows and different types of TCP connections; (3) nominal fixed values areassumed of the number of connectionsN , the round trip timeR and the link capacityC. These can all vary in realnetworks.Despite its limitations, the linear model has been successfully used to design effective AQM control schemes [1], [8], [6],[9], [7], [10], [19], [18].The AQM control strategies briefly outlined below (see [9], [10] for further details) are designed using the models (3) and(4) respectively. The robust nature of the strategy presented here will not only reduce the sensitivity to network parametersbut also eliminate inaccuracies due to the use of a linear model.

III. A STATE FEEDBACK APPROACH

In what follows we will carry out the design of a state feedback control scheme for active queue management (AQM)which is robust to unwanted variations of parameter uncertainties as round trip time, load and link capacity. In so doingwe will proceed in two stages. Firstly, a robust observer will be designed to estimate the transmission window online,avoiding the requirement for full state availability. Secondly, an appropriate robust controller for time-delay systems issynthesized to cope with round-trip time, load and link capacity variations.

δp

RobustController

RobustObserver

Lineariz edFluid

δw

δw

δqModel

Fig. 1. Block Diagram of the proposed Robust AQM control scheme.

The resulting control scheme is shown in Fig. 1. In order to synthesize the controller and the observer in Fig. 1 we adaptto AQM control the theoretical approaches recently presented in [29], [22] and [11]. A more detailed theoretical analysisof the schemes can be found in [9], [10].We wish to emphasize that by using a robust observer, we were able to synthesize a full state feedback controller which,because of its nature, can guarantee better performance than other control scheme. In our case, the control action is basedon the use of both states, the queue lengthq and the window sizeW .

A. Robust observer for time-delay systems

The control scheme introduced above requires full availability of the system states for feedback purposes. So we proposeto equip the control law with an appropriate robust observer for time-delay systems. In what follows, we adapt the resultspresented in [11] to the case of interest. The design uses a linear matrix inequality approach in order to guarantee thestability of the observer and reduce the effect of model uncertainties on the estimated state.We consider the following time delay state space system:

ẋ(t) = Ax(t) + Adx(t− τ(t)) + Bdu(t− τ(t)),y = Cx(t), (5)

with state vectorx(t) ∈ Rn, outputy(t) ∈ Rp, inputu(t) ∈ Rr, initial conditionsx(0) = Ψ(t), t ∈ [−τ, 0] andu(t) = Φ(t)for t∈ [−τ, 0].Let G(s) = C(sIn − A − Ade−sτ )−1Bde−sτ be the transfer function representation of system (5). SayG̃(s, z) =G(s, z)+Γ(s)∆(s, z) the transfer function of the system under the effect of additive uncertainties of the formΓ(s)∆(s, z)with z = e−sτ ; Γ(s) being a fixed stable weighting transfer function matrix and∆(s, z) a variable stable transfer functionwith ‖ ∆(s, z) ‖∞≤ 1. As discussed in [11], it is possible to show that ifZ(s) = HX(s) is the quantity to be observedthen,

Ẑ(s) = H(sI2 −A−Ade−sτ + LC)−1Bde−sτU(s)+ H(sIn −A−Ade−sτ + LC)−1LY (s) (6)

generates a robust estimation ofZ(s) if, given a scalarγo > 0, there exist a matrixS ∈ Rn×r and two symmetric positivedefinite matricesP andR such that:

M PAd 0 0 SD̄ATd P −R̄ 0 0 0

0 0 −R̄ 0 00 0 0 −Ir 0

D̄ST 0 0 0 −Ir

< 0

whereM = AT0 P + PA0 + R +1γ2o

HT H − SC − CT ST , R̄ = 12R, D̄ =‖ Γ(s) ‖∞. Y (s) is the Laplace transform ofthe measured output of system (13) and

L = P−1S. (7)

In particular it can be shown that the time evolution of the estimationẑ(t) given by (6) satisfies the following conditions:{

limt−→∞(z(t)− ẑ(t)) = 0, for Γ(s) ≡ 0,‖z(t)− ẑ(t)‖2, is bounded forΓ(s) 6= 0. (8)

We note that, as shown in [11], choosing a minimalγo reduces the effects of uncertainties on the estimation error.

IV. T WO ALTERNATIVE APPROACHES

In what follows, we will briefly present two alternative robust AQM controllers that coupled with the observer presentedin Sec. III-A can be used to implement the state feedback AQM control scheme shown in fig. 1. The first approach (see[10] for further details) is based on the use of a reduction method for time delay system, leading to a control law withmemory. The second (detailed in [9]) uses instead an H∞ theoretical synthesis approach resulting in a memoryless controllaw which is found to be more effective for practical applications through numerical and experimental tests. It is worthpointing out that, through the use of the observer, other state feedback approaches could be used as well to further improvethe performance of the existing schemes. An important constraint to be kept into account is the highly uncertain natureof communications over networks and the practical constraints on the controller implementation.

A. Robust reduction-based AQM Control (RRAQM)

The reduction transformation method presented in [21] can be briefly outlined as follows (see [21] for further details).Consider a time-delay system of the form

ẋ(t) = Ax(t) + Bdu(t− τ),y = Cx(t), (9)

with state vectorx(t) ∈ Rn, output y(t) ∈ Rp, input u(t) ∈ Rr, known and fixed nominal delayτ , initial conditionsx(0) = Ψ(t), t ∈ [−τ, 0] andu(t) = Φ(t) for t∈ [−τ, 0]. The idea is to compensate the effects of the delay and stabilize(9) by considering the linear transformation

z(t) .= x(t) +∫ t

t−τeA(t−τ−α)Bdu(α)dα.

Under this transformation, it can be shown (see [21]) that system (9) can be recast in terms of the new state variableszas the delay-free set of ODEs:

ż(t) = Az(t) + Bu(t), (10)

where the matrixB is defined asB = e−τABd. Under the assumption that(A,B) is stabilizable, system (9) can bestabilized using a classical state-feedback designed on the reduced model (10) given by

u(t) = Kz(t) = K(

x(t) +∫ t

t−τeA(t−τ−α)Bdu(α)dα.

)(11)

Consequently by using the reduction method approach, we can derive a stabilizing control for system (9) by designinga stabilizing controller for the ordinary system (10). Thus, the matrixK can be chosen according to one of the manyavailable methods for delay-free dynamical systems. The main disadvantage of the reduction approach presented sofar isthat it requires exact knowledge of the system equations. This is often impossible in real application, particularly in thecase of communication networks. So we consider parametric uncertainties in model (5) and designK in (11) that canrobustly stabilize the system following the strategy presented in [22].We assume that under the effect of time varying uncertainties∆A(t) and∆B(t), system (5) becomes:

ẋ(t) = (A + ∆A(t))x(t) + (Bd + ∆B(t))u(t− τ), (12)with ∆A(t) = DF (t)E, ∆B(t) = D1F1(t)E1, whereD, D1, E and E1 are real constant matrices with appropriatedimensions, andF (t), F1(t) are unknown matrices satisfying‖ F (t) ‖≤ 1, ‖ F1(t) ‖≤ 1. Then, the robust stabilizingfeedback gainK is designed according to the following theorem (see [22] for further details).Theorem If there exist matricesX > 0, J , T > 0 and scalarsλ > 0, γ > 0, ε > 0 such that

Ξ XET JT BT0EX −λI 0B0J 0 −γΘ

< 0;

[T JT ET1

E1J εI

]> 0,

whereΞ = XAT + AX + JT (e−AτBd)T + (e−AτBd)J + (λ + γ)DDT + εD1DT1 + T

and Θ is a positive definite matrix satisfyingΘ ≥ τ ∫ 0−τ eAT αET EeAαdα, then system (13) withK = JX−1 in (11) is

asymptotically stable.See [22] for a proof of this theorem.The approach presented above can now be used to obtain an improved robust AQM control scheme based on the reductionmethod (RRAQM) in the presence of parameter uncertainties as round trip time, load and bottleneck capacity. Firstly, noticethat the linearized time-delay model of TCP-IP given by (4) is equivalent to a pure system delay (9) withx = (δW δq)T ,u = δp, τ = R0 and:

A =

[− 2N0

R20C00

N0R0

− 1R0

]; Ad =

[0 00 0

]; Bd =

[−R0C20

2N200

]; CT =

[01

].

We consider for the parameter uncertainties in the model (4) the bounds|N − N0| ≤ ∆N , |R − R0| ≤ ∆R and|C − C0| ≤ ∆C which can be recast in the appropriate form as required in the previous sections. Thus applying thereduction transformation method, a control was chosen of the form (11) in order to locally stabilize system (4) in a robustmanner around its desired equilibrium point. A disadvantage of this control scheme is that it requires the window sizeW for feedback purposes as this is typically hard to measure. To overcome this problem, we design the robust observerintroduced in Sec. III-A to estimateδW online for the linear time-delay system (4) with uncertainties. (Note thatŴ canbe obtained by addingˆδW to the nominal valueW0). In particular, as explained in Sec. III-A, the observer generates arobust estimate of the window sizêW with zero estimation error during nominal operating conditions while guaranteeingbounded estimation error in perturbed conditions (see [10] for more details).

V. NUMERICAL VALIDATION

In what follows, the RRAQM strategy is tested and compared with a classical PI-AQM control strategy in different networkscenarios. All the simulations presented here are carried out using Simulink on the network topology introduced in [1].The round trip time variations are those obtained by varying the round trip propagation delayTp while the parameters ofthe PI controller are those suggested in [1]. In particular we will validate the effectiveness of RRAQM on two differentscenarios: 1)N(t) = N0 and slowly varyingR(t), 2) slowly varyingR(t) and loadN(t). More simulation results havebeen presented in [10].

A. Case 1: N(t) = N0 and slowly varyingR(t)We will consider the case where the round trip propagation delayTp depicted in Fig. 2 is slowly varying in the interval(0.2, 0.5) corresponding to a typical uncertainty interval about the nominal value of the round trip time in a communicationnetwork, also in the absence of strong congestion [12]. The evolution of the queue length for the robust RRAQM schemeand the PI controller, depicted in Fig.3(a) shows that the robust RRAQM scheme presents an improved transient andsteady-state performance. Namely, the PI controller shows consistent variations of the queue length (between minimumand maxim values) and so of the jitter delay. Moreover, we observe intervals where packets are lost due to queue overflows(q ≥ 800). The robust RRAQM, instead, guarantees an acceptable steady-state response and eliminates all packet lossesover the time range of interest while reducing queue variance. The resulting estimation error is bonded as expected fromthe theory presented in Sec. III-A.

B. Case 2: slowly varyingR(t) and N(t)Now as shown in Fig. 4 larger variations of the round-trip propagation delay (Tp ∈ (0.25, 0.92)) are considered togetherwith load variations (N variable∈ (30, 270) with bursts at10th, 30th, 70th seconds) representing a strong congestionscenario. We can observe in Fig. 5 that also in this case RRAQM performs significantly better than the PI scheme withno packet losses, low queue excursion, better queue utilization and an acceptable window estimation. Similar results areobserved when a slow link capacity variation ofC ∈ (3100, 3750)1 is introduced in the previous scenario. The variationof such link capacity might represent the reduction of bandwidth advertised from long-lived TCP flow connection due tothe presence of background unresponsive traffic flows as UDP and short-lived TCP.

1the variation inC is obtained by adding a square wave signal (period 30 s, amplitude of 700 packets) to a normal random signal with mean equalto 3750 packet, variance equal to300 and sample time0.1.

0 10 20 30 40 50 60 70 80 90 1000.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

Fig. 2. Case 1:Tp variations

0 10 20 30 40 50 60 70 80 90 1000

100

200

300

400

500

600

700

800

time [sec]0 10 20 30 40 50 60 70 80 90 100

0

2

4

6

8

10

12

14

16

18

time [sec]

(a) (b)

Fig. 3. Case 1:(a) Time evolution of the queue length for the PI-AQM (dotted line) and RRAQM schemes applied to the nonlinear model;(b) Timeevolution of the window size and estimated window size (dotted line) on nonlinear system.

The proposed controller can be implemented in NS-2 [20] and experimentally by using the suggestions in [23], [24] onhow to physically implement the integral term in (11). Preliminary investigations of the scheme in NS-2 confirm thesimulink results shown above. However, the practical implementation of the control strategy can be cumberson as thecomputation of the integral in the control law involves the storage of many additional variables depending on the choiceof the sample time. Thus, in what follows we summarize an alternative memoryless H∞ AQM control approach that isshown to be more effective in practical applications through NS-2 and experimental tests. (A more detailed analysis ofthe scheme presented here together with a thorough experimental and numerical validation will be presented in [9].)

C. H∞ state feedback control

The design of a robust H∞ state feedback controller for continuous time systems with time-varying delays was recentlydiscussed in [29]. The resulting controller was shown to guarantee quadratic stability of the closed loop system andacceptable error bounds. Moreover, the design can be easily extended to the case of time-delay systems with uncertainparameters. In particular, given the time-delay system:

ẋ(t) = (A + ∆A(t))x(t) + (Ad + ∆Ad(t))x(t− τ)+ (Bd + ∆Bd(t))u(t− τ),

y = Cx(t). (13)

it is possible to design a robust feedback controller for system (13) under the assumption that all states are measurableand the time-varying delayτ(t) satisfies the conditions:

0 ≤ τ(t) < ∞, τ̇(t) ≤ β < 1; (14)moreover it is supposed that the matrices representing uncertainties in (13) can be expressed as∆A(t) = H∆F (t)E∆,∆Ad(t) = H∆F (t)Ē∆ and∆Bd(t) = H∆F (t)Eu∆ whereH∆, E∆, Ē∆ andEu∆ are appropriate matrices, andF (t) is

0 10 20 30 40 50 60 70 80 90 1000.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 10 20 30 40 50 60 70 80 90 1000

50

100

150

200

250

300

(a) (b)

Fig. 4. Case 2:(a) Tp variations;(b)N variations

0 10 20 30 40 50 60 70 80 90 1000

100

200

300

400

500

600

700

800

time [sec] 0 10 20 30 40 50 60 70 80 90 1000

2

4

6

8

10

12

14

16

18

(a) (b)

Fig. 5. Case 2:(a) Time evolution of the queue length for the PI-AQM (dotted line) and RRAQM schemes applied to the nonlinear model;(b) Timeevolution of the window size and estimated window size (dotted line) on nonlinear system.

such thatF (t) : F (t)T F (t) ≤ I with its elements being Lebesgue measurable. Under these assumptions, it is possible toshow that, for a given positive constantγ, the controller

u(t) = Kx(t), (15)

renders the uncertain time-delay system (13) quadratically stable with an H∞-norm error boundγ if there exist positivedefinite matricesQ, S1, S2, a positive constantλ and a matrixM such that

U1 γλH∆ U2 MT Q

(γλH∆)T −γ2I 0 0 0UT2 0 −I 0 0M 0 0 −S2 0Q 0 0 0 −S1

< 0 (16)

where

U1 = QAT + AQ + (1− β)−1AdS1ATd+ (1− β)−1BdS2BTd , (17)

U2 = QCT + (1− β)−1AdS1C̄Td + (1− β)−1BdS2D̄Td ,U3 = −I + (1− β)−1C̄dS1C̄Td + (1− β)−1D̄dS2D̄Td ,C̄ = [C

1λ

E∆]T ; C̄d = [01λ

Ē∆]T ; D̄d = [01λ

Eu∆]T

M = KP−1,Q = P−1,Si = R−1i , i = 1, 2.

Note that the controller gainsK in Sec. (15) can be computed by using the relationshipM = KP−1 after solving theLMI problem (16) with respect toQ, S1, S2 andM (see [29] for further details).The linearized TCP fluid model (3) can be rewritten as the uncertain time-delay system (13) withx = (δW δq)T , u = δp,τ = R0 and

A =

(− N

R20C− 1

R20CNR0

− 1R0

), Ad =

(− NR20C

1R20C

0 0

)

Bd =(−R0C22N2

0

), (18)

initial conditionsx(0) = Ψ(t), t ∈ [−τ, 0], andu(t) = Φ(t) for t∈ [−τ, 0]. ∆A(t), ∆Ad(t) and ∆Bd(t) represent timevarying model uncertainties with appropriate form and dimensions depending on network parameters uncertainties.As discussed above, it is possible to show that, for a given positive constantγ, the controller

u(t) = Kx(t) (19)

with K obtained by solving a further appropriate LMI problem, renders the uncertain time-delay system (13) quadraticallystable with an H∞ norm boundγ. Besides as shown in Sec. III-A, it is possible to design a robust observer for system(13) by resolving an LMI problem.We consider for the parameter uncertainties in the model (3) the bounds|N − N0| ≤ ∆N , |R − R0| ≤ ∆R and|C − C0| ≤ ∆C which can be recast in the appropriate form as required in the controller and observer design. Notethat, as for all other AQM control schemes based on the linearized model of the TCP flow, the controller guaranteeslocal stability when applied to the nonlinear TCP fluid model. As will be shown in Sec. VI, the controller validationin network-simulator, shows that the controller presented here performs well when applied to control a realistic TCPconnection.

VI. CONTROLLER VALIDATION THROUGH NS-2

Now, we shall seek to validate the effectiveness of the H∞ AQM Controller (shortly labelled as RHC) derived above andcompare its performance with respect to other AQM schemes. To this aim, we used NS-2, a network simulator widely usedin the control and communication communities [20]. Simulations, unless otherwise stated, refer to the a single bottleneckedrouter running the AQM scheme connected by a link with a capacity of 15Mb/s. The propagation delays for the flowsis uniformly distributed between200 and280 ms. The target queue lengthq0 is set to200 packets while the maximumbuffer size is600. The average packet length is set to1 Kbyte. The default transport protocol is TCP-Reno with ECNmarking [26]. The control parameters are chosen as summarized in table I. They were selected for this network topologyin accordance to the guidelines given in the references reported in the table, where the control strategies considered werefirst presented.The parameters of the RHC controller are those summarized above. The sampling frequency of the RHC control schemeis fs = 160Hz as in [1]. For all other schemes the sampling frequency, such as the rest of unspecified parameters, arefixed to values recommended in the original papers and in NS-2. We report below a variety of numerical simulationsand experiments. Namely, we investigate (i) the observer effectiveness; (ii) the robustness of RHC to network parameteruncertainties; (iii) its dynamic behaviour for different network traffic scenarios; (iiii) the case of a tree topology withdifferent sources, links and multiple round-trip times. Then, in Sec VII we present some experimental results. Note that,for the sake of brevity, the comments to the figures reported below are kept to a minimum.

TABLE I

CONTROLLER PARAMETERS

Control typeRED (gentle) [30] minth = 100 maxth = 300 pmax = 0.1

PI [1] a = 0.00001822 b = 0.00001816 fs = 160REM [27] α = 0.1 γ = 0.001 φ = 1.002AVQ [28] α = 0.15 β = 0.8 γ = 0.98VRC [6] KP = 0.0024 KI = 0.0045 KD = 0.0003

A. Observer validation

To perform a first validation of the observation strategy, we consider a single bottlenecked router running the robust AQMscheme connected in the network scenario summarized above. We note that in the controller implementation we set theequilibrium probability value to zero (p0 = 0) as done in [3]. In this case, the observer estimates the variationδŴ aroundthe equilibrium point. The value ofW at the equilibrium can be obtained by using the mean value of the probabilityp0 = δp0, and using the expression:W 20 p0 = 2.Note that, for this value of the parameters, the expected mean value of the window size of the sources isŴ = 19.5packets. In Fig. 6(a) the estimated value of the window size variation from its nominal value,δŴ is presented. Asdepicted in Fig. 6(b), in this case we havep0 = 0.0046, hence the measured average window size at the equilibriumis given byW0 =

√2p0' 21. In so doing the value of the average window size estimated by the observer on-line is

W = δW + Wo = 21 packets, close to the expected average window sizeW̄ equal to19.5 packets.

X Graphpkt

time

-1.8000

-1.6000

-1.4000

-1.2000

-1.0000

-0.8000

-0.6000

-0.4000

-0.2000

-0.0000

0.0000 20.0000 40.0000 60.0000 80.0000 100.0000

X Graphp x 10-3

time0.0000

2.0000

4.0000

6.0000

8.0000

10.0000

12.0000

14.0000

16.0000

18.0000

20.0000

0.0000 20.0000 40.0000 60.0000 80.0000 100.0000

(a) (b)

Fig. 6. (a) Estimated value of the window size variation,δŴ ; (b) Packets probability mark,δp

B. Robustness to network parameters uncertainties

Now, we examine the robustness of the novel robust AQM scheme in the presence of variations of the loadN and the roundtrip propagation delayTp. In particular we evaluate average and standard deviation queue length in order to verify theAQM control effect on queue stabilization and therefore on queueing and jitter delays. In particular high queue standarddeviation implies variable round trip time for the sources and also low queue utilization if the queue goes frequentlyto zero. In order to better evaluate the differences in the controllers performance standard deviation we have chosen arelatively large queue (buffer size of600packets).1) Robustness to load variations:Considering the introduced single bottleneck topology we repeat simulation for differentN varying from 20 to 400.As shown in Figs 7-8, the VRC control scheme guarantees the best set point regulation thanks to its derivative componentthat causes high variance with queue oscillations for lower values of the load (down to60). The PI control scheme showshigh queue variance for both high and low load levels, while not achieving the desired set point from a medium load valueof about180 sessions. The RHC scheme presented in this paper guarantees a good queue stabilization with limited queueoscillations. As expected, RHC does not guarantee a perfect set point regulation because the principal aim of RHC controlis the queue stabilization in the presence of load and RTT variations. Anyway the set point error is never greater than 20percent with respect to the set point. Also, the REM scheme shows a relatively good behavior. The queue value underRED control is directly dependent upon theN value. AVQ control performs poorly. This can depend on the particular

(a) (b)

Fig. 7. Mean queue under N variations:(a) PI, RHC, VRC, REM;(b) RHC, RED, AVQ.

(a) (b)

Fig. 8. Queue standard deviation under N variations:(a) PI, RHC, VRC, REM;(b) RHC, RED, AVQ.

choice of topology. Indeed in [28], it is remarked that this control strategy is particularly suited for the case of low queuelength.2) Robustness to variations of the Round trip propagation delay:We consider again a single bottleneck topology andrepeat simulations for different values of the average round trip propagation delay varying from133 to 654 ms. ThecapacityC is 15Mb/s. We note that in the case of single bottleneck in order to guarantee a full queue for increasing delaywe have set a value of the load,N = 270, higher than the nominal value. This has also the effect of testing the controllersin the presence of aggressive sources when the delay is low.Here we have represented the queue statistics as function of one way delay given by adding average sourcesTp to q0/C.We note in Figs. 9-10 that, in this situation, the novel RHC scheme shows the best performance when compared with othercontrollers. In particular, as expected, the VRC controller shows a poor set-point regulation and high standard deviation,while the PI controller performs poorly for low delay values. Also, the RED controller presents an inverse dependencefrom delay variation while AVQ performance improves for increasing delays. In conclusion, when compared to otherexisting schemes, our simulations show that the RHC presents the best performance in term of robustness to networkparameter variations. Further tests and the robustness of the scheme to variations of the bottleneck link capacity have beenreported elsewhere [9].

C. Dynamic behavior validation: tree topology with different sources and multiple RTT

We consider the network tree-topology in Fig. 11 with many bottlenecks and sources located at each level. This isrepresentative of a more realistic network scenario where sources at different distances accessing the network, perceivedifferent values of the round trip time. In particular there are three levels. The first level is characterized by4 sources, thesecond level by16 and the third by64, resulting in84 sources overall. The RTT is uniformly distributed in the ranges

(a) (b)

Fig. 9. Mean queue underTp variations:(a) PI, RHC, VRC, REM;(b) RHC, RED, AVQ.

(a) (b)

Fig. 10. Queue standard deviation underTp variations:(a) PI, RHC, VRC, REM;(b) RHC, RED, AVQ.

[0.120, 0.200]s, [0.200, 0.320]s and[0.280, 0.440]s for the first, second and third level respectively. So the link at a greaterdistance from the tree root exhibits longer latency, assuming the link with the fastest bandwidth being located around thecenter of network. Beside in the experiment we add to the84 TCP flows for UDP flows transmitting on 1Mbit/s than are

Fig. 11. tree topology.

on from the0th s to 50th s. We note from Fig. 12 that VRC performs badly when UDP flows go down, AVQ is too

aggressive in presence of congestion increasing. Also RHC and PI are present better performances with the most fast timeresponse of RHC controller.We note that the effects remarked in the previous sections can be more impressive in the case of lower queue utilization.In particular high queue standard deviation would imply higher packet losses and low queue index utilization.

VII. E XPERIMENTAL VALIDATION

Some of the previous laws are tested and validated experimentally. In particular RHC, PI and RED controllers are comparedin single bottleneck scenario of10Mbit/s. Differently from what observed in the simulations, in the experimental runs theround trip time value is not uniformly distributed but has a specific settled value. The mean dimension of each packet is1500 byte and the transport protocol is not based on ECN marking. In the experiments, the controllers are implementedby usingFreeBSD4.7 operating system. For flows generation, thenetperf and iperf softwares were used. Destination andsource hosts are linux based.

A. Experimental validation - Nominal case

A single bottleneck is considered withN = 60 TCP flows with Tp = 0.240 s. In Figs. 13-15, the controlled queueevolution under the effects of PI, RED and RHC schemes are reported. We note that RHC guarantees a good set pointregulation with less oscillations and packet losses, and a faster time response when compared to the dynamics of PI andRED based schemes. We note that RED does not guarantee the set point regulation.

B. Experimental validation - Different load and Round trip propagation delay

We test the AQMs controllers for higher round trip time variations with respect to the nominal case. We note in Fig. 16-18a better performance of the RHC strategy in terms of queue variance, time response and queue regulation.

C. Experimental validation - Dynamic load variation

Now we consider the load variationN in tab. II.

TABLE II

DYNAMIC LOAD VARIATION

Time interval [s] Nt=0-20 60t=20-60 90t=60-120 40t=120-180 160t=180-250 60

As shown in Figs. 19-21, RHC control maintains a better set point regulation when compared to other schemes and iseffective in reducing queue variations.Another advantage is the reduced number of packet losses under RHC. Further experiments will be reported in [9].

VIII. C ONCLUSIONS

We have discussed the design of improved AQM control schemes to address the issue of variations of the networkparameters unavoidable in practical applications. Robust state feedback controllers were synthesized which take explicitlyinto account the uncertain, time-delay nature of the system under investigation. A robust observer was used in the schemespresented to estimate the window size online and therefore synthesize a full state feedback controller. The control synthesiswas carried out on a linearized fluid model of a TCP connection often used in the literature. The strategies were shown tobe particularly effective on realistic network models through extensive simulations in Network Simulator. The proposedcontrollers are designed on a single bottleneck topology and they would still behave reasonably in presence of differentnetwork topology [30]. It is worth mentioning here that the control law can be implemented as a distributed algorithm ina multi-bottleneck scenario as the presence of the observer makes the controller dependent only upon local information.This is the subject of ongoing research.

REFERENCES

[1] C. V. Hollot, V. Misra, D. Towsley, W. Gong,Analysis and Design of Controllers for AQM Routers Supporting TCP Flows, IEEE ACM Trans onAutomatic Control, Vol 47, no. 6, pp. 945-959, 2002.

[2] Y. Chait, C. V. Hollot, V. Misra, S. Oldak, D. Towsley, W. Gong,Fixed and Adaptive Model-Based Controllers for Active Queue Management,Proc. American Control Conference, pp 2981-2986, 2001.

[3] C. V. Hollot, V. Misra, D. Towsley, W. Gong,A control theoretic analisys of RED., IEEE INFOCOM 2001.[4] V. Misra, W. B. Gong, and D. Towsley,Fluid-based analysis of a network of AQM routers supporting TCP flows with an application to RED,

ACM SIGCOMM, 2000.[5] S.H. Low, F. Paganini, J.C. Doyle,Internet Congestion Control, IEEE Control Systems Magazine, pp. 28-43, 2002.[6] E.C. Park, H. Lim, K.J. Park, and C.H. Choi,Analysis and Design of the Virtual Rate Control alghorithm for stabilization Queues on TCP networks,

Comput. Networks, Vol.44, N.1, pp.17–41 2003[7] S. Manfredi, M. di Bernardo, F. Garofalo,A Robust Approach to Active Queue Management Control in Networks, to appear on Proc. IEEE

International Symposium on Circuits and Systems, 2004[8] P.F. Quet and H. Ozbay,On the Design of AQM Supporting TCP Flows Using Robust Control Theory, submitted for publication at IEEE Transactions

on Automatic Control, September 2002.[9] S. Manfredi, M. di Bernardo, F. Garofalo,Robust AQM Control for Queue Stabilization in TCP Networks, submetted to Automatica, 2004

[10] S. Manfredi, M. di Bernardo, F. Garofalo,Reduction-based Robust Active Queue Management Control, submitted to System Control Letters, 2004[11] A. Fattouh, O. Sename, J. M. Dion,A LMI approach to robust observer design for linear time-delay systems, Proc. of the39th IEEE Conference

on Decision Control, Sydney, Australia, pp. 1495-1500, 2000.[12] Larry L. Peterson, Bruce S. Davie,Computer Networks: a system approach, Morgan Kaufnann 2003.[13] L. Massoulie,Stability of distribuited congestion control with heterogeneous feedback delays, Microsoft Research Technical Report, 2000.[14] G. Vinnicombe,On the stability of end to end congestion control for the internet, University of Cambridge Technical Report, 2001.[15] H. Ozbay, S. Kalyanaraman and A. IftarOn rate based congestion control in high speed networks: design of an H∞ based flow controllerfor

single bottleneck, American Control Conference, vol.4 pp. 2376-2380, 1998.[16] P.H. Quet, B. Ataslar, A. Iftar, H. Ozbay, T. Kang and S. KalyanaramanRate based floe controllers for communication networks in the presence

of uncertain time varying multiple time delays, Automatica, vol.38, pp. 917-928, 2002.[17] P.H. Quet, P. Yan, and H. OzbayH∞ and LMMSE based capacity predictors for flow control in communication networks, submetted for pubblication

to Automatica, February 2002.[18] R. Fengyu, L. Chuang, Y. Xunche, S. Xiuming, W. FubaoA robust Active Queue Management Algorithm based on Sliding Mode Variable Structure

Control, Infocom’02, New York, pp. 64-79, 2002.[19] Z. Heying, L. Baohong, D. WenhuaDesign of a Robust Active Queue Management Algorithm based on Feedback Compensation, Sigcomm’03,

2003.[20] J. Fall, K. Varadhan,The ns manual, 2001.[21] Know, W.H. and A.E. Pearson,Feedback stabilization of linear systems with delayed control, IEEE Trans. Automatic Control, Vol. 25, no. 2, pp.

266-269, 1980.[22] Y. S. Moon, P. Park, W. H. Kwon,Robust stabilization of uncertain input-delayed systems using reduction method, Automatica, vol.37, pp. 307-312,

2001[23] V.V. Assche, M.Dambrine, J.F. Lafay, J.P. Richard,Some problems arising in the implementation of distribuited delay control laws, 38a Conference

on Decision and Control, Phoneix, Arizona USA, pp.4668-4672.[24] Q.C. Zhong,Implementation of Distribuited Delay in Control Laws, preprint submitted to Elsevier Science.[25] J. Aweya, M. Ouelette and D.Y. Montuno,A control theoretic approach to active queue management, Computer Newtworks, vol.36, pp. 203-235,

2001.[26] K.Ramakrishnam and S.Floyd,A proposal to add Explicit Congestion Notification (ECN)to ip, RFC 2481, 1999.[27] S. Athuraliya, S.H. Low, V.H. Li, Q. Yin,REM: Active queue managment, IEEE Network, 15 (3) pp. 48-53 2001.[28] S.Kunniyur and R.Srikant,Analysis and design of an adaptive queue (AVQ) algorithm for active queue management, in the proceedings of ACM

Sigcomm 2001, San Diego, California, pp.123-134, 2001.[29] J.H. Kim, H.B. Park,H∞ state feedback control for generalized continuous/discrete time-delay system, Automatica 35, pp. 1443-1451, 1999.[30] S. Floyd and V. Jacobson,Random Early Detection gateways for congestion avoidance, IEEE/ACM Transactions on Networking, Vol.1, no.4, pp.

397-413 1993.

(a) (b)

(c) (d)

(e) (f)

Fig. 12. Tree-topology. Time evolution of the queue length (ecn):(a) RHC; (b) PI; (c) VRC, (d) AVQ; (e) RED; (f) REM.

Fig. 13. Experiment:N = 60, RTT = 0.240s. Time evolution of the queue length for PI controller.

Fig. 14. Experiment:N = 60, RTT = 0.240s. Time evolution of the queue length for RED controller.

Fig. 15. Experiment:N = 60, RTT = 0.240s. Time evolution of the queue length for RHC controller.

Fig. 16. Experiment: N=180,RTT = 0.500s. Time evolution of the queue length for PI controller.

Fig. 17. Experiment: N=180,RTT = 0.500s. Time evolution of the queue length for RED controller.

Fig. 18. Experiment: N=180,RTT = 0.500s. Time evolution of the queue length for RHC controller.

Fig. 19. Experiment: dynamic load variations in Tab. II. Time evolution of the queue length for PI controller.

Fig. 20. Experiment: dynamic load variations in Tab. II. Time evolution of the queue length for RED controller.

Fig. 21. Experiment: dynamic load variations in Tab. II. Time evolution of the queue length for RHC controller.