traffic matrix estimation in non-stationary environments

Advanced Technology LaboratoriesAdvanced Technology Laboratories

Traffic Matrix Estimation in Non-Stationary Environments

Presented by R. L. Cruz

Department of Electrical & Computer EngineeringUniversity of California, San Diego

Joint work with

Antonio NucciNina Taft

Christophe Diot

NISS Affiliates Technology Day on Internet TomographyMarch 28, 2003


The Traffic Matrix Estimation Problem

• Formulated in

Y. Vardi, “Network Tomography: Estimating Source-Destination Traffic From Link Data,” JASA, March 1995, Vol. 91, No. 433, Theory & Methods



ingress

egress

Xj

Xj

Yi

PoP (Point of Presence)

Y = A X

Link Measurement Vector

Routing Matrix

“Traffic Matrix”



• Importance of Problem: capacity planning, routing protocol configuration, load balancing policies, failover strategies, etc.

• Difficulties in Practice– missing data – synchronization of measurements (SNMP)– Non-Stationarity (our focus here)

• long convergence time needed to obtain estimates


What is Non-Stationary?

•Traffic Itself is Non-Stationary


What is Non-Stationary?

• Also, Routing is Non-Stationary– e.g. Due to Link Failures– Essence of Our Approach

• Purposely reconfigure routing in order to help estimate traffic matrix

– More information leads to more accurate estimates

• Effectively increases rank of A• We have developed algorithms to

reconfigure the routing for this purpose (beyond the scope of this talk)


Outline of Remainder of Talk

• Describe the “Stationary” Method– Stationary traffic, non-stationary routing– Stationary traffic assumption is reasonable if we

always measure traffic at the same time of day (e.g. “peak period” of a work day)

• Briefly Describe the “Non-Stationary” Method– Both non-stationary traffic and non-stationary

routing– More complex but allows estimates to be obtained

much faster


Network and Measurement Model

• Network with L links, N nodes, P=N(N-1) OD pair flows

– K measurement intervals, 1 ≤ k ≤ K – Y(k) is the link count vector at time k: (L x 1)– A(k) is the routing matrix at time k: (L x P)– X(k) is the O-D pair traffic vector at time k: (P

x 1)• X(k) = (x1(k) , x2(k) , … xP(k))T

Y(k) = A(k) X(k)

k [1,K]

Y(k) and A(k) can be truncated to reflect missing and redundant data


Traffic Model: Stationary Case

x i(k) x i wi(k) k [1,K]

k

)(kxi

ix

)(kwi

• X(k) is the O-D pair traffic vector at time k: (P x 1)

X(k) = (x1(k) , x2(k) , … xP(k))T

X(k) = X + W(k)

W(k) : “Traffic Fluctuation Vector• Zero mean, covariance matrix B• B = diag(X)


Matrix Notation

CWAXY

KkkWkAXkAkY

],1[)()()()(

)(

...

)1(

...

)(000

0...00

00)2(0

000)1(

)(

...

)1(

)(

...

)1( 1

KW

W

W

x

x

X

kA

A

A

C

kA

A

A

kY

Y

Y

P

where:

Linear system of equations:

[LK] [LK][P] [LK][KP] [KP][P]

Choose Routing Configurations such that

Rank(A) = P


Traffic matrix Estimation-Stationary Case

• Initial Estimate: Use Psuedo-Inverse of A:

- does not require statistics of W (covariance B)• Gauss-Markov Theorem: Assume B is known

- Unbiased, minimum variance estimate- Coincides with Maximum Likelihood Estimate if W is Gaussian

Y = AX + CW

ˆ X (0) (AT A) 1 ATY

ˆ X (AT (CBCT ) 1 A) 1 AT (CBCT ) 1Y



Y = AX + CW

• Minimum Estimation Error:

(assumes B is known)

11 ))((])ˆ)(ˆ[( ACBCAXXXXE TTT

ˆ X (AT (CBCT ) 1 A) 1 AT (CBCT ) 1Y



• Recall we assume B = cov(W) satisfies B = diag(X)• Set

ˆ B (k ) diag( ˆ X (k ))

ˆ X (k1) (AT (C ˆ B (k )CT ) 1 A) 1 AT (C ˆ B (k )CT ) 1Y

• Recursion for Estimates:



• Our estimate is a solution to the equation:

ˆ X (AT (C diag( ˆ X ) CT ) 1 A) 1 AT (C diag( ˆ X ) CT ) 1Y

• Open questions for fixed point equation:- Existence of Solution?- Uniqueness?- Is solution an un-biased estimate?


Numerical Example-Stationary case

N=10 nodes, L=24 links and P=90 connections.

Three set of OD pairs with mean x equal to:

– 500 Mbps, 2 Gbps and 4 Gbps.

Gaussian Traffic Fluctuations:

bx 20


Stationary case: b=1 Samples/Snapshot=1


Stationary case: b=1 Samples/Snapshot=15


Stationary case: b=1.4 Samples/Snapshot=1


Stationary case: b=1.4 Samples/Snapshot=15


Stationary and Non-Stationary traffic

20 snapshots / 4 samples per snapshot / 5 min per sample

• Stationary Approach: 20 min per day (same time) / 20 days• Non-Stationary Approach: aggregate all the samples in one window time large 400 min (7 hours)


Traffic Model: Non-Stationary Case

)()( kTxkx ii

• Each OD pair is cyclo-stationary:

• Each OD pair is modeled as:

• Fourier series expansion:

],1[)()()( Kkkwkmkx iii

)/2sin(

)/2cos()(

)(ˆ)(ˆ2

0

Nkn

Nknkb

kbmkm

n

N

nnini

b

]2,1[],0[

],1[],0[

bb

b

NNnNk

NnNk

)(kxi )(kxi

kk

)(kwi )(kwi

)(kmi)(kmi


Mean estimation Results-Non Stationary case

Three set of OD pairs

where are linear independent Gaussian variables with:

)()2sin()( twfaxtx iiiiii

bix 20

3/2

3/

0

4

2

500

ii

Gbps

Gbps

Mbps

x

)(twi


Non Stationary case: b=1 Link Count


Non Stationary case: b=1 Mean estimation

traffic matrix estimation in non-stationary environments

Documents