Multiscale Traffic Processing Techniques for Network Inference and Control

Richard Baraniuk Edward Knightly Robert Nowak Rolf Riedi

Rice University INCITE ProjectApril 2001

Rice University | INCITE.rice.edu | April 2001

INCITEInterNet Control and Inference Tools at the Edge

• Overall Objective:

Scalable, edge-based tools for on-line network analysis, modeling, and measurement

• Theme for DARPA NMS Research:

Multiscale traffic analysis, modeling, and processing via multifractals

• Expertise:

Statistical signal processing, mathematics, network QoS

Rice University | INCITE.rice.edu | April 2001

Technical Challenges

Poor understanding of origins of complex network dynamics

Lack of adequate modeling techniques for network dynamics

Internal network inaccessible

Need: Manageable, reduced-complexity models with characterizable accuracy

Rice University | INCITE.rice.edu | April 2001

Multiscale modeling

Rice University | INCITE.rice.edu | April 2001

Multiscale Analysis

Time

Scale

Analysis: flow up the tree by adding

Start at bottom with trace itself

Var1

Var2

Var3

Varj

Multiscale statistics

Rice University | INCITE.rice.edu | April 2001

Multiscale Synthesis

Time

Scale

Synthesis: flow down via innovations

Start at top with total arrival

Signal: bottom nodes

Var1

Var2

Var3

Varj

Multiscale parameters

Rice University | INCITE.rice.edu | April 2001

Multifractal Wavelet Model (MWM)

• Random multiplicativeinnovations Aj,k on [0,1]

eg: beta

• Parsimonious modeling(one parameter per scale)

• Strong ties with rich theory of multifractals

Rice University | INCITE.rice.edu | April 2001

Multiscale Traffic Trace Matching

4ms

16ms

64ms

Auckland 2000 MWM matchscale

Rice University | INCITE.rice.edu | April 2001

Multiscale Queuing

Rice University | INCITE.rice.edu | April 2001

Probing the Network

Rice University | INCITE.rice.edu | April 2001

Probing

• Ideally:

delay spread of packet pair spaced by T sec

correlates with

cross-traffic volume at time-scale T

Rice University | INCITE.rice.edu | April 2001

Probing Uncertainty Principle

• Should not allow queue to empty between probe packets

• Small T for accurate measurements– but probe traffic would disturb

cross-traffic (and overflow bottleneck buffer!)

• Larger T leads to measurement uncertainties– queue could empty between probes

• To the rescue: model-based inference

Rice University | INCITE.rice.edu | April 2001

Multifractal Cross-Traffic Inference

• Model bursty cross-traffic using MWM

Rice University | INCITE.rice.edu | April 2001

Efficient Probing: Packet Chirps

• MWM tree inspires geometric chirp probe• MLE estimates of cross-traffic at multiple scales

Rice University | INCITE.rice.edu | April 2001

Chirp Probe Cross-Traffic Inference

Rice University | INCITE.rice.edu | April 2001

ns-2 Simulation

• Inference improves with increased utilization

Low utilization (39%) High utilization (65%)

Rice University | INCITE.rice.edu | April 2001

ns-2 Simulation (Adaptivity)

• Inference improves as MWM parameters adapt

MWM parameters Inferred x-traffic

Rice University | INCITE.rice.edu | April 2001

Adaptivity (MWM Cross-Traffic)

Eg: Route changes

Rice University | INCITE.rice.edu | April 2001

Comparing Probing schemes

Rice University | INCITE.rice.edu | April 2001

Comparing probing schemes

• `Classical’: Bandwidth estimation by packet pairs and trains

• Novel: Traffic estimation, probing best by Uniform? Poisson? Chirp?

Rice University | INCITE.rice.edu | April 2001

Model based Probing

Chirp: model based, superior

Uniform: Uncertainty increases error

Rice University | INCITE.rice.edu | April 2001

Impact of Probing on Performance

Heavy probing - reduces bandwidth - increases loss - inflicts time-outs

NS-simulation: Same `web-traffic’ with variable probing rates

Heavy

Light

Rice University | INCITE.rice.edu | April 2001

Influence of probing rate on error

• Chirp probing performing uniformly good• Uniform requires higher rates to perform

Rice University | INCITE.rice.edu | April 2001

Synergies

• SAIC (Warren): MWM code for real time simulator

• SLAC (Cottrell, Feng):Modify PingER for chirp-probingHigh performance networks

• Demo: C-code for real world chirp-probingusing NetDyn (TCP) + simple Daemon at receiver(INRIA France, UFMG Brazil, Michigan State)

Rice University | INCITE.rice.edu | April 2001

INCITE: Near-term / Ongoing

Verification with real Internet experiments– Rice testbed (practical issues)

– SAIC (real time algorithms) – SLAC / ESNet (real world verification)

Enhancements: rigorous statistical error analysis deal with random losses multiple bottleneck queues (see demo)

passive monitoring (novel models)

closed loop paths/feedback (ns-simulation)

Rice University | INCITE.rice.edu | April 2001

INCITE: Longer-Term Goals

• New traffic models, inference algorithms– theory, simulation, real implementation

• Applications to Control, QoS, Network Meltdown early warning

• Leverage from our other projects– ATR program (DARPA, ONR, ARO)

– RENE (Rice Everywhere Network:NSF)

– NSF ITR– DoE

Rice University | INCITE.rice.edu | April 2001

Stationary multifractals

Rice University | INCITE.rice.edu | April 2001

Stationary multiplicative models

j(s): stationary, indep., E[j(s)]=1

• A(t) = lim 0t 1(s) 2(s)… n(s) ds

– May degenerate (compare: MWM is conservative)– stationary increments

• Assume j(2j s) are i.i.d.; Renewal reward

– Compare MWM: j(2j s) constant over [k,k+1]

– If Var()<1: Convergence in L2 ; E[A(t)]=t

– Multifractal function: T(q)=q-log2E[q]

Rice University | INCITE.rice.edu | April 2001

Simulation

• L2 criterion for convergence translates to

T(2)>0

• Conjecture: For q>1 converge in Lq if T(q)>0

Thus non-degenerate iff T’(1)>0, ie E[ log ( /2) ] >0

Rice University | INCITE.rice.edu | April 2001

Parameter estimation

• No conservation: can’t isolate multipliers• Possible correlation within multipliers

• IID values:

– Z(s) = log [ 1(s) 2(s)… n(s) ]

– Cov(Z(t)Z(t+s))= i=1..n exp(-is)Var i(s)

• `LRD-scaling’ at medium scales, but SRD. Multifractal subordination -> true LRD.

Rice University | INCITE.rice.edu | April 2001

INCITE.rice.edu