multiscale traffic processing techniques for network inference and control
DESCRIPTION
Multiscale Traffic Processing Techniques for Network Inference and Control. Richard Baraniuk Edward Knightly Robert Nowak Rolf Riedi Rice University INCITE Project April 2001. INCITE. I nter N et C ontrol and I nference T ools at the E dge. - PowerPoint PPT PresentationTRANSCRIPT
Multiscale Traffic Processing Techniques for Network Inference and Control
Richard Baraniuk Edward Knightly Robert Nowak Rolf Riedi
Rice University INCITE ProjectApril 2001
Rice University | INCITE.rice.edu | April 2001
INCITEInterNet Control and Inference Tools at the Edge
• Overall Objective:
Scalable, edge-based tools for on-line network analysis, modeling, and measurement
• Theme for DARPA NMS Research:
Multiscale traffic analysis, modeling, and processing via multifractals
• Expertise:
Statistical signal processing, mathematics, network QoS
Rice University | INCITE.rice.edu | April 2001
Technical Challenges
Poor understanding of origins of complex network dynamics
Lack of adequate modeling techniques for network dynamics
Internal network inaccessible
Need: Manageable, reduced-complexity models with characterizable accuracy
Rice University | INCITE.rice.edu | April 2001
Multiscale modeling
Rice University | INCITE.rice.edu | April 2001
Multiscale Analysis
Time
Scale
Analysis: flow up the tree by adding
Start at bottom with trace itself
Var1
Var2
Var3
Varj
Multiscale statistics
Rice University | INCITE.rice.edu | April 2001
Multiscale Synthesis
Time
Scale
Synthesis: flow down via innovations
Start at top with total arrival
Signal: bottom nodes
Var1
Var2
Var3
Varj
Multiscale parameters
Rice University | INCITE.rice.edu | April 2001
Multifractal Wavelet Model (MWM)
• Random multiplicativeinnovations Aj,k on [0,1]
eg: beta
• Parsimonious modeling(one parameter per scale)
• Strong ties with rich theory of multifractals
Rice University | INCITE.rice.edu | April 2001
Multiscale Traffic Trace Matching
4ms
16ms
64ms
Auckland 2000 MWM matchscale
Rice University | INCITE.rice.edu | April 2001
Multiscale Queuing
Rice University | INCITE.rice.edu | April 2001
Probing the Network
Rice University | INCITE.rice.edu | April 2001
Probing
• Ideally:
delay spread of packet pair spaced by T sec
correlates with
cross-traffic volume at time-scale T
Rice University | INCITE.rice.edu | April 2001
Probing Uncertainty Principle
• Should not allow queue to empty between probe packets
• Small T for accurate measurements– but probe traffic would disturb
cross-traffic (and overflow bottleneck buffer!)
• Larger T leads to measurement uncertainties– queue could empty between probes
• To the rescue: model-based inference
Rice University | INCITE.rice.edu | April 2001
Multifractal Cross-Traffic Inference
• Model bursty cross-traffic using MWM
Rice University | INCITE.rice.edu | April 2001
Efficient Probing: Packet Chirps
• MWM tree inspires geometric chirp probe• MLE estimates of cross-traffic at multiple scales
Rice University | INCITE.rice.edu | April 2001
Chirp Probe Cross-Traffic Inference
Rice University | INCITE.rice.edu | April 2001
ns-2 Simulation
• Inference improves with increased utilization
Low utilization (39%) High utilization (65%)
Rice University | INCITE.rice.edu | April 2001
ns-2 Simulation (Adaptivity)
• Inference improves as MWM parameters adapt
MWM parameters Inferred x-traffic
Rice University | INCITE.rice.edu | April 2001
Adaptivity (MWM Cross-Traffic)
Eg: Route changes
Rice University | INCITE.rice.edu | April 2001
Comparing Probing schemes
Rice University | INCITE.rice.edu | April 2001
Comparing probing schemes
• `Classical’: Bandwidth estimation by packet pairs and trains
• Novel: Traffic estimation, probing best by Uniform? Poisson? Chirp?
Rice University | INCITE.rice.edu | April 2001
Model based Probing
Chirp: model based, superior
Uniform: Uncertainty increases error
Rice University | INCITE.rice.edu | April 2001
Impact of Probing on Performance
Heavy probing - reduces bandwidth - increases loss - inflicts time-outs
NS-simulation: Same `web-traffic’ with variable probing rates
Heavy
Light
Rice University | INCITE.rice.edu | April 2001
Influence of probing rate on error
• Chirp probing performing uniformly good• Uniform requires higher rates to perform
Rice University | INCITE.rice.edu | April 2001
Synergies
• SAIC (Warren): MWM code for real time simulator
• SLAC (Cottrell, Feng):Modify PingER for chirp-probingHigh performance networks
• Demo: C-code for real world chirp-probingusing NetDyn (TCP) + simple Daemon at receiver(INRIA France, UFMG Brazil, Michigan State)
Rice University | INCITE.rice.edu | April 2001
INCITE: Near-term / Ongoing
Verification with real Internet experiments– Rice testbed (practical issues)
– SAIC (real time algorithms) – SLAC / ESNet (real world verification)
Enhancements: rigorous statistical error analysis deal with random losses multiple bottleneck queues (see demo)
passive monitoring (novel models)
closed loop paths/feedback (ns-simulation)
Rice University | INCITE.rice.edu | April 2001
INCITE: Longer-Term Goals
• New traffic models, inference algorithms– theory, simulation, real implementation
• Applications to Control, QoS, Network Meltdown early warning
• Leverage from our other projects– ATR program (DARPA, ONR, ARO)
– RENE (Rice Everywhere Network:NSF)
– NSF ITR– DoE
Rice University | INCITE.rice.edu | April 2001
Stationary multifractals
Rice University | INCITE.rice.edu | April 2001
Stationary multiplicative models
j(s): stationary, indep., E[j(s)]=1
• A(t) = lim 0t 1(s) 2(s)… n(s) ds
– May degenerate (compare: MWM is conservative)– stationary increments
• Assume j(2j s) are i.i.d.; Renewal reward
– Compare MWM: j(2j s) constant over [k,k+1]
– If Var()<1: Convergence in L2 ; E[A(t)]=t
– Multifractal function: T(q)=q-log2E[q]
Rice University | INCITE.rice.edu | April 2001
Simulation
• L2 criterion for convergence translates to
T(2)>0
• Conjecture: For q>1 converge in Lq if T(q)>0
Thus non-degenerate iff T’(1)>0, ie E[ log ( /2) ] >0
Rice University | INCITE.rice.edu | April 2001
Parameter estimation
• No conservation: can’t isolate multipliers• Possible correlation within multipliers
• IID values:
– Z(s) = log [ 1(s) 2(s)… n(s) ]
– Cov(Z(t)Z(t+s))= i=1..n exp(-is)Var i(s)
• `LRD-scaling’ at medium scales, but SRD. Multifractal subordination -> true LRD.
Rice University | INCITE.rice.edu | April 2001
INCITE.rice.edu