introduction jiří navrátil slac. rice university richard baraniuk, edward knightly, robert nowak,...
TRANSCRIPT
Introduction
Jiří Navrátil
SLAC
Rice University
Richard Baraniuk, Edward Knightly, Robert Nowak, Rudolf Riedi
Xin Wang, Yolanda Tsang, Shriram Sarvotham, Vinay Ribeiro
Los Alamos National Lab (LANL)
Wu-chun Feng, Mark Gardner, Eric Weigle
Stanford Linear Accelerator Center (SLAC)
Les Cottrell, Warren Matthews, Jiri Navratil
INCITE: Edge-based Traffic Processing and Service Inference for High-Performance Networks Richard Baraniuk, Rice University; Les Cottrell, SLAC; Wu-chun Feng, LANL
Project Partners and Researchers
Project GoalsINCITE: Edge-based Traffic Processing and Service Inference for High-Performance Networks Richard Baraniuk, Rice University; Les Cottrell, SLAC; Wu-chun Feng, LANL
• Objectives– scalable, edge-based tools for on-line
network analysis, modeling, and measurement
• Based on– advanced mathematical theory and methods
• Designeted for– support high-performance computing
infrastructures, such as computational grids,– ESNET, Internet2 and other HPNetworking
project
Project ElementsINCITE: Edge-based Traffic Processing and Service Inference for High-Performance Networks Richard Baraniuk, Rice University; Les Cottrell, SLAC; Wu-chun Feng, LANL
• Advanced techniques – from networking, supercomputing, statistical signal
processing, applied mathematics
• Multiscale analysis and modeling– understand causes of burstiness in network traffic– realistic, yet analytically tractable, statistically robust, and
computationally efficient modeling
• On-line inference algorithms – characterize and map network performance as a function of
space, time, application, and protocol
• Data collection tools and validation experiments
Scheduled AccomplishmentsINCITE: Edge-based Traffic Processing and Service Inference for High-Performance Networks Richard Baraniuk, Rice University; Les Cottrell, SLAC; Wu-chun Feng, LANL
• Multiscale traffic models and analysis techniques– based on multifractals, cascades, wavelets– study how large flows interact and cause bursts– study adverse modulation of application-level traffic by
TCP/IP
• Inference algorithms for paths, links, and routers– multiscale end-to-end path modeling and probing– network tomography (active and passive)
• Data collection tools– add multiscale path, link inference to PingER suite– integrate into ESnet NIMI infrastructure– MAGNeT – Monitor for Application-Generated Network Traffic – TICKET – Traffic Information-Collecting Kernel with Exact
Timing
Future Research PlansINCITE: Edge-based Traffic Processing and Service Inference for High-Performance Networks Richard Baraniuk, Rice University; Les Cottrell, SLAC; Wu-chun Feng, LANL
New, high-performance traffic models– guide R&D of next-generation protocols
• Application-generated network traffic repository – enable grid and network researchers to test and evaluate
new protocols with actual traffic demands of applications rather than modulated demands
• Multiclass service inference– enable network clients to assess a system's multi-class
mechanisms and parameters using only passive, external observations
• Predictable QoS via end-point control– ensure minimum QoS levels to traffic flows– exploit path and link inferences in real-time end-point
admission control
(From Papers to Practice)
MWFS, TOMO, TOPO
20 ms
~300 ms
40 T for new set of values (12 sec)
First results
What has been done
• Phase 1 - Remodeling
- Code separation (BW and CT)
- Find how to call MATLAB from another program
- Analyze Results and data
- Find optimal params for model• Phase 2
- Webing of BW estimate
Data Dispersions from sunstats.cern.ch
pcgiga.cern.ch
sunstats.cern.ch
ccnsn07.in2p3.fr
plato.cacr.caltech.edu
pcgiga.cern.ch
default WS
BW ~ 70Mbps
pcgiga.cern.ch
WS 512K
BW ~ 100 Mbps
Reaction to the network problems
After tuning
MF-CT Features and benefits
• No need access to routers ! – Current monitoring systems for Load of traffic are
based on SNMP or Flows (needs access to routers)
• Low cost:– Allows permanent monitoring (20 pkts/sec ~ overhead
10 Kbytes/sec)– Can be used as data provider for ABW prediction
(ABW=BW-CT)
• Weak point for common useMATLAB code
Future work on CT
• Verification model– Define and setup verification model (S+R)– Measurements (S)– Analyze results (S+R)
• On-line running on selected sites– Prepare code for automation and Webing (S)– CT-Code modificaton ? (R)
SNMP counter
SNMP counter
MF-CT Simulator
SNMP counter
SNMP counter
UDP echo
UDP echo
SLAC IN2P3
CERN
CT RE-ENGINEERING
For practical monitoring would be necessary to do modification for using it in different modes:
– Continuos mode for monitoring one site in Large time scale (hours)
– Accumulation mode (1 min, 5 min, ?) for running for more sites in parallel
– ? Solution without MATLAB ?
Rob Nowak (and CAIDA people) say:
www.caida.org
Network Topology Identification
Pairwise delay measurements reveal topology
Ratnasamy & McCanne (99)Duffield, et al (00,01,02)
Bestavros, et al (01)Coates, et al (01)
Network Tomography
Measure end-to-end (from source to receiver) losses/delays
Infer link-level (at internal routers) loss rates and delay distributions
receivers
source
router / node
link
Unicast Network Tomography
Measure end-to-end losses of packets
‘0’ loss‘1’ success
‘0’ loss‘1’ success
Cannot isolate where losses occur !
Packet Pair Measurements
measurement packet pair
cross-traffic(2)packet (1)packet
(2)packet (1)packet
nearly experience packetandpacket (2)(1)
delay
delaysand/or losses identical
Delay Estimation
Measure end-to-end delays of packet-pairs
Packets experience thesame delay on link 1
0d min2 d min3d d
Extra delay on link 3
Packet-pair measurements
)(packet (1) n)(packet (2) n
Key Assumptions:
• fixed routes
• iid pair-measurements
• losses & delays on each link are mutually independent
• packet-pair losses & delays on shared links are nearly identical
)()2( ny )()1( ny
Nnnynyy 1)2()1( )(),(
"success"1or"loss"0
units"delay " 0,1,...,K )()( ny p
}2,1{p record occurrencesof losses and delays
1 0.5
10
2
2
2
1 0.5
10
10
5
Test network showinglink bandwidths (Mb/s)
cross-traffic link 9
• 40-byte packet-pair probes every 50 ms• competing traffic comprised of:
on-off exponential (500 byte packets) TCP connections (1000 byte packets)
Kbytes/s
time (s)
ns Simulation
Future work on TM and TP
• Model in frame of Internet (~100 sites)– Define verification model (S+R)– Deploy and install code on sites (S)– First measurements (S+R)– Analyze results (form,speed,quantity) (S+R)– ? Code modificaton (R)
• Production model ? – Compete with Pinger, RIPE, Surveyor, Nimi ? – How to unify VIRTUAL structure with Real