scalable integrated performance analaysis of multi-gigabit networks
DESCRIPTION
Scalable Integrated Performance Analaysis of Multi-Gigabit Networks. Ezra Kissel, U. Delaware Ahmed El-Hassany, Guilherme Fernandes, Martin Swany, Indiana U. Dan Gunter , Taghrid Samak, LBNL Jen Schopf, WHOI. What I hope you learn. Why we care about bulk data transfer at multi-gigabit rates - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/1.jpg)
Scalable Integrated Performance Analaysis of Multi-Gigabit Networks
Ezra Kissel, U. DelawareAhmed El-Hassany, Guilherme Fernandes,
Martin Swany, Indiana U.Dan Gunter, Taghrid Samak, LBNL
Jen Schopf, WHOI
![Page 2: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/2.jpg)
What I hope you learn1. Why we care about bulk data
transfer at multi-gigabit rates2. Why and how detailed monitoring is
helpful3. How dynamic control of monitoring
is related to Session Layer protocols
4/16/12 2
![Page 3: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/3.jpg)
Bulk data transfer needs
• Some domains of interest:– Climate simulation (Earth System Grid)– Genomics (JGI)– High-energy physics (Large Hadron
Collider)– Astronomy (Large Synoptic Survey
Telescope)– Astrophysics (FLASH)
Huge data
Analysis
sites
4/16/12 3
![Page 4: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/4.jpg)
Multi-gigabit rates• Networks connecting national labs
and universities have 10Gb/s and soon 100Gb/s capability. one PB = one day at 100Gb/s
• Rarely achieved due to bottlenecks:– Host: Application or Disks– Campus/local networks–Wide area networks
• Hard to tell why, where, or even if there is a problem4/16/12 4
![Page 5: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/5.jpg)
SolutionMonitor all the timeAnalyze all the time
.. but much more when something interesting is happening
Use analysis results as feedback
4/16/12 5
![Page 6: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/6.jpg)
System components• eXtensible Session Protocol (XSP)– Associate multiple TCP connections, L2 circuits,
as a "session"– Provide channels for bi-directional metadata
• NL-Calipers– Summarize in situ timings of every read/write
• BLiPP– Host and TCP stack info. using XSP channels
• PerfSONAR– Standard information formats and exchange
protocols4/16/12 6
![Page 7: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/7.jpg)
Dynamic Session Monitoring
User
(1) Start xfer
(2) Open session3) data
(3) NL-calipers data
(4) Signal TCP (4) Signal TCP(5) data
(5) data
Look at the performance Networkengineer
4/16/12 7
![Page 8: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/8.jpg)
Bottleneck detection
4/16/12 8
Triangles give "instantaneous" throughput
On fixed intervals, summarize all measurements into mean, min, max, variance for both rate and #bytes
Instrumentatio
n
Analysis: pick lowest mean value as bottleneck, apply t-test
![Page 9: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/9.jpg)
TCP throughputTime series of throughput* for representative TCP experiments: (a) 1 stream memory-to-disk with 100ms latency, (b) 1 stream memory-to-memory with no latency, (c) 1 stream disk-to-disk with no latency, (d) 4 streams memory-to-disk with 100ms latency and 1% loss added at 60 seconds.
4/16/12 9
![Page 10: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/10.jpg)
UDT throughputTime series of throughput* for representative UDT experiments: (a) 4 streams memory-to-disk with 100ms latency, (b) 4 streams memory-to-disk with 100ms latency and 1% loss added at 60 seconds, (c) 4 streams disk-to-disk with 100ms latency, (d) 4 streams memory-to-memory with 100ms latency.
4/16/12 10
![Page 11: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/11.jpg)
Wait, what?
4/16/12 11
![Page 12: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/12.jpg)
Half as many read()s.Others return zero, not counted
Variance
Less work being done
4/16/12 12
![Page 13: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/13.jpg)
Review• Why we care about bulk data
transfer at multi-gigabit rates• Why and how detailed monitoring is
helpful• How monitoring is related to Session
Layer protocols– and how that might integrate with a
management framework• Questions?4/16/12 13
![Page 14: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/14.jpg)
Related projects• NetLogger netlogger.lbl.gov• perfSONAR perfsonar.org• XSP damsl.cis.udel.edu/• GENI geni.net• CEDPS cedps-scidac.org
4/16/12 14
![Page 15: Scalable Integrated Performance Analaysis of Multi-Gigabit Networks](https://reader036.vdocuments.site/reader036/viewer/2022062302/568166ec550346895ddb3626/html5/thumbnails/15.jpg)
Topology-aware Monitoring
4/16/12 15