1 end-to-end detection of shared bottlenecks sridhar machiraju and weidong cui sahara winter retreat...
Post on 21-Dec-2015
222 views
TRANSCRIPT
![Page 1: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/1.jpg)
1
End-to-End Detection of Shared Bottlenecks
Sridhar Machiraju and Weidong Cui
Sahara Winter Retreat 2003
![Page 2: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/2.jpg)
2
Problem Statement
• Given 2 end-to-end flows f1 and f2, do they share a bottleneck (a congested link i.e., link with packet drops)
(OR)
• Given 2 routes R1 and R2 on the Internet, do they share a bottleneck link?
![Page 3: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/3.jpg)
3
Why is this hard?
• No information from the network• Only information available – delay and
drops. • Lots of noise – delay from intermediate
links and drops on other links• Bottlenecks may change over time
![Page 4: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/4.jpg)
4
Why solve this problem?
• Overlays – – RON - Decide if rerouting flows bypasses
congestion points or not– RON – Does such rerouting affect existing
flows? Which ones?– Cooperative overlays – overlay does not
want to share bottleneck with a “friendly overlay”
– OverQoS – Useful to cluster together overlay links based on shared bottlenecks
![Page 5: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/5.jpg)
5
Why solve this problem (cont.)?
• Other applications– Massive backups of data from different
servers – do them in parallel?– Content distribution – is the use of multipath
going to improve performance?– Kazaa – parallel downloads from peers– Multihomed ASs can evaluate the
“orthogonality” in terms other than fault-tolerance
![Page 6: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/6.jpg)
6
Related Work• Past work done only with Y or Inverted-Y
topologies using Poisson probes, packet pairs and inter-arrival times.
Receivers
Senders
![Page 7: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/7.jpg)
7
Goals
• Provide a general solution for double-Y topology
• Work with multiple bottlenecks and provide an indicator of shared congestion
• Be able to use active probe flows and also passively observed (TCP) flows
• Complexity issues for clustering flows
![Page 8: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/8.jpg)
8
Motivation of Our Techniques
• Droptail queues + TCP – queues exhibit bursty loss periods + no losses
• Queues build-up until bursty losses and decrease in sizes before increasing again
• Provides motivation for correlating periods of drops and delays (proportional to queue sizes)
• But…
![Page 9: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/9.jpg)
9
Synchronization Lag
0
T
d1 d2+
Flow 1
Flow 2
Time
Sender 1
Sender 2
0 1 2 3 4 5 6 7 8
0 1 2 3 4 5 6 7
0 1 2 3 4 5 6
0 1 2 3 4
Synchronization Lag = 3T
Note: is bounded by RTTmax/2
![Page 10: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/10.jpg)
10
Overview of Our Techniques
• We propose 2 techniques – – Probability Distribution (PD) technique – Cross-Correlation (CC) technique
• PD is based on getting the peak of the discrete probability distribution of, minimum time between drop of a flow and drop of the other
• CC is based on getting the maximum cross-correlation assuming various synch. lags
![Page 11: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/11.jpg)
11
PD Technique
• For each dropped packet of a flow, plot PD of minimum of the time differences between its sending time and the sending times of dropped packets of other flow
• If shared bottleneck, we expect (ideally) a 1 at d2- d1+ ; All flows may not see drops during same burst, so use threshold < 1 for peak
• We may see more than 1 drop in a burst; cluster drops into bursts and use time differences between starts of bursts
![Page 12: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/12.jpg)
12
PD technique (contd.)
• Robustness issues: synch. lag must be smaller than the time difference between consecutive drops of a flow
Delay1
Delay2
Packet Loss
![Page 13: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/13.jpg)
13
Cross-Correlation (CC) Technique
• Key ideas– Two “back-to-back” packets from two
different flows will experience similar packet drop/delay at the bottleneck
– If we can generate two sequences of “back-to-back” packets from two different flows, then we can calculate their cross-correlation coefficient of losses or delays to measure their “similarity”.
– If the cross-correlation coefficient is greater than some threshold, then the two flows share a bottleneck.
Network
![Page 14: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/14.jpg)
14
Questions about the CC Technique
• How to generate two sequences of “back-to-back” packets?– UDP probes with a constant interval T
• average interval <= T/2
– Shift the sequence to overcome the synch. lag
• How long should the two sequences be to get a significant result?– When the CC coefficient becomes relatively stable– But no less than a minimum period of time
• What should the threshold be?– Use 0.1 in the experiments– Why 0.1?
![Page 15: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/15.jpg)
15
Overcome the Synchronization Problem
Delay1
Delay2
Shift 2 packets
Packet Loss
• Find the max cross-correlation by shifting one of the two sequences within some range
• The value of the optimal shift is an estimation of the synchronization lag.
![Page 16: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/16.jpg)
16
Wide-Area Experiments
• Challenges– Access to hosts distributed globally?– How to verify our experimental results?
• Solutions– PlanetLab (http://www.planet-lab.org)– Set up an overlay network with double-Y
topology– Application-level routers monitor losses and
delays
![Page 17: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/17.jpg)
17
Topology with Shared Bottleneck (I)
Vancouver
Seattle Wisc
Atlanta
Bologna
Sydney
![Page 18: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/18.jpg)
18
Topology without Shared Bottleneck (II)
Vancouver
Seattle Wisc
Atlanta
Bologna
Sydney
![Page 19: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/19.jpg)
19
Experimental Setup
• Active Probing– 40 bytes per packet– Every 10ms
• Log packet arrival times on every node– Also can get information of losses from these
logs
• Traces from 10mins to 60mins• Threshold = 0.1 for the PD and CC
techniques
![Page 20: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/20.jpg)
20
Overall ResultsExp # Packet Drops PD Technique
Loss CC Technique
Delay CC Technique
shared
Non-shared
Peak Value
Est. Lag
CC Coeff.
Est. Lag
CC Coeff.
Est. Lag
1(20mins)
3 2096 < 0.1 - < 0.1 - < 0.1 -
2(10mins)
6772 165 0.21 60ms 0.22 50ms 0.12 50ms
3(10mins)
2070 32 0.45100m
s0.81 80ms < 0.1 -
4(10mins)
81 2252 < 0.1 - 0.38 -1.17s 0.99 -1.17s
5(30mins)
0 5565 < 0.1 - < 0.1 - < 0.1 -
6(60mins)
10272 1127 <0.1 - 0.23 6s < 0.1 -
7(10mins)
1592 57 < 0.1 - 0.75 -1.15 < 0.1 -
8(10mins)
1895 112 0.11180m
s0.55
300ms
< 0.1 -
Failed Cases
![Page 21: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/21.jpg)
21
Why the Delay CC Technique fails?
• Delay spikes at the non-shared part.
![Page 22: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/22.jpg)
22
Why the PD Technique fails?
• Large synchronization lag• Few number of drops at the bottleneck
![Page 23: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/23.jpg)
23
Open Issues
• Parameter Selection– What should the thresholds be?
• Active vs. Passive Probing– Active probing: waste network resources– Passive probing: cannot control the size/rate of the
probing sequences.
• Multiple Bottlenecks– Our techniques are not limited to the cases of single
bottlenecks.– But need more quantitative evaluations
• Probability of sharing a bottleneck– How often should we generate probing sequence to
detect if two flows share a bottleneck?– Can we give a probability rather than a 0-1 decision?
![Page 24: 1 End-to-End Detection of Shared Bottlenecks Sridhar Machiraju and Weidong Cui Sahara Winter Retreat 2003](https://reader035.vdocuments.site/reader035/viewer/2022081519/56649d585503460f94a38438/html5/thumbnails/24.jpg)
24
Conclusions
• Problem– Detect if 2 end-to-end flows share a bottleneck
• Challenge– Synchronization lag in double-Y topology
• Techniques– The Probability Distribution Technique– The Loss/Delay Cross-Correlation Technique
• Experimental Results– The Loss CC technique succeeds with all experiments– The Delay CC technique fails in some experiments
due to delay spikes at the non-shared part – The PD technique fails in some experiments due to
large synch. Lag and few number of losses at the bottleneck