12-21-2001 cs 838-21 an overlay routing scheme for moving large files su zhang kai xu
TRANSCRIPT
12-21-2001 CS 838-2 1
An Overlay Routing Scheme For Moving Large Files
Su ZhangKai Xu
12-21-2001 CS 838-2 2
Outline
Motivation Design and Implementation Evaluation Conclusions
12-21-2001 CS 838-2 3
Motivation
Transferring large amount of data across Internet is challenging– Long period of transferring vs. Problematic
underlying Internet paths [Paxson 96]• Path/node failures• Temporary path outages • Rapid route alternation• Temporary routing loops
An overlay routing scheme can help
12-21-2001 CS 838-2 4
Overlay Routing Scheme
A group of application-layer routers– Build on existing Internet routing substrate
Choosing “good” transferring path – Avoid problematic underlying paths
Caching on intermediate routers – Help on retransferring
12-21-2001 CS 838-2 5
The Generic Design
Router
Router
Router$
Router $
$
$
File TransferServer
File TransferServer
Application Application
12-21-2001 CS 838-2 6
File Transfer Routers
Link-state based routing protocol– Send “Hello” periodically
• Exchange link-state info.• Detect degraded path performance and failures
– Build forward table dynamically Flexible path metrics
– Latency, available throughput, packet loss rate– Application-specific metrics
• Network conditions fatal for one application, may not acceptable for another one
12-21-2001 CS 838-2 7
File Transferring
File transfer servers find the closest router– Propagate “who owns cache” queries and get
metric info.
Large files are split into chunks– Each chunk is transferred independently
• Over underlying Internet path directly• Or, via transfer routers
– “Best” path under current network situation
Caching policy enforced
12-21-2001 CS 838-2 8
Caching on Routers
Chunks are cached on intermediate routers– Cache policies decided by application– Build cache info table on each router
On retransferring – Cached chunks transferred from intermediate
routers
Caching policies – two layer– How to distribute chunks among routers– How to share the cache storage on each router
12-21-2001 CS 838-2 9
Evaluation Limited experimental environment
– Tux lab– Simulated network latency and degraded link
performance Illustrate potential performance advantages of
this routing scheme Experiments
– Overcoming degraded performance– Caching improvement– Flexible caching policy
12-21-2001 CS 838-2 10
Overcoming Degraded Performance Transferring a 10MB file Underlying links experiencing path outages or failures
– 10% of transferring time– Degraded performance: 10% - 100%
Routing through intermediate routers during performance failures ( + 5% vs. direct link)
Source
Router
Destination
Direct link
12-21-2001 CS 838-2 11
98%
99%
100%
101%
102%
103%
104%
105%
0% 10% 25% 50% 75% 100%
% of degraded performance
norm
aliz
ed t
ran
sfer
tim
e
W/O FTR
W/ FTR
Overcoming Degraded Performance
12-21-2001 CS 838-2 12
Caching Performance
Limited caching capacity on intermediate routers
Testing 10% - 100% data cached on the way– Caching improved transferring time greatly
Need more flexible cache policy!– Spread cached chunks over multiple routers– Drag frequently accessed chunks near destination
12-21-2001 CS 838-2 13
Caching Performance
0%
20%
40%
60%
80%
100%
120%
0% 10% 20% 40% 60% 80% 100%
% of cached data
norm
aliz
ed tr
ansf
er ti
me
12-21-2001 CS 838-2 14
Application Cache Policy Lottery vs. Round Robin
R
R
R R R R
RR
A B
C
A->B: Using Round Robin to leave caches (the first router caches seq#1, the second router caches seq#2… and wrap back)
A->C: 73.9% improvement vs. w/o caching
A->B: Using Lottery based on the distance to B (according to the hop number, generate possibility)
A->C: 55.1% improvement vs. w/o caching
12-21-2001 CS 838-2 15
Conclusions
Using overlay routing can greatly improve the performance and reliability of transferring large files over problematic underlying Internet links
Dynamically selecting path based on different metrics to adapt to application requirement
Using cache to speed up multiple transferring Flexible cache policy
12-21-2001 CS 838-2 16
Future Work How to setup the nodes on the Internet?
– Real experiments
How to get network metrics (bandwidth, loss rate etc.) accurately?
How to share the cache storage on each node for files in an efficient way?– More caching policies
How to recover transferring big files from interruptions?
12-21-2001 CS 838-2 17
Thank You!
Questions?
12-21-2001 CS 838-2 18
Important Reference:
“Resilient Overlay Networks” http://nms.lcs.mit.edu/ron “End-to-End Routing Behavior in the Internet”, Paxson, 96
Sigcomm “The End-to-End Effects of Internet Path Selection”, U of
Washington