12-21-2001 cs 838-21 an overlay routing scheme for moving large files su zhang kai xu

18
12-21-2001 CS 838-2 1 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu <[email protected]> <[email protected]>

Upload: buddy-cunningham

Post on 27-Dec-2015

215 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 1

An Overlay Routing Scheme For Moving Large Files

Su ZhangKai Xu

<[email protected]><[email protected]>

Page 2: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 2

Outline

Motivation Design and Implementation Evaluation Conclusions

Page 3: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 3

Motivation

Transferring large amount of data across Internet is challenging– Long period of transferring vs. Problematic

underlying Internet paths [Paxson 96]• Path/node failures• Temporary path outages • Rapid route alternation• Temporary routing loops

An overlay routing scheme can help

Page 4: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 4

Overlay Routing Scheme

A group of application-layer routers– Build on existing Internet routing substrate

Choosing “good” transferring path – Avoid problematic underlying paths

Caching on intermediate routers – Help on retransferring

Page 5: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 5

The Generic Design

Router

Router

Router$

Router $

$

$

File TransferServer

File TransferServer

Application Application

Page 6: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 6

File Transfer Routers

Link-state based routing protocol– Send “Hello” periodically

• Exchange link-state info.• Detect degraded path performance and failures

– Build forward table dynamically Flexible path metrics

– Latency, available throughput, packet loss rate– Application-specific metrics

• Network conditions fatal for one application, may not acceptable for another one

Page 7: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 7

File Transferring

File transfer servers find the closest router– Propagate “who owns cache” queries and get

metric info.

Large files are split into chunks– Each chunk is transferred independently

• Over underlying Internet path directly• Or, via transfer routers

– “Best” path under current network situation

Caching policy enforced

Page 8: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 8

Caching on Routers

Chunks are cached on intermediate routers– Cache policies decided by application– Build cache info table on each router

On retransferring – Cached chunks transferred from intermediate

routers

Caching policies – two layer– How to distribute chunks among routers– How to share the cache storage on each router

Page 9: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 9

Evaluation Limited experimental environment

– Tux lab– Simulated network latency and degraded link

performance Illustrate potential performance advantages of

this routing scheme Experiments

– Overcoming degraded performance– Caching improvement– Flexible caching policy

Page 10: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 10

Overcoming Degraded Performance Transferring a 10MB file Underlying links experiencing path outages or failures

– 10% of transferring time– Degraded performance: 10% - 100%

Routing through intermediate routers during performance failures ( + 5% vs. direct link)

Source

Router

Destination

Direct link

Page 11: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 11

98%

99%

100%

101%

102%

103%

104%

105%

0% 10% 25% 50% 75% 100%

% of degraded performance

norm

aliz

ed t

ran

sfer

tim

e

W/O FTR

W/ FTR

Overcoming Degraded Performance

Page 12: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 12

Caching Performance

Limited caching capacity on intermediate routers

Testing 10% - 100% data cached on the way– Caching improved transferring time greatly

Need more flexible cache policy!– Spread cached chunks over multiple routers– Drag frequently accessed chunks near destination

Page 13: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 13

Caching Performance

0%

20%

40%

60%

80%

100%

120%

0% 10% 20% 40% 60% 80% 100%

% of cached data

norm

aliz

ed tr

ansf

er ti

me

Page 14: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 14

Application Cache Policy Lottery vs. Round Robin

R

R

R R R R

RR

A B

C

A->B: Using Round Robin to leave caches (the first router caches seq#1, the second router caches seq#2… and wrap back)

A->C: 73.9% improvement vs. w/o caching

A->B: Using Lottery based on the distance to B (according to the hop number, generate possibility)

A->C: 55.1% improvement vs. w/o caching

Page 15: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 15

Conclusions

Using overlay routing can greatly improve the performance and reliability of transferring large files over problematic underlying Internet links

Dynamically selecting path based on different metrics to adapt to application requirement

Using cache to speed up multiple transferring Flexible cache policy

Page 16: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 16

Future Work How to setup the nodes on the Internet?

– Real experiments

How to get network metrics (bandwidth, loss rate etc.) accurately?

How to share the cache storage on each node for files in an efficient way?– More caching policies

How to recover transferring big files from interruptions?

Page 17: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 17

Thank You!

Questions?

Page 18: 12-21-2001 CS 838-21 An Overlay Routing Scheme For Moving Large Files Su Zhang Kai Xu

12-21-2001 CS 838-2 18

Important Reference:

“Resilient Overlay Networks” http://nms.lcs.mit.edu/ron “End-to-End Routing Behavior in the Internet”, Paxson, 96

Sigcomm “The End-to-End Effects of Internet Path Selection”, U of

Washington