optimal scheduling in peer-to-peer networks lee center workshop 5/19/06 mortada mehyar (with prof....
TRANSCRIPT
![Page 1: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/1.jpg)
Optimal Scheduling in Peer-to-Peer Networks
Lee Center Workshop 5/19/06
Mortada Mehyar(with Prof. Steven Low, Netlab)
![Page 2: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/2.jpg)
Outline
Brief description of p2p file sharing and Bittorrent protocol
Our model for Bittorrent-like file sharing
Efficiency of scheduling algorithms with respect to different optimality criteria.
![Page 3: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/3.jpg)
About Bittorrent
A p2p protocol started ~ 2002 The most popular p2p system. It accou
nts for 35% of all Internet traffic! (according to British Web analysis firm CacheLogic)
Warner Brothers to distribute films through Bittorrent (May 2006)
![Page 4: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/4.jpg)
Bittorrent Basics
Divide file into small pieces (256KB).
Utilize all peers’ upload capacities
server
client
client
client
Problem: large file (~GB) and large demand (10s, 1000s or more clients.) It is not feasible to set up infrastructure for traditional client-server download.
![Page 5: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/5.jpg)
Bittorrent schematicSeed (peer with entire file)
peer
peer
peer
new peer(with torrent file)
tracker
![Page 6: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/6.jpg)
Bittorrent algorithms: who to upload to?
Tit-for-tat: upload to peers from which most data downloaded in last 30 seconds (4 peers by default.)
Therefore: incentive to upload in order to be chosen by other peers!
![Page 7: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/7.jpg)
Bittorrent Algorithms: What piece to send?
Rarest-first: upload the piece that is rarest among your neighbors first
11 22
11 1122
33
![Page 8: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/8.jpg)
The ‘Broadcasting Model’
t = 0 11
t = 1 11
t = 2 11 11
11 11 11 11t = 3
M = 1, N = 7, all upload capacities are 1 piece per unit time
![Page 9: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/9.jpg)
Example: M = 2 , N = 3
t = 0
t = 1 11
t = 2 11 11
1122
t = 3 22
11
22 11
1122
22 2211
NM log(rarest first!)NM log
22 22t = 4
![Page 10: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/10.jpg)
Equal capacities, general M, N
Theorem 1:There exists a schedule for a server to broadcast M messages to N nodes in M+logN time [Bar-Noy et al, 2000]
However, it is very difficult to extend the result to networks of different capacities
![Page 11: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/11.jpg)
‘Uplink Sharing Model’
1 server, N peers with possibly different capacities.
Suppose upload capacities are the only bottleneck.
Suppose M >> 1SC
1C
2C3C
FF
![Page 12: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/12.jpg)
Optimal Last Finish Time
Theorem 2:the minimal time for all N peers to obtain a file F (optimal last finish time) from a server is
*L
*L
*L T, ... ,T,T
where F is the file size and Cs, C1,…,CN are the upload capacities. There always exists a schedule S0 such that the finish time vector is
N
j jSS CC
NF
C
F
1
*L ,maxT
![Page 13: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/13.jpg)
Example (Zero Peer Capacities) Suppose all peers have 0 capacity,
consider the following two strategies Divide capacity equally among peers:
Upload to peers one by one:
SSS C
NF
C
F
C
F,....,
2,
SSS C
NF
C
NF
C
NF,....,,
The last finish time is the same, but the latter is obviously better! In fact, the latter can be shown to be ‘average finish time’ optimal.
![Page 14: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/14.jpg)
Optimal Average Finish Time (N=3)
SC2
321 CCC 2
321
CCC 0
t1 t2 t30 t1 t2 t3 t1 t2 t3
SC
F
finish time
*LT
*LT
![Page 15: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/15.jpg)
Conclusion and Ongoing Work
Simple model with rich structure for understanding efficiency of p2p file sharing
It captures many issues Bittorrent addresses (e.g. favoring fast peers, rarest first policy)
Lots of questions remain open: understanding fairness-efficiency tradeoff other kinds of optimality criteria
![Page 16: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/16.jpg)
Netlab’s other research projects http://netlab.caltech.edu
More details about this work [email protected]
![Page 17: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/17.jpg)
Thank You!
![Page 18: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/18.jpg)
Backup slides start here
![Page 19: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/19.jpg)
Another way to look at Ts
1 if ,
1 if ,
T
1
1
1*S
N
CC
C
F
N
CC
C
NF
N
j j
SS
N
j j
SN
j j
![Page 20: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/20.jpg)
Previous Bittorrent Modeling Work
Qiu & Srikant [Sigcomm’04] Predator-prey-like fluid models Assumes equal capacities among peers Assumes rates of peer joins/leaves and stu
dies equilibrium and stability
![Page 21: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/21.jpg)
Proof of Theorem 2
N
j jSS CC
NF
C
F
1
*S ,maxT
First notice that the two terms have to be lower bounds of the optimal last finish time
So it remains to show that the equality is achievable. Here’s a strategy for that:
![Page 22: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/22.jpg)
Proof of Theorem 2
11
N
CC
N
j j
S
1NCi
When
the server allocates to peer i:
N
CC
N
C
NN
CC N
j jS
N
j j
N
j j
S
11
1
11
Each peer therefore receives:
N
j jSSN
j jS CC
NF
C
F
CC
NF
11
*S ,maxT
![Page 23: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/23.jpg)
Proof of Theorem 2
11
N
CC
N
j j
S
N
j j
Si
C
CC
1
When
the server allocates to peer i:
S
N
iN
j j
Si CC
CC
11
Each peer therefore receives:
N
j jSSS CC
NF
C
F
C
F
1
*S ,maxT
![Page 24: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/24.jpg)
Bittorrent Basics Torrent file:
Meta data about the file: filename, size, author, etc.
Hash info for each file piece to verify integrity Link to centralized tracker Published on the Web
Tracker: Keeps track of the IPs of peers ‘Bootstraps’ new peers Centralized, but does not coordinate data trans
fer among peers
![Page 25: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/25.jpg)
P2P systems
Napster (centralized directory) Kazza (semi-decentralized system with super peers) Gnutella (e.g. Limewire, Bearshare, decentralized) Bittorrent (most popular and successful for distribut
ion of large files)
![Page 26: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/26.jpg)
Another way to look at TL
1 if ,
1 if ,
T
1
1
1*L
N
CC
C
F
N
CC
C
NF
N
j j
SS
N
j j
SN
j j
![Page 27: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/27.jpg)
Non-Zero Peer Capacities
If the peer capacities are not all 0, then the “upload one by one” strategy can be shown to result in these finish times:
However… this is not average finish time optimal!
1
1211
,....,3
,2
,1
N
j jSSSS CC
N
CCCCCC
![Page 28: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/28.jpg)
Comparing finish time vectors
Definition: a finish time vector v1 is strictly better than another finish time vector v2 if no component of v1 is larger, and some component of v1 is smaller than the corresponding component of v2
- (2, 3, 3) strictly better than (3, 3, 3) - (1, 2, 3) (2, 2, 2) cannot be compared
with respect to this
![Page 29: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/29.jpg)
The ‘Broadcasting Model'
Assume discrete-time, synchronous system where N nodes have equal upload capacity of 1 “message” per unit time
Objective: find a schedule such that every node receives all M messages in minimal time
![Page 30: Optimal Scheduling in Peer-to-Peer Networks Lee Center Workshop 5/19/06 Mortada Mehyar (with Prof. Steven Low, Netlab)](https://reader035.vdocuments.site/reader035/viewer/2022062417/551a3eee5503463e778b4e7f/html5/thumbnails/30.jpg)
Assumptions reasonable for p2p
Size of file pieces (256KB for BT) is usually much smaller than total size of file (~GB). Namely, the number of pieces M >> 1.
Upload links are usually much slower (e.g. DSL lines), so assume upload capacities are the only bottleneck.