an experimental analysis of joost peer-to-peer vod service
TRANSCRIPT
Peer-to-Peer Netw. Appl. (2010) 3:351–362DOI 10.1007/s12083-009-0063-5
An experimental analysis of Joost peer-to-peerVoD service
Jun Lei · Lei Shi · Xiaoming Fu
Received: 17 March 2009 / Accepted: 22 September 2009 / Published online: 6 October 2009© The Author(s) 2009. This article is published with open access at Springerlink.com
Abstract Despite strong interest in peer-to-peer (P2P)Video-on-Demand (VoD) services, existing studiesmostly focus on peer-to-peer or overlay protocol de-sign based on simulations under various topologicalconstraints. We believe experimental studies on a real-life P2P VoD system will provide valuable informationto ISPs, network administrators, and content owners.In this paper we present a comprehensive analyticaland experimental study on Joost, one of the first com-mercial P2P VoD systems used for distributing variousforms of video over the Internet. Our extensive experi-ments prove that Joost is a server-assisted peer-to-peerVoD system. With several envisioned typical scenarioswe have further investigated the peer management interms of time pattern, bandwidth consumption andlocality considerations. Our major findings include: (1)the current Joost system is capable of providing high-quality VoD service through the use of an overlaynetwork deployed with a set of centralized contentservers; (2) inter-continental links are often used re-gardless of the number of local users, which may pose ahigh burden on the network providers; (3) easily reach-able, high-capacity nodes are selected as main relayingnodes, similar to super nodes in Skype, to facilitate the
J. Lei (B) · L. Shi · X. FuInstitute of Computer Science, University of Göttingen,Göttingen, Germanye-mail: [email protected],[email protected]
L. Shie-mail: [email protected]
X. Fue-mail: [email protected]
traversal of symmetric NATs and firewalls. We alsoprovide insights on the potential ways to construct moreefficient P2P VoD systems (e.g. considering topologicallocality-awareness, using adaptive/layered video).
Keywords Peer-to-peer (p2p) ·Video-on-demand (VoD) · Measurement
1 Introduction
In the recent few years, IPTV has gained a tremen-dous popularity in operators and users as well as a lotof attention from the research community [9, 11, 12].For residential users, such service is often providedin conjunction with VoD and may be bundled withother Internet services such as Voice over IP (VoIP).Traditionally, when a user selects a program, a point-to-point unicast connection is established between adecoder (aka set top box) and media server, which lacksefficiency and scalability. Most existing VoD servicesmainly rely on content distribution networks (CDNs)[16] or local streaming proxies to increase system scal-ability as well as to alleviate the delay experiencedby end users. However, their system performance anddeployment still faces a key challenge as the number ofusers increases. Especially, if a flash crowd [14] occurs,servers can be easily overloaded. The similar phenom-enon occurs when a web site catches the attention of alarge number of people, and gets an unexpected volumeand possibly overloading surge of traffic.
To address above issues, peer-to-peer technologies(e.g. swarming [7]) have been recently employed tosupport VoD services. However, it is more challengingto design a P2P VoD system than any other P2P media
352 Peer-to-Peer Netw. Appl. (2010) 3:351–362
streaming systems because, in addition to providinglow playback delays, the system allows users arrivingat arbitrary time to watch videos. The heterogeneousarrivals reduce sharing opportunities and increase thecomplexity of video distribution mechanisms. Besides,the system requires a certain local space to store thedownloaded video. Thus, another issue is how to allo-cate and use such storage in an efficient way to sup-port VoD functionalities. As current P2P VoD systemshave not been widely deployed, it is essential to un-derstand how to design a VoD architecture that scalessmoothly to support a large number of users, whilemaintaining high video quality and reasonable opera-tional costs. It is also critical for ISPs, network admin-istrators, and content owners to consider the networkand operational requirements for supporting P2P VoDsystems [22].
Most P2P VoD studies use simulations under var-ious common assumptions to evaluate their designs.However, due to a lack of large-scale deployed VoDsystems, few of these assumptions could be validatedthrough real measurement data. Our analysis on Joostseeks to validate and adapt existing assumptions aboutP2P VoD data. We choose Joost as the target for thestudy of peer-to-peer VoD services due to the followingreasons. As one of the earliest and large-scale com-mercial P2P VoD products, Joost has the potential tobecome popular following a successful story of Skype.The Joost architecture and many technologies it usesare proprietary although it is known to be built ontop of several open software such as Mozilla/xulrunner[1]. Except for the limited knowledge of the used opensoftware, the underlying P2P architecture and detailedmechanisms/techniques used in Joost, like when Skypewas new, are still unrevealed. While getting deep in-sights into various aspects of Joost has been challeng-ing, our results may provide valuable information toISPs, network administrators and content owners for abetter understanding of service requirements for build-ing and managing a scalable and efficient P2P VoDsystem.
The rest of the paper is organized as follows.Section 2 briefly introduces the overall architecture andidentifies the functionalities performed by the com-ponents of the Joost system. Section 3 describes ourexperimental setup. As the performance aspect playsan important role in P2P VoD acceptance, in Section 4we envision three typical usage scenarios and use themto study more closely on the peer behavior and ser-vice performance in terms of time pattern, bandwidthconsumption and locality considerations. Section 5presents related works. In Section 6 we conclude thispaper and plan future work.
Fig. 1 Joost architecture
2 Joost overview
Joost [1], created by N. Zennström and J. Friis, co-founders of Skype [21] and Kazaa [17], is one of firstP2P VoD systems for providing high-quality and com-prehensive VoD services using P2P TV technologies.Based on our preliminary experiments [18] using atestbed depicted in Section 3 and information in [2], wededuced the Joost system architecture using a top-downapproach: we firstly abstracted a high-level hierarchyfrom the overall architecture and then investigated thefunctionalities performed by each component of theP2P hierarchy.
Figure 1 shows five types of servers (cf. Section 2.1),one Joost peer manager,1 and various Joost clients.There are other servers taking charge of value-added services, for example, instant chat service (scd.joost.com) and advertisement server (lux-cdn-lo-4.joost.net). Because the fundamental functions are ourfocus, these additional servers have been omitted fromthe subsequent discussion.
2.1 Joost servers
Joost is rather a complicated system, and identifyingthe Joost servers facilitates our understanding of thepeer management mechanisms because their behav-iors can be differentiated from that of arbitrary peers.In this paper, we use the term of peer and clientinterchangeably.
As shown in Fig. 1, lux-www-lo4.joost.net is versionserver that is responsible for checking the current ver-sion of the software during login. For instance, Joost
1Although peer manager is named as super node in [2], in our ex-periments we identify that its functionality is peer management.It is not responsible for relaying/forwarding media data to otherpeers. Therefore, we call it peer manager in this paper.
Peer-to-Peer Netw. Appl. (2010) 3:351–362 353
Clients (JCs) sent HTTP 1.1 GET requests for gettingthe latest software version. The second type of server iscalled tracker server (i.e. lux-www-lo2.joost.net) whosesole responsibility is to keep track of its group membersand helps bootstrapping new peers.
Channel management in Joost differs largely fromany other P2P VoD system as it builds an APIfor the channel list using XULRunner that pro-vides Joost clients more interactive experience butmakes the system more complex. Due to the com-plexity, the channel management is not performed bya single server in Joost, but a server cluster. Thatis, the backend server, lux-backend-lo-1.joost.net, an-swers for controlling channel list requests and keep-ing load balance among cluster servers, whereas othercluster servers perform particular tasks (e.g. channelgraphs downloading). Some of them can be namedchannel graphics servers. For instance, there weretwo servers, lux-backend-13-bond0.joost.net and orsna-www.static-1-bond0.joost.net from which JC down-loaded the channel graphics instead of directly from thebackend server. Nevertheless, the scalability might be amajor concern for the future development if the num-ber of users dramatically increases in a short period.
The last type of Joost server is content server.During our experiments, we observed the followingserver sites: (1) 4.71.105.0/24 (sna-Itsnode-x-bondx-x.joost.net); (2) 4.71.174.0/24 (IPsoft); (3) 212.187.185.0/24 (Icy-Itsnode-x-bondx-x.joost.net). Here, x variesfrom 0 to 10. The first and third IP address site is ownedby Level 3 Communication INC [19] which has beenselected by Joost to support on demand Internet TV.The second IP space group belongs to IPsoft serviceprovider.
2.2 Peer manager
Another major distinguished design from other P2Pnetworks (e.g. Skype) or overlay multicast solutions[6] is that all peer managers in Joost are only usedfor controlling and helping new peers find availablecontributing peers. Based on above understanding, thecontrol traffic from peer managers can be easily differ-entiated from other media data traffic in Section 4.3.
In fact, it is quite efficient and reasonable that thepeer management is isolated from the media distribu-tion. According to [20], some universities have alreadybanned Skype from their campuses, while some otheruniversities and government agencies require that theirusers disable supernode functionality to avoid relayingtraffic outside the stub Autonomous Systems (ASs).Apparently, Joost designers have taken the issue intoconsideration. Moreover, strategically deployed peer
managers not only ease the membership managementbut also improve the reliability of transmission. Forexample, if a super node in Skype leaves ungracefully,all the other peers relying on it will be unavoidablyaffected.
2.3 Joost client
While active a Joost Client (JC) performs one of the fol-lowing actions: listen on particular ports for incomingtraffic; store media data into its local cache; maintain atable of other peers called a host cache, uses AdvancedVideo Codec (AVC); determine if it is behind a NAT orfirewall; and functions required by additional features,such as instant messaging. Without a surprise, Joost andSkype have some p2p mechanisms and techniques incommon. Detailed information can be found in [18].
2.4 Protocols
In order to evaluate the efficiency of video transmissionit is necessary to identify the protocols used for controltraffic and media data traffic.
Table 1 depicts the main protocols used in the Joost.All video packets are encoded in UDP and the sizeis exactly 1104 bytes. Using UDP for video transmis-sion is not reliable, however, it’s quite cost-effectiveand time-efficient especially for large-scale peer-to-peer networks. Besides the media transmission, peersfrequently communicate with each other by sendingUDP probes (64 bytes). Since April 2007, when portnumber 4166 was assigned by IANA as the official UDPport used for Joost, all media data and some controlmessages (e.g. peer management) are sent from Joostservers through 4166. Tracing these specific port num-bers facilitates our following performance experiments.
Table 1 Main protocols in the Joost system
Protocol Functionality Packet size
UDP Video distribution 1104 bytesContent probe (peer to peer) ∼ 64 bytesChannel switching < 1000 bytesVoD interactions < 150 bytes
HTTP Software versionClient → server ∼ 64 bytesServer → client < 500 bytes
Channel managementClient → server ∼ 64 bytesServer → client <= 1518 bytes
HTTPS Administrative managementClient → server 64 bytesServer → client < 500 bytes
354 Peer-to-Peer Netw. Appl. (2010) 3:351–362
HTTP is used for checking software version andupdating the channel list during bootstrapping and ini-tialization phases. When the JC browses the channelcategory, the channel graphs are downloaded in realtime through HTTP from the graphics servers.
When the JC re-connects to the Joost system,HTTPS is used for the administrative management du-ties which include checking software version, channellist updating, obtaining trackers.
2.5 Local video cache
So far, we identified the Joost architecture and associ-ated key components, which are most relevant to ouranalysis in Section 4. As supporting VoD functionalitiesrequires a local storage, different cache managementstrategy has a great impact on peer management. Wedid some experiments to identify its impacts.
It has been observed that during the first threemonths of our experiments, one peer’s the cache sizegrew up to 3.52GB (FAT 32) and has not been dwin-dled, however, in January 2008 Joost changed the de-sign to clean up the local cache after rebooting theclient. On the one side, such a change is rational, other-wise, the user’s resources will be significantly occupiedif the JC continuously switches to different channels.On the other side, it largely reduces the resource shar-ing opportunities among peers since newcomers cannotget video data from those who have downloaded videoin their local cache during the past logins.
JCs store the media data in their local caches as“anthill_cache” [18]. Our initial assumption was thatthe local cache did completely store the played videoand thus the JC should watch the old program directlyfrom the local cache. However, once we disabled theInternet connection, the program surprisingly stopped,even if the same program has just been watched. There-fore, although media data has been stored locally, play-back still requires a kind of codec from the remoteserver or an encryption key (e.g. AES key) authorizedby the Joost server to access the video file. To prove theconjecture, we performed following experiments.
We launched a new channel and at that moment, thelocal cache was empty. After the whole channel waswatched, the size of cache file grew up to 1.7 GB andthe average download speed was 518 kbps. When weswitched channel and then watched the same channel,the download speed dramatically dropped down to11 kbps. Moreover, the size of local cache increasedonly 0.1 GB (< 6.0%) during the second watching pe-riod. It is very likely that the slightly increased cachenow contains either a type of codec or an encryptionkey to allow the client to access the cached video file.
3 Experiment setup
There are three main Joost server sites: the USA,Europe and Asia [2]. Since all mechanisms and tech-nologies are assumed to be used in the same way,our experiments explore European site also due to theauthors’ location in Germany (GMT+01:00).
All experiments were performed between Septem-ber 2007 and February 2008, in which we used WindowsXP SP2 machines at various geographical locations.Note that we also used Windows Vista and MACOS X machines for the testing, however, their resultshaven’t shown significant difference from the followingresults. To illustrate our major findings, two Universitynodes configured with public IP addresses, another twoUniversity nodes located behind a NAT/Firewall (N/F-behind nodes); and two ADSL users were selected toset up an experimental testbed. As depicted in Fig. 2,university nodes were equipped with the same process-ing power and connected to 100 Mbps full-duplexuniversity LAN. ADSL users were supported with2 Mbps download and 1 Mbps upload bandwidth. SinceDecember 2007, Joost has been open to public althoughit was still called Beta version (Beta v1.0) during thetime of our experiments.
For data collection, we used Wireshark [5] andOmnipeek [4]. Tools like WhereIsIP [23] were used toperform reverse country, city and ISP lookups for an IPaddress when Omnipeek failed to return a DNS PTRrecord.
Fig. 2 Experimental testbed design
Peer-to-Peer Netw. Appl. (2010) 3:351–362 355
4 Inferring Joost peer management
Through three envisioned scenarios, we investigatedthe P2P management mechanisms which are the mostimportant and complicated aspect in the Joost system.We resort to passive measurements and data-drivenanalysis in order to reveal the aspect that a purely activemethodology alone cannot capture.
4.1 Experimental methodology
To facilitate our measurements, we analyze the majorfactors in which the peer management may involve.
• Time pattern: The different user distribution duringa day or a week may have great impacts on theperformance (e.g. contributions from peers highlydepend on the number of peers).
• Upload and download capacity: The peer manage-ment can be benefit from the efficient bandwidthusage if peer is given some incentives to contributemore to the network.
• Popularity impacts: The number of users may belargely determined by the popularity of the on-demanded programs.
• Locality considerations: One of the main challengesin P2P VoD system is the efficient allocation of theavailable resources. Thus, it is generally desirablethat data exchange be made preferably betweennodes that are placed “close by” in the underlyingnetwork to reduce the redundant usage of long-haul network links and to save local resources fornetwork providers.
4.2 Designed scenarios
• Scenario 1: To get a broader view of the time pat-tern, we monitored public nodes and NAT nodesover a period of three weeks (7–28 January, 2008)and captured over 78 GB data. Those test nodeswere equipped with the same processing power andbandwidth support as described in Section 3. Toalleviate the popularity impacts, we repeatedly rana 1063-min channel created by our own at bothnodes, including popular and unpopular programsselected from existing program list. Note how tocreate a new channel is out of scope of our paper,but can be found in [1]. Once the channel wasfinished playing, we dumped their local cache (c.f.Section 2.5) and re-started playing.
• Scenario 2: We randomly chose programs rankingin the “most popular programs” list and unpopularprograms with the same length. In this experimen-
tal scenario, a public node and a N/F-behind nodecontinuously ran popular and unpopular programsover two-week period (14–28 January, 2008). Wecaptured 24 GB data.
• Scenario 3: Two test nodes were located behinda Network Address Translator (NAT) and config-ured with non-routable, private IP addresses. Oneof them started to randomly choose one channel.After a short period (e.g. 5 minutes), the other nodeselected the same channel.
Scenario 1 is mainly designed to explore above twofactors, namely, time pattern and upload and downloadcapacity in Section 4.1. The second scenario is usedto investigate whether popularity impacts have beenconsidered in Joost’s P2P management mechanisms.In the last scenario, it would be interesting to see ifthe locality-awareness has been carefully consideredin Joost system. As two NAT nodes were physicallylocated next to each other, it would be possible that thesecond test node receive a large portion of data fromthe first node.
As media data is encapsulated in UDP (cf. Sec-tion 2.4), we only captured UDP packets and isolatedmedia data from other control traffic. Nevertheless,Joost encrypts all UDP payloads and therefore, ouranalysis on the collected data is restricted to IP andUDP header fields including the source and destinationIP address and port, and packet lengths.
4.3 Experimental results
Our initial experiments [18] indicated that Joost re-lies on a plenty of dedicated infrastructure nodes (e.g.content servers) to distribute video. However, sinceDecember 2007 during our experiments, the contribu-tions of peers largely increased and varied according tothe time/life pattern. The following experiments illus-trate our recent findings.
4.3.1 Time pattern—NAT/FW-behind node
Figures 3 and 4 illustrates typical segments of firstscenario results of N/F-behind node. They plot theaverage percentage of media contributions from Joostservers (cf. Section 2.1) and peers. We separate week-day trace from weekends trace since they have differentdistributions.
The first main observation from above two figuresis that Joost servers delivered a majority portion ofmedia data to Joost clients during the entire week(85% in average for weekday trace, 91% in averagefor weekends). The deviations identify that both traces
356 Peer-to-Peer Netw. Appl. (2010) 3:351–362
0:00 2:00 4:00 6:00 8:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 24:00
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
11 January, Friday
Ave
rage
Per
cent
age
of C
ontr
ibut
ions
Time (7-11 January, 2008, GMT+01:00)
From Content Servers From Peers
Fig. 3 Weekday trace of NAT/FW-behind node
follow the similar pattern except for two time slotsin the weekdays (16:00 and 22:00, 11 January, Friday)when there were a great number of peers contributedto the test nodes. In fact, it is understandable since bythat time the weekends had already started in someEuropean countries. Note that our experimental lo-cation was in Germany. To verify our conjecture, wefurther traced these contributed peers at these two par-ticular time slots. Among the total peer contributions(63.3% in average), 38% of the media data was con-tributed by European peers and 25.3% was transmittedfrom the US.
0:00 2:00 4:00 6:00 8:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 24:00
-0.1
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Ave
rage
Per
cent
age
of C
ontr
ibut
ions
Time (12-13 January, 2008, GMT+01:00)
From Content Servers From Peers
Fig. 4 Weekend trace of NAT/FW-behind node
The second interesting observation is that there weretwo user peeks in weekday traces, 6:00 and 12:00, whena lot of contributions (50–60%) were from other peersinstead of content servers. According to what has beenobserved in [10] that the number of users drops gradu-ally during the early morning and climbs up to a peakwhen users are in noon break, the number of Europeanusers is expected to be the least in both time slots,namely, 4:00–6:00 and 10:00–12:00 in our experiments.In other words, if the contributors were located inEurope, it would be against the daily life pattern sincethe first time slot (4:00–6:00) will be sleeping time and10:00–12:00 is working time. In contrast to the weekdaytrace, during 0:00–2:00 in the weekends there was a userpeek, which is possible that most of the contributionscame from European countries.
To discover the reasons and identify our conjecture,we further analyzed user peeks during these two timeslots. As shown in Fig. 5, from 4:00–6:00 most of thedata was contributed from the U.S. and 10:00–12:00European peers contributed the most (60–65%) but USpeers still contributed over 20%. It indicates that inter-continental links (between the U.S. and Europe) areoften used regardless of the number of local peers. Forthe third slot 0:00–2:00, peers from the U.S., Europeand others shared the similar portion of the contribu-tions.
As network operators struggle to control the overallusage of bandwidth, it would be advisable that P2Pservice provider could constrain the media data trans-mission within a locality without frequent use of inter-continental links [15]. Otherwise, the inter-continentalbandwidth usage will become a non-marginal issue
4:30 5:00 5:30 10:30 11:00 11:30 00:30 1:00 1:30
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
From US From Europe From Asia From UK From South Africa From others (e.g. Australia)
Per
cent
age
of C
ontr
ibut
ions
Time (8 January,2008, GMT+01:00)
Fig. 5 Timeslot trace of NAT/FW-behind node
Peer-to-Peer Netw. Appl. (2010) 3:351–362 357
for the network providers. To verify whether locality-awareness has been carefully considered in Joostsystem, we studied its performance additionally inSection 4.3.5.
4.3.2 Time pattern—public node
Since public nodes and N/F-behind nodes ran the samechannel, the available contributions were assumed tobe similar. Surprisingly, public IP address configurednodes relied great heavily on the Joost content servers.As depicted in Figs. 6 and 7, most of the time contentservers contributed over 60% of the media data. Com-paring with Figs. 3 and 4, we conjecture that Joost usesa peer management algorithm similar to the one usedin Skype that easily reachable nodes (e.g. public nodes)with high capacity are used to relay traffic for otherpeers, so-called super nodes in Skype. Actually, thedifference of upload throughput between public nodesand N/F-behind nodes is significant (see Section 4.3.3).Besides, we believe that the available bandwidth, per-formance (e.g. CPU, memory size) are considered inthe peer selection phase, which has been identifiedin [18].
If we consider the deviation of the weekly trace inFigs. 6 and 7, except for the time during 4:00–6:00, therest hourly trace followed the similar pattern. To revealthe cause of difference, we tracked this particular timeduration. It was noticed that on average 26.4% (total49.3%) of contributions came from the US (local time:20:00–1:00), 17.5% from Europe and rest of peers (e.g.
0:00 2:00 4:00 6:00 8:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 24:00
0.0
0.2
0.4
0.6
0.8
1.0
7 January (Monday)
Ave
rage
Per
cent
age
of C
ontr
ibut
ions
Time (7-11 January, 2008, GMT+01:00)
From Content Servers From Peers
Fig. 6 Weekday trace of public node
0:00 2:00 4:00 6:00 8:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 24:000.0
0.2
0.4
0.6
0.8
1.0
Ave
rage
Per
cent
age
of C
ontr
ibut
ions
Time (12-13 January, 2008, GMT+01:00)
From Content Servers From Peers
Fig. 7 Weekend trace of public node
Australia (GMT+10:00)) contributed 5.43%. Again,it conforms to the above observation that most con-tribution were transmitted through inter-continentallinks.
Furthermore, the weekend trace performed quitedifferently from weekday trace. Particularly, the timeslots of 2:00 and 17:00 respectively reached the peercontribution peeks (over 50% of the contributions).Among these contributions, European peers transmit-ted on average 34.72% of the media data, US peersforwarded 11.0% and others contributed 6.48%. Nev-ertheless, Joost servers still contributed 47.8% which ismuch higher than any of the above contributions.
Lastly, the number of contributing peers is expectedto be even higher at weekend nights but for both publicand N/F-behind nodes most of the contributions camefrom content servers. As observed in [18], the possibleexplanation is that the local Joost users in Germanywere limited likely due to the current programs aremostly only in English without German subtitles. Ifthe Joost client can select preferred language in thesecondary audio tracks, it will attract more clients dur-ing their relaxing time. The other explanation couldbe that most Internet TV fans are night owls since forboth public node and N/F-behind node between 0:00and 2:00 on weekends the peer contributions reachedthe highest peak. Another possible reason is that mostGerman resident users were slow-speed ADSL/DSLusers (e.g. 2 Mbps download and 1 Mbps upload band-width) and thus, their contributions might not be suffi-cient to support the high-quality playback requirementat other users.
358 Peer-to-Peer Netw. Appl. (2010) 3:351–362
0 200 400 600 800 1000
0
50
100
150
200
250
300
350
400
450
500
550
600
Thr
ough
put (
kbps
)
Time (minutes)
Download (public)
Download (NAT-behind)
Upload (public)
Upload (NAT-behind)
Fig. 8 Data throughput of public and NAT/FW-behind node
4.3.3 Bandwidth consumption
Having isolated the UDP packets, we examined theaverage throughput of public node and N/F-behindnode for each period of 1000 minutes through fourweeks. Figure 8 shows that the public node’s uploadthroughput is on average 67% higher than that of theN/F-behind node although they have the same capacityregarding its bandwidth support and processing power.Such an observation suggests that public node is likelychosen as relaying nodes for other peers. Moreover,the average download throughput of the public nodeis 15% higher than that of the N/F-behind node sincemost of the data was directly transmitted from contentservers. The reason why the throughput of the publicnode was not stable is that the content servers may notbe able to contribute with the same amount of mediadata when simultaneously serving a large amount ofpeers.
We observed that the N/F-behind node downloadedand uploaded media data in a fairly constant speedcompared with the public node. The download through-put of the N/F-behind node was 438 kbps, that is,200 MB per hour, and 22 MB per hour for uploading.For public nodes, the average download throughputwas 493 kbps, namely, 225 MB for one hour, and 68 MBper hour for uploading (3 times more than that of N/F-behind nodes). Through the experiments, we found thatthe average percentage of control traffic among thetotal traffic was 15%. Thus, public nodes only need tosupport 580 kbps downlink capacity and 84 kbps foruplink although they were connected to 100 Mbps full-
duplex university LAN. Thus, we suggest that Joost canbe a little aggressive especially to high-capacity nodewhen they are available.
Further, we explicitly investigated low capacitynodes with ADSL connections in [18]. Unfortunately,these clients could be weakly (e.g. occasionally stalled)supported in the Joost system since current Joost sys-tem only provides the same quality for any video. Wewould suggest using layered or adaptive mechanismsfor more efficient video distribution. For example,servers or some high-capacity nodes are responsiblefor transmitting the basic layer of the encoded videoand other available peers could be used to transmit theenhanced layers in order to improve the video quality.Although it might introduce some complexities intothe peer management, the Joost system could supportmuch more users including some low-capacity nodeswithout wasting network resources.
4.3.4 Popularity impacts
As identified that pubic nodes actively participate inrelaying media data for other peers, we only tracedtwo public nodes running different types of programsas defined in Scenario 2.
As with many P2P media streaming applications, thenumber of users is largely determined by the popularityof selected program. For popular channels, the uploadthroughput should be much higher than that of unpop-ular channels. What is observed in Fig. 9 is consistentwith the speculation that popular channel node’s up-load throughput was much higher (over 150%) thanthat of unpopular channel node.
0 100 200 300 400 500 600 700 0
20
40
60
80
100
Upl
oad
Thr
ough
put (
kbps
)
Time (Minutes)
Popular Channels
Unpopular Channels
Fig. 9 Upload throughput of popular channels vs. unpopularchannels
Peer-to-Peer Netw. Appl. (2010) 3:351–362 359
For popular channels, at the initial phase thethroughput increased dramatically to reach thethroughput peek 85 kbps. Then, it decreased till35 kbps and increased again till 60 kbps and keptrelatively low. The near constant upload throughputduring the late stage suggests that Joost P2P systemscales well since more contributors are able to forwardmedia data after they receive the data.
During our experiments, the throughput of unpopu-lar channels is always low (in average 2.34 kbps) and theunpopular channel node comes into the stable phasemuch earlier than the popular channel node. We con-jecture that there are much less requests in the systemfor the unpopular programs.
4.3.5 Locality considerations
After three-day repeated experiments of Scenario 3,we analyzed the collected data from both test nodes.Although the two test nodes were watching the samechannel and geographically locating near each other,the second test node only received in average 1.3%of the data from the first node (96.2 MB out of ap-proximately 7.4 GB). Note that there are basicallythree different levels of locality: 1) geographical (e.g.same continent area); 2) AS-level (e.g. same AS); and3) topological (e.g. same access network). The resultscan identify that the topological locality have not con-sidered in the peer management. Otherwise, the secondtest node would receive most of the data from the firstnode instead from other remote nodes.
To further verify above observations, we parsed theIP address of contributed peers from which our testnodes received data. Totally, we identified 1210 distinctpeers which provided inneglectable contents to our testnodes. These peers were located in over 54 countries.As shown in Fig. 10, the major sources of peers areEurope and the United States. Of all the data collectedfrom the test nodes, 45% (547) came from Europeancountries, 24% (293) came from the United States,8.2% (99) came from Asian countries, 7.9% from SouthAmerica, 3.6% from other countries. Besides, 45 IPaddresses were not traceable and therefore we markedthem as “unknown”.
Moreover, sources of JCs from Germany were 130(19% of Europe). Since our host was located inGermany, we suspect that the geographical distance(e.g. from specific continent) may have been consideredin Joost. For example, the prefix awareness may havebeen considered during the peer selection. Note that ahigh-level (geographical locality) is not enough for peermanagement since resource can be still wasted for being
3.6%3.8%
7.1%
7.9%
8.2%24%
45%
Europe North America Asia South America United Kingdom Others Unknown
Fig. 10 Geographic location
transferred to a remote user in the same geographicallocation.
In summary, we ascertain that the locality-awarenessis not well designed in the Joost system and can beimproved with more considerations on topological levelawareness.
5 Related work
Most of existing work about P2P VoD systems wasconcentrated on the protocol design and the implemen-tation [8, 9, 12]. Other existing experimental analysisis based on Enterprise-supported data and focusing ontranslating daily “log” file into user arrival and de-parture distribution. Besides, the video data providedby other P2P media streaming systems [3, 10] is notseriously encrypted and the corresponding architectureis much easier.
Hei et al. [11] provided a comprehensive overviewof P2P streaming system and characterized P2P IPTVbehavior and traffic profiles at packet, connection andapplication levels. Their observations provided us astarting point towards comprehensive understanding ofpeer management mechanisms in Joost. Nevertheless,through experiments we identified the Joost architec-ture which is quite different from, even more com-plex than, any media streaming architecture describedin [11]. The VoD functionalities give the users moreflexibilities, and hence make the system more difficultto design as well as its reliability is more difficult tosupport. With regard to peer management, each Joostclient is more “selfish” since it only cares about thecontent behind its current playback position.
Huang et al. [13] presented a general architecture forsupporting P2P VoD services based on PPLive [3]. The
360 Peer-to-Peer Netw. Appl. (2010) 3:351–362
paper focused on introducing some important buildingblocks of PPLive system as well as evaluating usersatisfaction and health of the systems. However, theirfocus is apparently different from ours, for instance,their measurement analysis does not consider popular-ity impacts and locality considerations.
Hall et al. [24] performed a measurement study ofJoost in May, 2007. The authors showed a prelimi-nary understanding of Joost’s application behavior andnetwork behavior. However, there are several majordifferences between their work and our work. First,their experiments were taken based on Joost version0.9.2 which is now out-of-date. For instance, sinceJanuary 2008 Joost changed the design to clean upthe local cache after rebooting the client. Exactly be-cause of that, the peer management can be very differ-ent from previous experiments. Through our extensiveexperiments we inferred the Joost architecture andkey components. Moreover, we designed some typicalscenarios in order to investigated the performance ofpeer management in terms of time pattern, bandwidthconsumption and locality considerations, which was notprovided in [24].
6 Conclusions and future work
Joost is one of the first commercial P2P VoD systemswhich can provide high quality on-demand TV based onP2P technologies. Unlike live media streaming system,each VoD client is more “selfish” in the sense thatit only cares about contents after its current playingposition, which is often different from other peers. Thepeer can only download from those whose playbackpositions are ahead, or from who have already down-loaded the program. Instead, itself can help peers whichjoin later than itself. However, as each client can changeits playback position at any time, which differs frommany other P2P streaming systems, it becomes difficultto optimize the overall VoD system. For example, the“rarest-first” strategy [7] in BitTorrent is not applicablehere.
In this paper we have made a first step towardsdiscovering various aspects of the Joost functions andbehavior by analyzing the network traffic and by beingacquainted with some of the open software used inJoost. We resort to data-driven analysis upon passivemeasurements to real the peer management mecha-nisms used in Joost. Our analysis demonstrates thatwith some dedicated infrastructure the current Internetinfrastructure is capable of meeting performance re-quirements of high-quality VoD. Although large-scaleP2P VoD systems are potentially deployed in today’s
Internet, the performance could be improved, for ex-ample, based on the following observations:
• The current Joost system relies on an overlay net-work deployed with a set of centralized contentservers as identified in Sections 4.3.1 and 4.3.2,which may still raise a scalability issue in the nearfuture. However, it is very useful to separate mediadistribution from controlling peer-to-peer hierar-chy, which makes the Joost system relatively stableand potentially scalable.
• As indicated in Sections 4.3.2 and 4.3.3, we believethat public nodes with high capacity may be se-lected as main relaying nodes to ease the traversalof symmetric NATs and firewalls. However, in ourexperiments their uplink capacity usage is still quitelow (on average 84 kbps). Thus, the Joost systemcould be aggressive to these nodes, together withcertain incentive mechanisms to encourage themcontribute more to the network, which may help thesystem overcome the scalability issue.
• Section 4.3.5 identified that the geographical dis-tance may have been considered in the peermanagement of Joost. However, the lower-levellocality-awareness (e.g. topological locality) maystill be missing in the peer management. Besides,the inter-continental links are often used to trans-mit media data regardless of the number of localusers, which may overload the network provider’scosts. If the P2P service could be AS-/network levellocality awareness, it would be beneficial for bothcustomers and service providers.
• Joost currently provides each client with the samequality of video. This may result in an inefficientresource utilization if some clients are unable tosupport the desired video quality. Hence, layeredvideo or adaptive mechanisms could be introducedinto Joost.
As our experiments were conducted through passivemeasurements and data-driven analysis, above obser-vations could be potentially limited in some perspec-tives. For example, the selected test-bed is relativelysmall due to the time and experimental limitations.Although we believe the major findings would be con-sistent with the fact behind the Joost system, it wouldbe favorable to establish a large-scale testbed. Further,we mainly performed our measurements in Germany;however, it would be more interesting to conduct ex-periments in different locations (e.g. other Europeanand Asian countries).
Besides, there are several avenues left for futurework. For example, the exact peer lookup and selectiontechniques are still unclear. Our guess is that it uses a
Peer-to-Peer Netw. Appl. (2010) 3:351–362 361
combination of swarm mechanisms in BitTorrent andprefix awareness. Therefore, we intend to take a closeinvestigation on Joost VoD functionalities, thoughdifficult, which could provide more useful hints on howto provide efficient peer management methods.
Open Access This article is distributed under the terms of theCreative Commons Attribution Noncommercial License whichpermits any noncommercial use, distribution, and reproductionin any medium, provided the original author(s) and source arecredited.
References
1. Joost (2009) Joost homepage. http://www.joost.net/2. scaryideas.com (2009) Joost network architecture. http://
www.scaryideas.com/video/2362/3. PPLive (2009). PPLive homepage. http://www.pplive.com4. WildPackets (2009) WildPackets homepage. http://www.
wildpackets.com/5. Wireshark (2009) Wireshark homepage. http://www.
wireshark.org6. Abad CL, Yurcik W, Campbell RH (2004) A survey and
comparison of end-system overlay multicast solutions suit-able for network-centric warfare. In: Battlespace digitizationand network-centric systems IV, pp 215–226
7. Cohen B (2003) Incentives build robustness in bittorent.In: 1st workshop on the economics of peer-2-peer systems,Berkley, CA
8. Do T, Hua K, Tantaoui M (2004) P2vod: providing faulttolerant video-on-demand streaming in peer-to-peer environ-ment. In: Proceedings of ICC’04
9. Guo Y, Suh K, Kurose J, Towsley D (2007) P2cast: peer-to-peer patching for video on demand service. Multimedia ToolsAppl 33(2):109–129
10. Yu H, Zheng D, Zhao BY, Zheng W (2006) Understandinguser behavior in large-scale video-on-demand systems. In:Proceedings of EuroSys’06
11. Hei X, Liang C, Liu Y, Ross K (2007) A measurement studyof a large-scale p2p iptv system. In: The 6th internationalworkshop on peer-to-peer systems
12. Huang C, Li J, Ross K (2007) Peer-assisted vod: makinginternet video distribution cheap. In: The 6th internationalworkshop on peer-to-peer systems
13. Huang Y, Fu T, Chiu D-M, Huang C (2008) Challenges,design and analysis of a large-scale p2p vod system. In:Proceedings of ACM SIGCOMM
14. Norros I, Prabhu BJ, Reittu H (2006) Flash crowd in a filesharing system based on random encounters. In: Workshopon interdisciplinary systems approach in performance evalu-ation and design of computer and communication systems
15. IETF (2009) IETF application-layer traffic optimization(ALTO). http://www.ietf.org/html.charters/alto-charter.html
16. Almeida JM, Eager DL, Frris M, Vernon MK (2002) Provi-sioning content distrition networks for streaming media. In:Proceedings of INFOCOM
17. Kazaa (2009) Kazaa homepage. http://www.kazaa.com/18. Lei J, Shi L, Fu X (2007) An experimental analysis of joost
peer-to-peer vod service. Technical report no. IFI-TB-2007-03, Institute for Computer Science, University of Goettingen,Germany. ISSN 1611-1044
19. Level 3 Communications (2009) Level 3 communicationshomepage. http://www.level3.com/
20. Paul R (2006) More universities banning Skype. http://arstechnica.com/news.ars/post/20060924-7824.html
21. Baset SA, Schulzrinne HG (2006) An analysis of the Skypepeer-to-peer internet telephony protocol. In: Proceedings ofINFOCOM, Barcelona, Spain
22. Hilt V, Rimac I, Tomsu M, Gurbani V, Marocco E (2008)A survey on research on the application-layer traffic op-timization (ALTO) problem. Internet draft (draft-ietf-alto-survey-00), ALTO
23. Jufsoft (2009) WhereIsIp. http://www.jufsoft.com/whereisip/24. Hall YJ, Piemonte P, Weyant M (2007) Joost: a measurement
study. Carnegie Mellon University, Pittsburgh
Jun Lei ([email protected]) received her Dr. rer. nat inComputer Science from the University of Goettingen, Germanyin July 2008, and the M.E. degree in Engineering of Sciencefrom Zhejiang University, China, in 2004. She is also awardedwith “Buchpreis der Fakultaet fuer Mathematik und Informatik”.She is currently a Post-doctoral research fellow in the Insti-tute of Computer Science, University of Goettingen, Germany.Her research interests include overlay networks, peer-to-peernetworking, multimedia systems and mobile networks. She hasserved on a number of conferences as TPC member, includingGLOBCOM, WiMob, SIMUTools, IWCMC and ICCCN.
Lei Shi ([email protected]) received his B.S. degreefrom Chongqing University, China, and his M.S. degree fromUppsala University, Sweden, both in Computer Science. He iscurrently a Ph.D. student at University of Goettingen. His re-search interests include network processors, P2P networks andTCP enhancements.
362 Peer-to-Peer Netw. Appl. (2010) 3:351–362
Xiaoming Fu ([email protected]) received his PhD degreein Computer Science from Tsinghua University, Beijing, China in2000. After working at Technical University Berlin as a research
staff for almost 2 years, he joined the University of Goettingen in2002 as Assistant Professor, leading a research team working onnetworking research. Since April 2007 he has been a Professorand Head of the Computer Networks Group at the UniversityGoettingen. His research interests lie in the architectures, proto-cols and mechanisms for QoS, firewalls, multicast, peer-to-peer,overlay, mobile networking and security related issues. He haspublished about 50 referred papers in international conferencesand journals, as well as several RFCs/I-Ds. He has also served ona number of conferences as session chair, program chair or TPCmember, including INFOCOM, ICDCS, ICNP, GLOBECOM,ICC, MobiArch. He chaired the 1st and 2nd ACM Workshopon Mobility in the Evolving Internet Architecture (MobiArch)and currently serves on its steering committee. He is a mem-ber of editorial board of Elsevier’s Computer CommunicationsJournal, and a guest editor of the IEEE Network’s forthcomingSpecial Issue on Implications and Control of Middleboxes in theInternet.