p2p and more …

2012 © R. Stainov 1

P2P and more …

Rumen Stainovhttp://www2.hs-fulda.de/~stainov/erasmus/TUSofia/

L: TKSoftP: [email protected]

Peer-to-Peer – what is this ?

2012© R. Stainov 2


2012© R. Stainov 3

4

System Architectures

Client-Server Data Centers / Cloud Computing

Peer-to-peer (P2P) Hybrid Client-Server and P2P

2012 © R. Stainov

Client-Server Architecture

Server: Always-on Host Permanent IP address Server Farms for Scaling

Clients: communicate with server may be intermittently

connected may have dynamic IP

addresses do not communicate directly

with each other

client/server


Google Data Centers Estimated cost of data center: $600M Google spent $2.4B in 2007 on new data centers needs 50-100 Megawatt electricity


Pure P2P Architecture

no always-on server arbitrary end systems

directly communicate peers are

intermittently connected and change IP addresses

Highly scalable but difficult to manage

peer-peer


8

0

0.5

1

1.5

2

2.5

3

3.5

0 5 10 15 20 25 30 35

N

Min

imu

m D

istr

ibut

ion

Tim

e P2P

Client-Server

Server-client vs. P2P: File Distribution Time

Hybrid of Client-Server and P2P

Skype voice-over-IP P2P application centralized server: finding address of remote

party: client-client connection: direct (not through

server) Instant messaging

chatting between two users is P2P centralized service: client presence

detection/location user registers its IP address with central

server when it comes online user contacts central server to find IP

addresses of buddies 9


First Idea – Direct downloading of (music) files

Shawn Fanning, Boston Student, called Napster

102012© R. Stainov

Napster File Sharing: hybrid center + edge

“slashdot”•song5.mp3•song6.mp3•song7.mp3

“kingrook”•song4.mp3•song5.mp3•song6.mp3

•song5.mp3

1. User starts Napster and connects to the server

3. beastieboy enters search criteria

4. Napster shows the hits to beastieboy

2. Napster creates a Dynamic Directory of the personal .mp3 collection of the user

Title User Speed song1.mp3 beasiteboy DSLsong2.mp3 beasiteboy DSLsong3.mp3 beasiteboy DSLsong4.mp3 kingrook T1song5.mp3 kingrook T1song5.mp3 slashdot 28.8song6.mp3 kingrook T1song6.mp3 slashdot 28.8song7.mp3 slashdot 28.8

5. beastieboy connects direcly to kingrook for file transfer

s o n g 5

“beastieboy”•song1.mp3•song2.mp3•song3.mp3

11

Peer-to-Peer Systems

Second Idea, Using Fat Clients for Resources Sharing Thin client Fat client

2012© R. Stainov 12

IBM 8-bit PC

@ 4.77MHz360k

Diskette

64-bit PC @ 4-core 4GHz1 TB HD

DEC’S VT100No Memory

Peer-to-Peer (P2P)

Resources Sharing

What can we share? Computer-related resources

Shareable related-computer resources: CPU cycles - seti@home, GIMPS Bandwidth - PPLive, PPStream Storage Space - OceanStore, Murex Data - Napster, Gnutella People - Buddy Finder Camera, Microphone, Sensor, Service???

13

SETI@Home

SETI – Search for Extra-Terrestrial Intelligence @Home – On your own computer A radio telescope in Puerto Rico scans the sky

for radio signals Fills a DAT tape of 35GB in 15 hours That data have to be analyzed

14

SETI@Home – stat MUX using parallel computing resources


Third Idea – Searching for (music) files on all peers

Distributed Hash Table (DHT) - BitTorrent, Kademlia, eMule

16

DHT

2012 © R. Stainov


17

10 15 6 23 8 19

0 1 2 3 4 5

Hash function f(x) = x mod 5Hash function f(x) = x mod 6

2012 © R. Stainov


18

10 15 6 23 8 19

0 1 2 3 4 5

Hash function f(x) = x mod 5Hash function f(x) = x mod 6

2012 © R. Stainov


19

10 15 6 23 8 19

0 1 2 3 4 5

The virtual DHT address space

DHT

2012 © R. Stainov

Peer-to-Peer Systems: DHTIdea: 1. A lookup service similar to a hash table: (key, value) pairs are stored in the DHT,

2. Responsibility for maintaining the mapping from keys to values is distributed among the nodes,

3. Scales to extremely large numbers of nodes

Tasks of DHT are: To find the location of the searched resource (data). Equal distribution of the resources to the nodes. Routing to the node storing the resource from any other node.


Peer-to-Peer Systems: DHT

Each node has a search operation (lookup) Find the value associated with the key

The nodes maintain routing pointers (Finger poiters) to other nodes if the resource is not local.

Key Value

Alexander Berlin

Ivan Moskow

Marina Fulda

Peter Avignon

Rumen Fulda

Stefan Sofia

21


22

The virtual DHT address spaceAdvantages: Load Balancing. Equal distribution of objects and peer in

the DHT address space. Scalability. Adding and removing of peers concerns the

neighbors only . Fast Lookup. The object‘s ID defines the peer ID Availability. No central component (server) exists, which

could fail. Anonymity. In P2P we address not a computer (e.g.

server), but contents. 0 1 2 3 4 5

DHT

2012© R. Stainov


Lookup in P2P Applications

2012© R. Stainov 23


24

The virtual DHT address space

Disadvantages: Geographic or network location is irrelevant (Overlay)

Routing and data transfer over continents (speed) Dealing with disconnected (crashed) nodes

Especially in mobile networks

0 1 2 3 4 5

DHT

2012 © R. Stainov


Consistentent Hashing


The steps for look up in Chord are logarithmic

The size of the routing table is logarithmic

Example:

log2(1000000)≈20

Effective !

Peer-to-Peer Systems: Chord

The nodes build a modulo 2m ring (Chord Ring). The node Ids are created by hashing of IP addreses and the resource IDs by hashing of the recource key. Those IDs build a ID ring modulo 2m.

26

H(a) = 6H(b) = 5H(c) = 0H(d) = 11H(e) = 2

Each node a,b,c,d,e calculates an ID using the hash function H

26

2

11

6

5

01

3

4

789

10

15

14

13

12


The succesor (succ) of an Identificator is the first node clockwise. Example:

27

27

2

11

6

5

01

3

4

789

10

15

14

13

12

succ(12)=14 succ(15)=2 succ(6)=6

2012 © R. Stainov


Each resource calculates an ID using the hash function H. Example: H(“Marina”)=12, H(“Peter”)=14, H(“Rumen”)=4, H(“Stefan”) = 9.

Resource-Ids are maped in the same or in successor (succ) node and are stored in this node.

28

“Marina” succ(12) = 0“Peter” succ(14) = 0“Rumen” succ(4) = 5“Stefan” succ(9) = 11

28

2

11

6

5

01

3

4

789

10

15

14

13

12

2012 © R. Stainov


Each node points to his successor (succ), the so called succ pointer. Each node points to his predecessor (pred), the so called pred pointer.



Lookup of the key “Rumen” from node 2:

2012 © R. Stainov

Calculate H(“Rumen”) = 9 Go to succ pointers until „9“ is

found, i.e. Lookup the nodes 2, 5,6,11 (BINGO) Value „Fulda“ is sent back to node

2 .

30

30

2

11

6

5

01

3

4

789

10

15

14

13

12

Peer-to-Peer Systems: ChordSpeed up of the lookup. Using Succ pointers in succ(n+1), we need in the worse case N steps for N

nodes. We can achieve a speed up, if we use a pointer in succ (n+2M-1). In our example the node 0 should point to the nodes succ (0+1)=2, succ (0+2)=2, succ (0+4)=5, succ (0+8)=11 zeigen.

succ (0+1)=2succ (0+2)=2succ (0+4)=5succ (0+8)=11

The distance is first the half.

Pointer to succ(n+2M-1)

31

31

2

11

6

5

01

3

4

789

10

15

14

13

12

The steps for look up in Chord are logarithmic

The size of the routing table is logarithmic

Example:

log2(1000000)≈20

Effective !

Finger table node n:Finger points to the succesor of n + 2i

finger[i] = successor (n + 2i)


Routing to 15 (Get (15)) goes in considerable less steps (log2(N) steps insteat of N steps in the worst case)

2012 © R. Stainov

32


Prinzip of the lookup – find the next predecessor.

2012 © R. Stainov

33

8

2

6

7

4

3

5

10

9

14

15 0 1

11

13

12

15

Get(15)

8

2

6

7

4

3

5

10

9

14

15 0 1

11

13

12

8

Get(8)


A new peer joins and informs his successor.


1513


2012 © R. Stainov

8

2

6

7

4

3

5

10

9

14

15

11

13

01

12

2

129

8

14

Join

The new peer takes the corresponding resources from his successor.

35


2012 © R. Stainov

8

2

6

7

4

3

5

10

9

14

15

11

13

01

12

2

12

14

9

8

98

36

The new peer takes the corresponding resources from his successor.

MOBILITY SUPPORT


What is the problem ?

1.Internet access over different networks on different cost.

2.The communication with the mobile device could be temporarily (or even for a longer period) interrupted.

3.Heterogeneous mobile devices according to their computing, transmission and storage capacity.

Existing Solutions

Proprietary add-on layer to the existing P2P systems for caching or buffering of data units at the powerful peers

broadcast channels, feeds throw boxes data MULEs Emails per POST

38

0 1 2 3 4 5

DHT

Our Idea

39

DHT

Peer Port

Data Object

Sending Peer Receiving Peer

PUSH

Peer Port represents "ordered message buffer" for a redundant persistent P2P communication.

Difference to the existing approaches: Peer Port is

redundant to the primary P2P channel and is used only in case of interruption

Peer Port is a part of the P2P system and continues receiving data after the interruption

Peer Port

40

DHT

Peer Port

Data Object


PUSH

Normal operation with redundant buffering (PUSH)

Registration with the Peer Group:creates a peer port. defines his size and validity date/time. sets-up a unique sequence number for messages.creates a ticket for accessing it over PULL.

Peer Port

41

Data recovery (PULL) after interruption.

DHT

Peer Port

Data Object


PUSH

Bang

PULL

Peer Group

PULL the missing

messages at

“low cost”messages ID = [peer port ID, message sequence number]

Continue with persistent communication over the Peer Port

42

Peers as relays - Example

Problem when both Alice and Bob are behind “NATs”.

NAT prevents an outside peer from initiating a call to insider peer

Solution: Using Alice’s and Bob’s

SNs, Relay is chosen Each peer initiates

session with relay. Peers can now

communicate through NATs via relay

Use Case A famous orthodontist is traveling

His colleague in the practice needs frequently consultations over video & VoIP, but:Because of entering a tunnel, the WiMAX connection is interrupted shortly. A normal P2P channel will lose information, so that this part has to be repeated. Solution: the missed information will be taken from the peer port and can be played, even couple of minutes later. The connection over WiMAX is replaced by 3G/GPRS. A normal P2P channel will be disconnected, and after reconnection the missing audio and video transmission has to be repeated. Solution: the transmission continues over the peer port.

43

WiMAX

Advantages of Peer Ports

P2P technology. The peer port inherits the P2P advantages like low cost, availability, high bandwidth, scalability and anonymity.

Delay tolerant. The peer ports overlay will be tailored to the context of the delay tolerant networking and will be used for add-on persistent communication with the goal that each message will eventually be delivered to its destination.

Low cost recovery after interruption. The peers belonging to the group are chosen to be “near” the receiver, building on this way a location aware group.

Network independent recovery. Sending peers will push the data into the peer ports; receiving peers will pull data from them, if necessary.

44

PEER PORTS FOR LAYERED P2P STREAMING

Peer Ports Implementation

46

New Problems: •received and processed data should be removed •missing data could be requested from Peer Port

A new Peer Port Sliding Window Protocol is needed.

New Ideas:

Peer Ports for Layered Streaming.

Peer Port Sliding Window

47

DHT

Peer Port

Data Object


PUSH ACK + miss. data

control + req.

Peer Group

During the Normal Operation:•Transient (direct) communication between Sending and Receiving Peers•Sliding Window between the Receiving Peer and the Peer Port.

Peer Port Sliding Window

48

After Interruption:•Persistent (indirect) communication between Sending and Receiving Peers •Sliding window between the Receiving Peer and the Peer Port.

DHT

Peer Port

Data Object


PUSH PUSH data + ack

control + req.

Peer Group

Peer Ports for Layered Streaming

49

Layered coding: •The basic media stream is first encoded into a base layer providing the minimum quality and requiring the minimum bandwidth. •The enhancement layers are encoded separately into separate packets, where each additional layer contributes to the improving the quality of the media played.

L4

L3

L2

L1

L2

L1

More enhancement layers introduce better video qualityBase layer


50

Existing Application in P2P:•Supplier & Receiver Side Schedulers•Scheduling mechanism to request absent blocks, based on 3 buffering windows: free stage, decision stage and remedy stage.

Our Approach:

Parallel transmission over 2 Peer Ports – one for basic layer stream, the another for enhancement layers.


51

During Normal Operation : •The Base Peer Port is used mainly for back up of the packets of the base layer, and therefore for requesting of missed base layer messages. The enhancement layer packets are buffered separately, and deleted first.

DHT

Base Peer Port

Data Object



control + req.

Peer Group

Enhancement Peer Port

control + req.ACK + miss. data

• If the “control + req.” message requests missing base layer and enhancement layer packets, the reply message “ACK + miss. Data” should include them.


52

During Normal Operation : •The Enhancement Peer Port is used mainly for back up of enhancement packets layer, and therefore for requesting of missed enhancement layer messages. The base layer packets are buffered separately, and deleted first.

DHT

Base Peer Port

Data Object



control + req.

Peer Group



• Control messages are used to request missing enhancement packets and to implement the sliding window protocol to the Receiving Peer.


53

Persistent Operation (In case of interruption): •The Base Peer Port will start playing the role of a proxy and will push the packets to the Receiving Peer, using the Sliding Window Protocol.

… But, the Extra Hop Problem …

DHT

Base Peer Port

Data Object


PUSH PUSH. Data + ack

control + req.

Peer Group


control + req.PUSH. Data + ack

Solution: Pushing Base Layer Packets trough the Base Peer Port in parallel to pushing Enhancement Layer Packets trough the Enhancement Peer Port.


54

Persistent Operation (In case of interruption):

The Enhancement Peer Port plays the role of backup for Base Layer Streaming packets.

DHT

Base Peer Port

Data Object



control + req.

Peer Group



Implementation: The request for missing base layer streaming packets will be sent to both Peer Ports, and both will deliver the missing base layer streaming packets. The first message with the base layer packet will be processed, the second will be ignored.

The Peer Port solution for Data Sharing Networks

1. The Peer Port is representative of the mobile Peer, and forwards the data chunks received to the mobile Peer.

2. If several mobile Peers are sharing the same data a swarm of Peer Ports, like BitTorrent, Gnutella, etc.

3. The sliding window protocol:

55

DHT

Peer Port m

mobile Peer m

PUSH. Data + ack

control + req.BitTorrentSwarm

control + req.

mobile Peer n

PUSH. Data + ack

Peer Port n

• Bit vector with missed data chunks for each data sharing session.

• The mobile Peer sends periodically control messages with the last received chunks.

• The missing chunks will be selectively sent.

Peer Port Problems with mobile Peer-to-Peer Multimedia:1. Limitations in the throughput and

Quality of Service (QoS) over the wireless interface. gaps in an audio/video presentation Layered streaming coding, The basic media stream is first encoded into a

base layer providing the minimum quality and therefore requiring the minimum bandwidth.

The enhancement layers are encoded separately into special packets, where each additional layer contributes to the improving the quality of the media played.

56

Peer Port Support for streaming multimedia.

The transient communication channel is used for the base layer and for as many enhancement layer packets as possible.

When the mobile connection deteriorates, the lost packets can be recovered eventually on-time from the Base Peer Port (Base Packets), and/or from Enhancement Port (Enhancement packets).

57

DHT

Base Peer Port

Data Object

Sending Peer mobile Peer


control + req.

Peer Group



Peer Port Support for streaming

multimedia. Interruption Persistent Communication Base Peer Port for storing forwarding mainly base layer

packets. The Enhancement Peer Port for handling mainly

enhancement layer packets.

58

DHT

Base Peer Port

Data Object



control + req.

Peer Group



Peer Port Support for streaming

multimedia. Layered Streaming using a Swarm of Peer

Ports When a live multimedia stream has to be sent to group of

mobile Peersa class listening on a distance education lecture, or a live IPTV

The one-to-many multimedia streaming seems to be a problem in the traditional Client-Server networks because of

Bandwidth bottleneck at the sending node. Bad scaling Pure efficiency

P2P Networks in a swarm are: Self-scaling The content delivery can effectively use the outgoing bandwidth

of all the peers – statistical multiplexing.59

Peer Port Support for streaming multimedia.

Layered Streaming using a Swarm of Peer Ports

The first layer represents the swarm of Peer Ports only.

Peer Ports only on powerful stationary computers. Efficient delivery and scheduling of live video chunks. Statistical multiplexing and therefore efficiency of the video

data sent to a group of peers. The second layer represents the mobile

Peers receiving the same multimedia content.

Do not participate directly in the swarm. Order to the corresponding Peer Port to gather, and to

forward to him the desired multimedia stream. The Base Peer Port forwards mainly the base layer packets. The Enhancement Peer Port mainly the enhancement layer

packets.

60

Problems with Ambient Assisted Living (AAL):

61

SENDING• Alternative comm.

media• Can be interrupted

for a short, or long period

Mobile Mobile Care attendants

DoctorsFamily

membersSending/Receiving

• Alternative comm. media

• Can be interrupted for a short, or long period

Peer Port The Peer Port solution for AAL1. Delay tolerant communication with the mobile

SENDING device. Sending device pushes data via normal transient P2P

communication (enhanced security). Back-up during the normal communication locally (in the

mobile device) and to the Peer Port. Add-on persistent communication and recovery in case of

interruption.

62

DHT

Peer Port

Cache Data Object Mobile


PUSH

ACK + miss. datacontrol + req.

Peer Group

Peer Port

The Peer Port solution for AAL2. Network independent recovery for the mobile RECEIVING

device. Sending Peer pushes the data into the Peer Ports and to the receiver. Mobile receiving peers will pull data from Peer Ports, only if some data are

missing. Disconnection: The Peer Ports are using for persistent communication.

63

DHT

Peer Port

Data Object



control + req.

Peer Group

Peer Port The Peer Port solution for AAL3. Direct mobile-to-mobile communication in

case of emergency Mobile receiving peers will pull emergency data directly

from the Peer Port of the mobile sending device, if the stationary assistance system at home is not accessible.

64

DHT

Peer Port

Data Object

Mobile Sending Peer Mobile Receiving Peer


control + req.

Peer Group

Implementation

We are using Chord to implement experimentally the above ideas. Our approach however, can be applied to an arbitrary P2P system.

We started to experiment with the peer port for two different P2P applications:

(1) Data messages and file transfer

(2) Audio and video transmission.

65

Future Work

The efficiency of P2P is the stat MUX at the application layer.

The future work will include algorithms for performance improvements using a swarm of Peer Ports.

66

p2p and more …

Documents