1

The Design of a RobustPeer-to-Peer System

Rodrigo Rodrigues, Barbara Liskov, Liuba Shrira

Presented by Yi Chen

Some slides are borrowed from the authors’

   

2

Talk Outline

Existing P2P systems Motivation for a robust P2P system The new P2P architecture Conclusion and future work Discussions

   

3

Existing P2P Systems

Started with explosion of file sharing apps Existing P2P systems:

• All nodes have identical responsibilities • All communication is symmetric• Volunteer nodes join and leave system at any time

Advantages:• Harness idle storage and network resources• Automatic load balancing • Self-organization• Desirable properties scale as we add nodes

   

4

Motivation: Support More Applications

Current applications that use P2P storage systems• Assume states are in secure place• Use untrusted P2P storage system to publish data

E.g. content sharing

What if we want to store the state of the application in a P2P system (e.g. support writes)?

E.g. digital library archiving, mail service

   

5

Motivation: Better Fault-Tolerance

Current P2P systems tolerate failstop failures• Faulty nodes stop

Current P2P systems does not handle Byzantine failures• faulty node may behave arbitrary and malicious

e.g. an attacker can join the system with multiple Ids, provide fake information

   

6

Motivation: Efficient Lookups

Current approach: Chord Every node maintains

a routing table about O(logN) entries

Routing algorithm locates data with latency O(logN),

N8

N51

N42

N1

N38

N32N21

N14

LookupK36

K36 N8+1: N14N8+2: N14

N8+8: N21N8+4: N14

N8+16: N32

N8+32: N42

   

7

Talk Outline

Existing P2P systems Motivation for a robust P2P system The new P2P architecture

• Configuration Service• P2P nodes

Conclusion and future work Discussions

   

8

Architecture

Configuration Service (CS): • can run on subset of P2P nodes or be separate from P2P nodes• determines current configuration and propagate the information

P2P nodes (server machines)

ConfigurationService P2P Nodes

   

9

Configuration Service

Four main functions:• Admission control• Node monitoring• Deciding on a new configuration• Propagating configuration information

   

10

Admission Control

Prevents a single user controlling large fraction of the ID space

Hard to do in volunteer-based system Approach:

• Maintain a list of authorities who are permitted to add nodes to the system

• Each new nodes inform CS its ID, IP and public key

   

11

Node Monitoring

How to determine a node should be evicted? Failstop: ping nodes periodically Byzantine: hard!

• Probing: must be indistinguishable from client requests

• Still, faulty nodes may lie in wait• Rely on proactive recovery

   

12

Proactive Recovery

All nodes in the system are recovered proactively and frequently.

Approach:• Each node has a watchdog timer to trigger recovery• During a recovery, a node reboots from the saved

correct state. Then, in order to bring the states up-to-date, it sends status message to replicas, who detect missing messages (if any) and retransmits them.

   

13

Proactive Recovery (cont.)

To ensure correctness, choose recovery frequency, such that:

There are (3f+1) the number of replicas for a data item. For any moment, there are at most f faulty replicas (see algorithm in [2])

Byzantine faulty nodes may only present temporarily in the system

   

14

Assumptions of the model

To apply proactive recovery, P2P nodes are server machines, such that• less failure-prone• dedicated to handle distributed applications. not

constantly joining and leaving the system, and routing states changes infrequently

• still can run symmetric protocols, and distributed across a wide area

   

15

Deciding on a New Configuration

Include newly joined nodes Evict potentially faulty nodes

• Byzantine faulty nodes only present temporarily, therefore do need to detect and remove them.

• Remove potential failstop nodes that are not reachable for certain time period

Produce new configuration periodicallye.g. once every hour

   

16

Propagating Configuration

What to propagate? Disseminate entire config• Assumption: few nodes joining/leaving• Advantage: enable efficient lookup• Overhead:

• Local storage of configuration information

100,000 nodes -> 14.7 MB• Transmit using diffs and compressions

How to propagate? Will be discussed later

   

17

Talk Outline

Existing P2P systems Motivation for a robust P2P system The new P2P architecture

• Configuration Service• P2P nodes

Conclusion and future work Discussions

   

18

Two Layers on P2P Nodes

Lookup layer:

Receive/maintain configuration info

Storage layer:

Store/retrieve/transfer data items• Upon configuration changes, need state transfer• During lookup, ask lookup layer for the replicas that

hold the data of interest, then contact the storage layer at the replicas directly

   

19

Two Types of Data

Immutable, self-verifying data.Use the hash of its content as ID

Mutable data• Extend the data with a monotonically increasing

version number

• Retrieval needs to make sure to be the latest version

   

20

Data Management of Storage Layer

Storage/Retrieval with static configuration• Immutable data• Mutable data

Storage/Retrieval during re-configuration• State transfer• Configuration propagation

   

21

Storage/Retrieval with Static Configuration

Need to ensure that 2f+1 replicas claim to have stored the data

Assume no concurrent writes to the same data items

   

22

Immutable Data

Adversary cannot forge them Not updateable – never out-of-date Write to and receive acknowledgement from at

least 2f + 1 replicas Read from only one replica, if the content hash

matches the ID

   

23

Mutable data

Replay attack Must include version number Write to 2f+1 replicas

• One round-trip if version number is known• Otherwise learn current version number by an extra

round-trip

Read from 2f+1 replicas and choose the one with highest version number to guarantee correctness

   

24

Store / Retrieve Algorithms

Write2f+1 replicas

Read2f+1 replicas

In total n=3f+1 replicas At any moment, at most f faulty replicas Therefore read at least 1 correct replica

   

25

Storage/Retrieval during Re-configuration

When configuration changes, the set of replicas of a data item may change

Need to • State transfer

Bring the up-to-date version of data to P2P nodes that are responsible for them

• Configuration propagation

Propagate new configuration to all nodes over some time

   

26

State Transfer

If a node receives a new configuration, and realizes it is responsible for new data items

Then it pulls the up-to-date data from replicas of last configuration• Immutable data: read one replica if the content hash

matches id• Mutable data: read 2f+1 replicas to ensure

correctness

   

27

Configuration Propagation

Nodes include id of the latest configuration they know in the messages they exchange

If a node detects a new configuration, it requests for a copy of that configuration

   

28

Talk Outline

Existing P2P systems Motivation for a robust P2P system The new P2P architecture Conclusion and future work Discussions

   

29

Conclusions

A hybrid system consists of a set of servers as P2P nodes, and a CS.

P2P nodes are sever machines that dedicated for the storage application

CS are responsible for computing and propagating the current configuration

A Byzantine fault-tolerant system by applying proactive recovery

   

30

Future Work

Implementation Improve algorithms – delete, quotas, … Eliminate rebooting requirement

   

31

Discussions

Compare its underlying assumptions with existing P2P systems

Proposed improvements over existing P2P systems• Support more applications • Better fault-tolerance• Efficient lookups

   

32

Discussion: Assumptions

Compare its underlying assumptions with existing P2P systems• All nodes have identical responsibilities? Yes • All communication is symmetric? Yes• Volunteer nodes join / leave system at any time? No

• Assume participating nodes are dedicated to handle distributed applications, and do not frequently join/leave

• Node join under admission control

Similar as a distributed system?

   

33

Discussion: Support More Applications

Current P2P systems use P2P storage to publish data, only useful for content sharing

This system supports state changes of the data (writes), useful for archiving

Read/write 2f+1 replicas, big overhead?

   

34

Discussion: Better fault-tolerance

Current P2P systems: failstop fault-tolerant

This system: Byzantine fault-tolerant• By applying proactive recovery • Only suitable for server machines

   

35

Discussion: Efficient Lookups

Existing P2P systems:A node is responsible for the routing states of a subset

of the nodes. Routing requires several lookups

This system: Every node stores the whole configuration, enables

direct lookups• Pros: efficient lookups• Cons: big storage; big transfer; many routing state

changes when nodes join/leave

   

36

Questions?

Thank you!