scalable peer-to-peer network for highly synchronized simulations shun-yun hu institute of physics,...

42
Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Upload: meredith-nash

Post on 12-Jan-2016

217 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Scalable Peer-to-peer Network for Highly Synchronized Simulations

Shun-Yun Hu

Institute of Physics, Academia Sinica

2005/03/11

Page 2: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Outline

Introduction Voronoi-based Overlay Network (VON) Simulation Results Conclusion

Page 3: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

A Look at Simulations

Simulations are important tools in scientific research

Larger scale and higher resolution (more accurate and detailed simulations) are constantly sought

However, computational resource can be limited

Page 4: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

An Untapped Potential

300 Million PCs on the Internet (2000 est.)

Up to 80% to 90% of CPU is wasted

Large supply of computing resource, growing rapidly

Page 5: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

An Example: SETI@Home

Search for Extraterrestrial Intelligence (SETI) UC Berkeley Project launched in May 1999

PC User downloads a screen saver Calculations are done using idle CPU time

2005/03 statistics (in 6 years) 5.3 M world-wide participants 2.2 M years of single-processor CPU 54 teraflop machine (current top 3: 70.72, 51.87, 35.86)

Page 6: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Simulation: Folding@Home

Stanford Project launched in Sept. 2000 Seeks to determine protein’s 3D structure

Screensaver that downloads “work units” 2002 Statistics:

30,000 volunteers 1 M days of single-processor CPU

Published 23 papers in: Science, Nature, Nature Structural Biology, PNAS, JMB, etc.

Page 7: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

The Grand Question

Can we build the ultimate simulator for large-scale simulation utilizing millions of computers world-wide?

Potential applications: Nuclear reaction Star clusters Atomic-scale modeling in material science Weather, earthquakes Biology (protein, ecosystem, brain, ...)

Page 8: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Current Limitations

Current methodology Centralized server + many clients Client requests “work unit” to process Communication is minimized Clients do not communicate

Issues: Only suitable for “embarrassingly parallel” simulations Sophisticated server-side algorithm and management required

An alternative: peer-to-peer (P2P) computing

Page 9: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

What is Peer-to-Peer (P2P)?

[Stoica et al. 2003] Distributed systems without any centralized control

or hierarchical organization Runs software with equivalent functionality

Examples File-sharing: Napster, Gnutella, eDonkey VoIP: Skype DHT: Chord, CAN, Pastry

Page 10: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Peer-to-Peer Overlay

A P2P overlay network source: [Keller & Simon 2003]

Page 11: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Promise & Challenge of P2P

Promises Growing resource, decentralized

Scalable Commodity hardware Affordable

Challenges Topology maintenance dynamic join/leave Efficient content retrieval no global knowledge

Page 12: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

A Simulation Scenario

How can we utilize P2P for simulation-purpose?Answer: depends on what you want to simulate

We observe that many simulations… are spatially-oriented (i.e. based on coordinate systems) run in discrete time-steps require synchronization at each time-step exhibit localized interaction (i.e. short-range interaction)

example: molecular dynamics (MD) simulation

Page 13: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Scenario Defined for P2P

Many simulated entities (nodes) on a 2D plane ( > 1,000) Positions (coordinates) may change at each time-step How to synchronize positions with those in Area of Interest

(AOI)?

Area of Interest

Page 14: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

P2P Design Goals

Observation: the contents are information from AOI neighbors P2P content discovery is a neighbor discovery problem

Solve the Neighbor Discovery Problem in a fully-distributed, message-efficient manner.

Specific goals: Scalable Limit & minimize message traffics Fast Direct connection with AOI neighbors

Page 15: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Outline

Introduction Voronoi-based Overlay Network (VON) Simulation Results Conclusion

Page 16: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Voronoi Diagram

2D Plane partitioned into regions by sites, each region contains all the points closest to its site

Can be used to find k-nearest neighbor easily

Neighbors

Site

Region

Page 17: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Design Concepts

Identify enclosing and boundary neighbors Each node constructs a Voronoi of all AOI neighbors Enclosing neighbors are minimally maintained Mutual collaboration in neighbor discovery

Circle Area of Interest (AOI)

White self

Yellow enclosing neighbor (E.N.)

L. Blue boundary neighbor (B.N.)

Pink E.N. & B.N.

Green AOI neighbor

D. Blue unknown neighbor

Use Voronoi to solve the neighbor discovery problem

Page 18: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Procedure (JOIN)

1) Joining node sends coordinates to any existing node

Join request is forwarded to acceptor

2) Acceptor sends back its own neighbor list

joining node connects with other nodes on the list

Acceptor’s region

Joining node

Page 19: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Procedure (MOVE)

1) Positions sent to all neighbors, mark messages to B.N.

B.N. checks for overlaps between mover’s AOI and its E.N.

2) Connect to new nodes upon notification by B.N.

Disconnect any non-overlapped neighbor

Boundary neighbors

New neighbors

Non-overlapped neighbors

Page 20: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Demonstration

Simulation video General movements (30 nodes, 800x600 world) Local vs. global view

Page 21: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11
Page 22: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Outline

Introduction Voronoi-based Overlay Network (VON) Simulation Results Conclusion

Page 23: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Simulation Method

Condition World-size: 1000x1000 AOI: 150 Trials: 10 ~ 250 nodes Time-steps: 1000

Behavior model Random movement: random direction Constant velocity: 5 units/step Movement duration: random (1-25 steps)

Page 24: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Consistency Metrics

Topology Consistency [Kawahara, 2004]

Number of observed AOI neighbors

Number of actual AOI neighbors

Drift Distance [Diot, 1999]Distance between observed position and actual position

(average over all nodes)

Page 25: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Topology Consistency

Topology Consistency Measurements

90

9192

9394

95

9697

9899

100

0 50 100 150 200 250

Number of Nodes

To

po

log

y C

on

sis

ten

cy

(%

)

Page 26: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Drift Distance

0

10

20

30

40

50

60

70

80

90

100

0 50 100 150 200 250

Number of Nodes

Dri

ft D

ista

nc

e

average

maximum

Page 27: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Scalability (1)

Transmission Size Per Node Per Second

0.0

1.0

2.0

3.0

4.0

5.0

6.0

0 25 50 75 100 125 150 175 200 225 250

Number of Nodes

Siz

e (

kb

)

send (max)

send (avg)

recv (max)

recv (avg)

Page 28: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Scalability (2)

Average Neighbor Size Measurements

0

2

4

6

8

10

12

14

16

18

0 50 100 150 200 250

Number of Nodes

Nei

gh

bo

r S

ize

connected

AOI

Page 29: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Scalability (3)

Comparison of Voronoi-based P2P and Client-Server

0

20

40

60

80

100

120

140

160

180

0 25 50 75 100 125 150 175 200 225 250

Number of Nodes

Siz

e (

kb

)

send (avg)

recv (avg)

CS-send (avg)

CS-recv (avg)

Page 30: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Outline

Introduction Voronoi-based Overlay Network (VON) Simulation Results Conclusion

Page 31: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Summary

Idle CPU and networks are untapped potential resources for large-scale simulation

Current approaches do not support simulations that require frequent synchronization / updates

A promising solution: Voronoi-based P2P Overlay Leverage knowledge of each peer to maintain topology Properties: scalable, efficient, fully-distributed Enable simulations with frequent localized synchronization

Page 32: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Future Works

3D Voronoi

Heterogeneous node capacities

Node failures

Application to actual research problems

Page 33: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Acknowledgements Dr. Jui-Fa Chen (陳瑞發老師 ) Dr. Wei-Chuan Lin (林偉川老師 ) Members of the Alpha Lab, TKU CS

Guan-Ming Liao (廖冠名 ) Dr. Chin-Kun Hu (胡進錕老師 ) LSCP, Institute of Physics, Academia Sinica

Joaquin Keller (France Telecomm R&D, Solipsis) Bart Whitebook(butterfly.net) Jon Watte (there.com)

Dr. Wen-Bing Horng (洪文斌老師 ) Dr. Jiung-yao Huang (黃俊堯老師 )

Page 34: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Protein Folding Problem

Find native state (lowest free energy) 3D structure given a 1D sequence of amino acids

Timescale limitation of classical MD methods Secondary structure folds in 0.1 ~ 10 s Small protein folds in tens of s Current record: 1s (villin headpiece) full-atomic simulation of 1 ns takes one CPU day 100 ~ 10,000 gap (it might take decades)

Page 35: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Folding@Home Parallelization Dynamics of complex

system involves crossing of free energy barriers

Most time is spent in free energy minimum “waiting”

Possible to simulate using trajectories much shorter than folding time

“ensemble dynamics” (same coords, different velocities)

Page 36: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Simulation Specifics

free energy barrier crossing is identified by spike in energy variance

Fs peptide (5-residue) (fold time 10ns and 160 +/-10ns)

Artificial mini-protein BBA5 (23-residue) Tens of thousands of 5-20ns trajectories (total of 700us) Mean folding time is 10s, 10 out of 10,000 folds in 10ns

Page 37: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Procedure (LEAVE)

1) Simply disconnect

2) Others then update their Voronoi

new B.N. is discovered via existing B.N.

Leaving node (also a B.N.)

New boundary neighbor

Page 38: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11
Page 39: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Scalability (1)

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

0 25 50 75 100 125 150 175 200 225 250

Number of Nodes

Siz

e (

kb

)

send (basi c)

recv (basi c)

send (dAOI)

recv (dAOI)

Average transmission size per node per second

Page 40: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Scalability (2)Maximum transmission size per second among all nodes

0.0

1.0

2.0

3.0

4.0

5.0

6.0

0 25 50 75 100 125 150 175 200 225 250

Number of Nodes

Siz

e (

kb

)

send (basi c)

recv (basi c)

send (dAOI)

recv (dAOI)

Page 41: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Scalability (3)Average neighbor size for basic and dynamic AOI models

0

2

4

6

8

10

12

14

16

18

0 50 100 150 200 250

Number of Nodes

Neig

hb

or

Siz

e

connected (basic)

connected (dAOI)

AOI (basic)

AOI (dAOI)

Page 42: Scalable Peer-to-peer Network for Highly Synchronized Simulations Shun-Yun Hu Institute of Physics, Academia Sinica 2005/03/11

Problems of Voronoi Approach

Message traffic Circular round-up of nodes Redundant message sending

(inherent to fully-distributed design)

Incomplete neighbor discovery Can happen with inconsistent / incorrect neighbor list Fast moving node