polystyrene: survivable shape for self-organising data
DESCRIPTION
SocioPlug. Polystyrene: Survivable Shape for Self-Organising Data. François Ta ïani Joint work with: Hoel Kervadec (INSA Rennes) Simon Bouget (ENS Rennes ) Anne Marie Kermarrec (ASAP). Focus. Epidemic Topology Construction algorithms Decentralized, fast, scalable - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/1.jpg)
SocioPlug
Polystyrene: Survivable Shape for Self-Organising Data
François Taïani
Joint work with:
Hoel Kervadec (INSA Rennes)Simon Bouget (ENS Rennes)Anne Marie Kermarrec (ASAP)
![Page 2: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/2.jpg)
F. Taiani 2
Focus Epidemic Topology Construction algorithms
Decentralized, fast, scalable Fundamental building block to higher-level services
(DHT, Multicast, Pub-Sub, Recommendations)
Taken from [JMB09]
![Page 3: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/3.jpg)
Problem: Catastrophic Failure
The topology heals But the overall shape is lost
How to recreate whole shapefrom surviving nodes?
?
![Page 4: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/4.jpg)
F. Taiani 4
Outline
Background:Decentralized Topology Construction
Polystyrene:Architecture and Protocol
Evaluation Outlook
![Page 5: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/5.jpg)
Decentralized Topology Const. Each node : some data Find k “closest” nodes in system
Decentralized approach, asynchronous rounds
random sampling (RPS)
topology layer gossip-based topology construction (e.g. T-Man)
topology linkrandom link
AB
C
D
E
AB
C
D
E
node node position
![Page 6: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/6.jpg)
Main idea: greedy neighbourhood optimization
Decentralized Topology Const.
exchange ofneighbors lists
neighborhoodoptimization
1 2
F C
E D
AB
C
D
F
E
AB
C
D
F
E
![Page 7: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/7.jpg)
Polystyrene
Topology Construction(T-Man, Vicinity, Gossple)
Neighbours
Peer Sampling Service(RPS, Cyclon, SCAMP)
Node position
Polystyrene’s Architecture
![Page 8: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/8.jpg)
Polystyrene Protocolrecovery
2
FD
Topology Construction
migration
4
projection3
Node position
ghosts
backup (outgoing)
1
backup (incoming)
1’
Neighbours3’
guests
![Page 9: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/9.jpg)
F. Taiani 9
The Migration Process
q.post
p.post
p.guestst
q.guestst
a
b
c
df
e
p q
![Page 10: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/10.jpg)
F. Taiani 10
The Migration Process
Bi-clustering of guest points Heuristics : diameter
a
b
c
df
e
p q
![Page 11: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/11.jpg)
F. Taiani 11
The Migration Process
Bi-clustering of guest points Heuristics : diameter
a
b
c
df
e
points closer to b
points closer to d
p q
![Page 12: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/12.jpg)
F. Taiani 12
The Migration Process
Bi-clustering of guest points Heuristics : diameter + minimum move
q.post+1
p.post+1
q.guestst+1
a
b
c
df
e p.guestst+1
points closer to b
points closer to d
p q
![Page 13: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/13.jpg)
F. Taiani 13
Evaluation Shape : 2D 40x80 logical torus Round 20 : 50% correlated node crashes
Polystyrene recreates shapewith surviving nodes
After failure (r=20) (r=22) (r=28)
![Page 14: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/14.jpg)
Eval: Quality of Neigborhoods
Polystyrene maintains good neighborhoods
![Page 15: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/15.jpg)
Eval: Quality of Shape
And the torus gets restored!
![Page 16: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/16.jpg)
F. Taiani 16
Eval: Scalability Time (rounds) until homeogeneity less than
Logarithmic convergence!
![Page 17: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/17.jpg)
Outlook An example of advanced topology construction
Replicated, highly robust, self-organising
Potential extension to load-balancing Good for plug heterogeneity
Concrete application on top of polystyrene DHT, recommendation, queries, search
Larger picture Self-organising data primitives for plug infrastructures
![Page 18: Polystyrene: Survivable Shape for Self-Organising Data](https://reader036.vdocuments.site/reader036/viewer/2022062806/56816187550346895dd12064/html5/thumbnails/18.jpg)
18
References [KMG03] Kermarrec A.-M., Massoulie L., Ganesh, A.J., Reliable Probabilistic
Communication in Large-Scale Information Dissemination Systems, IEEE Transactions on Parallel and Distributed Systems, March 2003, (14:3)
[JGK04] Jelasity, M., Guerraoui, R., Kermarrec, A.-M., and van Steen, M. (2004). The peer sampling service: experimental evaluation of unstructured gossip-based implementations. Middleware ’04, pages 79–98, New York, NY, USA. Springer- Verlag New York, Inc.
[VS05] Voulgaris, S. & Steen, M. V. Epidemic-style Management of Semantic Overlays for Content-Based Searching. Proc. of the 11th Int. Euro-Par Conf. on Parallel Processing (Euro-Par'05), Springer, 2005, 1143-1152
[Jelasity, Alberto Montresor, and Ozalp Babaoglu. 2009. T-Man: Gossip-based fast overlay topology construction. Comput. Netw. 53, 13 (August 2009), 2321-2339.
[BFG+10] Bertier, M.; Frey, D.; Guerraoui, R.; Kermarrec, A.-M. & Leroy, V.The GOSSPLE anonymous social network. Proc. of the ACM/IFIP/USENIX 11th Int. Conf. on Middleware, 2010, 191-211
[TLB14] Taiani, F., Lin, S. and Blair, G. S. (2014) GossipKit: A Unified Component Framework for Gossip. IEEE TSE, Preprint, doi: 10.1109/TSE.2013.50