rambo: reconfigurable atomic memory for dynamic networks
DESCRIPTION
RAMBO: Reconfigurable Atomic Memory for Dynamic Networks. Seth GilbertNancy LynchAlexander Shvartsman Presenter: Anastasia Braginsky (December 2013) Slides partially borrowed from Seth Gilbert (DSN ’03) and Edward Bortnikov (talk). RAMBO name. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/1.jpg)
RAMBO: Reconfigurable Atomic Memory for Dynamic
NetworksSeth Gilbert Nancy Lynch Alexander Shvartsman
Presenter: Anastasia Braginsky (December 2013)
Slides partially borrowed from Seth Gilbert (DSN ’03) and Edward Bortnikov (talk)
![Page 2: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/2.jpg)
RAMBO name» Reconfigurable Atomic Memory for Basic Objects
![Page 3: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/3.jpg)
Outline» Introduction» Background
˃ Static Quorum Systems˃ Consensus
» RAMBO high level overview» Preliminaries» The RAMBO algorithm» The reconfiguration service» Conclusions
![Page 4: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/4.jpg)
Distributed Shared MemoryReadWrite(7)
Write(0)
![Page 5: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/5.jpg)
Atomic Consistency (linearizability)
» Definition: Each operation appears to occur at some point between its invocation and response
» Sufficient condition: For each object x, all the read and write operations for x can be partially ordered by , so that:
1. No operation has infinitely many other operations ordered before it
2. is consistent with the order of invocations and responses:
there are no operations such that 1 completes before 2 starts, yet 21
3. All write operations are ordered with respect to each other and with respect to all the reads.
4. Every read returns the value of the last write preceding it in
![Page 6: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/6.jpg)
op A completes before op B begins, then B returns the results of A
Read 7
Write(7)
ReadWrite(7)
Write(0)
![Page 7: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/7.jpg)
Suggestions?» Central server?
˃ Performance bottleneck
˃ Single point of failure
» So multiple servers need to replicate the content
˃ And do not stop the world if some reconfiguration is needed
» But now how to find the latest value of replicated object?
![Page 8: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/8.jpg)
Distributed Networked System» All-to-all connectivity, but messages can be lost,
delayed, or re-ordered
» No global clock or synchronization mechanism - asynchrony
» Nodes can fail
» A distributed networked system can be static (fixed set of participating nodes) or dynamic
![Page 9: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/9.jpg)
And if everything fails?» Memory access operations are guaranteed to
terminate under certain assumptions
» Static: ˃ The majority of replicas need to be active
˃ Network delays are bounded
» Dynamic:˃ Dynamically changing subset of replicas need to be active during certain periods
» Otherwise… Sorry… ˃ Operations may not terminate
![Page 10: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/10.jpg)
10
Dependable Systems and Networks 2003
Quorums
Write(7)Read
![Page 11: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/11.jpg)
11
Dependable Systems and Networks 2003
Dynamic Atomic Memory
![Page 12: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/12.jpg)
Outline» Introduction» Background
˃ Static Quorum Systems˃ Consensus
» RAMBO high level overview» Preliminaries» The RAMBO algorithm» The reconfiguration service» Conclusions
![Page 13: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/13.jpg)
Static Quorum Systems» Upfal and Wigderson (85)
˃ First general scheme for emulating shared-memory in the message-passing system
˃ majority sets of readers and writers
…» Attiya, Bar-Noy and Dolev (90/95)
˃ Dijkstra Award in 2011˃ Including extensions to the original algorithm [N. Lynch and A.
Shvartsman. Robust emulation of shared memory using dynamic quorum-acknowledged broadcast. 1997]
![Page 14: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/14.jpg)
A(ttiya) B(ar-Noy) D(olev)
» Algorithm uses replication to achieve fault-tolerance and availability
» n nodes
» The system tolerates at most n/2-1 crashes
![Page 15: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/15.jpg)
ABD for a single register» Each node i maintains the local value of the register
» valuei and tagi = <seq, pid>
» Tags are compared lexicographically
» Each new write assigns a unique tag (pid to break ties)
» Read and write operations have two phases
˃ Query replicas for information
˃ Propagate information to replicas
» Send to everyone, majority should response
![Page 16: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/16.jpg)
Read: Phase IValue = 5Tag = <4,j>Node i
Value = 5Tag = <4,j>Node j
Value = 3Tag = <5,k>Node k
Value = 6Tag = <5,l>Node l
Query
Query
Query
![Page 17: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/17.jpg)
Read: Phase IValue = 5Tag = <4,j>Node i
Value = 5Tag = <4,j>Node j
Value = 3Tag = <5,k>Node k
Value = 6Tag = <5,l>Node l
Response3< ,5,k>
Response 6, <5,l>
![Page 18: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/18.jpg)
Read: Phase IIValue = 6Tag = <5,l>Node i
Value = 5Tag = <4,j>Node j
Value = 3Tag = <5,k>Node k
Value = 6Tag = <5,l>Node l
Propagate 6, <5,l>
Propagate 6, <5,l>
Propagate 6, <5,l>
![Page 19: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/19.jpg)
Read: Phase II
Value = 5Tag = <4,j>Node j
Value = 6Tag = <5,l>Node k
Value = 6Tag = <5,l>Node i
Value = 6Tag = <5,l>Node l
Acknow-legment
Acknow-legment
![Page 20: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/20.jpg)
Consistency» Two majorities have non-empty intersection
» There is at least one node participating in Propagation phase of previous operation and in Query phase of this one
» All writes ordered by their tags
![Page 21: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/21.jpg)
Too long waiting for the majority?
» Use quorum systems
» Quorum is a subset of nodes
» Any two quorums intersect
» The size of the set can be much less than the majority
» The majority-based implementations tolerate crashes of any minority
» The quorum-based implementations require that the nodes in at least one quorum do not crash
![Page 22: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/22.jpg)
Consensus» Set of processes need to agree an a value
» Nodes propose several values for consideration
» Any solution must satisfy:
˃ Agreement: no two processes decide on different values
˃ Validity: the value decided was proposed by some node
˃ Termination: all correct processes reach a decision
» Consensus termination can not be guaranteed in the presence of even a single process crash
» Paxos is an implementation of a consensus
![Page 23: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/23.jpg)
Outline» Introduction» Background
˃ Static Quorum Systems˃ Consensus
» RAMBO high level overview» Preliminaries» The RAMBO algorithm» The reconfiguration service
![Page 24: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/24.jpg)
RAMBO multi-reader, multi-writer» Short term: Quorum-based Replication – to provide
fault tolerance˃ Read- and write- quorums collected into configurations˃ Any quorum-configuration can be installed in any time
» Long term: Reconfiguration – to cope with changing participants
» Participants can join and fail
![Page 25: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/25.jpg)
26
Tues
day,
June
24
Rambo» Decouple read/write ops and reconfiguration
˃ fast read/write ops, even if recon slow
» A stable state (no reconfigurations) is similar to the static two-phase ABD, but ˃ Extended for multi-writer registers˃ Generalized to use quorum systems
» New participants can join the service by contacting at least one existing participant
![Page 26: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/26.jpg)
Quorums Reconfigurations
» Performed concurrently with any ongoing reads and writes
» Multiple reconfigurations can be in progress concurrently
» Reconfiguration involves˃ Introduction of a new configuration˃ Garbage collection of obsolete configuration(s)
![Page 27: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/27.jpg)
Dependable Systems and Networks 2003
28
Tues
day,
June
24
Rambo stabilization» frequent reconfiguration?
» clocks out of synch?
•messages lost?
•messages delayed?
Network stabilizesRambo stabilizes
![Page 28: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/28.jpg)
Three Sub-Protocols» Joiner
˃ Joiner is notified by a device that the device wants to join˃ The device provides the initial world view (set of devices that this
device thinks has already joined)˃ Joiner contacts this world and retrieves the information necessary for
the new device to participate
» Reader-Writer: Executing read-write operations and old configurations garbage-collection
» Recon: Producing new configurations
![Page 29: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/29.jpg)
Configuration map» Each participant maintain a configuration map –
cmap – to store the sequence of configurations
» For node i, cmapi(k) is ˃ the configuration number k if configuration is active˃ or a notification that this configuration doesn’t yet exist ˃ or a notification that this configuration was already garbage collected
» This sequence evolves as new configurations are introduced by Recon and as all configurations are garbage collected
![Page 30: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/30.jpg)
CMAP Evolution c0
c0 c1
c0 c1 c2
± c1 c2
± ± c2
. . .
. . .
. . .
. . .
. . .
± ± ± c3 . . .. . .
![Page 31: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/31.jpg)
Reader-Writer» Each read or write executes in the context of one or more active
configurations (must use all active configurations)
» Reads and writes proceed concurrently with ongoing reconfigurations
» Two phases» Query phase – information is retrieved from one (or more) read-quorums of all
active configurations» Propagate phase – information is updated in one (or more) write-quorums of all
active configurations
» Garbage-Collection (GC) – removing old configurations » Notifying about old configuration(s)» Propagating information from old configuration to the next
![Page 32: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/32.jpg)
RAMBO Assumptions » Assumptions regarding RAMBO behavior:
˃ Regularly sends gossip messages to the participants
˃ The initial world views overlap sufficiently such that every node that has joined the system is aware about every other node soon enough
˃ Every configuration remains viable until sufficiently long after the next new configuration is installed
˃ Reconfigurations are not initiated too frequent
![Page 33: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/33.jpg)
Outline» Introduction» Background
˃ Static Quorum Systems˃ Consensus
» RAMBO high level overview» Preliminaries» The RAMBO algorithm» The reconfiguration service
![Page 34: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/34.jpg)
The system» Set of devices
communicating via all-to-all asynchronous message-passing network
» I : totally ordered set of device identifiers
2
1
5
7
6
4
3
A node or a participant
![Page 35: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/35.jpg)
The system» Set of devices communicating
via all-to-all asynchronous message-passing network
» I : totally ordered set of device identifiers
» Nodes may fail by stopping (all components) without worning
2
1
5
7
6
4
3
JoinerRead-Write
Recon
JoinerRead-Write
Recon
JoinerRead-Write
Recon
JoinerRead-Write
Recon
JoinerRead-Write
Recon
JoinerRead-Write
Recon
JoinerRead-Write
Recon
![Page 36: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/36.jpg)
Shared Memory Read/Write Objects
» X : set of object identifiers
» For each object xX, Vx is the set of values that x may take
on
» (v0)x – the initial value of object x
» (i0)x – the initial creator of object x, the node that is initially
responsible for object x (this responsibility can be delegated)
» T = N x I : set of tags, used to order the values written to the
system
![Page 37: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/37.jpg)
Configurations» C : set of configuration identifiers
» Each identifier cC is assosiated with unique configuration consisting of:˃ members(c) – a finite subset of I˃ read-quorums(c) – a set of finite subsets of members(c)˃ write-quorums(c) – a set of finite subsets of members(c)
» For every cC, for every Rread-quorums(c), and for every Wwrite-quorums(c) : RW≠
![Page 38: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/38.jpg)
RAMBO APIDomains» I = set of Nodes » V = set of Values» C = set of Configurations
Input (Request) Join(J) // J – initial world view
Read
Write(v)
Recon (c, c’) // reconfiguration request
Fail
Output (Response) Join-ack
Read-ack(v)
Write-ack
Recon-ack // request has been proceeded
Report (c) // new configuration
Inputs and Outputs are all asynchronous
per node iI and object xX
![Page 39: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/39.jpg)
Requests’ Well-Formedness» No requests after fail
» Each client issues at most one join request and waits for
acknowledgement before any further requests
» Before issuing a new read/write/recon wait for previous
acknowledgment
» Each client issues at most one recon(*,c) request (configuration
identifiers are unique)
» Client can request reconfiguration from c to c’ only if c was
installed and all members of c’ have already joined
![Page 40: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/40.jpg)
Responses’ Well-Formedness» No responses after fail
» Responses comes only upon requests
![Page 41: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/41.jpg)
Reconfiguration service APIDomains» I = set of Nodes » V = set of Values» C = set of Configurations
Input (Request) Join
Recon(c,c’)
Request-config (k)// the client has learned of every configuration preceding k
Fail
Output (Response) Join-ack
Recon-ack
New-config(c,k)// the kth configuration has been agreed upon
Report(c)
Inputs and Outputs are all asynchronous
per node iI and object xX
![Page 42: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/42.jpg)
Recon Service Specification» Recon˃ Chooses configurations˃ Tells members of the previous and new configuration.˃ Informs Reader-Writer components (new-config).
» Behavior (assuming well-formedness):˃ Agreement: Two configs never assigned to same k.˃ Validity: Any announced new-config was previously requested
by someone.˃ No duplication: No configuration is assigned to more than one
k.
![Page 43: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/43.jpg)
Outline» Introduction» Background
˃ Static Quorum Systems˃ Consensus
» RAMBO high level overview» Preliminaries» The RAMBO algorithm» The reconfiguration service
![Page 44: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/44.jpg)
Suppress explicit mention of x
» The shared memory is described as the composition of a separate implementation for each object xX
» V, v0, c0, and i0 as shorthand for
» Vx, (v0)x, (c0)x, and (i0)x
![Page 45: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/45.jpg)
Recon
Read-Write
User of node i
Joiner
Recon
Read-Write
User of node j
Joiner
![Page 46: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/46.jpg)
Joiner automata state
» status {idle, joining, active, failed}, initially idle
» others-status, a mapping from Recon and Reader-Writer to {idle, joining, active}, initially everywhere idle
» initial-world (iw) I, initially
![Page 47: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/47.jpg)
Join(J)
Joiner automataReco
n
RWUser
of node i
Joiner status=idle
iw=
Recon
Read-Write
User of node j
Joiner
![Page 48: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/48.jpg)
Join(J)
Joiner automata
Recon
Read-Write
User of node j
Joiner
join
Recon
RWUser
of node i
Joiner status = joining
iw=J
Hope at least one will
answer…
![Page 49: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/49.jpg)
Join(J)
Joiner automata
Recon
Read-Write
User of node j
Joiner
join
join
Recon
RWUser
of node i
Joiner status = joining
iw=J
join
![Page 50: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/50.jpg)
Recon
RWUser
of node i
Joiner status = joining
iw=J
Joiner automata
Recon
Read-Write
User of node j
Joiner
![Page 51: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/51.jpg)
Joiner automata
Recon
Read-Write
User of node j
Joiner
Join-ack
Recon
RWUser
of node i
Joiner status = active
iw=J
![Page 52: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/52.jpg)
R/W Automata State» status {idle, joining, active, failed}, initially idle» world I, initially » value V, initially v0 + tag T, initially (0,i0)
» cmap - what configurations are currently active according to this node understanding
» op – record that keeps track of the status of a current locally initiated read/write operation˃ op includes acc the set of clients that have sent responses
» pnum-local – counts phases of locally-initiated operations» pnum-vector[] – records latest known phase numbers for all
locations» gc - record that keeps track of the status of a current locally-
initiated GC operation
![Page 53: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/53.jpg)
“Recent” messages» A message from i to j is deemed “recent” by j if i
knows about j’s current phase
i jpnum-vector[j]i = pnum-localj
» i has received a message from j that was sent after j began the new phase and was received prior to i sending message to j
![Page 54: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/54.jpg)
Reader-Writer automata: joinReco
n
Read-Write
world=world{k}
User of
node i
Joiner
Recon
Read-Write
User of node j
Joiner
Join(k)
if (status == idle) {if (k==i) status=active;else status=joining;
}
once status is
active
Join-ack
![Page 55: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/55.jpg)
Reader-Writer of node i» In each phase Reader-Writer contacts a set of
quorums ˃ At least one for each active configuration
» To obtain recent value, tag and cmap
» This by sending and receiving messages in the background
![Page 56: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/56.jpg)
Send periodically from i to j
» If status is active than send to every j in your world
world, value, tag, cmap, pnum-local, pnem-vector[j]
![Page 57: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/57.jpg)
Receive from j, if i isn’t idle or failed:status = active; // in case it was joiningworld=world world-received;if (tag-received > tag) (value,tag)=(value-received, tag-received);
// learn about new and old configurationscmap = update(cmap-received);pnum-vector[j] = max(pnum-vector[j],pnum-received);
if (ongoing-operation and message-is-recent)if (op.configuration is consistent)op.acc = op.acc {j};else restart operation;
else if (ongoing-gc and message-is-recent)gc.acc = gc.acc {j};
![Page 58: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/58.jpg)
Read/Write Query PhaseReco
n
RW:Wait for at least one
quorum from each active c to update you
User of
node i
Joiner
Recon
Read-Write
User of node j
Joiner
ReadWrite(v)
if (status ≠ idle) and (status ≠ fail) {pnum-local++;op.type = read/write;op.phase = query;op.acc = ;
}
![Page 59: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/59.jpg)
Read/Write Propagation PhaseReco
n
R: propagate value to be
returnedW: update
value and tag and propagate
User of
node i
Joiner
Recon
Read-Write
User of node j
Joiner
Enoughupdates
if (status ≠ idle) and (status ≠ fail) {pnum-local++;op.type = read/write;op.phase = propagate;op.acc = ;
}
once at leastone
write-quorumfrom eachactive c
updates you
Read/write-ack
![Page 60: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/60.jpg)
Garbage-collection (GC)» Old configurations identifiers are garbage-collected at
each node i
˃ If node i hears that another node has already garbage-collected some configuration
˃ If node i itself initiated garbage-collection of configurations with identifiers l<k if it has learned about configuration k
» During GC information is propagated from one configuration to the next
» GC proceeds concurrently with reads and writes on the same node
![Page 61: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/61.jpg)
GC in Two Phases» Query phase
˃ Communicate with both a read-quorum and a write-quorum of every active configuration l<k
˃ Accomplish two tasks:+ Ensure that all nodes with “old” configurations learn about
existence of configuration k and learn that that all configurations smaller than k are garbage-collected
+ Collect recent tag and value from old configurations. By the end of the Query Phase we have received a value as recent as any written prior to the GC beginning
» Propagation phase˃ Propagate value and tag to a write-quorum of configuration k
![Page 62: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/62.jpg)
GC termination» No “fixed point” test
» Do not evolve with newer learned configurations
» GC has fixed amount of work
» GC can terminate also if discover that someone
else has already garbage-collected all the
configurations smaller than k
![Page 63: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/63.jpg)
Proof Sketch» ≤ ordering of tags between sequential GC operations
˃ ∩ between the R-quorum of CMAP[k] and W-quorum of CMAP[k+1]
» Ordering between sequential GC and R/W˃ ≤ ordering of tags between the GC and READ operations˃ < ordering of tags between the GC and WRITE operations
» Ordering between sequential R and W˃ ≤ ordering between */R˃ < ordering between */W˃ Either there is a common configuration C
+ Tag conveyed through the quorum ∩ property ˃ … or the tag info is conveyed through the GC of some
configuration in between
![Page 64: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/64.jpg)
Outline» Introduction» Background
˃ Static Quorum Systems˃ Consensus
» RAMBO high level overview» Preliminaries» The RAMBO algorithm
» The reconfiguration service
![Page 65: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/65.jpg)
Sequence of configurations» Recall, for node i, cmapi(k) is the configuration
number k» Recon always emits a unique new configuration k to
be stored at cmapi(k)» Any node i that is member of its latest known
configuration c=cmapi(k) can propose a new configuration at any time
» Different proposals are reconciled by executing consensus among the members of c (e.g. using a version of Paxos)
![Page 66: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/66.jpg)
Although consensus may be slow
» In fact, in some situations, it may not even terminate…
» Reconfiguration doesn’t delay reads and writes˃ Provided at least one quorum set is alive for
each active configuration
![Page 67: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/67.jpg)
The reconfiguration service» Built using a collection of global consensus services Cons(k,c)
˃ One for each k>0 and for each cC
» Goal – to reach agrement among members of configuration c
» API for Cons(k,c) for fixed k and c˃ Input
+ init(c’)k,c,i c, c’C, imembers(c) at most one event per k,c,i+ faili cC, imembers(c)
˃ Output+ decide(c’)k,c,i c,c’C, imembers(c)
![Page 68: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/68.jpg)
Recon Automata State» status {idle, active, failed}, initially idle» rec-status {idle} (active x N+), initially idle
˃ Whether a reconfiguration request has been submitted, and if so, with respect to which configuration index
» rec-cmap - what configurations are assigned to each index k
» cons-data (N+(C x C)) : initially everywhere
![Page 69: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/69.jpg)
Recon:If active, then
among all configuration indexes in rec-cmap find the
next one
Read-Write
User of node i
Joiner
Recon
Read-Write
User of node j
Joiner
Recon(c, c’)
init(c’)k,c,i
![Page 70: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/70.jpg)
Recon:Update rec-
cmap
Read-Write
User of node i
Joiner
Recon
Read-Write
User of node j
Joiner
decide)c(’’k,c,i
Rec-ack
new-config(c,k)
Report(c)
![Page 71: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/71.jpg)
CONCLUSIONS» RAMBO – an algorithm for implementing a
reconfigurable read/write shared memory in an asynchronous message-passing system
» RAMBO guarantees atomicity, regardless of network instability and timing asynchrony
» When network is stable, RAMBO guarantees good performance under reasonable assumptions
» RAMBO can be viewed as a framework for refinements and optimizations
![Page 72: RAMBO: Reconfigurable Atomic Memory for Dynamic Networks](https://reader036.vdocuments.site/reader036/viewer/2022062323/568166ba550346895ddac177/html5/thumbnails/72.jpg)
THANK YOU !