distributed systems group membership and view …7 group membership • like with a failure...
TRANSCRIPT
![Page 1: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/1.jpg)
Distributed Systems
Group Membership and View Synchronous Communication
Prof R. Guerraoui Distributed Programming Laboratory
![Page 2: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/2.jpg)
2
Group Membership
B
A
C
Who is there?
![Page 3: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/3.jpg)
3
Group Membership
• In some distributed applications, processes need to know which processes are participating in the computation and which are not
• Failure detectors provide such information; however, that information is not coordinated (see next slide) even if the failure detector is perfect
![Page 4: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/4.jpg)
4
p1
p2
p4
crash
Perfect Failure Detector suspect(p2,p3)
suspect(p2)
suspect(p2,p3)
suspect(p3) suspect()
p3 crash
![Page 5: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/5.jpg)
5
p1
p2
p4
crash
Group Membership V1 = (p1,p4)
V1 = (p1,p4)
p2 crash
![Page 6: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/6.jpg)
6
Group Membership • To illustrate the concept, we focus here on a
group membership abstraction to coordinate the information about crashes
• In general, a group membership abstraction can also typically be used to coordinate the processes joinning and leaving explicitly the set of processes (i.e., without crashes)
![Page 7: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/7.jpg)
7
Group Membership • Like with a failure detector, the processes are
informed about failures; we say that the processes install views
• Like with a perfect failure detector, the processes have accurate knowledge about failures
• Unlike with a perfect failure detector, the information about failures are coordinated: the processes install the same sequence of views
![Page 8: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/8.jpg)
8
Group Membership
Memb1. Local Monotonicity: If a process installs view (j,M) after installing (k,N), then j > k and M < N
Memb2. Agreement: No two processes install views (j,M) and (j,M’) such that M ≠ M’
Memb3. Completeness: If a process p crashes, then there is an integer j such that every correct process eventually installs view (j,M) such that p ∉ M
Memb4. Accuracy: If some process installs a view (i,M) and p ∉ M, then p has crashed
![Page 9: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/9.jpg)
9
Group Membership
Events
Indication: <membView, V>
• Properties:
• Memb1, Memb2, Memb3, Memb4
![Page 10: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/10.jpg)
10
Algorithm (gmp) Implements: groupMembership (gmp).
Uses:
PerfectFailureDetector (P).
UniformConsensus(Ucons).
upon event < Init > do
view := (0,S);
correct := S;
wait := true;
![Page 11: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/11.jpg)
11
Algorithm (gmp – cont’d) upon event < crash, pi > do
correct := correct \ {pi};
• upon event (correct < view.memb) and (wait = false) do
• wait := true;
• trigger<ucPropose,(view.id+1,correct) >;
![Page 12: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/12.jpg)
12
Algorithm (gmp – cont’d)
• upon event < ucDecided, (id, memb)> do
• view := (id, memb);
• wait := false;
• trigger < membView, view>;
![Page 13: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/13.jpg)
13
p1
p2
p4
crash
Algorithm (gmp)
UCons((p1,p2,p4);(p1,p2,p4))
p3 crash
UCons((p1,p4);(p1,p4))
UCons((p1,p3,p4);(p1,p2,p4))
UCons((p1,p4);(p1,p4))
![Page 14: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/14.jpg)
14
Group Membership and Broadcast
crash
membView(p1,p3)
p1
p2
p3
m
m
membView(p1,p3)
![Page 15: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/15.jpg)
15
• View synchronous broadcast is an abstraction that results from the combination of group membership and reliable broadcast
• View synchronous broadcast ensures that the delivery of messages is coordinated with the installation of views
View Synchrony
![Page 16: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/16.jpg)
16
View Synchrony
Besides the properties of group membership (Memb1-Memb4) and reliable broadcast (RB1-RB4), the following property is ensured:
VS: A message is vsDelivered in the view where it is vsBroadcast
![Page 17: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/17.jpg)
17
Events
Request:
<vsBroadcast, m>
• Indication:
• <vsDeliver, src, m>
• <vsView, V>
View Synchrony
![Page 18: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/18.jpg)
18
View Synchrony
If the application keeps vsBroadcasting messages, the view synchrony abstraction might never be able to vsInstall a new view; the abstraction would be impossible to implement
We introduce a specific event for the abstraction to block the application from vsBroadcasting messages; this only happens when a process crashes
![Page 19: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/19.jpg)
19
Events
Request:
<vsBroadcast, m>; <vsBlock, ok>
Indication:
<vsDeliver, src, m>; <vsView, V>; <vsBlock>
View Synchrony
![Page 20: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/20.jpg)
20
Algorithm (vsc) Implements: ViewSynchrony (vs).
Uses:
GroupMembership (gmp).
TerminatingReliableBroadcast(trb).
BestEffortBroadcast(beb).
![Page 21: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/21.jpg)
21
upon event < Init > do
view := (0,S); nextView := ⊥;
pending := delivered := trbDone := ∅;
flushing := blocked := false;
Algorithm (vsc – cont’d)
![Page 22: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/22.jpg)
22
Algorithm (vsc – cont’d)
upon event <vsBroadcast,m> and (blocked = false) do
delivered := delivered ∪ ⎨ m ⎬
trigger <vsDeliver, self, m>;
trigger <bebBroadcast, [Data,view.id,m>;
![Page 23: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/23.jpg)
23
Algorithm (vsc – cont’d)
upon event<bebDeliver,src,[Data,vid,m]) do
If(view.id = vid) and (m ∉ delivered) and (blocked = false) then
delivered := delivered ∪ ⎨ m ⎬
trigger <vsDeliver, src, m >;
![Page 24: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/24.jpg)
24
upon event < membView, V > do addtoTail (pending, V);
Algorithm (vsc – cont’d)
upon (pending ≠ ∅) and (flushing = false) do
nextView := removeFromhead (pending);
flushing := true;
trigger <vsBlock>;
![Page 25: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/25.jpg)
25
Algorithm (vsc – cont’d)
Upon <vsBlockOk> do
blocked := true;
trbDone:= ∅;
trigger <trbBroadcast, self, (view.id,delivered)>;
![Page 26: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/26.jpg)
26
Algorithm (vsc – cont’d)
Upon <trbDeliver, p, (vid, del)> do
trbDone := trbDone ∪ ⎨p⎬;
forall m ∈ del and m ∉ delivered do
delivered := delivered ∪ ⎨ m ⎬;
trigger <vsDeliver, src, m >;
![Page 27: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/27.jpg)
27
Upon (trbDone = view.memb) and (blocked = true) do
view := nextView;
flushing := blocked := false;
delivered := ∅;
trigger <vsView, view>;
Algorithm (vsc – cont’d)
![Page 28: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/28.jpg)
28
Consensus-Based View Synchrony
Instead of launching parallel instances of TRBs, plus a group membership, we use one consensus instance and parallel broadcasts for every view change
Roughly, the processes exchange the messages they have delivered when they detect a failure, and use consensus to agree on the membership and the message set
![Page 29: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/29.jpg)
29
Algorithm 2 (vsc) Implements: ViewSynchrony (vs).
Uses:
UniformConsensus (uc).
BestEffortBroadcast(beb).
PerfectFailureDetector(P).
![Page 30: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/30.jpg)
30
upon event < Init > do
view := (0,S);
correct := S;
flushing := blocked := false;
delivered := dset := ∅;
Algorithm 2 (vsc – cont’d)
![Page 31: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/31.jpg)
31
Algorithm 2 (vsc – cont’d)
upon event <vsBroadcast,m) and (blocked = false) do
delivered := delivered ∪ ⎨ m ⎬
trigger <vsDeliver, self,m>;
trigger <bebBroadcast,[Data,view.id,m] >;
![Page 32: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/32.jpg)
32
Algorithm 2 (vsc – cont’d) upon event<bebDeliver,src,[Data,vid,m]) do
if (view.id = vid) and m ∉ delivered and blocked = false then
delivered := delivered ∪ ⎨ m ⎬
trigger <vsDeliver, src, m >;
![Page 33: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/33.jpg)
33
upon event < crash, p > do correct := correct \ ⎨ p ⎬; if flushing = false then
flushing := true; trigger <vsBlock>;
Algorithm 2 (vsc – cont’d)
![Page 34: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/34.jpg)
34
Algorithm 2 (vsc – cont’d)
Upon <vsBlockOk> do
blocked := true;
trigger <bebBroadcast, [DSET,view.id,delivered] >;
![Page 35: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/35.jpg)
35
Algorithm 2 (vsc – cont’d)
Upon <bebDeliver, src, [DSET,vid,del] > do
dset:= dset ∪ (src,del);
if forall p ∈ correct, (p,mset) ∈ dset then trigger <ucPropose, view.id+1, correct, dset >;
![Page 36: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/36.jpg)
36
Algorithm 2 (vsc – cont’d) Upon <ucDecided, id, memb, vsdset > do
forall (p,mset) ∈ vsdset: p ∈ memb do
forall (src,m) ∈ mset: m ∉ delivered do
delivered := delivered ∪ ⎨m⎬
trigger <vsDeliver, src, m>;
view := (id, memb); flushing := blocked := false; dset := delivered := ∅;
trigger <vsView, view>;
![Page 37: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/37.jpg)
37
Uniform View Synchrony
We now combine the properties of
group membership (Memb1-Memb4) – which is already uniform
uniform reliable broadcast (RB1-RB4) – which we require to be uniform
VS: A message is vsDelivered in the view where it is vsBroadcast – which is already uniform
![Page 38: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/38.jpg)
38
Uniform View Synchrony
Using uniform reliable broadcast instead of best effort broadcast in the previous algorithms does not ensure the uniformity of the message delivery
![Page 39: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/39.jpg)
39
Uniformity?
crash
vsView(p1,p3)
p1
p2
p3
m
m
vsView(p1,p3)
vsDeliver(m) m
m
![Page 40: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/40.jpg)
40
upon event < Init > do
view := (0,S);
correct := S;
flushing := blocked := false;
udelivered := delivered := dset := ∅;
for all m: ack(m) := ∅;
Algorithm 3 (uvsc)
![Page 41: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/41.jpg)
41
Algorithm 3 (uvsc – cont’d)
upon event <vsBroadcast,m) and (blocked = false) do
delivered := delivered ∪ ⎨m⎬;
trigger <bebBroadcast,[Data,view.id,m] >;
![Page 42: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/42.jpg)
42
Algorithm 3 (uvsc – cont’d) upon event<bebDeliver,src,[Data,vid,m]) do
if (view.id = vid) then
ack(m) := ack(m) ∪ ⎨src⎬;
if m ∉ delivered then
delivered := delivered ∪ ⎨ m ⎬
trigger <bebBroadcast, [Data,view.id,m] >;
![Page 43: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/43.jpg)
43
Algorithm 3 (uvsc – cont’d) upon event (view ≤ ack(m)) and (m ∉ udelivered) do
udelivered := udelivered ∪ ⎨ m ⎬
trigger <vsDeliver, src(m), m >;
![Page 44: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/44.jpg)
44
upon event < crash, p > do correct := correct \ ⎨ p ⎬; if flushing = false then
flushing := true; trigger <vsBlock>;
Algorithm 3 (uvsc – cont’d)
![Page 45: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/45.jpg)
45
Algorithm 3 (uvsc – cont’d) Upon <vsBlockOk> do
blocked := true;
trigger <bebBroadcast, [DSET,view.id,delivered] >;
Upon <bebDeliver, src, [DSET,vid,del] > do
dset:= dset ∪ (src,del);
if forall p ∈ correct, (p,mset) ∈ dset then trigger <ucPropose, view.id+1, correct, dset >;
![Page 46: Distributed Systems Group Membership and View …7 Group Membership • Like with a failure detector, the processes are informed about failures; we say that the processes install views](https://reader034.vdocuments.site/reader034/viewer/2022051917/600919a0e940e438b627bc51/html5/thumbnails/46.jpg)
46
Algorithm 3 (uvsc – cont’d) Upon <ucDecided, id, memb, vsdset > do
forall (p,mset) ∈ vs-dset: p ∈ memb do
forall (src,m) ∈ mset: m ∉ udelivered do
udelivered := udelivered ∪ ⎨m⎬
trigger <vsDeliver, src, m>;
view := (id, memb); flushing := blocked := false; dset := delivered := udelivered := ∅;
trigger <vsView, view>;