distributed systems total order broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 ....
TRANSCRIPT
![Page 1: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/1.jpg)
Distributed systems
Total Order Broadcast
Prof R. Guerraoui Distributed Programming Laboratory
![Page 2: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/2.jpg)
Overview Intuitions: what is total order broadcast?
Specifications of total order broadcast
Consensus-based total order algorithm
![Page 3: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/3.jpg)
Broadcast
B
A
C
m
m
deliver
broadcast
deliver
![Page 4: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/4.jpg)
Intuitions (1) In reliable broadcast, the processes are free to deliver messages in any order they wish In causal broadcast, the processes need to deliver messages according to some order (causal order) The order imposed by causal broadcast is however partial: some messages might be delivered in different order by the processes
![Page 5: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/5.jpg)
Reliable Broadcast
p1
p2
p3
m2
m1
m1
m2
m1
m3
m1 m2 m3
m3
m3
m2
![Page 6: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/6.jpg)
Causal Broadcast
p1
p2
p3
m2
m1
m1
m2
m1
m3
m1 m2 m3
m3
m3
m2
![Page 7: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/7.jpg)
Intuitions (2) In total order broadcast, the processes must deliver all messages according to the same order (i.e., the order is now total) Note that this order does not need to respect causality (or even FIFO ordering) Total order broadcast can be made to respect causal (or FIFO) ordering
![Page 8: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/8.jpg)
Total Order Broadcast (I)
p1
p2
p3
m2
m1
m3
m2
m1
m3
m1 m2 m3
m1
m3
m2
![Page 9: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/9.jpg)
Total Order Broadcast (II)
p1
p2
p3
m1
m2
m3
m1
m1
m3
m2 m1 m3
m2
m3
m2
![Page 10: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/10.jpg)
Intuitions (3) A replicated service where the replicas need to treat the requests in the same order to preserve consistency
(we talk about state machine replication) A notification service where the subscribers need to get notifications in the same order
![Page 11: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/11.jpg)
Modules of a process
request
indication
indication
indication
request
![Page 12: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/12.jpg)
Overview Intuitions: what is total order broadcast?
Specifications of total order broadcast
Consensus-based algorithm
![Page 13: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/13.jpg)
Total order broadcast (tob) Events
Request: <toBroadcast, m>
Indication: <toDeliver, src, m>
• Properties:
• RB1, RB2, RB3, RB4
• Total order property
![Page 14: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/14.jpg)
Specification (I)
Validity: If pi and pj are correct, then every message broadcast by pi is eventually delivered by pj
No duplication: No message is delivered more than once
No creation: No message is delivered unless it was broadcast
(Uniform) Agreement: For any message m. If a correct (any) process delivers m, then every correct process delivers m
![Page 15: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/15.jpg)
Specification (II)
(Uniform) Total order:
Let m and m’ be any two messages.
Let pi be any (correct) process that delivers m without having delivered m’
Then no (correct) process delivers m’ before m
![Page 16: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/16.jpg)
Specifications
Note the difference with the following properties:
Let pi and pj be any two correct (any) processes that deliver two messages m and m’. If pi delivers m’ before m, then pj delivers m’ before m.
Let pi and pj be any two (correct) processes that deliver a message m. If pi delivers a message m’ before m, then pj delivers m’ before m.
![Page 17: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/17.jpg)
p1
p2
p3
m1
m2 p4
crash
crash m2
m1
![Page 18: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/18.jpg)
p1
p2
p3
m1
m2 p4
crash
crash m2
m1
![Page 19: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/19.jpg)
Overview Intuitions: what total order broadcast can bring?
Specifications of total order broadcast
Consensus-based algorithm
![Page 20: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/20.jpg)
(Uniform) Consensus In the (uniform) consensus problem, the processes propose values and need to agree on one among these values
C1. Validity: Any value decided is a value proposed
C2. (Uniform) Agreement: No two correct (any) processes decide differently
C3. Termination: Every correct process eventually decides
C4. Integrity: Every process decides at most once
![Page 21: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/21.jpg)
Consensus Events
Request: <Propose, v>
Indication: <Decide, v’>
• Properties:
• C1, C2, C3, C4
![Page 22: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/22.jpg)
Modules of a process
request
indication
indication
indication
request indication request
![Page 23: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/23.jpg)
Algorithm Implements: TotalOrder (to).
Uses:
ReliableBroadcast (rb).
Consensus (cons);
upon event < Init > do
unordered: = delivered: = ∅;
wait := false;
sn := 1;
![Page 24: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/24.jpg)
Algorithm (cont’d) upon event < toBroadcast, m> do
trigger < rbBroadcast, m>;
upon event <rbDeliver,sm,m> and (m ∉ delivered) do
unordered := unordered U {(sm,m)};
upon (unordered ≠ ∅) and not(wait) do
wait := true:
trigger < Propose, unordered>sn;
![Page 25: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/25.jpg)
Algorithm (cont’d) upon event <Decide,decided>sn do
unordered := unordered \ decided;
ordered := deterministicSort(decided);
for all (sm,m) in ordered:
trigger < toDeliver,sm,m>;
delivered := delivered U {m};
sn : = sn + 1;
wait := false;
![Page 26: Distributed systems Total Order Broadcast€¦ · p1 p2 p3 m2 m1 m3 m2 m1 m3 m2 m1 m3 m1 m3 m2 . Total Order Broadcast (II) p1 p2 p3 m1 m2 m3 m1 m1 m3 m1 m2 m3 m2 m3 m2 . Intuitions](https://reader033.vdocuments.site/reader033/viewer/2022051607/60360809cd515813016554a5/html5/thumbnails/26.jpg)
Equivalences 1. One can build consensus with total order
broadcast
2. One can build total order broadcast with consensus and reliable broadcast
Therefore, consensus and total order broadcast are equivalent problems in a system with reliable channels