aran bergman & eddie bortnikov & alex shraer, principles of reliable distributed systems,...

Aran Bergman & Eddie Bortnikov & Alex Shraer, Principles of Reliable Distributed Systems, Spring 20081

Principles of Reliable Distributed Systems

Recitation 2: Broadcast Services

Spring 2009

Alex Shraer


Broadcast Service for Replication

• Primitives: broadcast(m), deliver(m).– For simplicity, assume m is unique.

Network

BroadcastAlgorithm

Application

deliverbroadcast

receivesend

BroadcastAlgorithm

Application

deliverbroadcast

receivesend


Reliable Broadcast Specifications

• Validity: if a correct process broadcasts m then all correct processes eventually deliver m

• Agreement: if a correct process delivers m then all correct processes eventually deliver m– Uniform Agreement: if any process delivers m then all

correct processes eventually deliver m

• Integrity: m is delivered by a correct process at most once, and only if it was previously broadcast


Reliable Broadcast - Quiz

• What happens if a process fails during the broadcast of a message?

• Does a message delivery by a faulty process require the delivery of this message by correct processes?


FIFO Broadcast• Why is FIFO important?• FIFO Order: If a process broadcasts a message m before it

broadcasts a message m’, then no correct process delivers m’ unless it has previously delivered m.

• FIFO Broadcast: Reliable broadcast + FIFO Order

• Alternative definition of FIFO Order?– “all messages broadcast by the same process are delivered to all

processes in the order they are sent”

• Quiz: Are these definitions equivalent?


Example

m1

m2

m3

p (fau lty)

q (correct)

• Also, this alternative definition forces faulty processes to deliver messages. (impossible)


Causal Broadcast• Why is causality important?• Event e causally precedes event f (e→f) iff:

– a process executes both e and f, in that order, or– e is the broadcast of some message m and f is the delivery of m,

or– There is an event h, such that e→h and h→f.

• Causal Order: If the broadcast of a message m causally precedes the broadcast of a message m’, then no correct process delivers m’ unless it has previously delivered m.

• Causal Broadcast: Reliable broadcast + Causal order


Atomic Broadcast and Uniformity

• Why would we want more than Causal Broadcast? • Atomic Broadcast: Reliable Broadcast + Total Order• Total Order: if correct processes p and q both deliver

messages m and m’, then p delivers m before m’ if and only if q delivers m before m’.


Broadcast Primitives


Uniformity• Agreement, Integrity and Order place no restrictions on the behavior of faulty processes.• Uniform – limit the behavior of faulty processes

• Example 1: Agreement allows a faulty process to deliver a message that is never delivered by correct processes

• Uniform Agreement: If a process (whether correct or faulty) delivers a message m, then all correct processes eventually deliver m.

• Example 2: Integrity allows a faulty process to deliver a message more than once, and to deliver messages ‘out of thin air’

• Uniform Integrity: For any message m, every process (whether correct or faulty) delivers m at most once, and only if some process broadcast m.

• Likewise, we can strengthen the Order properties:• Uniform FIFO Order: If a process broadcasts a message m before it broadcasts a

message m’, then no process (whether correct of faulty) delivers m’ unless it has previously delivered m.

• Uniform Causal Order: If the broadcast of a message m causally precedes the broadcast of a message m’, then no process (whether correct or faulty) delivers m’ unless it has previously delivered m.

• Uniform Total Order: if any processes p and q (whether correct or faulty) both deliver messages m and m’, then p delivers m before m’ iff q delivers m before m’.


Crash Failures

• Suppose processes are only subject to crash failures– They operate correctly up to the time they crash (by

definition).

• Can we assume that the message deliveries that a process makes before crashing are always ‘correct’ (consistent with those of correct processes)?– No


Crash Failures (cont’d)

• Coordinator-based algorithm:– When a process intends to broadcast a message m, it

first sends m to a coordinator.

– The coordinator delivers messages in the order in which it receives them, and periodically informs the other processes of this message delivery order.

– Other processes deliver messages according to this order.

– If the coordinator crashes, another process takes over as coordinator.


Crash Failures (cont’d)• The algorithm satisfies the specification Atomic

Broadcast• Suppose a coordinator delivers m before m’ and

crashes.• A new coordinator could think m’ is before m. • All correct processes follow the new coordinator• Thus, the old coordinator delivered messages out

of order before it crashed.– Inconsistency can occur even when there are only crash

failures. – Protocols should explicitly prevent inconsistency even

when there are only crash failures


LTS Broadcast Algorithm - code for process pi

Logical Clock Assignment: TS[j] ← 0, j=0,…,n pending ← empty

broadcast (m) TS[i] ← TS[i] + 1 send (m, TS[i], i) to all

upon receive (m, t, j) TS[j] ← t add (m, t, j) to pending TS[i] ← max (TS[i], t) + 1

Delivery Rule let (m, t, j) be the entry in pending with the smallest t, j if t, j TS[k],k k=0,…n then

deliver (m) remove (m, t, j) from pending


p1

p2

p3

m1

m1

m2

m2

m3

m3 m4

m4

t=0 t=5 t=10

0

0

1,2

2

3,1

3,3

4

5

6,2

7

14

3

6

האם הרשת מעבירה את כל ההודעות בסדר שמשמר את יחס •)? happens-before (כלומר causalityה-

בכל פעם שהוא משתנה. סמנו את ערך logical clockסמנו את ערך • המצורף לכל הודעה.LTSה-

. אם m3 ו m1 להודעות delivery) כל תהליך מבצע tציינו מתי (באיזה • ציינו זאת בטבלה.deliveryבריצה המתוארת לא מתבצע

m3 m1

p1

p2

p3

Example Exam Question

0 2 73 8

1212

77

710

Delivery according to LTS


Vector Clocks

• At process pi, on broadcast(m)

– VC[i] := VC[i]+1– use reliable broadcast to send m with VC to all– deliver m locally

• Upon receive m– place in message buffer

• Deliver m from pj from buffer if

– VC[j] = m.VC[j] – 1 – forall k≠j : VC[k] ≥ m.VC[k]

• Upon deliver – VC[j] := VC[j] + 1

VC[j] is the number of messages of pj that causally precedepi’s subsequent

messages

FIFO


Example Exam Question – Cont.

שהיה מצורף לכל הודעה אם היינו משתמשים בהם.Vector Clockסמנו את ה-•. אם בריצה m3 ו m1 להודעות delivery) כל תהליך מבצע tציינו מתי (באיזה •

ציינו זאת בטבלה.deliveryהמתוארת לא מתבצע

p1

p2

p3

m1

m1

m2m

2

m3

m3 m4

m4

t=0 t=5 t=10

[0,0,0]

[0,0,0]

[0,0,0]

[0,1,0]

[0,1,0]

[0,1,0]

[1,1,0]

[0,1,1]

[1,1,1]

[1,1,1]

[1,1,0]

[1,2,1]

[1,2,1]

[0,1,0][0,1,1]

[1,1,0]

[0,1,1]

[1,2,1]

[1,2,1]

m3 m1

p1

p2

p3

34

17

27

Delivery according to VC

aran bergman & eddie bortnikov & alex shraer, principles of reliable distributed systems,...

Documents

broadcast slide

fifo broadcast

causal broadcast

reliable broadcast quiz

messages broadcast

atomic broadcast

broadcast primitives

broadcast receivesend