distributed algorithms – 2g1513 lecture 10 – by ali ghodsi fault-tolerance in asynchronous...

Distributed Algorithms – 2g1513

Lecture 10 – by Ali GhodsiFault-Tolerance in Asynchronous Networks

2

Consensus Problems

Consensus problems very important in DS Distributed Databases

All processes must agree whether to commit or abort a transaction

If any process says abort, all processes should abort

Atomic Broadcast All processes receive the same set of messages coming

from correct processes only Can be used to implement consensus, vice versa

3

Fischer, Lynch, Paterson 1983/85 Consensus cannot be solved in

asynchronous model With possibility of one process crashing

http://www.sics.se/~ali/flp85.pdf

Most influential paper award PODC 2001

4

Modified Model

To proof the result, we will modify our model of a distributed system slightly

Processes execute local algorithms, modeled by a STS

But, given any state, a correct process can always execute a “dummy” instruction For any state in a process, there exists a transition There exists always an applicable event on every process

A crashed process, cannot make any transitions

5

Definition: T-crash fair executions A t-crash-robust algorithm is a consensus algorithm

if it satisfies:

Termination All correct processes eventually decides

Agreement In every configuration, the decided processes should have decided

for the same value (0 or 1)

Non-triviality There exists at least one possible input configuration where the

decision is 0 There exists at least one possible input configuration where the

decision is 1 Example, maybe input “0,0,1”->0 while “0,1,1”->1

6

Definitions

0-decided configuration A configuration with decide ”0” on some process


0-valent configuration A configuration in which every reachable decided configuration is a 0-decide

1-valent configuration A configuration in which every reachable decided configuration is a 1-decide

Bivalent configuration A configuration which can reach a 0-decided and 1-decided configuration

7

Definitions Illustrated 1(4)


0-decided configuration

{ STATE2,

STATE,5

DECIDE-0,

STATE7

{msg1, msg2}

}

At least of them is in state DECIDE-0

msg1

msg

2

P1 state2

P2 state5

P4 state7

P3 decide0

8

Definitions Illustrated 2(4) 0-valent configuration

No 1-decided configurations are reachable Future determined, means ”everyone will decide 0”

0- valent configuration

{ P1_state,

P2_state,

P3_state,

P4_state,

{msg1}

}

0-valent configuration

{ P1_state,

P2_state2,

P3_state,

P4_state,

{msg1}

}


{ decide-0,

P2_state,

P3_state,

P4_state,

{msg1, msg2}

}


{ decide-0,

P2_state2,

P3_state2,

P4_state,

{msg1, msg2}

}


{ decide-0,

P2_state,

P3_state,

decide-0,

{ msg2}

}


{ decide-0,

P2_state2,

P3_state2,

decide-0,

{ msg2}

}


{ decide-0,

P2_state,

decide-0,

P4_state,

{msg1, msg2}

}


{ decide-0,

P2_state3,

P3_state,

decide-0,

{}

}

9

Definitions Illustrated 3(4) 1-valent configuration

No 0-decided configurations are reachable Future determined, means ”everyone will decide 1”

0- valent configuration

{ P1_state,

P2_state,

P3_state,

P4_state,

{msg1}

}


{ P1_state,

P2_state2,

P3_state,

P4_state,

{msg1}

}


{ decide-1,

P2_state,

P3_state,

P4_state,

{msg1, msg2}

}


{ decide-1,

P2_state2,

P3_state2,

P4_state,

{msg1, msg2}

}


{ decide-1,

P2_state,

P3_state,

decide-1,

{ msg2}

}


{ decide-1,

P2_state2,

P3_state2,

decide-1,

{ msg2}

}


{ decide-1,

P2_state,

decide-1,

P4_state,

{msg1, msg2}

}


{ decide-1,

P2_state3,

P3_state,

decide-1,

{}

}

10

Definitions Illustrated 4(4) Bivalent configuration

Both 0 and 1-decided configurations are reachable Future undetermined, could go either way…

bivalent configuration

{ P1_state,

P2_state,

P3_state,

P4_state,

{msg1}

}


{ P1_state,

P2_state2,

P3_state,

P4_state,

{msg1}

}


{ decide-1,

P2_state5,

P3_state6,

P4_state5,

{msg1, msg3}

}


{ decide-0,

P2_state2,

P3_state2,

P4_state,

{msg1, msg2}

}


{ decide-1,

P2_state5,

P3_state6,

decide-1,

{ msg2}

}


{ decide-0,

P2_state2,

P3_state2,

decide-0,

{ msg2}

}


{ decide-0,

P2_state,

decide-0,

P4_state,

{msg1, msg2}

}


{ decide-1,

P2_state9,

P3_state6,

decide-1,

{}

}

11

Bivalent Initial Configuration

Theorem For any algorithm that solves the 1-crash

consensus problem there exists an initial bivalent configuration

12

Proof 1/(10)

We know that the algorithm must be non-trivial There should be some initial configuration that will lead to a

0-decide There should be some initial configuration that will lead to a

1-decide

Take two such configuration i1 and i2

E.g. 4 processes initial values (0,1,0,1,1) lead to 1 Initial values (0,0,1,0,0) lead to 0

13

Proof 2/(10)

We know there exists inputsp1, p2, p3, p4, p5

(0,1,0,1,1) leading to 1

(0,0,1,0,0) leading to 0

Lets look at other initial configurations by flipping the inputs transforming the upper input to the lower input

14

Proof 3/(10)


(0,1,0,1,1) leading to 1 (0,0,0,1,1) leading to ?

(0,0,1,0,0) leading to 0

Lets look at other initial configurations by

flipping the inputs transforming the upper

input to the lower input

15

Proof 4/(10)


(0,1,0,1,1) leading to 1 (0,0,0,1,1) leading to ? (0,0,1,1,1) leading to ?

(0,0,1,0,0) leading to 0




16

Proof 5/(10)


(0,1,0,1,1) leading to 1 (0,0,0,1,1) leading to ? (0,0,1,1,1) leading to ? (0,0,1,0,1) leading to ? (0,0,1,0,0) leading to 0




17

Proof 6/(10)


(0,1,0,1,1) leading to 1 (0,0,0,1,1) leading to ? (0,0,1,1,1) leading to ? (0,0,1,0,1) leading to ? (0,0,1,0,0) leading to 0

There must exist two neighboring configurations here, with two different outcomes




18

Proof 7/(10)


(0,1,0,1,1) leading to 1 (0,0,0,1,1) leading to 1 (0,0,1,1,1) leading to 1 (0,0,1,0,1) leading to 0 (0,0,1,0,0) leading to 0

Assume the following two

Lets look at other initial configurations by flipping the inputs

19

Proof 8/(10)


(0,1,0,1,1) leading to 1 (0,0,0,1,1) leading to 1 (0,0,1,1,1) leading to 1 (0,0,1,0,1) leading to 0 (0,0,1,0,0) leading to 0


Identical configurations except for process p4

20

Proof 9/(10)


(0,0,1,1,1) leading to 1 (0,0,1,0,1) leading to 0

The consensus algorithm should tolerate if p4 crashes! (0,0,1,X,1), leads to ? (either 0 or 1)


21

Proof 10/(10)


(0,0,1,1,1) leading to 1 (0,0,1,0,1) leading to 0

The consensus algorithm should tolerate if p4 crashes! (0,0,1,X,1), leads to ? (either 0 or 1)

If it leads to 1, then depending on whether p4 crashes or not (0,0,1,0,1) either leads to 0 or 1 (bivalent)

If it leads to 0, then depending on whether p4 crashes or not(0,0,1,1,1) either leads to 0 or 1 (bivalent)


22

Initial Bivalence

Intuition Given any algorithm, we can find some start state, that depending on the

failure of one process, will either lead to a 0-decide or a 1-decide

Bivalent Initial Config

{ P1_state,

P2_state,

P3_state,

P4_state,

{msg1}

}


{ P1_state,

P2_state2,

P3_state,

P4_state,

{msg1}

}


{ P1_state,

P2_state,

P3_state,

P4_state,

{msg1, msg2}

}


{ decide-1,

P2_state2,

P3_state2,

P4_state,

{msg1, msg2}

}


{ decide-0,

P2_state,

P3_state,

P4_state,

{ msg2}

}


{ P1_state,

P2_state,

decide-1,

P4_state,

{msg1, msg2}

}


{ decide-0,

decide-0,

P3_state,

decide-0,

{}

}

23

Coarse-grained Model of Distributed Systems In our model, we will now let each event be the

receipt of a message After the receipt of a message m, a process

deterministically makes all internal and send events it can do

In other words, we make our course-grained model a bit more fine-grained An event represents the receipt of a message, some internal

transitions and the sending of some messages

A receipt of message m at process p is always applicable if a message m with destination p is in the network

24

Intuition behind model

receive <tok, y> from q

for x:=1 to 3 do

beginy:=y+1;

send <tok, y> neighp[x];

end

receive <tok, z> from q;

print z+y

Receipt event e

Initial state of p

State of p after receipt of e

Deterministic transitions

Receipt event f

Deterministic transitionsState of p after receipt of f

25

Order of events

Intuition The order in which two applicable events are

executed is not important!

Order Theorem Let ep and eq be two events on two different

processors p and q which are both applicable in configuration . Then ep can be applied to eq(), and eq can be applied to ep().

Moreover, ep(eq()) = eq(ep() ).

26

Definitions

A sequence of events =( e1, e2,…,ek) is applicable in configuration if e1 is applicable in , e2 applicable in e1() ...

If the resulting configuration is we write ()= or

If only contains events of a subset of the processes P, we write P

27

Order of sequences

Diamond Theorem Let sequences 1 and 2 be applicable in

configuration , and let no process participate in both 1 and 2. Then 2 is applicable in 1(), 2 is applicable in 2(), and 1(2())=2(1())

Proof By induction using the order theorem

28

Illustration of the Diamond Theorem

1 2

1() 2()

2 1

=2(1())=1(2())

29

Bivalent Configuration

Any configuration of the 1-robust consensus algorithm is exactly one of these three Bivalent 0-valent 1-valent

Why? Any configuration leads to a decide because of termination We know bivalent configurations exist If it is not bivalent, it must lead to either 0-decide or 1-

decide, so it is either 0-valent or 1-valent

30

Bivalent Configurations

In any bivalent config , either one applicable event goes to a bivalent config, or there exists two applicable events, leading to a 0-

valent and 1-valent configurations (respectively)

Bivalent BivalentBivalent 1-valent

0-valent

Case 1 Case 2

31

Staying Bivalent

Theorem Given any bivalent config and an event e

applicable in There exists another reachable config where e is applicable,

and e() is bivalent

Bivalent …

Theorem Illustration

eBivalent …

e

…

…e

Bivalent

32

Proof definitions

Assume e involves process p

Call the set of all possible configs reachable from without applying e the set C

Apply event e to all configs in C and call the resulting configs D

Bivalent

…

e


…

…

…

…

……

…

ee

…

…e

…e

C

D

…e

33

Proof intuition

We will proof that D contains a bivalent config by contradiction

I.e., assume there exists no bivalent config in D, show that this will lead to a contradiction or absurdity

Bivalent

…

e


…

…

…

…

……

…

…

ee

e

…

…e

…e

C

D

34

Proof

Assume D contains no bivalent configs I.e. all configs in D are either 0-valent or 1-valent

Then it follows that there exists a 0-valent and a 1-valent config in D (next slides)

35

Proof

We know we can reach a 0-valent and 1-valent config from , call them 1 and 2 (non-triviality)

Either 1 and 2 are in C or they are not in C

If inside C, then e(1) and e(2) is in D and they are 0-valent/1-valent

Bivalent

…

e

1 and 2 are in C 1 and 2 are not in C

1

…

…

…

2 …

…

…

ee

e

…

…e

…e

C

Bivalent

…

e

…

…

…

2

1

…

ee

e

…

…e

…e

C

36

Proof

If not inside C, then some1 and 2 exists on the path to 1 and

2, such that e(1) and e(2) are in D and they are 0-valent/1-valent

[Remember we assumed no bivalent config available in D]

Bivalent

…

e

1 and 2 are in C 1 and 2 are not in C

1

…

…

…

2 …

…

…

ee

e

…

…e

…e

C

Bivalent

…

e1

…

…

…

2

2

1

…

ee

e

…

…e

…e

C

37

Reflection

We now know that D must always contain a 0-valent and 1-valent config, assuming no bivalent config exists in D

Lets call the two 0-valent and 1-valent configs in D, d0 and d1

We will now show that this situation is a contradiction itself. Hence, D must contain a bivalent config

38

f

Deriving the contradiction

There must exist two configs c0 and c1 in C

such that c1=f(c0), and d0=e(c0) and d1=e(c1)

c0 c1

d0 d1

e e

C

D Lets see why!

39

Proofing two neighbors exist 1(4) We know is bivalent, and e() is in D and is either 0-valent or

1-valent, assume 0-valent

0-valent

e

C

D

40



There is a reachable 1-valent config in D

f0 1

0-valent

e e

C 2 … m

1-valent

D

41




e is applicable in each i, and must be 0-valent or 1-valent

1

0-valent 1-valente e

C 2 … m

x-valent y-valent z-valent

D

e e e

f0

42

There exists two neighbors, one 1-valent and one 0-

valent

Proofing two neighbors exist 4(4)

1

0-valent 1-valente e

C 2 … m

0-valent 1-valent z-valent

D

e e e

f0 f1 f2 f3

We know is bivalent, and e() is in D and is either 0-valent or 1-valent, assume 0-valent


e is applicable in each i, and must be 0-valent or 1-valent

43


valent




e is applicable in each i, and is 0/1-valent

f1C 2

0-valent 1-valent

D

e e

44


valent

Neighbors lead to contradiction 1(3) We now know there exist two configs c0 and c1 in C such that

c1=f(c0), and d0=e(c0) and d1=e(c1)

Either the events e and f happen on the same processor or on different processors, both cases will lead to contradictions

f1C 2

0-valent 1-valent

D

e e

45

Neighbors lead to contradiction 2(3) We now know there exist two configs c0 and c1 in C such that

c1=f(c0), and d0=e(c0) and d1=e(c1)

Assume e and f happen on two different processes p and q Then, the order of their execution can be exchanged

fc0 c1

d1

e e

C

D0-valent 1-valent

fd0

Contradiction as d0 is 0-valent, but it can lead to a 1-

valent config, hence d0 must be bivalent, but we assumed

no bivalent configs exist in D

46

Neighbors lead to contradiction 3(3) We now know there exist two configs c0 and c1 in C such that c1=f(c0), and d0=e(c0) and

d1=e(c1)

Assume e and f happen on the same process p, the algorithm should still work if p is silent

fc0 c1 d1

e e

C

0-valent 1-valent

d0

Contradiction as A should be a 0/1-valent configuration, but we have shown

that A can lead to both 0 and 1

f2 ee A

If p is silent, the algorithm should continue and terminate with a decision in some config A

0

If p is silent, some

execution leading to 0

should exist

1

If p is silent, some

execution leading to 1

should exist

47

Proof Map

Assume there is no bivalent config in D

We know all configs in D are 0-valent or 1-valent

Show that we can find a 0-valent and 1-valent config in D

Show that two neighboring configs c0─e→c1 exist, where c0

─f→”0-valent config”, c1 ─f→”1-valent config”

Show this is a contradiction

Assumption must be incorrectD must contain a bivalent configuration

48

Final Theorem

No deterministic 1-crash-robust consensus algorithm exists for the asynchronous model

Proof1. Start in a initial bivalent config

2. Given the bivalent config, pick the event e that has been applicable longest

Pick the execution taking us to another config where e is applicable

Apply e, and get a bivalent config

3. Repeat 2.

49

Consensus not Impossible!

Lets do deterministic consensus algorithm for the a different failure model Initially dead processes

Assume t failures can happen initially

Where t=4 for N=10, t=5 for N=11

Let L denote L=6 for N=10, L=6 for N=11

2

1Nt

2

1NL

N=t+L

50

Intuition

Assume N processes are connected in a underlying graph, and at most t fail

We know L processes are alive after the start Broadcast your identity, and receive/collect L identities

For any two correct processes, their set of collected identities will overlap Quorom concept There are N nodes, any two processes have L identities

each, i.e. total

N+1 identities, total N nodes, at least two must be same (PHP)

12

122

NN

L

51

Initially Dead Consensus

Receive L messages

Initial state of p

Any two processes have overlapping Succ

Keep identity of senders in Rcvd

Wait until you’ve received a message from every process that is transitively in each Succ

Every process has the same set Alive

52

Summary

We have proved that a 1-crash resilient deterministic consensus algorithm does not exist

Hence, there exists always an execution which stays in bivalent configurations and still keeps applying all applicable events!

All correct processes execute infinite number of events, and still leads to no decision!

We have shown an algorithm for consensus which is for the initially dead processes model

distributed algorithms – 2g1513 lecture 10 – by ali ghodsi fault-tolerance in asynchronous...

Documents