1 causal delivery advanced networks phd. saúl pomares hernández

1

Causal DeliveryCausal Delivery

Advanced NetworksAdvanced Networks

PhD. Saúl Pomares HernándezPhD. Saúl Pomares Hernández

7-2

Clocks, events and process statesClocks, events and process states

A distributed system is defined as a collection P of N processes pi, i = 1,2,… N

Each process pi has a state si consisting of its variables (which it transforms as it executes)

Processes communicate only by messages (via a network) Actions of processes:

Send, Receive, change own state Event: the occurrence of a single action that a process carries out as it

executes e.g. Send, Receive, change state Events at a single process pi, can be placed in a total ordering

denoted by the relation i between the events. i.e.

e i e’ if and only if e occurs before e’ at pi

A history of process pi: is a series of events ordered by i

history(pi)= hi = <ei0, ei

1, ei2, …>

7-3

Logical time and logical clocks Happened-before(Lamport 1978)

Instead of synchronizing clocks, event ordering can be used

p1

p2

p3

a b

c d

e f

m1

m2

Physicaltime

1. If two events occurred at the same process pi (i = 1, 2, … N) then they occurred in the order observed by pi, that is

2. when a message, m is sent between two processes, send(m) happened before receive(m)

3. The happened before relation is transitive

a b (at p1) c d (at p2) b c because of m1 also d f because of m2

Not all events are related by consider a and e (different processes and no chain of messages to relate them)they are not related by ; they are said to be concurrent; write as a || e

7-4

Lamport’s logical clocks

A logical clock is a monotonically increasing software counter. It need not relate to a physical clock.

Each process pi has a logical clock, Li which can be used to apply logical timestamps to events LC1: Li is incremented by 1 before each event at process pi LC2:

• (a) when process pi sends message m, it piggybacks t = Li • (b) when pj receives (m,t) it sets Lj := max(Lj, t) and applies LC1

before timestamping the event receive (m)

a b

c d

e f

m1

m2

21

3 4

51

p1

p2

p3

Physical time

e e’ implies L(e)<L(e’)The converse is not true, that is L(e)<L(e') does not

imply e e’

7-5

Vector clocks

Vector clock Vi at process pi is an array of N integers VC1:initially Vi[j] = 0 for i, j = 1, 2, …N VC2:before pi timestamps an event it sets Vi[i] := Vi[i] +1 VC3:pi piggybacks t = Vi on every message it sends VC4:when pi receives (m,t) it sets Vi[j] := max(Vi[j] , t[j]) j = 1,

2, …N ( then before next event adds 1 to own element using VC2)

a b

c d

e f

m1

m2

(2,0,0)(1,0,0)

(2,1,0) (2,2,0)

(2,2,2)(0,0,1)

p1

p2

p3

Physical time

Note that e e’ implies V(e)<V(e’). The converse is also true

7-6

Causal Order deliver [BIR91]

Causal Ordering:

If send(m) send(m’), then k g

deliveryk(m) deliveryk(m’)

T ime

(0,0,0)

m2

A B C(0,0,0) (0,0,0)

(1,0,0)

(1,1,0)

(1,0,0)

m1

The message m2 arrives but can not be delivered

Only after the delivery of m1 , The message m2 also can be delivered.

Initial vector

Delivery condition

if (VT(m’ )[i] = VT(pj)[i] +1 and VT(m’ )[k] VT(pj)[k] (k i, k=1…n)then delivery(m)

7-7

Causal Order deliver

The general algorithm of vector time for causal delivery is as follows: Initially, VT(pi)[j] = 0 j=1…n. For each event send(m) at pi, VT(pi)[i] = VT(pi)[i] + 1. Each multicast message by process pi is timestamped with the

updated value of VT(pi). For each event deliveredj(m’), pj modifies its vector time in the

following manner: VT(pj)[k]=VT(m’)

For each reception receive(m’) à pj , ij, m’=(i,VT(m’),message) To enforce a causal delivery of m’ i. Delivery condition if not (VT(m’)[i] = VT(pj)[i] +1 and VT(m’)[k] VT(pj)[k] (k i,

k=1…n) then wait else ii delivery(m)

7-8

Problem

m1

m2

m3

m4

m5

t

7-9

Causal Order deliver, Multigroup Case

If sendi(m,g) sendj(m’,g’), then k g g’ deliveryk(m) deliveryk(m’)

p1

p2

p3

g1

g3 g2

t

m2

m3

m1

Delivery of event m3 must be delayed.

p2 g1 g2

(1,0,x)(x,0,0)(0,x,0)

(1,0,x)(x,0,0)(1,x,0)

(1,0,x)(x,0,1)(1,x,0)

g1={p1, p2} g2={p2, p3} g3={p1, p3}

m1m2 m3

Delivery condition

- VTa(m)[i] = VTa(pj)[i] +1

- k : (pk ga Λ ki): VTa(m)[k] ≤VT(pj)[k] and

- g : (g Gj): VTg(m) ≤VTg(pj)

g3

g1 g2

p1 p3

p2

7-10

Exercise

p1

p2

p3

ch1

ch3 ch2

e5

e4

e2e3

e1

Delivery of event e5 must be delayed.

p2 ch1 ch2

ch1={?} ch2={?} ch3={?}

?...

?

ch3

ch1 ch2

p1 p3

p2

e1

e5

7-11

The Basic Principles (cont.)The Basic Principles (cont.)

The causal relation, denoted by :

1. x, a y, b if x=y a < b

2. x, a y, b if x, a is the sending of an event and y, b is the delivery of that event .

3. x, a y, b if z, c | (x, a z, c z, c y, b)

7-12

Immediate Dependency Relation

Immediate Dependency Relation :

ee’[ (e e’) e” E, (e e” e’)]

The problem with causal ordering :

The amount of control information emitted for large values of n = |G| is prohibitively high.

7-13

Immediate Dependency Relation (cont.)

Causal Intra-Channel Ordering:

If send(e) send(e’), then k c

deliveryk(e) deliveryk(e’)

Proposition 1:

If e,e’ E send(e) send(e’), then k c deliveryk(e) deliveryk(e’)

7-14

Serial Events

Immediate Dependency Relation :ee’[ (e e’) e” E, (e e” e’)]

e4

e2

e3

e

p p p4 p5

t

1

S1S1

S2S2

S3S3

S4S4

e 2e 2

e 4 e 4

e 3e 3

p1 2 3 4 p5

t

e1 e2 e3 e4

e3 e4

IDR Graph

Immediate Dependency

7-15

Concurrent Events

IDR Graph

Immediate Dependency Relation :ee’[ (e e’) e” E, (e e” e’)]

e5

e2 e3

e 2 e 3

e 5

S1S1

S1S6

S1S8

e 1

S1S2 S1S3S1S4

S1S7

S1S5

e 4

p1 p2 p3 p4 p5

e 6

t

e5

e1

e2

e3 e4

e6

( e2 || ( e3 e4 ) ) e6

Immediate Dependency

7-16

Concurrent Relation || :

e || e’ (e e’ e’ e)

Immediate Dependency Relation :

ee’[ (e e’) e” E, (e e” e’)].

Observation :

(e’ e e” e) e’ || e” e5

e1

e2

e3 e4

e6

7-17

For Multi-Channel Case

Immediate Inter-Channel Dependency Relation :

(e,c)(e’,c’) [((e,c) (e’, c’))(e”, c’’) E,

((e,c) (e”, c’’) (e’, c’) c’’ c c’’c’)]

Observation:

If only one channel exists in the system, then

=

7-18

Causal Inter-Channel Ordering:

If send(e,c) send(e’,c’), then k cc’ deliveryk(e) deliveryk(e’)

Proposition 2:

If e,e’ E send(e,c) send(e’,c’), then k cc’ deliveryk(e) deliveryk(e’)

7-19

Inter-channel Dependency

Immediate Inter-Channel Dependency Relation :(e,c)(e’,c’) [((e,c) (e’, c’))(e”, c’’) E,

((e,c) (e”, c’’) (e’, c’) c’’ c c’’c’)]

Proposition 2: If e,e’ E send(e,c) send(e’,c’), then k cc’ deliveryk(e) deliveryk(e’)

((e1,ch1)(e2,ch3))(e3,ch2)

Events with IICDR to e3

p1

p2

p3

ch1

ch3 ch2

t

e2

e3

e1

Delivery of event e3 must be delayed.

p2 ch1 ch2

7-20

II. For each message diffused by pi into group d 1. VT(pi)[i] = VT(pi)[ i] +1

2. for all iCIci : ci=(k, x, c , ch_dests)

3. if destschcid _. then

4. )},,{( cxkHH ee

5. ddestschcidestschci \_._.

6. endif 7. if destsci. then

8. ciCICI ii \

9. endif 10. endfor 11. e=( i , t=VT(pi)[ i], d, message , He) 12. send(e) into the group d 13. )}\,],)[(,{( dCHdipVTiCICI iiii

The Multi-Channel Causal Algorithm

7-21


III. For each e = ),,,,( eHeventdtk received by pj

To impose a causal delivery Condition of Multi-group delivery 17. If not 1])[(( kpVTt j

18. jej CHcHcxllpVTx ),,(])[( ) then

19. wait 20. else 21. Delivery(message) 22. 1])[(])[( kpVTkpVT jj

23. if jCIdxkx ),,( then

24. jj CICI \ dxkci ,,

25. endif 26. jjj CHdtkCICI ,,,

7-22


I. Initially, 1. VT(pi)[k] = 0 k:1… gG |g| 2. CIi 3. He II. For each message diffused by pi into group d 4. VT(pi)[i] = VT(pi)[ i] +1



7. )},,{( cxkHH ee



11. ciCICI ii \






23. if then

24. jj CICI \

25. endif 26. jjj CHdtkCICI ,,,

27. for all eHcxl ),,(

28. if (c CHj) then 29. if jCIcyly ),,( then/*x ≤ y*/

30. if x < y then /* don´t do anything */ 31. endif 32. if x = y then 33. if (c ≠ d) then

34. MAJ( cylci ,, , d )

35. else /* c = d */ 36. cyljj ciCICI ,,\

37. endif 38. endif 39. endif 40. else /* c CHj */ 41. if then

42. if x < y then /* don´t do anything*/ 43. endif 44. if x = y then

45. MAJ( cylci ,, , d )

46. endif 47. if x > y then 48. VT(pj)[l] = x 49.

jj CICI \

50.

51. endif 52. else /* jCIcyly ),,( */

53. if (VT(pj)[l] < x) then 54. VT(pj)[l] = x 55.

56. endif 57. endif 58. endif 59. endfor 60. endif IV. Updating 61. MAJ(cik,x,c, d) { 62. if (c ≠ d) then 63. destschci cxk _.,, destschci cxk _.,, \ d

64. if then

65. cxkjj ciCICI ,,\

66. endif 67. else /* c = d */ 68. cxkjj ciCICI ,,\

69. endif 70. }




7. )},,{( cxkHH ee



11. ciCICI ii \






23. if then

24. jj CICI \

25. endif 26.


28. if (c CHj) then 29. if then/*x ≤ y*/


34. MAJ( cylci ,, , d )




45. MAJ( cylci ,, , d )


jj CICI \

50.




64. if then



69. endif 70. }

7-23





7. )},,{( cxkHH ee



11. ciCICI ii \






23. if then

24. jj CICI \

25. endif 26.




34. MAJ( cylci ,, , d )


37. endif 38. endif 39. endif 40. else /* c CHj */ 41. if jCIcyly ),,( then


45. MAJ( cylci ,, , d )


jj CICI \ cylci ,,

50. jjj CHcxlCICI ,,,


53. if (VT(pj)[l] < x) then 54. VT(pj)[l] = x 55. jjj CHcxlCICI ,,,


64. if then



69. endif 70. }

7-24





7. )},,{( cxkHH ee



11. ciCICI ii \






23. if then

24. jj CICI \

25. endif 26.




34. MAJ( cylci ,, , d )




45. MAJ( cylci ,, , d )


jj CICI \

50.




64. if destschci cxk _.,, then



69. endif 70. }

7-25

Implementation

The MCP General Structure

ChannelChannel

SessionSession ConsistentSession

ConsistentSession

ParticipantParticipant

Causal Ordering Channel

Causal Ordering Channel

*

1

**

*

1

1 *

* *

**

1 1

*1

JSDT

7-26

Membership

Membership service

The rest of Participants

Participant pk

req_join(ch, pk)

serv_join(ch,pk,np)

init_join(pk, pi,VT(pi)[i])

Wait forserv_join

Wait for np-1 messages init_join.

Actualization of its VT

join(ch, pk)

Memory reservation for a new participant pk

Only after the reception of join, we consider pk like a member of channel ch

The only no causal message

Join Procedure

7-27

Membership

Membership service

The rest of Participants

Participant pk

Leave Procedure

req_leave(ch, pk)

Wait for serv_leave

leave(ch, pk)

leaving notification of pk

serv_leave(ch,pk)

Only after the reception of leave, on efface toute information concernant pk

7-28

Implementation (cont.)

The MCP Architecture

Network

Cooperative Distributed Engineering System

g channels 1 2 . . .

Multi-Channel Coordination Protocol

Java Shared Data Toolkit

Light Reliable Multicast Protocol

1 causal delivery advanced networks phd. saúl pomares hernández

Documents