distributed snapshot (continued). example 1: count the tokens let us verify that chandy-lamport...
TRANSCRIPT
![Page 1: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/1.jpg)
Distributed Snapshot (continued)
![Page 2: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/2.jpg)
Example 1: Count the tokens
Let us verify that Chandy-Lamport snapshot algorithm correctly counts
the tokens circulating in the system
A
B
C
How to account for the channel states? Use sent and received variables for each process.
token no token
token
token
no token
no token
A
B
C
no token
no token
token
Are these consistent cuts?
1
2
3
![Page 3: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/3.jpg)
Example 2: Communicating State Machines
ch1
ch2
i j
up
down
up
state machine
i
state machine
j
send
M
send
M'
down
global state i ch1 j ch2
S0 down φ down φ
1 S up M down φ
2 S up M up M'
3 S down M up φ
receive
M'
receive
M
![Page 4: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/4.jpg)
Something unusual
Let machine i start Chandy-lamport snapshot before it
has sent M along ch1. Also, let machine j receive the
marker after it sends out M’ along ch2. Observe that
the snapshot state is
down up M’
Doesn’t this appear strange? This state was never
reached during the computation!
![Page 5: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/5.jpg)
Understanding snapshot
S0
S1
i sends M
j sends M'
j receives M
j sends M'i receives M'
S1'
S2
S2'
i sends M
j receives M
i receives M'
i receives M' j receives M
S3
S0
S3'
recorded
state SSS
![Page 6: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/6.jpg)
Understanding snapshot
The observed state is a feasible state that is reachable
from the initial configuration. It may not actually be visited
during a specific execution.
The final state of the original computation is always
reachable from the observed state.
![Page 7: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/7.jpg)
Discussions
What good is a snapshot if that state has never been visited by the system?
- It is relevant for the detection of stable predicates.
- Useful for checkpointing.
![Page 8: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/8.jpg)
Discussions
What if the channels are not FIFO?
Study how Lai-Yang algorithm works. It does not use any marker
LY1. The initiator records its own state. When it needs to send a
message m to another process, it sends a message (m, red).
LY2. When a process receives a message (m, red), it records its state
if it has not already done so, and then accepts the message m.
Question 1. Why will it work?
Question 1 Are there any limitations of this approach?
![Page 9: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/9.jpg)
Food for thought
Distributed snapshot = distributed read.
Distributed reset = distributed write
How difficult is distributed reset?
![Page 10: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/10.jpg)
Global State Collection
![Page 11: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/11.jpg)
Global state collection
Some applications
- computing network topology
- termination detection
- deadlock detection
Chandy-Lamport algorithm does a partial job. Each process
generates a fragment of the global state, but these pieces have
to be “stitched together” to form a global state.
![Page 12: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/12.jpg)
A simple exercise
Once the pieces of a consistent
global state become available,
consider collecting the global state via
all-to-all broadcast
At the end, each process
will compute a set V, where
V= {s(i): 0 ≤ i ≤ N-1 }
i
k
j
l
s(i) s(j)
s(k) s(l)
![Page 13: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/13.jpg)
All-to-all broadcast
Program broadcast (for process i}define V.i, W.i : set of values;initially V.i={s(i)}, W.i =
and every channel is empty
do V.i ≠ W.i send (V.i \ W.i) to every outgoing channel; W.i := V.i
[] j: ¬ empty (j, i) receive X from channel(j, i); V.i := V.i X
od
V.iW.i
V.kW.k
(i,k)
Acts like a “pump”
Assume that the topology is a strongly connected graph
V.jW.j
(j,i)
![Page 14: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/14.jpg)
Proof outline
Lemma. empty (i, k) W.i V.k. (Why?)
Lemma. The algorithm will terminate in a bounded number of steps. (Why?)
(we will discuss these in class)
(Upon termination) i: V.i = W.i, and all channels are empty.So, V.i V.k.
In a strongly connected graph, every node is in
a cycle On a cyclic path, V.i = V.k must be
true. Since s(i) V.i, s(i) V.k.
V.iW.i
V.kW.k
(i,k)
i k
![Page 15: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/15.jpg)
Termination detection
During the progress of a distributed computation,
processes may periodically turn active or passive.
A distributed computation termination when:
(a) every process is passive,
(b) all channels are empty, and
(c) the global state satisfies the desired postcondition
![Page 16: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/16.jpg)
Visualizing diffusing computation1
2
3
4
5
1
2
3
4
5
1
2
3
4
5
(a) (b) (c)
Notice how one process engages another process. Eventually all processes turn white, and no message is in transit -- this signals termination. How to develop a signaling mechanism to detect termination?
passive
active
initiator
![Page 17: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/17.jpg)
Dijkstra-Scholten algorithm
An initiator initiates termination detection
by sending signals (messages) down the
edges via which it engages other nodes.
At a “suitable time,” the recipient sends an
ack back.
When the initiator receives ack from every
node that it engaged, it detects termination.
Node j engages node k.
j k
j k
j k
The basic scheme
signal
ack
![Page 18: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/18.jpg)
Dijkstra-Scholten algorithmDeficit (e) = # of signals on edge e - # of ack on edge e
For any node, C = total deficit along incoming edges
and D = total deficit along outgoing edges edges
For the initiator, by definition, C = 0
Dijkstra-Scholten algorithm used the following two
invariants to develop their algorithm:
Invariant 1. (C ≥ 0) (D ≥ 0)
Invariant 2. (C > 0) (D = 0)
0
1
2
4
3
5
![Page 19: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/19.jpg)
Dijkstra-Scholten algorithm
The invariants must hold when
an interim node sends an ack.
So, acks will be sent when
(C-1 ≥ 0) (C-1 > 0 D=0)
{follows from INV1 and INV2}
= (C > 1) (C ≥1 D=0)
= (C > 1) (C =1 D=0)
0
1
2
4
3
5
![Page 20: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/20.jpg)
Dijkstra-Scholten algorithm
program detect {for an internal node i}initially C=0, D=0, parent = i
do m = signal (C=0) C:=1; state:= active; parent := sender{Send signals to engage other nodes if necessary, or turn passive}
[] m = ack D:= D-1[] (C=1 D=0) state = passive
send ack to parent; C:= 0; parent := i[] m = signal (C=1)
send ack to the sender;od
0
1
2
4
3
5
Note that the parent relation induces a spanning tree
![Page 21: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/21.jpg)
Distributed deadlock
When each process waits for some other process (to do something), a deadlock occurs.
Assume each process owns a few resources. Review how resources are allocated, and how a deadlock is created.
Three criteria for the occurrence of deadlock
- Exclusive use of resources- Non-preemptive scheduling- Circular waiting by all (or a subset of) processes
![Page 22: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/22.jpg)
Distributed deadlock
Three aspects of deadlock
– deadlock detection– deadlock prevention– deadlock recovery
![Page 23: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/23.jpg)
Distributed deadlock
• May occur due to bad designs/bad strategy
[Sometimes prevention is more expensive than
detection and recovery. So designs may not care
about deadlocks, particularly if it is rare.]
• Caused by failures or perturbations in the system
![Page 24: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/24.jpg)
Distributed Deadlock Prevention uses pessimistic strategies
An example from banker’s problem (Dijkstra)
A banker has $10,000. She approves a credit line of $6,000 to each of
the three customers A, B, C. since the requirement of each is less than
the available funds.
1. The customers can pay back any portion of their loans at any time. Note
that no one is required to pay any part of the loan unless (s)he has
borrowed up to the entire credit line.
2. However, after the customer has borrowed up to the entire credit line
($6000) (s)he must return the entire money is a finite time.
3. Now, assume that A, B, C borrowed $3000 each.
The state is unsafe since there is a “potential for deadlock.” Why?
![Page 25: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/25.jpg)
Banker’s Problem
Questions for the banker
Let the current allocations be A = $2000, B = $2400,
$C=$1800.
1. Now, if A asks for an additional $1500, then will the
banker give the money immediately?
2. Instead, if B asks for $1500 then will the banker give the
money immediately?
![Page 26: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/26.jpg)
Wait-for Graph (WFG)
• Represents who waits for whom.
• No single process can see the WFG.
• Review how the WFG is formed.
p0 p1
p2p3
p4
![Page 27: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/27.jpg)
Another classification
• Resource deadlock
[R1 AND R2 AND R3 …]
also known as AND deadlock
• Communication deadlock
[R1 OR R2 OR R3 …]
also known as OR deadlock
p0 p1
p2p3
p4
![Page 28: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/28.jpg)
Detection of resource deadlock
Notations
w(j) = true (j is waiting)
depend [j,i] = true j succn(i) (n>0)
(i’s progress depends on j’s progress)
P(i,s,k) is a probe (i=initiator, s= sender, r=receiver)
3
2
4
1
P(4, 4, 3)
initiator
![Page 29: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/29.jpg)
Detection of resource deadlock
{Program for process k}
do P(i,s,k) received
w[k] (k ≠ i) ¬ depend[k, i]
send P(i,k,j) to each successor j;
depend[k, i]:= true
[] P(i,s,k) received w[k] (k = i)
process k is deadlocked
od
1
2 3
4
0
![Page 30: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/30.jpg)
Observations
To detect deadlock, the initiator must be in a cycle
Message complexity = O(|E|)
(edge-chasing algorithm)
1
2 3
4
0
E=set of edgesof the WFG
![Page 31: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/31.jpg)
Communication deadlock
0 1 2 3 4
This WFG has a resource deadlock but no communication deadlock
5
![Page 32: Distributed Snapshot (continued). Example 1: Count the tokens Let us verify that Chandy-Lamport snapshot algorithm correctly counts the tokens circulating](https://reader030.vdocuments.site/reader030/viewer/2022032703/56649d215503460f949f723c/html5/thumbnails/32.jpg)
Detection of communication deadlock
A process ignores a probe, if it is not waiting for any process. Otherwise,
• first probe mark the sender as parent;forwards the probe to successors
• Not the first probe Send ack to that sender
• ack received from every successor send ack to the parent
Communication deadlock is detectedif the initiator receives ack.
0 1 2 3 4
Has many similarities with Dijkstra-Scholten’s termination detection algorithm